Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Ghent University Compact hardware for real-time speech recognition using a Liquid State Machine Benjamin Schrauwen – Michiel D’Haene David Verstraeten – Jan Van Campenhout Electronics and Information Systems Department Ghent University – Belgium IJCNN 2007 Intro • Goal: • Create isolated digits speech recognition in digital FPGA hardware • Real-time processing • As small as possible • First introduce LSM based speech recognition • Investigate two existing hardware architectures (to fast, to large) • Introduce new hardware architecture Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 2/18 LSM based speech recognition Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 3/18 Ear model Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 4/18 Liquid State Machine • Recurrent structures without the training: Reservoir Computing •Jaeger (2001): Echo State Networks (engineering) •Maass (2002): Liquid State Machines (neuroscience) •Steil (2003): weight dynamics of Atiya-Parlos equivalent • Fixed, random topology operated in correct dynamic regime • Different node types possible: THG, linear, tanh, spiking, … • Linear “readout” function which is trained (No local minima, no problems with recurrent structure, one shot learning) • On-line computing: prediction at every time-step • Any time-invariant filter with fading memory can be learned (with output feedback, universal computing) Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 5/18 Reservoir computing Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 6/18 error error Influence of parameters dynamic regime reservoir size error Not very important: •Connection fraction •Exact topology •Weight distribution timescale Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 7/18 Spiking Neural Networks • Incoming spikes influence the membrane potential • When certain threshold θ is reached: reset and fire membr θ t input output Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 8/18 Readout and post-processing Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 9/18 Digital spiking neurons • SNN: mathematically a more complex model than ANN • But: better implementable in hardware • No weight multiplications: table look-up • Filtering can be implemented using shifts and adds • Interconnection only single bit, and sparse communication • Asynchronous communication easily implementable Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 10/18 Digital spiking neurons • Hardware can take advantage of parallelism • But area-speed trade-off: we don’t have to make the implementation faster than needed by the application • For trade-off: different implementations with other area-speed needed • Possible parallelisms: • • Network parallelism • Neuron/synapse parallelism • Arithmetic parallelism We implemented: • SPPA: network parallel, neuron serial, arithmetic parallel • PPSA: network parallel, neuron parallel, arithmetic serial • SPSA: network serial or parallel, neuron serial, arithmetic serial Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 11/18 SPPA • Much like classical CPU • [Roggen 2003][Upegui 2005] Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 12/18 Serial adder PPSA • Use FPGA features • [Girau2006] • [Schrauwen2006] SRL16 Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 13/18 SPSA • Everything serial • PEs are very small (4 LUTs) • Many parallel PEs possible • SIMD controller architecture • Memory based interconnect Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 14/18 Area-speed trade-off for speech task •Speech task in hardware •LSM with 200 neurons •16 kHz processing speed •Real-time requirement LUTs memory Real-time SPPA 13812 900 kbit 347 PPSA 13426 58 kbit 205 SPSA 10PE 488 144 kbit 2.2 SPSA 5PE 489 144 kbit 1.1 SPSA 1PE 489 144 kbit 0.23 Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 15/18 RC Toolbox • Freely available RC Matlab toolbox (www.elis.ugent.be/rct) • Simulation models for hardware quantization • HW design methodology: • Generic network and node settings • Readout pipeline • Generate network with hardware constraints • Evaluate node quantization effects • Automatically export to HW description Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 16/18 System 12 MHz 50 MHz Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 100 MHz 150 MHz interc Xilinx ML 401 17/18 Conclusions • Hardware real-time speech recognition in HW is possible with very limited hardware • Presented novel architecture for SNN implementation in HW • Enlarges area/speed design space drastically • Uses RC toolbox simulation environment for easy porting • Future work: experiment with different applications; add further SNN features such as SDTP, IP, dyn. synapses; and add the possibility to change the weights Compact hardware for real-time speech recognition using a LSM IJCNN – August 13, 2007 18/18