Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
ge Miao Hai Yiran Qing and Garrett S. 1.Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA 15261. 2.2. Air Force Research Laboratory, Information Directorate, 525 Brooks Road, Rome, New York, USA 13441. 1 Chen , 2 Wu 2 Rose Convergence Speed Analysis Memristor-based BSB Recall Circuit In this work, we propose a hardware realization of the Brain-Statein-a-Box (BSB) neural network model training algorithm. This method can be implemented as an analog/digital mixed-signal circuit to train memristor crossbar arrays within BSB circuits. The training effect is demonstrated through experimentation and the quality as an autoassociative memory is also analyzed and compared with software based training methods. The impacts of non-ideal device characteristics and fabrication defects in crossbar arrays are discussed. Our hardware architecture shows great potential for low power, high speed, small hardware size computations, and provides inherent security features. Simulation Results The training effect is achieved. An clear convergence gap can be observed between trained pattern and other untrained patterns. Physical constraints in hardware training Voltage L Memristor Theory z h Voltage Voltage Voltage Voltage L TiO L z Pt 2Pt RL hL h z TiO L Pt Pt TiO 2-x2TiO2 z RL RL Pt h h TiO TiO 2 TiO222-x TiO RL2-x Doping RL TiO RHL R Pt TiO front TiO2-x 2-x TiO Doping Doping 2-x R R H H Pt Pt front front Pt Doping (a) (b) Doping RH R H Pt Ptfront R H TiO2 R Pt front L (a) (a) (b) (b) TiO2-x (a) (b) (b) (a) (b) In 2008, HP Lab demonstrated the first memristive device, in which the RH was achieved by moving the doping front along a TiO Pt memristive effect 2 thin-film. Figure (a) and(b)(b) illustrate the conceptual view of the TiO2 thin-film memristor and the corresponding variable resistor model, which is equivalent to two serially-connected resistors. Here, RL (RH) is used to denote the low (high) resistance state (LRS and HRS). The overall resistance or memristance can be expressed as: z M ( ) RH (1 ) RL where is the relative doping front position, which is the ratio of doping front position over the total thickness of the TiO2 thin-film. When the electrical excitation through a memristor is greater than the threshold voltage, the memristance changes (in training). Otherwise, a memristor behaves like a resistor. POSTER TEMPLATE BY: www.PosterPresentations.com 60% Hardware (linear) Hardware (nonlinear) Lillo (1994) Perfetti (1995) Park (2010) 40% 20% 0% 1 100% Embedded BSB Training Circuit wij (t j y j ) i wij sign (t j y j ) sign ( i ) Vin(i)>0 Training starts M1& M2, ST Initialization ST = m YES Training ends Vout(i)>0 Vout(i) > Vth_h Vin BSB Recall Circuit R/W Control 60% 40% Hardware (linear) Hardware (nonlinear) Lillo (1994) Perfetti (1995) Park (2010) r1 r2 r3 20% 0% Prototype patterns γ(k), k =1,...,5 Diff(i)[0] Vout(i) > Vth_l Vout Vout(i) > Vth_h BSB recall circuit ST & Arbiter Diff = 0 NO ST = 0 r4 r5 Table 1. Quality of domain of attraction. Msel Vin Training signal generation & program memristor crossbar 80% Uniform size of domain of attraction. Training pattern selection Error detection, refresh Diff 3 Error correction rate. Vop , if V Vop S ' (v) V , if Vop V Vop . V , if V V op op Algorithm modification: 2 Hamming distance (l) V(t 1) S ' (G1VA (t ) G1VA (t ) G2 V(t )) NO Pt 80% 0 Registers γ(k), k=1...m Limited data access: A crossbar array represents the terms in a weight matrix by analog states of memristors – memristance. Reading the exact memristance of the entire array is significantly costly in terms of design complexity as well as performance overhead. Limited accuracy of signal detection: The output signals of the memristor crossbar in analog format shall be used to control the training scheme. The accuracy in detecting these output signals affects the quality, speed, and cost of the overall training procedure. Non-ideal device characteristics: For example, the evolution of the memristance for actual memristive devices tends to be nonlinear. Therefore, applying the same excitation can result in different memristance change, depending on the state of memristor. Process variations and defects: Due to process variations, memristors in a crossbar array are not always the same. For example, the upper- and lower-bounds of memristance varies [16][17]. Moreover, “dead cells” stuck at high-resistance state (HRS) or low-resistance state (LRS) could exist. 100% Uniform size of domain of attraction Abstract 1 Li , Error correction rate 1 Hu , Training Signal Generate Error Detection ± Vth_l ± Vth_h Vout(i) > Vth_l YES ST++ Diff Training complete (a) STEP 1: Initializing the crossbar arrays. At the beginning of a training procedure, all memristance values in M1 and M2 are initialized to approximate the average of RL and RH. The initialization doesn’t have to be precisely accurate. Indeed, even when all the memristors are all at LRS or HRS, the crossbar arrays can still be successfully trained but it requires more time to reach convergence. STEP 2: Selecting a prototype pattern (k) Bn(k=1,…,m). Here, B is the n-dimension binary space (1, 1). Assume a training set includes m prototype patterns and each pattern (k) has the same probability to be chosen every time. Here, we use a counter ST to record the number of patterns that have passed training in sequence. When ST > 0, the corresponding patterns that passed training are excluded from the selection. STEP 3: Sending (k) to BSB recall circuit. We convert (k) in binary space (1, 1) to a set of input voltages within the boundary (0.1V, 0.1V). These input signals are supplied to the two memristor crossbars simultaneously. The output signals can be obtained at the end of the BSB recall circuit. Diff(i)[1] (b) Diff(i)[1] Diff(i)[0] Diff(i) XX 11 00 0 11 11 00 11 -1 1 (c) STEP 4: Error detection. Since an input signal Vin(i) is ±0.1V, the preset threshold voltages can be used to compare Vout(i) and λ∙ Vin(i),. We employ four threshold voltages, including: Vth _ h 0.1 Vth_ l 0.1 th _ h V 0.1 th _ l V 0.1 Here, θ represents the tolerable difference. STEP 5: Program memristor crossbar. Note that training the memristor crossbar array is conducted column by column. To train the jth column, the polarity and amplitude of the training pulse is determined by Diff(j). The training pulses are supplied on the rows of memristor crossbar arrays. The jth column is connected to ground and all the other columns are floating. For a column, the training pattern is either the current selected prototype pattern or the opposite pattern. STEP 6: If training is completed? The counter ST increases by 1 if a prototype pattern goes through STEP 25 and reports no error without further tuning M1 and M2. Otherwise, ST is reset to 0 whenever an error is detected. Repeat STEP 25 until ST reaches m. Hardware (linear) Hardware (nonlinear) Lillo (1994) Perfetti (1995) Park (2010) Best 419 465 164 478 502 Good 6 0 1 34 10 LRS/LRS (G1=30) HRS/HRS (G1=30) HRS/HRS (G1=50,digonal) Negative 465 473 859 512 512 Bad 134 86 0 0 0 LRS/HRS (G1=30) HRS/HRS (G1=50) 3000 Quality of Domain of attraction ge Training Scheme Analysis for Memristor-Based Neuromorphic Design 2500 2000 1500 1000 500 0 1 2 3 4 5 Dead cell percentage (%) Acknowledgement This material is based upon work supported by the National Science Foundation under Grant No. EECS-1311747, ECCS-1202225, and CNS-1253424. Received and cleared for public release by AFRL on May 1, 2012, case number 88ABW-2012-2568. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF, AFRL or its contractors.