Download Memristive Devices in Analog Neuromorphic Circuits Hermann Kohlstedt Nanoelektronik Technische Fakultät

Memristive Devices in Analog Neuromorphic Circuits
Hermann Kohlstedt
Nanoelektronik
Technische Fakultät
Christian-Albrechts-Universität zu Kiel
1
NanoNetwork Workshop_Bergen_June 2013
A Brain replaced by Computer Chips
c
V = 1000 cm3
b
A chip 1 cm
N
a
2
4
2 10 chips
2 104 1010
N transistors
P total
6
10 W
P MOSFET
2 1014
1 MW (!)
5 nW
approx. number of synapses
P brain
P synapse
25 W
250 fW
2
Contents
• Introduction
• Neurobiology – A few Milestones
• Neuromorphic Electronics
• Two examples: Pavlov`s Dog and an Amoeba
• A memresistive Flash cell
• Summary
3
Introduction
Computer - Brain
Computer: Arithmetic operation
2376492
= 1541,5875
Computing Gap
Brain: Pattern Recognition / Associative Memory
Vacation:
4
Introduction
Neurons for information processing
Synapse
Dendrite
Data spikes
Bible of analog VLSI for Neural Circuits:
Analog VLSI and Neural Systems
Carver Mead, Addison‐Wesley 1989, p. 44
M. Mahowald, R.Rodney Douglas, A silicon neuron, Nature 1991
Soma
Axon
A survey of Bio‐Inspired and other alternative Architectures
D. Hammertrom in:
Nanotechnology, Vol. 4, Ch. 10, p. 252
Wiley, 2008, ed. by R. Waser
Review: G. Indiveri et al. Neuromorphic silicon neuron circuits,
frontiers in Neuroscience 5, article 73 (2011).
5
Introduction
Spikes – the information units
Pulse duration:
3, 5 ms (in electronics: 60 ns)
Signal speed ‐ along the axon:
100 m/s
(in electroncis 2.4 x 108 m/s)
6
Neurobiology – A few Milestones
Santiago Ramón y Cajal
Cajal: Learning means, that the synaptic interconnection are not fixed. They adjust in correspondence to the input signals from the environment.
In other words: He suggest already that something like a synaptic cleft must exist! (in 1890!!)
In Search of Memory, Eric R. Kandel, W. W. Norton & Company, New York 2006.
S. R. Cajal , La fine structure des Centres Nerveux, Proc. R. Soc. London (B) 1894 , 55 , 444
7
Neurobiology – A few Milestones
“The memory in brains is distributed over
the whole “system” but certain regions store different aspects!”
Donald O. Hebb.
Hebbs learning rule: When an Axon of cell A excites cell B and repeatedly or persistently takes part in it's firing,
some growth process of metabolic changes take place in one or both cells. Thus, that's efficiency is increased!
"Cells that fire together, wire together."
D. O. Hebb , The Organization of Behavior , John Wiley , New York 1949 .
8
Neurobiology – A few Milestones
Long Term Depression (LTD)
Long Term Potentiation (LTP)
Hippocampal Brain Slice
T. V. P. Bliss and T. LØmo, Long‐Lasting Potentiation of Synaptic Transmission in the Dentate Area of the anaesthetized Rabbit Following Stimulation of the Perforant Path, J. Physiol. 232, 331 (1973).
From Molecules to Networks, Ed. John H. Byran and James L. Roberts, Academic Press 2009:J. H. Byren et al., Learning and Memory Basic Mechanisms: , Chap 19 p. 541
9
What means learning in biological Systems?
Three Levels
1
Behavior
Psychology
Implicit learning
Explicit learning
Automatic in quality:
habituation, sensitization,
classical conditioning
Conscious or declarative:
Recall people, places, facts, and events etc.
2
Networks
Architecture
3
Nerve Cells
Biochemistry
10
A reductionistic Principle
Aplysia California: a Snail
E. R. Kandel, Science 294, 1030 (2001).
In Search of Memory
Eric R. Kandel
W. W. Norton Company 2006
To bridge the Gap between Behavior and Cell Biology
11
A reductionistic Principle
In Search of Memory, Eric R. Kandel, W. W. Norton & Company, New York 2006.
12
Leon Chua`s Memristor
I
V
L. O. Chua,
Memristor – the missing circuit element, IEEE Trans. Circuit Theory 18, 507 (1971).
See also: materials today Dec. 2011 Memory matters and MRS Bulletin, Resistive switching phenomena in thin films, Feb. 2012
13
You have the choice – a few Examples
Which memristive Device should I use?
Reviews: Doo Seok Jeong et al. Rep. Prog. Phys. 75 (2012)
S. D. Ha and S. Ramanathan, JAP 110 (2011)
Ferroelectric Tunnel Junctions
Andrè Chanthbouala, et al.
Nature Nanotechnology 2012
Ti‐Oxide
Ionics and Tunnel Barriers
D. S. Jeong et al. Solid‐State Electronics 63, 1 (2011)
Nanoinonics
D. B. Strukov, G. S. Snider, D. R. Stewart, R. S. Williams, Nature 2008, 453, 80. MgO
Spin Transfer Torque Devices
P. Krzysteczko et al., Adv. Mater. 2012
Nanoionics
R. Waser , R. Dittmann , G. Staikov , K. Szot
Adv. Mater. 2009
14
Memristive Devices for Neuromorphic Systems
Memristive devices as artifical synapses
• synaptic plasticity: spike timing dependent plasticity (STDP)
Sung Hyun Jo et al., Nano Lett. 10, 1297‐1301 (2010).
• precondition of learning: long term potentiation
T. Ohno et al., Nature Materials 10, 591–595 (2011).
15
Pavlov`s Dog: Classical Conditioning
Image No. 0030628
Credit:
The Granger Collection, NYC — All rights reserved.
16
Pavlov`s Dog: Classical Conditioning
Associative Learning
IVAN PETROVICH PAVLOV (1905)
• Experiment to understand implicit learning in biological systems. after conditioning
before conditioning
Experimental Psychology and Psychopathology in Animals, Vol. 1 p. 47‐60, Ivan P. Pavlov, Lectures on Conditioned Reflexes, International Pub., New York 1928
17
Neural mediating circuit for associative learning
• Electrical circuit layout: single memristive device implemented in an analogue circuitry
Adder
Comparator
Unconditional stimulus (UCS)
Reference set-point:
Threshold Vcth
+
OP1
Vcth
Conditional stimulus (CS)
+
R1
OP2
-
Vmth
Alertness Vout
RM
VM
• Voltage divider
comprizing a memristive device
M. Ziegler, et al., Advanced Functional Materials, 22, 2744 (2012)/Experimental
O. Bichler et al. Neural Computation 25, 549 (2013)/Experimental
Y. V. Pershin and M. Di Ventra, Neural Networks 23 (2010)/ Emulator
18
Implicit learning
• circuit with threshold voltages for the comparator and mem device
Vbell < Vcth & Vfood> Vcth
Vbell + Vfood > Vpmth (before conditioning) & Vbell > Vcth (after conditioning)
19
Device requirements
• Pt/SiO2/Ge0.3Se0.7/Cu memristive device in voltage divider
R. Soni et al., J. Appl Phys. 110, 054509 (2011).
• synaptic potentiation via transition LRS to HRS
Current (m A)
0.5
Vpmth= 0.33V
Vnmth= -0.18V
1
3
0.0
2
4
-0.5
0.47 k Ω
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
Voltage (V)
• effective threshold voltage of the device
20
Amoeba: Physarum polycephalum
• Anticipation to enviormental changes for periodic repetition
z
z
Unicellular organism, able to solve mazes
Interesting candidate to study basic cognitive functions
T. Ueda, Hokkaido University
Learning behavior in biological systems
21
Amoeba Anticipate Periodic Events
T. Saigusa, A. Tero, T. Nakagaki, Y. Kuramoto, Phys. Rev. Lett, 100, 018101 (2008)
Biological experiment:
Humidity
Temperature
favorable
unfavorable
anticipated events
22
A memristive circuit model to mimic an amoeba
Y. V. Pershin, S. La Fontaine and M. Di Ventra, Phys. Rev. E 80, 021926, 2009.
Simulation: Electronic Emulator
R
memristive device
L
resonant circuit
C
Problems for experimental implementation:
z
Circuit parameter
R = 1 Ω L = 2 H C = 1 F
z
Using a real memristive device
23
Amoebae anticipation
Simulation
Periodic input pattern needed for learning
Y. V. Pershin and M. Di Ventra, Adv. Phys. 60 (2011) and references therein
24
Electronic circuit
R = 100 Ω
L = 100mH
memristive device
C = 50 nF
I Output current
R = 10 kΩ
25
Requirements for the memristive device
z
High off‐resistance for an ideal LC circuit
z
Change to on‐state requires a threshold voltage
z
High Reset voltage in respect to set voltage
1.
Al
TiO2‐x
Ag
26
Experimental implementation:
M. Ziegler, et al. An electronic implementation of amoeba anticipation
Applied Physics A (2013) anticipated events
27
Amoeba anticipation Periodic Events
T. Saigusa, A. Tero, T. Nakagaki, Y. Kuramoto, Phys. Rev. Lett, 100, 018101 (2008)
z
Better anticipation to environmental changes for periodic repetition
28
Amoeba anticipation
Non-periodic
pattern
Response (µA)
Periodic
pattern
at resonance
frequency
Vin (V)
Vin (V)
29
You have the choice – a few Examples
Which memristive Device should I use?
Reviews: Doo Seok Jeong et al. Rep. Prog. Phys. 75 (2012)
S. D. Ha and S. Ramanathan, JAP 110 (2011)
Ferroelectric Tunnel Junctions
Andrè Chanthbouala, et al.
Nature Nanotechnology 2012
Ti‐Oxide
D. S. Jeong et al. Solid‐State Electronics 63, 1 (2011)
Nanoinonics
D. B. Strukov, G. S. Snider, D. R. Stewart, R. S. Williams, Nature 2008, 453, 80. MgO
Spin Transfer Torque Devices
P. Krzysteczko et al., Adv. Mater. 2012
Nanoionics
R. Waser , R. Dittmann , G. Staikov , K. Szot
Adv. Mater. 2009
30
Floating Gate Transistor as Memristive Device?
H. C. Card and W.R. Moore, Electronic Letters 25, 805 (1989).
C. Diorio, P. Hasler, B.A. Mimich, and C. A. Mead, IEEE Trans. on Elec. Dev. 43, 1972 (1996).
Three terminal devices:
Write
Read
Erase
What about:
• Memristive operation mode of a single EEPROM cell • Reduction to a two‐terminal device: simultaneous read/write
31
A two‐terminal MemFlash‐cell
M. Ziegler, et al., Appl Phys. Lett. 101, 263504 (2012). • Reduction to a two‐terminal device: simultaneous read/write
32
MemFlash
33
How large is the benefit of memristive Devices for Neuromorphic Electronics?
• Memristive devices with improved performance:
Yield, parameter spread, retention, etc. • System architecture:
Mixed signal circuits including memristive devices
•Which neurobiological schemes are essential ?
Long Term Potentiation, Spike Time Dependent Plasticity, Feedback Loops, Coding, Encoding etc.
Doo Seok Jeong et al. Towards artificial and synapses: a material point of view
RSC Advances 3, 3169 (2013).
34
Thanks to …
Martin Ziegler, Mirko Hansen, Christoph Riggert, Rohit Soni
and Marina Ignatov
Karlheinz Ochs, Thomas Mussenbrock
Wolfgang Krautschneider, Dietmar Schröder Thorsten Bartsch Doo Seok Jeong Paul Meuffels
AG Nanoelektronik 2012
Financial support from Schleswig‐Holsteins Landesgraduiertenförderung is gratefully acknowledged.
35
…and Axel as Pavlov`s Dog,…
…my daughter Nora for painting her dog
36