Download nn2new-02

Formal neurons I
Before we start artificial NN, we look at single neuron first
Topics:
Neuronal Activity
Dendritic tree: receive spikes
Soma: generate spikes
Axon: sent out spikes
Purkinje neuron
Pyramidal neuron
Neurons
Hillock
input
output
Single neuron activity
• Membrane potential is the voltage difference between neuron
and its surrounding (0 mV)
Membrane potential
Cell
Cell
Cell
Cell
0 Mv
Single neuron activity
•If you measure the membrane potential of a neuron and print it out
on the screen, it looks like (from time 0 to 60 minutes)
spike
Single neuron activity
•A spike is generated when the membrane potential is greater than
its threshold
Single neuron activity
•We can forget all sub-threshold activity and concentrate on
spikes, which are signals sent to other neurons
Spikes
Spike trains
• Sequence of spikes, generated due to simulations (inputs to a
neuron)
Spikes
• Only spikes are important since other neurons receive them
(signals)
•
Neurons communicate with spikes
•
Information is coded by spikes
• If we can manage to measure the spiking time, we decipher
how the brain works
Life is not so simple
• spiking time in the cortex is random
With identical input
for the identical neuron
spike patterns are similar, but not identical
Recording from a real neuron: membrane potential
Single spiking time is meaningless
To extract useful information, we have to average
 for a group of neurons in a local circuit where neuron
codes the same information
 over a time window
to obtain the firing rate r
Single spiking time is meaningless
To extract useful information, we have to average
 for a group of neurons in a local circuit where neuron
codes the same information
 over a time window
to obtain the firing rate r
r =
=
Local circuit
Single spiking time is meaningless
To extract useful information, we have to average
 for a group of neurons in a local circuit where neuron
codes the same information
 over a time window
to obtain the firing rate r
r =
=
Local circuit
= 6 Hz
Time window = 1 sec
Hence we have firing rate of a group of neuron
Next we turn our attention to how to go from a group of local
neurons to neural networks
Topics:
Neuronal Activity
From neurons to neural
networks
From group of neurons to networks
R = f ( w j rj )
r1
w1: synaptic strength
wn
rn
ri is the firing rate of input local circuit
The neurons at output local circuits receives signal in the form
N
wr
i =1
i
i
The output firing rate of the output local circuit is then given by
N
R
R = f (  wi ri )
i =1
where f is the activation function, generally taking as the
Sigmoidal or other forms
wi weight, (synaptic strength) measuring how strong is the
interaction between neurons.
Therefore the key for neural networks
is to understand the local input-output relationship
m
yi = f ( wij x j  bi )
j =1
A single neuron have 5 components
1. Input x
2. Weight w
m
yi = f ( wij xj  bi )
j =1
A single neuron have 5 components
1. Input x
2. Weight w
3. Bias b
m
yi = f ( wij xj  bi )
j =1
A single neuron have 5 components
1.
2.
3.
4.
Input x
Weight w
Bias b
Activation function f
m
yi = f ( wij xj  bi )
j =1
A single neuron have 5 components
1.
2.
3.
4.
5.
Input x
Weight w
Bias b
Activation function f
Output y
m
yi = f ( wij xj  bi )
j =1
A single neuron have 5 components
1.
2.
3.
4.
5.
Input x
Weight w
Bias b
Activation function f
Output y
y = f ( S xi wi + b )
m
yi = f ( wij xj  bi )
j =1
Artificial Neural networks
Local circuits (average to get firing rates)
Single neuron (send out spikes)