Download 6.2. How does nonlinear neuron work?

6.2. How does nonlinear neuron work?
(Translated by Joanna Masapust, [email protected])
Let’s start form a simple program which shows how one nonlinear neuron works. I remind you the scheme of
that neuron in the figure 6.1.
Fig. 6.1. The structure of a nonlinear neuron constructed by adding nonlinear transfer function to a linear neuron.
The program, which models this neuron (Example 06a) is analogous with the program Example 01c,
which you used for exploring how a linear neuron works. This example will help you to understand step by step
how a simple nonlinearly characterized (in this example it is threshold characteristics) neuron works. In the
program you can choose between so called unipolar and bipolar characteristics, so at the beginning the program
asks you to choose one of them - see fig. 6.3a - field Neuron type (additionally you can set the amount of
neuron’s input but at first I would recommend accepting default value 4 and going to next window by clicking
Next). Despite the fact that certain names sound complicated, the thing is in fact simple: the unipolar
characteristic determines that the output signal is always nonnegative (usually it is 1 or 0 but certain nonlinear
neurons can also give output values included somewhere between these extremes). On the other hand, using the
bipolar characteristics it is possible to get either positive or negative signal values (usually it is +1 or
-1 but here also the intermediate values are possible). A comparison between the unipolar and bipolar
characteristics is shown in the fig. 6.2, so you can learn the exact difference. There is only one thing I want to
add.
The real, biological neurons do not know the term “negative signal”. All signals in your brain can be
only positive (or zero if you are being lazy!), therefore the unipolar characteristic is definitely more appropriate
in terms of the biological reality. However, considering technical neural networks, more important for us is to
get an useful computational tool, than to try copying the biological model with maximal accuracy. That is why
considering the characteristics used in neural technology we usually are making some “misuse”. It is because we
are trying to avoid situation, during which 0 signals would appear in the network, especially when signals
generated by one neuron are the input signals for other neurons in the same network. As you probably remember
- network learns badly if it encounters 0 signals. (Do you remember maybe the play with filtering signals using
neural network which I proposed you in the last section? It was plainly visible there!). Therefore, beside a
network composed of elements accepting 0 and 1, bipolar structures are being implemented in the neural
networks technology. In these types of structures we also have two types of signals, but they are labelled as +1
and -1. Simple, isn’t it? Look at it more closely in the figure 6.2.
Neuron
response
Neuron
response
+1
+1
excitation
excitation
reaction threshold
-1
reaction threshold
-1
Fig. 6.2. Nonlinear neuron characteristics: unipolar (left side) and bipolar (right side)
The usage of the programme Example 06a is simple and intuitively obvious. Playing with this programme you
will probably notice that it is more categorical than the linear neuron, which you found out earlier (cf. fig. 6.3b).
That previously considered neurons (as well as networks built from them) react to input signals in a subtle and
balanced way: certain combinations of input signals induce a strong reaction (high output signal), whereas other
result in much weaker reactions or the network seems to be almost totally unresponsive to them (the output
signal is close to zero). As opposed to these subtle linear neurons - nonlinear neurons work according to “all-ornone” law. To a certain combination of signals the neuron can react definitely negative (the output signal reaches
-1), but sometimes a minimal change in input signals is enough for the neuron output signal to become definitely
and completely positive (the output signal changes to +1) - fig. 6.4.
Fig. 6.3a. The beginning of conversation with the programme Example 06a
The categorical
response of a
neuron according to
„all-or-none” law.
Fig. 6.3b. The next stage of conversations with the programme Example 06a
A slight change
in the input signal
x(3) caused
positive neuron
reaction.
The. 6.4. The sensitivity of a nonlinear neuron to the change of input signals.
The programme Example 06a described above is very poor and primitive. Despite this - from all programmes
considered until now - the technique it uses and the way it works are very similar to these in the real brain. A few
simple experiments, similar to these which you had carried out with linear neurons, will help you to understand
how this programme works. It is not a big thing, nothing especially interesting. Just a simple function. But, using
your imagination a bit, you can see that there you are experimenting on a real nerve cell…
…In a nickel-plated handle of an instrument an open scull is locate, and inside it, grey, wrinkled
surface of brain pulses in time with heart beats. A microscopic electrode immerses in the tissue and
characteristic shapes of nerve signals start to appear on monitors - always equally steep as an impulse’s spike,
invariable in different brain regions and in different situations - but still always having different meaning.
A slight movement of the simulator’s knob which sends electric signals into the tissue - impulses
disappear. A little different combination of signals - and the impulse reappears. It is hard to believe, but it is true
- this is the way how real human brain works. Your brain. And even at this very moment. In trillions of cells
electrical signals appear and disappear. It is just this and so much at the same time.
It contains everything: a solution of the equation and true love. The understanding of existence and the
desire to commit the worst crime. The scent of meadow in the spring and the view of the starry sky. All the
unspoken words and hidden desires. An idea, which has just come into your head and the childhood memory
which comes back in dreams… Impulses, all these are just impulses - or lack of them. All - or none. 1 or 0, just
like on computer...
Fantasy is a beautiful and inspiring thing but it doesn’t add anything to the knowledge. Let’s go back
then to the reality and to further experiments.