Download 1 - AGH

1.4. Why should neural networks consist of layers?
(translation by Krzysztof Królczyk, [email protected])
Similarly to earlier mentioned simplification of biological information (about other real brain properties), this
also applies to space layout of neurons, and connections they create – the whole science complexity,
neuroanatomic and cytological knowledge were reduced to absolute minimum. Neural net designers focused
heavily on implementing working model - practical, tough extremely truncated. It appears, we could observe
regular pattern, which neurons tend to create in several brain areas. Below, we can see few examples of such
layer-like structure (Fig. 1.15).
Fig. 1.15. Layered structure of the human brain cortex
Retina is another example of such structure (Fig. 1.16); as for embryologists – being transformed part of
cerebral cortex.
Fig. 1.16. Retina (part of the eye) is also organized as layered structure
It’s safe to imply, that neural networks, designed as a multilayer structure are quite convenient - technically it’s
the easiest way; however, neural nets are biologically “crippled” models of actual tissue, nevertheless functional
enough to assume that results obtained are fairly correct – at last in context of neurophysiology. According to
words of one green ogre “Ogres have layers. Like onions.”. Neural networks have layers also.
Typical neural network, therefore, has structure shown in Fig. 1.17.
Fig. 1.17. Typical neural network structure
What are main advantages of such (layer) approach? It’s very simple to create a model, and simulate it‘s
behavior using various computer programs. That’s why researchers adopted such structure and use it from there
on, in every neural net. Let’s say it again – it’s very inaccurate if considered as a biological model, however
main idea is preserved. There were, of course, remarks, how much better would have networks operate if model
closer resembled its origin, real tissue, or how could it be adjusted to perform specific tasks, but as for now, none
worries about it.
Another problem is with connecting layers. For example, in real brain, schematic of neural connections is quite
complicated, and differs depending on which brain area is considered. Therefore, in XIX century, first
topological brain map was created, dividing it in view of identical neural connections templates. (Fig. 1.18).
Fig. 1.18. Map of regions with different cell connections in the brain (by K. Brodmann)
Here, with same color, are marked fragments with microscopically examined similar connections, whereas
different colors corresponds to substantial differences. This map, with rather historical meaning, was called
Brodmann’s areas. Brodmann divided the cortex into 52 regions. Currently we treat brain much more subtle;
however – this is a good example of problem we’re facing when analyzing “how are neurons connected into
net”, the answer varies with different brain part.
If we were thinking about the sole purpose of building artificial neural networks – one may think it’s essential to
adapt its connection structure to single problem we’re dealing with. That’s true, it’s been proven that well chosen
structure can greatly increase speed of net’s learning. The problem is situated, unfortunately, elsewhere. In most
problems we are trying to solve – we can’t really tell what is best way to work the problem out. If we can’t even
guess which algorithm is suitable, and which one is network going to employ after learning process, the less
could we be capable of selecting (a priori), network elements which are necessary, from useless.
Therefore – the decision about connecting layers, and single elements in networks are arbitrary, and usually it’s
full connection – each element is connected to all other. Once again – such idea of homogeneous, full connection
schematic – reduces effort required to define network, however increases computing complexity, i.e. higher
memory usage, or chip complexity, needed to recreate all connections between elements. It’s worth noting, that
without such simplification, network definition would require thousands parameters, surely causing employment
such structure, to be a programmer’s worst nightmare. Whereas using fully connected elements, again, is basic
thoughtless designer’s will. It’s almost a practice; also, causes no real harm, since learning process eliminates
unnecessary connections from whole bunch.