Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Recursive InterNetwork Architecture (RINA) wikipedia , lookup
Zero-configuration networking wikipedia , lookup
Distributed firewall wikipedia , lookup
Computer network wikipedia , lookup
Cracking of wireless networks wikipedia , lookup
Piggybacking (Internet access) wikipedia , lookup
Neural Networks: An Overview There are many different kinds of Neural Network they include to name a few: The Back Propagating Networks Kohonen (Self-Organising) Networks Neural Networks: Hopfield Networks AnMachine Overview The Boltzmann All Neural Networks use the concept of Neurons which are gradually trained to give a more appropriate response to input. This project has used a Kohonen Network. I shall explain the properties of such a network and why it is best for a small scale project such as this. What So what is aisKohonen a Neuron? Network? Neurons have inputs and outputs. The inputs are generally weighted so that some inputs to the neuron may have a greater affect than others. The inputs are added together and this result is used to generate an output for the neuron. x1 w1 Kohonen (Self-Organising) Networks x2 x3 w2 ∑ U w3 A neuron of such as this one is called an Adaptive Linear Element (or Adaline) and may be found own it’s own … or may be found as part of a larger “neural network” where the output of a neuron may be the input for the next “layer”. This project has used a Kohonen Network. I shall explain the properties of such a network and why it is best for a small scale project such as this. Kohonen (Self-Organising) Networks What is a Kohonen Network? A Kohonen network uses these basic principles however in a Kohonen network there is only one layer of neurons which are arranged in a 2D lattice. There is an input layer with inputs which feed into every neuron. x1 x2 Kohonen (Self-Organising) Networks How does training work? What is a Kohonen Network? The A Kohonen key to the network neuraluses network theseare basic theprinciples weights ashowever it is theyinthat a Kohonen dictate which network neuron there will is win onlywhen one layer the network of neurons is presented which arewith a arranged ininput particular a 2D pattern. lattice. There is an input layer with inputs which feed into every neuron. Individual neuron output =∑(xi – wik) 2 (for all i) x1 Where wik is the weight between input i and neuron k. Each neuron evaluates the sum of the weighted inputs and uses this as it’s own output. This is the Euclidian distance of x2 from the “input pattern”. the neuron The neuron with the smallest Euclidian distance is the one which best represents the input pattern and is known as the winner. Kohonen (Self-Organising) Networks How does training work? The All the use weights of a Neighborhood in the network during are initialised training. to small random values. An input pattern is presented to the network and the winning neuron is found. The weights to the winning node are updated using the following equation. The updated weight between input i and neuron k The gain term Input i wik(t+1) = wik(t) + G(t) (xi(t) – wik(t)) The original weight between input i and neuron k The gain term is a value between 0 and 1 which starts high and as more patterns are presented to the network is decreased. This promotes large changes away from the initial state of the network but smaller changes later when the network is some way through training. This is referred to as course mapping and fine mapping. Kohonen (Self-Organising) Networks How does training work? The use of a Neighborhood during training. Usually the weights of more neurons than the winner are changed. Once the winner has been identified a neighbourhood of a predefined shape and size is drawn around it. x1 x2 Kohonen (Self-Organising) Networks How does training work? The A real useoutput of a Neighborhood lattice will typically duringhave training. hundreds of neurons. Usually the weights of more neurons than the winner are changed. Once the winner has been identified a neighbourhood of a predefined shape and size is drawn around it. All the neurons in the neighbourhood have their weights updated to better represent the input pattern. This promotes clustering of similar results. The neighbourhood size starts very large (typically the entire network) and is decreased during training in a similar way to the gain term. Kohonen (Self-Organising) Networks How does training work? A Then real all output that is lattice left iswill to present typically some haveinput hundreds patterns of neurons. to the network with 2) so that the defined areas of the known realthe world values (such as PCO Over time lattice of neurons will organise its self into areas of neurons lattice be found the individual problem and toThis giveself the organisation areas real which can respond morefor highly to certain input patterns. world values. gives the network it’s name and leads to the lattice of nodes often being referred to as a self organising matrix. The PCO2 problem on a 302.1 Kohonen network 320.8 295.6 254.3 Now when an input pattern is presented to the network the winning neuron should be in or close to a defined area so a value can be assigned. The PCO2 problem on a Kohonen network The aim of this project is to program, implement and test a Kohonen network designed to predict PCO2 levels at the surface of the ocean. The inputs Lattice size The network has been mainly programmed in Fortran and is made up of many console programs each with a particular function to perform on the network. Where ever possible the programs are arbitrary and so such properties as the number of input in an input pattern can be set for each network. The latter networks have had 5 inputs in each pattern, these are: 2 for month , 1 latitude, 1 longitude and 1 sea surface temperature. These where used as they where thought to have the greatest affect on the PCO2 concentration. To encourage the network to distinguish between months far apart and close in the calendar and to give a greater emphasis on the month input it was included twice in each pattern. However one was an input based on the Sine of the month and one on the Cosine making the month cyclical. The PCO2 problem on a Kohonen network 2 data PCOaim The of in this the project NorthisAtlantic to program, drift area implement from a and neural testnetwork a Kohonen network designed to predict PCO2 levels at the surface of the ocean. Lattice size Typically a square lattice size of 70x70 neurons was used. Of the 4900 neurons in such a lattice only 18.06% of them will be given exact PCO2 values to report when they are found to be the winner. If an “unclassified” neuron wins then a function must be employed to find the most appropriate PCO2 value. As the network must find the output of every neuron in the lattice to find the winner the size of the lattice greatly affects the time the network takes to train or run. To illustrate the properties of lattice sizes several sizes have been tested. Lattice size Classed Apr May Jun Jul Aug Sep Oct Nov Average This is a table of Χ where Χ2=Σ (model-data) Number of Data points 2 175x175 (30625) 6.13% 30.58 4.81 4.75 2.78 2.12 2.73 2.16 2.12 6.5080 70x70 (4900) 18.06% 35.68 4.74 6.93 3.33 2.52 3.13 2.24 2.22 7.6000 28x28 (784) 46.68% 31.82 6.16 6.57 3.75 2.62 4.27 2.36 2.31 7.4833 11x11 (121) 86.78% 11.54 8.86 10.70 4.62 2.91 5.00 4.07 2.82 6.3161 4x4 (16) 100.00% 30.49 13.22 14.19 5.58 11.43 10.98 16.52 3.59 13.2505 Using data as model - 11.54 15.84 15.69 10.57 11.53 9.41 11.77 4.31 11.3331 All data is restricted to the north Atlantic drift area. The PCO2 problem on a Kohonen network PCO2 data in the North Atlantic drift area from a neural network Using Levitus data (which gives a uniform grid of latitude and longitude with temperature values for any month) a map of PCO2 values can be constructed. This was done with MatLab. Each point on the maps has an inside colour set by the PCO2 value given by the network and an outside colour set by the data the network was trained on, this gives a visual grade of how well the network predicts the PCO2 levels. The network which created these maps had a lattice size of 70x70 Created by a 70x70 lattice Created by a 175x175 lattice Created by a 28x28 lattice Created by a 11x11 lattice The PCO2 problem on a Kohonen network Training method Created by a 4x4 lattice The PCO2 problem on a Kohonen network Training method Long As mentioned Train 3000000 before the 80.00 gain and80.00 the neighbourhood 100.00 75.00size change during the training 3000000 process. 65.00 65.00 75.00is stepped 60.00 down while the Usually the gain 52.00 smoothly. 52.00 60.00 50.00 program looks neighbourhood3000000 size gets smaller The training 40.00 40.00these 50.00 33.00of the network. for a file which3000000 tells it how to change properties 30.00 20.00 33.00 The file used in6000000 training the last networks was as 20.00 follows. 6000000 20.00 16.00 3000000 20.00 65.00 10.00 65.00 100.00 30.00 3000000 6000000 The number of runs. This is the number of times a random input pattern is chosen and presented to the network. 40.00 20.00 40.00 10.00 30.00 20.00 Gain. This is expressed as a percentage. The gain goes from the first of these to the second over the number of runs. Mostly it is fixed. 20.00 16.00 Neighbourhood diameter. This is expressed as a percent of the lattice width. The size goes from the first value to the second smoothly over the number of runs. So two more tests where performed this time changing the training method. The PCO2 problem on a Kohonen network Training method Long Train 3000000 3000000 3000000 3000000 6000000 6000000 80.00 65.00 52.00 40.00 30.00 20.00 80.00 100.00 75.00 65.00 75.00 60.00 52.00 60.00 50.00 40.00 50.00 33.00 20.00 33.00 20.00 10.00 20.00 16.00 Quick Train 1000000 65.00 65.00 100.00 35.00 500000 40.00 40.00 35.00 25.00 1500000 20.00 20.00 25.00 18.00 These are designed to do the same as original training method but with more or less runs i.e. the patterns are presented to the network more or less times. The networks both had a 70x70 output lattice so a comparison with the 70x70 size network trained with the original method is valid. This is a table of Χ where Classed Apr May Jun Jul Aug Sep Oct Nov Average Long 18.00% 34.83 4.80 6.08 3.45 2.37 3.37 2.07 2.26 7.4038 Original 18.06% 35.68 4.74 6.93 3.33 2.52 3.13 2.24 2.22 7.6000 Quick 19.84% 18.38 4.30 4.99 3.13 2.38 3.24 2.14 2.22 5.0963 Χ2=Σ (model-data) Number of Data points 2 Created by a 70x70 lattice using the Long method Created by a 70x70 lattice using the Original method Created by a 70x70 lattice using the Quick method