Download Supervised Learning

The Cogs of
C
Cognition
iti
Tom Hartley
[email protected]
y@p y
gy y
Neural Network Practical:
Handout III
Next: Run the Simulations
• In this section we will take the data from
our spreadsheets and use it as the basis
for some neural network simulations
illustrating the ideas covered in the talks.
Supervised Learning
•
•
•
•
For supervised learning your spreadsheet needs to specify both
input and target output patterns
patterns.
If your stimuli fall into different categories, you could try using your
output units to represent the category – see if the network can learn
to categorize the input patterns. The easiest way to represent the
categories
t
i iis tto h
have one output
t t unitit (column)
( l
) for
f each
h category.
t
The value of this unit will be 1 for the items in the category and 0 for
the other items.
g
p
perhaps
p they
y lie on a
If yyour stimuli don't fall into categories
continuum. You could use one output unit to represent each items
position on the continuum with a value of 0 standing for one end, 0.5
in the middle, and a value of 1 for the other.
What other output representations might be useful/interesting for
your network to learn?
Supervised Learning
Learning. Step 1
•
Create a spreadsheet
p
containing
g yyour input
p
and output patterns in excel.
–
–
–
–
–
Each row represents a set of input patterns (neuron
activation levels) and the corresponding output
patterns.
Columns that stand for inputs should have headings
start with i_
Columns that stand for outputs should have
headings
g start with o_
Numbers in each row should be between 0 and 1.
Make a note of the number of input and output
columns.
columns
Supervised Learning: Step 2
C
Create
an NNS pattern fil
file
• Create a nns pattern file
– type the following command into Matlab
xls2nns([])
– choose the Excel file to open.
– the Matlab program should create a new file
with the same name as your spreadsheet but
with a .pat
p extension
Select the spreadsheet
file…
Creates a new .pat file
in the same folder
(should be your My
Documents folder)
Supervised Learning: Step 3a
• Start Java NNS.
NNS
– Double-click on my computer.
– Navigate to E:\Cogs\JavaNNS
– Double-click on run_simulator.bat
Supervised Learning: Step 3b
• Load the pattern file in JavaNNS
– Go to File/Open, then find your .pat file
Your .pat file should be in your My Documents folder
Supervised Learning: Step 4a
• Create input units
– In JavaNNS go to Tools/Create/Layers
– Change width to be the number of units
(model neurons) in your input layer (number
of columns starting with ii_))
– Change unit type to be input
– Click create – this creates your input layer
Supervised Learning: Step 4b
• Create output units
– Change width to be the number of units in
your output layer (number of columns starting
with o_)
– Change top left to read 1 5 1
– Change unit type to be output
Supervised Learning: Step 4c
• Connect input
p to output
p units and Label them
–
–
–
–
Go to Tools/Create/Connections
Select connect feedforward
Click connect
Draw a box around the top row of units (the inputs).
Theyy should turn yyellow. Right
g click on the yyellow
units and select edit units…
– Change the name of these units from NoName to
"input"
input
– Repeat for output units changing the name to "output"
this time.
Supervised Learning: Step 5a
• Go to tools/Control panel
p
• Go to the patterns tab
• You should see your patterns file listed under training set
and validation set
set, if not repeat step 3
3.
• By clicking on the arrow buttons at the bottom right of the
window, you can move through the patterns you created,
activating
ti ti the
th input
i
t and
d output
t t units
it according
di tto the
th
patterns in the rows of your spreadsheet. The colour of
the units changes to represent the activation level (green
means very active,
ti
bl
blue means resting,
ti
red
d means
suppressed/inhibited).
Supervised Learning: Step 5b
• Go to the initializing tab
tab.
• With initializing function set to “Random
Weights” (don
Weights
(don’tt change anything) press
the “Init” Button.
• This
Thi sets
t th
the connection
ti weights
i ht iin your
network to random starting values.
Supervised Learning: Step 5c
• Go to the updating tab. By clicking the arrows at
the bottom right you can again move through the
patterns in the rows of your spreadsheet. This
time only the input units change according to the
values you set. The output units change
according
di tto th
the activity
ti it propagated
t d tto them
th
via
i
the weighted connections. These are initially
random so the output patterns will not change in
random,
a systematic way as you change the input
p
patterns.
Supervised Learning: Step 5d
• Go to the learning
g tab. Press learn all. This cycles
y
through all the patterns in your network 100 times. Every
time, the output of the network is compared with the
g output
p ((i.e., the p
pattern specified
p
in yyour
target
spreadsheet, columns beginning with o_). The weights
are adjusted by a small amount to reduce the difference
p and the target
g output.
p You
between the current output
should notice that when you go to the updating tab and
flick through the patterns, the changes are more
systematic
y
– and closer to those specified
p
in the target
g
output patterns.
• You can see how the error changes during learning by
looking at View/Error graph
graph.
Properties you can explore
•
•
•
•
Neural networks can use distributed representations; a particular
object concept or action is represented by the pattern of activity
object,
across a population of neurons. Note that this is very different to the
way conventional computers represent information using symbols.
The connections can learn to translate from one pattern of input to
another
th pattern.
tt
With enough
h layers
l
almost
l
t any translation
t
l ti can be
b
learned. Did your network succeed in learning the translation you
specified? If not, can you see where/why it is going wrong?
generalize: if the network learns to solve a p
problem
Neural networks g
with one set of input patterns, it will probably produce a sensible
response to a new, unseen input. Can you figure out how to test this
with your network?
Neural networks are resistant to damage
damage. The network does not
suddenly fail when e.g., some of the connections are cut or when
some of the neurons are removed – performance degrades
gracefully. Try changing the weights on some of the connections in
your network
network, and see if you can break itit. How does it go wrong?