Download Neural Networks in Computer Science

Methodology of Simulations



CS/PY 399 Lecture Presentation # 19
February 21, 2001
Mount Union College
Simulating Systems (Ch. 2,
Plunkett & Elman)


To study complex systems of interacting
components, we can:
1) Study real systems
– can we do this without disturbing the
system to the point that results are flawed?

2) Build a model of the system and
simulate actual system operation
– does model include all important aspects of
the actual system?
Advantages of Modeling and
Simulation

correlation between model components
and brain structure
– we can include more relevant details than
in a verbal description of system operation


non-linear nature of Neural Networks
makes purely mathematical analysis
difficult or impossible
surprising behavior can be uncovered
– leads to new experimental hypotheses
Advantages of Modeling and
Simulation

It is easier to dissect a model than a
brain!
– model will work after dissection

We can begin to discover why some
behavior arises from connected neuron
clusters by studying node activations
Simulation as Experiment


Several factors must be defined before
we use the computer to simulate a
network
Define problem or goal
– purpose of the experiment
– e.g., to see if a network can learn letter
sequences
Simulation as Experiment

Formulate a hypothesis
– e.g., a feed-forward network can learn
letter sequences

Design a mechanism for testing the
hypothesis
– e.g., see if training data can train network
with a global error rate < ε

Evaluate results of simulation
Decisions when Designing
Connectionist Simulations

Task
– what is the desired behavior (output) for
the network
– defined by the training environment
• input/output patterns in training set
– network is taught by example
• does not learn rules; learns to recognize
patterns
• is this what happens with humans?
– e.g., multiplication tables
Decisions when Designing
Connectionist Simulations

Stimulus Representation
– our models will use a numerical
representation of input stimuli
– vector notation used for multiple inputs
• defines an n-dimensional space, for n inputs
– output also numeric
Decisions when Designing
Connectionist Simulations

Network Architecture
– number and arrangement of nodes
– one large, completely connected network?
• simplest to design
– several sub-networks connected in some
configuration?
• this is more like brain structure
– feed-forward vs. recurrent connections
– these decisions will powerfully affect the
network’s capabilities
Completely-Connected Network
Layered Feedforward Network
Recurrent Network
Analyzing the Simulation after
Training

Global Error
– output produced vs. output desired
averaged over all patterns
– perfect learning: error = 0
– can be misleading, especially if there are
relatively few patterns that give bad results
• e.g. Quicksort average vs. worst-case behavior
– avg. time for 1 million names:  20,000,000 ops.
– worst-case:  1,000,000,000,000 (1 trillion) ops.
Analyzing the Simulation after
Training

Individual Pattern Error
– looking at specific interesting patterns
• too many patterns, in general, to examine all of
them
– also, we may look at novel patterns (not
part of the training set)
• spontaneous generalization, etc.
– what has the network learned?
• rote information vs. generalized knowledge
Analyzing the Simulation after
Training

Inspecting weights and internal network
settings
– to see how the network is arriving at
conclusions
– comparison of starting weights with final
trained weight set is often illuminating
– after training, it is often useful to examine
specific hidden node activations for a
selected input pattern
• what part of the network is triggered?
Methodology of Simulations



CS/PY 399 Lecture Presentation # 19
February 21, 2001
Mount Union College