Download Introduction to Financial Prediction using Artificial Intelligent Method

Back-propagation network (BPN)
Student : Dah-Sheng Lee
Professor: Hahn-Ming Lee
Date:20 September 2003
Outline




What is a Neural Network ?
Artificial Neural Network (ANN) property
Back-propagation network (BPN)
Reference
What is a Neural Network ?

Neural Networks are a different paradigm for
computing:



von Neumann machines are based on the
processing/memory abstraction of human information
processing.
neural networks are based on the parallel architecture of
animal brains.
Neural networks are a form of multiprocessor
computer system, with




simple processing elements
a high degree of interconnection
simple scalar messages
adaptive interaction between elements
What is a Neural Network ? (cont…)
A biological neuron may have as many as 10,000
different inputs, and may send its output (the presence
or absence of a short-duration spike) to many other
neurons. Neurons are wired up in a 3-dimensional
pattern. Real brains, however, are orders of magnitude
more complex than any artificial neural network so far
considered.
神經樹突（dendrites）
(dendrites）
神經突觸（synapses）
（synapses）
神經軸突（axon）
神經核（soma）
What is a Neural Network ? (cont…)
The dendrites are extensions of a neuron which connect to other
neurons to form a neural network, while synapses are a gateway which
connects to dendrites that come from other neurons.
A biological neuron may thus be connected to other neurons as well
as accepting connections from other neurons, and so we have the basis
of a network.
Through those connections, electrical pulses are transmitted, and
information is carried in the timing and the frequency with which these
pulses are emitted.
So, our neuron receives information from other neurons, processes it
and then relays this information to other neurons.
輸入訊號
X1
X2
Xi
Xn
閥值
處理單元淨值
轉移函數
W1j
W2j
:
:
:
Wij
Wnj
θ i netj
鏈結加權值
f
Yj
輸出訊號
n
Y j  f (  Wij X i  
i
j
)
Artificial Neural Network (ANN) property
ANN運作所需的範例資料有




Training example
Testing example
待推案例
ANN characteristics




Input : training set,testing set
output
Processing Element(PE)
Artificial Neural Network (ANN) property
(cont…)

ANN Type




Supervised learning :Perceptron, BPN, PNN,LVQ,CPN
Unsupervised learning :SOM, ART
Associate learning :Hopfield,
Bidirectional Associative Memory(BAM),
Hopfield-Tank
Optimization application :HTN,
ANN(Annealed Neural Network)
Artificial Neural Network (ANN) property
(cont…)

ANN structure

X1
One way feedforward
X2
Xn
Y
‧
‧
‧
X1

Two way feedforward
X2
Xn
Y
‧
‧
‧
X1

Feedback
X2
Xn
Y
‧
‧
‧
Artificial Neural Network (ANN) property
(cont…)

ANN Basic Model






Processing Element(PE) : summation fc.,Activity
fc.,transfer fc.
Input layer : [x1.....xi](training set,testing set)
Hidden layer : present PE's internal relationship
Output layer : normalize output,competitive
output,competitive learning
Network : Learning,Recalling
Weights : connecting between layers
Artificial Neural Network (ANN) property
(cont…)

Transfer Function Type

Discrete Type






Linear Type



Perceptron/step fc.
Signum fc.
Signum 0 fc.
Hopfield-Tank fc.
BAM fc.
threshold line fc.
straight linear fc.
Nonlinear Type


sigmoid fc.
Hyperbolic tangent fc.
Artificial Neural Network (ANN) property
(cont…)
(Single-layer Perceptrons)
Back-propagation network (BPN)
The network model “BPN” is
 Supervised learning
 Feedforward
 Multilayer Perceptrons
(Special case: no hidden layer)
(Multilayer Perceprons)
Back-propagation network (BPN)
Training algorithm





Step 1: Initialize the network synaptic weights to small
random value.
Step 2:Form the set of training input/output pairs, present
an input pattern and calculate the network response.
Step 3: The desire network response is compared with the actual
output of the network, and by using 1* and 2* all the local
errors can be computed
Step 4:The weight of the network are update according to 3*
Step 5:Until the network reaches a predetermined level of accuracy in
producing the adequate response for all the training pattern,
continue step 2 through 4
Back-propagation network (BPN)
advantage algorithm





Backpropagation Learning Algorithm
with Momentum Updating
Batch Updating
Search-Then-Converge Method
Batch Updating with Variable Learning
Rate
etc…
Reference


“Principles of Neuroncomputing for Science and
Engineer”
Fredric M. Ham; Ivica Kostanic;
McGRAW-HILL INTERNATIONAL EDITION, 2001
“類神經網路模式應用與實作” 8th
葉怡成
儒林出版社, 2003