Download Artificial Intelligence

AI – CS289
Machine Learning - Labs
Machine Learning – Lab 5
09th November 2006
Dr Bogdan L. Vrusias
[email protected]
AI – CS289
Machine Learning - Labs
Instructions
•
The following slides demonstrate the capabilities of Machine Learning for
building typical day-to-day applications.
•
The examples are taken from:
– Negnevitsky, M., "Artificial Intelligence: A Guide to Intelligent Systems", 2nd edn.
Addison Wesley, Harlow, England, 2005.
•
If you have not yet done so from the previous lab, then download and unzip
the following file, that contains all examples:
http://www.booksites.net/download/negnevitsky2/student_files/matlab/0321204662_matlab.zip
•
or alternately:
http://www.cs.surrey.ac.uk/teaching/cs289/lecturenotes/0321204662_matlab.zip
•
To run the examples, use Matlab (left hand side window called current
directory) to navigate to the directory where you have downloaded and
unzipped the files, and then simply type (case sensitive) the name of the file
without the “.m” extension.
– E.g. to run the digit_recognition.m you type digit_recognition on
Matlab’s command window.
09th November 2006
Bogdan L. Vrusias © 2006
2
AI – CS289
Machine Learning - Labs
Character recognition neural networks
•
•
•
Filename: digit_recognition.m
Matlab command: digit_recognition
Problem: A multilayer feedforward network is used for the recognition of digits from 0
to 9. Each digit is represented by a 5 x 9 bit map.
•
Run the file, follow the instructions, READ THE COMMENTS on each step, and
observe the following:
Noise level: 10%
Noise level: 0%
–
–
–
–
The training inputs (0-9) are represented in
45-dimensional vectors together with the target
outputs (10-dimensional).
We then set the training parameters (number of
neurons on the hidden and output layers, epochs, etc)
We test the network with a sample number.
Observe how the network recognise test inputs.
To improve the recognition we train the network with
noisy examples. The recognition accuracy has improved
dramatically (see final graph)!!!
1
2
2
3
3
4
4
5
5
6
6
7
7
8
8
9
9
10
1
2
3
4
5
6
10
1
2
3
4
5
6
Performance of the digit recognition network
60
Network trained with "perfect" examples
Network trained with "noisy" examples
50
Recognition error, %
–
1
40
30
20
10
0
09th
November 2006
Bogdan L. Vrusias © 2006
0
5
10
15
20
25
30
Noise level, %
35
40
45
50
3
AI – CS289
Machine Learning - Labs
Iris plant classification: back-propagation
algorithm
•
•
•
Filename: Iris_bp.m
Matlab command: Iris_bp
Problem: The Iris plant data set contains 3 classes, and each class is represented by 50
plants. A plant is characterised by its sepal length, sepal width, petal length and petal
width. A three-layer back-propagation network is required to classify Iris plants.
•
Run the file, follow the instructions, READ THE COMMENTS on each step, and
observe the following:
–
The system is trained with 102 examples and then
tested on 48 examples. The system has 5 hidden
neurons and 3 output neurons (one for each category)
After 1000 epochs the system can recognise the
type of each input with 95.83% accuracy!
Performance is 0.0185543, Goal is 0.001
0
10
-1
10
Training-Blue Goal-Black
–
-2
10
-3
10
-4
10
09th November 2006
Bogdan L. Vrusias © 2006
0
100
200
300
400
500
600
1000 Epochs
700
800
900
4
1000
AI – CS289
Machine Learning - Labs
Iris plant classification: back-propagation
algorithm
•
•
•
Filename: Iris_bp.m
Matlab command: Iris_bp
Problem: The Iris plant data set contains 3 classes, and each class is represented by 50
plants. A plant is characterised by its sepal length, sepal width, petal length and petal
width. A single-layer competitive network is required to classify Iris plants.
•
Run the file, follow the instructions, READ THE COMMENTS on each step, and
observe the following:
Input data for the Iris plant classification problem
–
The system is trained with 106 examples and then
tested on 44 examples. The system has 4 input
neurons and 3 output neurons (one for each category)
After the training, the system can recognise the
type of each input with 84.09% accuracy!
1
0.8
Petal length
–
0.6
0.4
0.2
0
1
1
0.5
0.5
0
Sepal width
0
Sepal length
Learning curve of the competitive network for Iris classification
0.25
What is the difference between supervised and unsupervised
learning?
0.2
Euclidean distance
–
0.15
0.1
0.05
09th November 2006
Bogdan L. Vrusias © 2006
0
0
10
20
30
40
50
Epoch
60
70
80
90
100
5