Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Artificial Neural Networks Group #4 John Dilag 108613685 Eric Loo 108818998 Jessica Zeng 108591189 Anita Wasilewska CSE 352: Artificial Intelligence References [1] http://www.psych.utoronto.ca/users/reingold/courses/ai/cache/neural2.html [2] http://cdn.grid.fotosearch.com/LIF/LIF114/sa302026.jpg [3] https://www.willamette.edu/~gorr/classes/cs449/ann-overview.html [4] https://www.youtube.com/watch?v=DG5-UyRBQD4 [5] https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Neural_Network_Basics [6] http://www.cs.utsa.edu/~bylander/cs6243/neural-networks.pdf [7] http://www.cob.calpoly.edu/~eli/pdf/neural.pdf [8] http://www.slideshare.net/Ahmed_hashmi/neural-network-its-applications [9]http://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/History/history1.html [10] http://psych.utoronto.ca/users/reingold/courses/ai/cache/neural4.html [11] http://www.alyuda.com/products/forecaster/neural-network-applications.htm [12] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.8217&rep=rep1&type=pdf 2 Overview 1. What are Artificial Neural Networks? 2. History 3. How do they work? 4. How do they learn? 5. Applications of ANNs 6. Conclusion 3 Artificial Neural Networks Why should we care? 4 5 What are Artificial Neural Networks (ANN)? • Part of machine learning and cognitive science • Modeled after the information processing and physical structure of neuron connections in our brain to solve problems that cannot be solved by traditional computing • Capable of generalization - the ability to recognize similarities among different input patterns (pattern recognition) • Can be trained to solve certain problems using a teaching method and sample data Sources: https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Neural_Network_Basics [5], https://shillyard.files.wordpress.com/2013/02/brain-rewired1.jpg 6 History of Neural Networks • • • • • • • • • • 1943 - Warren McCulloch and Walter Pitts; Paper on how neurons work, Modeled a simple Neural Network 1957/8 - Frank Rosenblatt, Charles Wightman and others developed the first successful neurocomputer, Mark I perceptron 1959 - ADALINE, new learning laws and binary pattern recognition and MADALINE an adaptive filter that eliminates echoes from phone lines 1975 - First multilayered network 1982 - John Hopfield renewed interest in the topic of NN 1986 - Publication of PDP (Parallel Distributed Processing) books sparked boom in NN 1987 - The IEEE annual international ANN conference was started for ANN researchers. 1988 - The International Neural Network Society (INNS) journal was founded 1989 - Neural Computation journal 1990 - IEEE Transactions on NN journal Sources: http://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/History/history1.html, http://www.springer.com/cda/content/document/cda_downloaddocument/ 9789401798150-c2.pdf?SGWID=0-0-45-1495021-p177264210 7 How do ANN work? • Neurons provides us with our abilities to remember, think, and apply previous experiences to our every action • Humans have many variations of neurons, but they all have the same four basic components: o Dendrites o Soma o Axon o Synapses Sources: http://www.psych.utoronto.ca/users/reingold/courses/ai/cache/neural2.html 8 How do ANN work: Neuron Sources: http://cdn.grid.fotosearch.com/LIF/LIF114/sa302026.jpg 9 How do ANN work: Artificial Node Each node takes in various inputs, and each input is multiplied by its associated weight, wi. The products are them summed, fed through a transfer function to set its value, and then outputted to the next set of nodes. Sources: http://www.psych.utoronto.ca/users/reingold/courses/ai/cache/neural2.html, https://www.willamette.edu/~gorr/classes/cs449/ann-overview.html 10 How do ANN work: Layers 11 How do they learn? Learning for ANN is finding values of weight connections that minimizes error by using backpropagation: 1. Initialize with random connection weights 2. Feed the training sample - set of given input and desired output 3. Let the network calculate the output with the given inputs forward 4. Calculate the error 5. Output nodes communicate to the hidden nodes the error and each pair adjusts the connection weights between. The nodes continue to push back until all nodes have been assigned an error 6. Network will try the original inputs again, and repeat this process until the output is desired Sources: https://www.youtube.com/watch?v=DG5-UyRBQD4, http://www3.cs.stonybrook.edu/~cse352/L12NN.pdf 12 Applications of ANNs ANNs are used in many applications in the recent years. Some examples include but are not limited to • Facial Recognition • Written number and letter Recognition • Stock market predictions • Staff Scheduling • Fraud Detection • Pattern Recognition • College Application Screenings • Fingerprint Recognition Sources:http://www.alyuda.com/products/forecaster/neural-network-applications.htm 13 Applications of ANNs: Written Number Recognition Sources: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.8217&rep=rep1&type=pdf 14 Applications of ANNs: Written Number Recognition Sources: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.8217&rep=rep1&type=pdf 15 Applications of ANNs: Written Number Recognition https://www.youtube.com/watch? v=QmNydKGBlxQ The size of the training sample of 5000 images. The total number of weights in a network 158 000. Digitize Checks Fillable Forms Papers Documents This fully-meshed network consists of three layers. The first layer contains 196 elements. The second layer contains 25 elements. The output layer includes ten elements Sources: https://www.youtube.com/watch?v=QmNydKGBlxQ 16 Future In ANN Recent experimental data has provided further evidence that biological neurons are structurally more complex than the simplistic explanation above. They are significantly more complex than the existing artificial neurons that are built into today's artificial neural networks. As biology provides a better understanding of neurons, and as technology advances, network designers can continue to improve their systems by building upon man's understanding of the biological brain. Sources: http://www.psych.utoronto.ca/users/reingold/courses/ai/cache/neural2.html 17 Thank you! 18