Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lateral computing wikipedia , lookup
Theoretical computer science wikipedia , lookup
Computational complexity theory wikipedia , lookup
Multiple-criteria decision analysis wikipedia , lookup
K-nearest neighbors algorithm wikipedia , lookup
Inverse problem wikipedia , lookup
Natural computing wikipedia , lookup
Machine learning wikipedia , lookup
Pattern recognition wikipedia , lookup
Artificial intelligence wikipedia , lookup
DEEP LEARNING: WHAT IS IT GOOD FOR? R. BURGMANN ITC571 EMERGING TECHNOLOGIES AND INNOVATION INTRODUCTION • A brief history. • What are neural networks? • What are artificial neural networks (ANN)? • Early problem, solutions, stagnation. • Where does deep learning (DL) fit in the toolbox? • Relationship between ANN and DL • When is it the right choice? • How close are we to human level artificial intelligence (are we there yet)? • Questions and Answers. WHAT ARE NEURAL NETWORKS? Image courtesy of Hagan, Demuth & Beal. Neural Network Design PWS Publishing. Boston. Page 1-8 WHAT ARE ARTIFICIAL NEURAL NETWORSK? Image courtesy of Hagan, Demuth & Beal. Neural Network Design PWS Publishing. Boston. Page 2-11 EARLY PROBLEMS, SOLUTIONS AND STAGNATION • Problem 1. Single layer networks could only solve linear problems. • Problem 2. At first no algorithm known to train multi-layer networks until backpropagation algorithm. • Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network. • Problem 4. Not enough training data for serious problems. • Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster. • Problem 6. If you could solve it with an ANN back in the 1980s and 1990s then there where easier ways. WHERE DOES DEEP LEARNING FIT IN THE TOOLBOX? • Relationship between ANN and DL • Rather than 3 to 5 layers, deep learning uses neural networks hundreds of layers deep. • How is this achieved? • • Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network. • Convolution • Pooling Problem 4. Not enough training data for serious problems. • • Unlimited training data (thank you internet) Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster. • Unlimited processing capacity (thank you computer games industry and graphical processing chips) WHERE DOES DEEP LEARNING FIT IN THE TOOLBOX? OTHER MACHINE LEARNING TECHNIQUES • Is deep learning the only game in town? • Well… no. • Bayesian Networks • Classification Rules • Linear Models • Clustering • Ensemble Learning WHEN IS IT THE RIGHT CHOICE? • Tens to hundreds of millions of training examples • Unknown features • Examples • Baldi et al, searching for exotic particles in high energy particle physics. • Druzhkov, et al, image classification and object detection. • Noda, et al, Audio-visual speech recognition. • Sheehan, et al, Population genetics • Suk, et al, Brain imaging of Alzheimers patients ARE WE THERE YET? HOW CLOSE ARE WE TO HUMAN LEVEL AI? • From deep learning to AIXI, “an optimal rational reinforcement learning agent” Hutter (2005) QUESTIONS AND ANSWERS • Thank you for your time.