Download Deep Learning - CSU Thinkspace

DEEP LEARNING:
WHAT IS IT GOOD FOR?
R. BURGMANN
ITC571 EMERGING TECHNOLOGIES AND INNOVATION
INTRODUCTION
• A brief history.
•
What are neural networks?
•
What are artificial neural networks (ANN)?
•
Early problem, solutions, stagnation.
• Where does deep learning (DL) fit in the toolbox?
•
Relationship between ANN and DL
• When is it the right choice?
• How close are we to human level artificial intelligence (are we there yet)?
• Questions and Answers.
WHAT ARE NEURAL NETWORKS?
Image courtesy of Hagan, Demuth &
Beal. Neural Network Design PWS
Publishing. Boston. Page 1-8
WHAT ARE ARTIFICIAL NEURAL NETWORSK?
Image courtesy of
Hagan, Demuth &
Beal. Neural
Network Design
PWS Publishing.
Boston. Page 2-11
EARLY PROBLEMS, SOLUTIONS AND STAGNATION
• Problem 1. Single layer networks could only solve linear problems.
• Problem 2. At first no algorithm known to train multi-layer networks until backpropagation algorithm.
• Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the
network.
• Problem 4. Not enough training data for serious problems.
• Problem 5. If you had a serious problem with lots of training data then you needed a super computer
cluster.
• Problem 6. If you could solve it with an ANN back in the 1980s and 1990s then there where easier ways.
WHERE DOES DEEP LEARNING FIT IN THE TOOLBOX?
• Relationship between ANN and DL
•
Rather than 3 to 5 layers, deep learning uses neural networks hundreds of layers deep.
• How is this achieved?
•
•
Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network.
•
Convolution
•
Pooling
Problem 4. Not enough training data for serious problems.
•
•
Unlimited training data (thank you internet)
Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster.
•
Unlimited processing capacity (thank you computer games industry and graphical processing chips)
WHERE DOES DEEP LEARNING FIT IN THE TOOLBOX?
OTHER MACHINE LEARNING TECHNIQUES
• Is deep learning the only game in town?
• Well… no.
•
Bayesian Networks
•
Classification Rules
•
Linear Models
•
Clustering
•
Ensemble Learning
WHEN IS IT THE RIGHT CHOICE?
• Tens to hundreds of millions of training examples
• Unknown features
• Examples
•
Baldi et al, searching for exotic particles in high energy particle physics.
•
Druzhkov, et al, image classification and object detection.
•
Noda, et al, Audio-visual speech recognition.
•
Sheehan, et al, Population genetics
•
Suk, et al, Brain imaging of Alzheimers patients
ARE WE THERE YET?
HOW CLOSE ARE WE TO HUMAN LEVEL AI?
• From deep learning to AIXI, “an optimal rational reinforcement learning agent” Hutter (2005)
QUESTIONS AND ANSWERS
• Thank you for your time.