Download ARM and Machine Learning

ARM and Machine Learning
Jem Davies
ARM Fellow & VP Technology
Media Processing Group, Imaging & Vision Group
December 12 2016
© ARM 2016
Who am I?

Jem Davies - ARM Fellow and VP of Technology - Media
Processing and Imaging & Vision Groups

Working on multimedia, computer vision and machine learning

13 years at ARM

Mathematician turned chemist, turned software engineer…




2
… turned architect
… turned tech future predictor
Glider pilot instructor, fireworks lighter
and scuba diver…
… what next?
© ARM 2016
Machine Learning overview


Machine Learning makes smart connections to previously
encountered concepts
It’s useful when:
•
•
We don’t have algorithms…
but we do have a lot of data
Image recognition
3
© ARM 2016
Robotics
Home security
Speech recognition
We use Machine Learning technology every day
Speech
recognition
Computational
photography
Predictive text
Face tracking
camera
4
© ARM 2016
Digital
assistant
Fingerprint ID
Definitions and
recent developments
5
© ARM 2016
The AI landscape
Machine
Learning
Constraints
and control
theory
Artificial
Intelligence
Computer
Vision
AI
Reasoning
6
© ARM 2016
Deep
Learning
Natural
language
processing
Search and
Optimization
Machine
Learning
Algorithms:
CNNs,
RNNs, etc.
Key terms and definitions
Artificial Intelligence
• The broadest term - applying to any technique enabling computers to mimic
human intelligence
Machine Learning
• A subset of AI including techniques enabling computers to improve at tasks
with experience. Includes deep learning
Deep Learning/Neural Networks
• A branch of machine learning that attempts to model real life ideas in data by
using a deep graph with multiple processing layers
Algorithms
• DNNs, CNNs, RNNs, SGEMM etc.
Computer Vision
• An interdisciplinary field that allows computers to gain understanding from
digital images or videos. Many computer vision applications use ML algorithms.
7
© ARM 2016
Recent developments in deep learning

The image detection benchmark ImageNet has
reached nearly 100% accuracy


Research groups feel it’s not useful to work on
ImageNet anymore (it’s a solved problem)


8
Largely due to Neural Networks
GoogLeNet, Inception, etc.
The successful approach used for ImageNet has
spread to other domains, yielding great
improvements
© ARM 2016
Why did this happen ?
The Machine Learning learning loop
9
© ARM 2016
Basic neural network
Squashing function
x1
x2
[Inputs]
x3
xn-1
10
© ARM 2016
w2
w1
n
w3
wn-1
xn
Sigmoid: y=1/(1+e^(-z))
wn
z = ∑ wi xi
i =1
y = H ( z)
y
[Output]
Machine Learning computation
in the cloud and on-device
11
© ARM 2016
Dissecting the Machine Learning process
Cloud side
Training engine
Generate inference engines
Inference engine
Heterogeneous tools
Distribute & optimize
Device side
ARM Cortex + Mali + Others
Platform acceleration libraries
12
© ARM 2016
Machine Learning process on ARM
Cloud services / applications
Cloud side
Training engine
Cloud service SDK
+ analytics
Cloud
Cloud applications
Cloud APIs
Metadata streams, thumbnails etc.
Device
Applications
Inference engine
Middleware
Heterogeneous tools
Device side
ARM Cortex + Mali + Others
Platform acceleration libraries
13
© ARM 2016
SW
HW
CPU
GPU
(Cache-coherent interconnect)
CV/ML IP
HW Cores
HW Interconnect
It is easy to recognise speech
It is easy to wreck a nice beach
14
© ARM 2016
Automatic speech recognition is not easy

Active research area for over 50 years!




Applications




15
Significant attention recently due to large amount of
cloud computing
Neural Networks has improved performance
Siri, Google Now, Cortana, Amazon Echo now creating
usable applications
Safety and security
Interactions with smaller or hands free devices (e.g. wearables)
Improved accessibility for hearing-impaired
Allows indexing of spoken words
© ARM 2016
ASR use cases and Machine Learning

Large Vocabulary Continuous Speech Recognition (LVCSR)



Keyword spotting of simple commands


“OK Google”, “Set alarm for 7”
Sound monitoring

16
Dictation/transcription, virtual assistant
Requires dictionary, knowledge of grammar
Early/automatic anomaly detection
© ARM 2016
Keyword spotting

Listen for certain words/phrases



Only need to learn certain words
“Okay Google”
“Play music”
Speech Signal
Feature extraction
Features
Acoustic model

Simpler algorithm



Algorithm must run locally on device


17
No knowledge of grammar
Only needs acoustic model
Potentially always listening
Power consumption vitally important
© ARM 2016
Words
Play music
Computer Vision
18
© ARM 2016
Computer vision pipeline
Output
Application
Pre-processing
Feature
extraction
Detection and
segmentation
High-level
processing
Image acquisition
ISP/GPU
19
© ARM 2016
GPU/CPU/DSP/Spirit CV
CPU
ARM and Machine Learning
20
© ARM 2016
Machine Learning runs on ARM-powered client
devices today
21
© ARM 2016
Caffe framework deep learning on ARM
Video hosted on youtube
https://www.youtube.com/watch?v=k4ovpelG9vs&t=13s
22
© ARM 2016
Deep learning frameworks on ARM
23

DNN used for real-time detection of objects

Based on the Caffe deep learning framework

Detection entirely on the mobile device – not
cloud based

Optimized for ARM Cortex CPU or Mali GPU

Running Machine Learning on the GPU frees
up the CPU for other tasks

Optimised libraries also being developed for
TensorFlow and OpenVX
© ARM 2016
2.7
64%
5.5
20%
Conclusions
Innovation
Growth
Maturity
Machine
Learning

24
The rapid uptake in Machine Learning is
going mainstream, affecting compute
everywhere

Machine Learning dramatically increases
compute demands

As much as possible, Machine Learning
workloads should run locally on device,
not on remote servers

Machine Learning is driving demand for advanced ARM
processors and accelerator IP

Machine Learning is having a significant impact on ARM’s
roadmap for future processors and architectures
© ARM 2016
Platforms
& Tools
Thank you for listening
The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited
(or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be
trademarks of their respective owners.
Copyright © 2016 ARM Limited
© ARM 2016