Download Machine Learning - A Computational Intelligence Approach

Machine Learning - A Computational
Intelligence Approach
PhD program in
Computer Science and System Engineering
Period : 20-23 June 2016
mask1.swf
mask2.swf
Hidden pattern
Natural Intelligence
Recognition (patterns: visual, odors, flavors,
sound, linguistic – objects, people, ....)
Reasoning on uncertain data
Optimization (biological/social)
Computation Intelligence
Computational Intelligence methodologies, partly inspired by
natural systems, are a family of powerful methods for data
analysis, able to transform the available heterogeneous data into
knowledge.
In many ways, Computational Intelligence, can be considered
another denomination of Soft Computing, Natural Computing...
or also of Cybernetics.
Computational Intelligence
Sensor information acquisition
Learning
Optimization
Recognition
Linguistic Information processing
Action
Computational Intelligence Paradigms
Neural Networks
Mathematical methods/algorithms for problem solving inspired to the
structure and physiological mechanisms of the nervous system
(parallelism, self-organization, associative memory, robustness,
learning, ...)
Fuzzy logic
Extension of traditional logic (based on true/false, 1/0) using degrees
of truth ranging between 0 and 1, in order to represent uncertainty
and qualitative reasoning
Evolutionary Computation
Simulation of biological evolution as a general optimization technique
Computational Intelligence Paradigms
• Interval
Mathematics and Rough sets (using objects incorporating the
concept of variability interval)
•Machine Learning
• Neuro-fuzzy systems (systems using fuzzy truth representation and
incorporating neural learning)
• Ant Colony Optimization, Particle Swarm Optimization, Simulated
Annealing, Quantum Computing, DNA Computing, Immune Systems,
Chaos computing, Interval Computation, Possibility theory, ...
Biological Neurons
Neural Networks
•Learning
•Generalization
y=H ( ∑
i
•ωOptimization
x −θ )
i i
•Universal Function
Approximation
Fuzzy Sets
Fuzzy Logic
Decision making theory able to handle the
imprecision of linguistic knowledge
Fuzzy rules:
IF a man is short THEN he will not make a very
good professional basketball player
Neuro-Fuzzy Approach to Learning Machines
Evolutionary Computation
Evolutionary algorithms are optimization and searches procedures
inspired by genetics and the process of natural evolution
Solution = Individual of a population
Parameter of a solution= Gene
Representation of a solution = Chromosome
Set of solutions= Population
Objective function = Fitness
Search operators:

Selection

Crossover (recombination)

Mutation
Generic Evolutionary Algorithm
1. Initialise population
2. Evaluate population
3. Repeat
3.1. Select sub-population for reproduction (Selection)
3.2. Recombine the ‘‘genes’’ of selected parents ( Crossover)
3.3. Mutate the mated population stochastically (Mutation)
3.4. Evaluate the fitness of the new population
3.5. Select the survivors from the actual fitness
Note 1: Not all these steps are present in all EAs.
Note 2: There are as many EAs as the researchers working in EC!
(Poli, 1996)
Swarm Intelligence
Data Processing Model
Inspiration
Applications
Neural Networks
Nervous system
Classification, regression,
etc
Evolutionary Algorithms
Biological evolution
Optimization, Authomatic
programming
Swarm Intelligence
Social behaviour
Optimization, collaborative
models
Simulated anealing
Physics of lattices
Optimization
Soft Computing
Soft computing differs from conventional (hard) computing
in that, unlike hard computing,
it is tolerant of imprecision, uncertainty, partial
truth, and approximation.
In effect, the role model for soft computing is the human
mind.
The guiding principle of soft computing is:
Exploit the tolerance for imprecision,
uncertainty, partial truth, and approximation
to achieve tractability, robustness and low
solution cost.
Computational Intelligence Paradigms
Over the past 30 years many Computational Intelligence methods
strategies have been applied successfully to the solution of complex
problems of many fields,
e.g.:

regression (prediction)

classification

clustering

signal and image processing

feature selection

data visualization

data mining

information fusion

etc.
Successful cases of Computational
Intelligence applications
•“artificial intelligence” in electronic games (Evolutionary algorithms)
•Computer mouses control (Neural Networks)
•Spam detection in email (Neural Networks)
•Intrusion detection in computer systems (Fuzzy Logic, Neural
Networks)
• Washer control (Fuzzy logic)
• Camera's autofocus (Fuzzy Logic)
• Image processing and compression (all)
•Bioinformatic data processin (all)
• Wiring of printed circuits of electronic board/ Silicon chip
optimization (Simulated Annealing, Evolutionary Algorithms)
• etc.
Machine Learning Tasks
IMMEDIATE TASKS FOR HUMANS

Face identification

Spoken language understanding

To recognize a flower from its smell
DIFFICULT TASKS FOR HUMANS

Weather forecasting

Protein secondary struct prediction

Fingerprint recognition

Medical Diagnosis

WEB search
BENEFITS OF ML
Speed
Standardization
The Century of data
Francis Diebold , Francis X. Diebold , F. X. Diebold (economists University of Pennsylvania) “Big Data Dynamic Factor Models
for Macroeconomic Measurement and Forecasting” (2000)
David L. Donoho (Dept Statistics - Stanford University) Lecture
at AMS Conf on Math Challenges of the 21st Century (2000)
“The coming century is surely the century of data.
A combination of blind faith and serious purpose makes our society
invest massively in the collection and processing of data of all
kinds, on scales unimaginable until recently. Hyperspectral
Imagery, Internet Portals, Financial tick-by-tick data, and DNA
Microarrays are just a few of the better-known sources, feeding
data in torrential streams into scientific and business databases
worldwide. ...
The trend today is towards more observations but even more so, to
radically larger numbers of variables – voracious, automatic,
systematic collection of hyper-informative detail about each
observed instance.
The Century of data
We are seeing examples where the observations gathered on
individual instances are curves, or spectra, or images, or
even movies, so that a single observation has dimensions in
the thousands or billions, while there are only tens or
hundreds of instances available for study.
Classical methods are simply not designed to cope with this
kind of explosive growth of dimensionality of the
observation vector.
We can say with complete confidence that in the coming
century, high-dimensional data analysis will be a very
significant activity, and completely new methods of highdimensional data analysis will be developed; we just don't
know what they are yet. ...”
The Curse of dimensionality
Richard Bellman, Adaptive Control Processes: A Guided Tour. Princeton
University Press, 1961, page 97:
“In view of all that we have said in the forgoing sections, the many
obstacles we appear to have surmounted, what casts the pall over
our victory celebration? It is the curse of dimensionality, a
malediction that has plagued the scientist from the earliest days”.
Impossibility of optimizing a function of many variables by a brute force search
on a discrete multidimensional grid, as the number of grids points increases
exponentially with dimensionality, i.e., with the number of variables.
Nowadays, the “curse of dimensionality” refers to any problem in data
analysis that results from a large number of variables (attributes).
Informatics in every day life

Ubiquity

Pervasivity
2013
BIG DATA
“There were 5 Exabytes of information
created between the dawn of civilization
through 2003, but that much information
is now created every 2 days” Google
CEO Eric Schmidt (2011)”
BIG DATA and Health
Most of BIG DATA are bionedical data.
In the next 3 years in the world there will be at least 1 billion
smartphones active and that at least 8 million patients
regularly use their devices for health parameters monitoring.
If today's global health data settle around 500 petabytes by
2020 this figure could swell up to 25,000 petabytes (25
billion gigabytes).
What are we to make of this huge amount of medical
data?
45
Where is the Life we have lost in living?
Where is the wisdom we have lost in
knowledge?
Where is the knowledge we have lost in
information?
from "The Rock"
by Thomas Stearns Eliot (1888-1965)
Knowledge is not wisdom
Information is not knowledge
Data is not information
Measurements are not data!
from IEEE-Lifesciences Jul 2013)
by Mathukumalli Vidyasagar
Where is the Life we have lost in living?
Where is the wisdom we have lost in
knowledge?
Where is the knowledge we have lost in
information?
from "The Rock"
by Thomas Stearns Eliot (1888-1965)
Knowledge is not wisdom
Information is not knowledge
Data is not information
Measurements are not data!
from IEEE-Lifesciences Jul 2013)
by Mathukumalli Vidyasagar
Machine Learning: machines
(algorithms / programs) that
learn from data
Computational Intelligence:
algorithms / programs inspired
by the natural computing
(neural networks, fuzzy logic,
genetic algorithms, etc.)
BIG DATA and Health
Using ML and CI we can combine

physiological parameters (blood count, body
temperature, heart rate, weight, etc..) of people with

their personal information (age, gender),

their geographical location

and data related to their lifestyle (frequency of
physical activity, diet, vices, etc.).
48
BIG DATA and Health
Asthmapolis, a startup that has developed a special device
mountable on any inhaler for asthmatics.
Among other things , this device is equipped with a GPS that
allows, every time the user performs an inhalation , to record
its position.
By combining geo-localized data of thousands of asthma is
possible to assess what influence has the environment in which
they live on the frequency of attacks.
49
BIG DATA and Health
CellScope (California) is developing an otoscope to
connect to the phone. It could be used at home to
diagnose ear infections of children, which cost millions
of visits to doctors each year.
Scanadu is testing a sensor, placed on the forehead of the
patient, is able to monitor heart rate, respiratory rate
and body temperature data that are processed and
stored directly on the phone.
50
BIG DATA and Health
Oncologists of Memorial Sloan- Kettering Cancer Centre in New
York are testing the IBM supercomputer Watson a program that can
be used not only for diagnosis in oncology, but also in medicine.
Watson uses a very comprehensive data collection , knows how to
manage the complexity of information and especially not swayed by
personal preference. Diagnosis modulated with probability scale .
Predictive Medical Technologies ( San Francisco ) predictive
algorithm (program) that analyzes the data of patients in intensive
care. This algorithm , to identify patients at risk of heart attack or
other diseases up to 24 hours before they are made .
51
BIG DATA and Health
Japan 6/6/2013
The project Google Flu Trends is active since 2006 and to date has
been monitoring the movements of influenza virus in 25 countries
on five continents.
Twitter as a " thermometer " of the health of Americans
Analyzing the tweets of users suffering from allergies can better
understand trends in the spread of the most common ailments of
52
the season .
BIG DATA and Health
2014: year of wearable sensors
53
BIG DATA and Health
In the next future we may be monitored digitally (bracelet that
constantly records the pressure or heart rate) in connection with
the health care system. Any anomaly would be transmitted
instantly to a health center that would experience the health
personnel,
The goal is to keep us healthy, rather than tackling the disease
IEO (Milan) is piloting a program of daily contact with the patient
after discharge. "This reduces the returns to the hospital for minor
problems."
54
BIG DATA and Health
Medicine more and more personalized, thanks to the spread of a
set of software that learn to "meet you" and then to correct you,
to advise and to warn you, in order to achieve better fitness,
better health, and why not, a mood better.
It is expected that the market for personalized medicine will
reach $ 500 billion by 2017
55
Well-Being Technologies
"Well-Being Technologies" WBT (or "Positive Computing"): using the
technologies of Game Design, Gamification, m-Health, sensors, IoT, Virtual
Reality, Computational Intelligence, to design systems supporting the
development of wellness and human potential.
WBT aims tol the technological support to the Positive Psychology that is a
recent approach of psychology directed to help in achieving a satisfying life
and personal growth, rather than to treat mental illness (pathology). The goal
of Positive Psychology and is the complete mental health seen as a
combination of emotional, psychological, and social well-being, along with
low mental illness.
56
Well-Being Technologies
Mental illness will represent the most costly diseases in the world
over the next two decades (2011-2030), exceeding the combined
cost of cancer, diabetes and chronic obstructive pulmonary
disease (World Economic Forum)
The Well-Being Technologies can give an answer to this serious
problem through
●
Early diagnosis of diseases such as autism, dyslexia, ADHD,
Alzheimer's, Parkinson's;
●
Tools/games for cognitive and /or motor rehabilitation, or to
●
Tools/games for reducing stress in patients who have to
undergo chemotherapy;
●
Tools/games for overcoming phobias.
57
Well-Being Technologies
Gross National Happiness (GNH) index to
measure the standard of view of a country
United Nations are monitoring the wellbeing publishing a World Happiness Report
reflecting a new worldwide demand for
more attention to happiness as criteria for
the governance policy.
58