Download Laboratorio di Intelligenza Artificiale e Robotica

Laboratorio di Intelligenza Artificiale e
Robotica
A.A. 2008-2009
Daniele Loiacono
Outline
2
‰ Machine Learning
Unsupervised Learning
Supervised Learning
Reinforcement Learning
‰ Genetic Algorithms
‰ Genetics-Based Machine Learning
‰ Applications
Computational Intelligence and Games
Machine Learning for Embedded Systems Design
Daniele Loiacono
What is
Machine Learning?
What is Machine Learning?
4
‰ “The field of machine learning is concerned with the question
of how to construct computer programs that automatically
improve with experience.” Tom Mitchell (1997)
‰ A program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with
experience E.
‰ A well-defined learning task is defined by P, T, and E.
Daniele Loiacono
Example: checkers
5
‰ Task T: playing checkers
‰ Artificial Intelligence
Design and implement a computer-based
system that exhibit intelligent action
‰ Machine Learning
Write a program that can learn how to play
It can learn from examples of previous games, by playing
against another opponent, by playing against itself
Daniele Loiacono
Unsupervised
Learning
7
Daniele Loiacono
Supervised
Learning
What is an apple?
Daniele Loiacono
9
What is an apple?
Daniele Loiacono
10
Reinforcement
Learning
RL framework
12
reward
state
Agent
...delay…
...delay…
Environment
Daniele Loiacono
action
Optimal value function
13
‰ The problem becomes learning how to maximize the amount
of reward collected (payoff)
‰ Usually RL algorithms try to learn the action-value function
Q(s,a) and derive from it the optimal policy
Daniele Loiacono
Genetic Algorithms
The population
15
Daniele Loiacono
Ranking by fitness
Daniele Loiacono
16
Mate Selection
17
Selection
probability is
proportional
to fitness
Daniele Loiacono
Crossover
18
Exploit
goodness of
parents
Daniele Loiacono
Mutation
19
Explore
unknown
Daniele Loiacono
Best solution
20
Daniele Loiacono
The GA recipe
21
Daniele Loiacono
Genetic Algorithms
+
Machine Learning
=
Genetic Based
Machine Learning
Examples of GBML
‰
‰
‰
‰
‰
‰
23
Neuroevolution
Classifiers structure and parameters optimization with GAs
Feature selection with GAs
Rule-based Evolutionary Systems
GA-based approach for training classifiers
…
Daniele Loiacono
GBML projects
24
‰ Goal
Implementation of a GBML approach, or
extension of an available implementation,
or comparison of different approaches
Experimental analysis
Technical report
‰ Skills
Basic Knowledge of EC and ML
Good analysis capabilities
Good programming skills (C++)
‰ Students: max 2
Daniele Loiacono
Computational
Intelligence
and
Games
Why games?
26
‰ Modern computer games are
ideal benchmarks for
computational intelligence
techniques
Inexpensive
Challenging
‰ Computational intelligence could
be useful for modern games
Speedup AI design
Improvement of AI
Adaptive AI
Customization of game content
Neverwinter Nights
Civilization II
Daniele Loiacono
Unreal Tournament
Urban Combat
Issues in CIG
27
‰ CIG is an attractive field but involves several technical
issues:
Interface
Experiments design
Simulation speed
Customization
Documentation
‰ Typical benchmarks are specifically designed for doing
research
‰ Modern computer games are very complex and with different
goals
Daniele Loiacono
Projects available on TORCS
‰ TORCS is an open source car
racing simulator
‰ Well suited for CIG research
Software API available
Examples available
Competitions
‰ Available projects
Automated generation of
tracks
Visual input for TORCS
C++ Learning API
Learn your bot!
Daniele Loiacono
28
SmarTrack
29
‰ Goal
Generation, evaluation and classification of tracks for TORCS
‰ Skills
Good programming skills (C++)
Good analysis capabilities
Basic knowledge of EC and/or ML
‰ Students: max 3
‰ Reference
http://cig.dei.polimi.it
http://torcs.sourceforge.net/
Daniele Loiacono
EyeBot
30
‰ Goal
Develop a TORCS controller
based on visual inputs
‰ Skills
Some knowledge/experience
with computer vision is
preferred
Good programming skills
(C++)
‰ Students: max 2
‰ Reference
http://cig.dei.polimi.it
http://torcs.sourceforge.net/
Daniele Loiacono
C++ Learning API
‰ Goal
Developing a learning
framework for TORCS
(partially as porting of an
existing JAVA API)
‰ Skills
Basic knowledge of EC and/or
ML
Good programming skills
(C++)
‰ Students: max 2
‰ Reference
http://cig.dei.polimi.it
http://torcs.sourceforge.net/
Daniele Loiacono
31
Learn your bot!
32
‰ Goal
Apply any evolutionary
computation or machine learning
techniques to develop a
controller for TORCS
Provide the software, the
controller and a presentation
‰ Skills
Knowledge of EC and/or ML
Good analysis capabilities
Basic programming skills (C++)
‰ Students: max 1
‰ Reference
http://cig.dei.polimi.it
http://torcs.sourceforge.net/
Daniele Loiacono
Racing Games for CIG research
‰ Goal
Analysis of racing games for
research purposes
Comparison of different games
Prototype and tutorial for a
specific game
‰ Skills
Good programming skill
Good analysis capabilities
‰ Reference
http://www.rfactor.net/
http://vdrift.net/
Daniele Loiacono
33
Projects on Unreal Tournament
‰ UT is a commercial FPS
‰ Easy to extend through
UnrealScript
‰ GameBots API available for
UT2004
‰ Projects
BotPrize
Analysis and extension
of GameBots
Daniele Loiacono
34
The Botprize
35
‰ A Turing test for FPS bots!
‰ Goal
Analysis of GameBots and of the state of the
art
Submit an entry to the competition
‰ Skills
Good programming skills (Java)
Strong commitment
‰ Students: 4-6
‰ Deadlines
Registration: Sunday 5 October
Intermediate deadline: Friday 31 October
Final deadline: 5 December
‰ Reference
http://botprize.org/
Daniele Loiacono
GameBots for CIG research
36
‰ Goal
Analysis of GameBots for
CIG research
Extending GameBots
Example of machine
learning application
‰ Skills
Good programming (Java)
Knowledge of EC and ML
‰ Reference
https://artemis.ms.mff.cuni.cz/pogamut/tiki-index.php
Daniele Loiacono
Machine Learning
For Embedded
Systems Design
Overview
38
‰ Embedded systems are today more and more complex
‰ There is need of automated tools to support their design
‰ We focus on
Performance models
Design space exploration for high level synthesis
Daniele Loiacono
Performance model
39
‰ Goal
Generation/analysis/pre-processing of the dataset
Application of a supervised learning technique
Experimental analysis
‰ Skills
Good knowledge of ML
Good analysis capabilities
‰ Reference
http://trac.elet.polimi.it/panda/
Additional contact: Ing. Marco Lattuada,
[email protected]
Daniele Loiacono
High Level Synthesis
“High-Level Synthesis means going from an algorithmic level
specification of a behaviour of a digital system to a register
level structure that implements that behavior”.
McFarland, et al., Proc. IEEE, February 1990.
Resource
Library
Behavioral
specification
Design
constraints
High-Level
Synthesis tool
Objectives
Scheduling
Allocation
Binding
Controller
Synthesis
Daniele Loiacono
Datapath
& Controller
EC approaches for HLS
41
‰ Goal
Integration of an EC-based optimization process into the HLS
framework
Experimental analysis
Technical report
‰ Skills
Good programming (C++)
Good analysis capabilities
‰ Reference
http://trac.elet.polimi.it/panda
Additional contact: Ing. Christian Pilato, [email protected]
Daniele Loiacono
Contacts
42
‰ Daniele Loiacono
Email: [email protected]
Web: http://home.dei.polimi.it/loiacono
Stanza 146, Primo Piano, DEI, Tel. 3540
‰ When and how to start ?
Have a look to the references in the slide
Drop me an email for additional info on a specific
project
Select one or two projects you are interested and send
me an email to arrange a meeting on them
Ideal start of the projects would be first week of
November
Daniele Loiacono