Download Folie 1 - Signal processing

Institute of Electrical Measurement and Measurement Signal Processing
VISION:
Axel Pinz
•
•
•
•
•
Paradigms
Systems
Algorithms
Applications
Evaluation
 A Graph-based structure !
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
1
Institute of Electrical Measurement and Measurement Signal Processing
The Graph of Vision History
Systems
1978
Pattern Recognition
Image Proc. 1970s
Artificial Intelligence
1971
Image Understanding
1980s
Cognitive Psychology
1985 ?
Neurophysiology
1963
Algorithms
1962
Recognition
1970s
Marr
1978
Robotics
1994 ?
R+R
1990
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Reconstruction
1980 ?
Axel Pinz
9.5.2008
2
Institute of Electrical Measurement and Measurement Signal Processing
Artificial Intelligence
• SHRDLU [Winograd 1971]
• Blocks world
original screen display (MIT AI Lab)
later color rendering (Univ. Utah)
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
3
Institute of Electrical Measurement and Measurement Signal Processing
Blocks World – 1970s
•
•
•
•
Natural language understanding
Spatio-temporal reasoning
LISP
Visual input would be nice
• Vision as an “enabling sensory module” in AI
– Patrick Henry Winston “The Psychology of Computer Vision” 1975
– Marvin Minsky “A framework for representing knowledge” 1975
• Frames
• Frame Representation Language FRL
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
4
Institute of Electrical Measurement and Measurement Signal Processing
Blocks World – Today
• Still an issue of ongoing research…
• EU FP 6 project MACS
• 2004-2008
 Learning !
 “Affordances” …
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
5
Institute of Electrical Measurement and Measurement Signal Processing
AI Methods for Vision – 1980s
• “Image Understanding”
• Expert systems
– SIGMA Aerial Image Understanding
– Matsuyama, Hwang, …
• Reasoning, Blackboard system
– VISIONS, Draper, OHM, …
• Schema learning
– SLS, Draper,…
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
6
Institute of Electrical Measurement and Measurement Signal Processing
SIGMA [Matsuyama]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
7
Institute of Electrical Measurement and Measurement Signal Processing
VISIONS [Hanson+Riseman], SLS [Draper]
A Massachusetts “road”-scene
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
8
Institute of Electrical Measurement and Measurement Signal Processing
“Image Understanding” – 1980s
• Integrating bottom-up and top-down
processes  combinatorial explosion
• “knowledge engineers”, hand-crafted
knowledge-base  learning is a must
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
9
Institute of Electrical Measurement and Measurement Signal Processing
Systems 1990 – 200+
• “Computer Vision Systems”
– [Hanson+Riseman 1978]
• UMass VISIONS  KBVision
– AAI Amerinex Artificial intelligence
– AAI Amerinex Applied Imaging
• Grimson  Cognex
– PatMax
• IUE
– TargetJr Joe Mundy
• Matlab
• OpenCV
• toolboxes, libraries
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
10
Institute of Electrical Measurement and Measurement Signal Processing
Robotics
• Brooks: “building brains for bodies”
– Autonomous Robots 1:7-25, 1994
• Learning!
• Interaction!
• Reward by survival…!
• Active, purposive, qualitative
– Bajcsy 1988
– Aloimonos 1989
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
11
Institute of Electrical Measurement and Measurement Signal Processing
Babybot – Lira Lab [Sandini]
learning to push
learnt
Babybot’s view
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
12
Institute of Electrical Measurement and Measurement Signal Processing
Neurophysiology
•
•
•
•
Retina  visual cortex
Layered representation in visual cortex
Receptive fields
Hubel+Wiesel
–
–
–
–
–
Gradients
Oriented “edgels”
At various scales
Bottom-up grouping  recognition
Visual pathway  stereo, reconstruction
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
13
Institute of Electrical Measurement and Measurement Signal Processing
Visual Pathway [Hubel+Wiesel]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
14
Institute of Electrical Measurement and Measurement Signal Processing
Receptive Fields [Hubel+Wiesel]
on-off
simple
complex
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
15
Institute of Electrical Measurement and Measurement Signal Processing
Pattern Recognition + Image Processing – PRIP
• Structural [Pavlidis 1972, Fu 1982]
• Statistical [Fukunaga 1990]
• Representation (regular, irregular, scale,…)
pyramids, scale space, graphs,…
• Feature sets, feature selection, classifiers,
learning,…
• 2D (discrete) signal processing
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
16
Institute of Electrical Measurement and Measurement Signal Processing
Maximum Likelihood Classification
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
17
Institute of Electrical Measurement and Measurement Signal Processing
Scale Space [Lindeberg]
Space + time
[Laptev + Lindeberg]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
18
Institute of Electrical Measurement and Measurement Signal Processing
Cognitive Psychology
• Perceptual grouping
– Lowe 1985
– Sarkar 1994
– LLVE Matsuyama
• Qualitative volumetric models (Geons)
– Biederman 1985, Bergevin+Levine 1988
– Dickinson 1992: Aspect Hierarchy
• A renaissance in today’s shape models for
category detection…
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
19
Institute of Electrical Measurement and Measurement Signal Processing
My Favorite Example [Lowe]
Very hard !!!
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
20
Institute of Electrical Measurement and Measurement Signal Processing
My Favorite Example [Lowe]
Very easy !!!
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
21
Institute of Electrical Measurement and Measurement Signal Processing
Geons [Biederman], Aspect Hierarchy [Dickinson]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
22
Institute of Electrical Measurement and Measurement Signal Processing
Geons [Biederman], Aspect Hierarchy [Dickinson]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
23
Institute of Electrical Measurement and Measurement Signal Processing
The Marr Paradigm – 1978-?
• Systems approach to vision  computational
model
– Algorithm, data, hardware implementation
– Low level modules
• Representational levels
– Primal sketch  saliency
– 2-1/2 D  reconstruction !
– 3D object model  is it really required?
•
•
•
•
“reconstruction school”
“visual modules”, “shape from X”
Zerroug 1994
Structure + Motion 200x
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
24
Institute of Electrical Measurement and Measurement Signal Processing
Marr (~1978) – “primal sketch”
“saliency” !
“Harris” corners
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
25
Institute of Electrical Measurement and Measurement Signal Processing
Marr (~1978) –
“2-1/2-D sketch”, “generalized cone”
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
26
Institute of Electrical Measurement and Measurement Signal Processing
Marr (~1978) – 3D hierarchical models
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
27
Institute of Electrical Measurement and Measurement Signal Processing
Marr ~1978, Aloimonos ~1988
“robustness”
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
28
Institute of Electrical Measurement and Measurement Signal Processing
Zerroug 1994
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
29
Institute of Electrical Measurement and Measurement Signal Processing
Algorithms
•
•
•
•
•
•
•
•
•
•
•
•
•
Convolution  kernels
Mathematical morphology  structuring elements Serra
Fourier transform  spectrum, phase, descriptors, Gabor filter bank
Hough transform  voting space
PCA
Pyramids (regular, irregular)  graphs
Scale space theory in CV Lindeberg
Algebraic projective geometry Hartley, Zisserman
Color (good chapters in Forsyth+Ponce, Burger+Burge)
Texture  invariant moments (Hu … Haralick … VanGool)
Interest point detection (Schmid, …)  affine covariance
Pattern Classification Duda+Hart
Machine Learning Hastie et al.
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
30
Institute of Electrical Measurement and Measurement Signal Processing
Reconstruction
•
•
•
•
Projective geometry
Photogrammetry
Essential, fundamental, multiview, …
Calibrated, uncalibrated, autocalibration,
calibration-free,…
• Structure and Motion
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
31
Institute of Electrical Measurement and Measurement Signal Processing
Structure and motion [Schweighofer 2008]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
32
Institute of Electrical Measurement and Measurement Signal Processing
Recognition
• Specific objects
– PCA [Murase+Nayar 95], Eigenfaces [Pentland 93]
• Categories
– Generative [Fergus 2003], discriminative [Opelt 2004]
• Recognition vs. localization
• Features (descriptors), grouping
– Shape, texture, color, proximity, similarity,…
• Saliency (detectors)
• Active
– Motion, space+time …
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
33
Institute of Electrical Measurement and Measurement Signal Processing
Shape-Based Category Detection [Opelt 2006]
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
34
Institute of Electrical Measurement and Measurement Signal Processing
“R + R”: Confluence of
Recognition and Reconstruction
• “recognition and reconstruction schools will merge”
[Aloimonos+Shulman 1989]
• “Marr paradigm – use of 2-1/2D complicates the problem”
[Medioni et al. 2000]  tensor voting
• Spatio-temporal reasoning and representation
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Axel Pinz
9.5.2008
35
Institute of Electrical Measurement and Measurement Signal Processing
The Graph of Vision History
Systems
1990s
Pattern Recognition
Image Proc. 1970s
Artificial Intelligence
1971
Image Understanding
1980s
Cognitive Psychology
1985 ?
Neurophysiology
1963
Algorithms
1962
Recognition
1970s
Marr
1978
Robotics
1994 ?
R+R
1990
HistoryHorst
of Computer
Vision Paradigms-Systems-Algorithms-Applications-Evaluation
Professor
Cerjak, 19.12.2005
Reconstruction
1980 ?
Axel Pinz
9.5.2008
36