Download Machine Vision - Intelligent Systems Laboratory

TECHNICAL UNIVERSITY OF CRETE
DEPARTMENT OF ELECTRONIC AND COMPUTER
ENGINEERING
MACHINE VISION
Euripides G.M. Petrakis
Michalis Zervakis
http://www.intelligence.tuc/~petrakis
http://courses.ece.tuc.gr
Chania 2010
E.G.M. Petrakis
Machine Vision (Introduction)
1
Machine Vision
• The goal of Machine Vision is to create a
model of the real world from images
– A machine vision system recovers useful
information about a scene from its two
dimensional projections
– The world is three dimensional
– Two dimensional digitized images
E.G.M. Petrakis
Machine Vision (Introduction)
2
Machine Vision (2)
• Knowledge about the objects (regions) in a
scene and projection geometry is required.
• The information which is recovered differs
depending on the application
– Satellite, medical images etc.
• Processing takes place in stages:
– Enhancement, segmentation, image analysis
and matching (pattern recognition).
E.G.M. Petrakis
Machine Vision (Introduction)
3
Illumination
Image
Acquisition
Scene
Machine
Vision System
2D
Digital Image
Image
Description
Feedback
The goal of a machine vision system is to compute a
meaningful description of the scene (e.g., object)
Machine Vision Stages
Image Acquisition
(by cameras, scanners etc)
Image Processing
Image Enhancement
Image Restoration
• Find regions (objects) in
the image
Image Segmentation
• Take measurements of
objects/relationships
Image Analysis
(Binary Image Processing)
Model Matching
Pattern Recognition
E.G.M. Petrakis
• Analog to digital
conversion
• Remove noise/patterns,
improve contrast
• Match the above
description with similar
description of known
objects (models)
Machine Vision (Introduction)
5
Image Processing
Image Processing
Input Image
Output Image
• Image transformation
– image enhancement (filtering, edge detection, surface detection,
computation of depth).
– Image restoration (remove point/pattern degradation: there exist a
mathematical expression of the type of degradation like e.g. Added
multiplicative noise, sin/cos pattern degradation etc).
E.G.M. Petrakis
Machine Vision (Introduction)
6
Image Segmentation
Image Segmentation
Input Image
Regions/Objects
• Classify pixels into groups (regions/objects of interest)
sharing common characteristics.
– Intensity/Color, texture, motion etc.
• Two types of techniques:
– Region segmentation: find the pixels of a region.
– Edge segmentation: find the pixels of its outline contour.
E.G.M. Petrakis
Machine Vision (Introduction)
7
Image Analysis
Image Analysis
Input Image
Segmented Image
(regions, objects)
Measurements
• Take useful measurements from pixels, regions, spatial
relationships, motion etc.
– Grey scale / color intensity values;
– Size, distance;
– Velocity;
E.G.M. Petrakis
Machine Vision (Introduction)
8
Pattern Recognition
Model Matching
Pattern Recognition
Image/regions •Measurements, or
•Structural description
Class identifier
• Classify an image (region) into one of a number of known
classes
– Statistical pattern recognition (the measurements form vectors
which are classified into classes);
– Structural pattern recognition (decompose the image into primitive
structures).
E.G.M. Petrakis
Machine Vision (Introduction)
9
Digital Image Representation
• Image: 2D array of gray level or color values
– Pixel: array element;
– Pixel value: arithmetic value of gray level or color
intensity.
• Gray level image: f = f(x,y)
- 3D image f=f(x,y,z)
• Color image (multi-spectral)
f = {Rred(x,y), Ggreen(x,y), Bblue(x,y)}
E.G.M. Petrakis
Machine Vision (Introduction)
10
What a computer “sees” is very different from what
a human sees. A computer sees pixels (arithmetic values)
while a human sees shapes, structures etc.
E.G.M. Petrakis
Machine Vision (Introduction)
11
Relationships to other fields
•
•
•
•
•
•
Image Processing (IP)
Pattern Recognition (PR)
Computer Graphics (CG)
Artificial Intelligence (AI)
Neural Networks (NN)
Psychophysics
E.G.M. Petrakis
Machine Vision (Introduction)
12
Image Processing (IP)
• IP transforms images to images
– Image filtering, compression, restoration
• IP is applied at the early stages of machine
vision.
– IP is usually used to enhance particular
information and to suppress noise.
E.G.M. Petrakis
Machine Vision (Introduction)
13
Pattern Recognition (PR)
• PR classifies numerical and symbolic data.
– Statistical: classify feature vectors.
– Structural: represent the composition of an
object in terms of primitives and parse this
description.
• PR is usually used to classify objects but
object recognition in machine vision usually
requires many other techniques.
E.G.M. Petrakis
Machine Vision (Introduction)
14
Statistical Pattern Recognition
• Pattern: the description of an an object
– Feature vector
– (size, roundness, color, texture)
• Pattern class: set of patterns with similar
characteristics.
• Take measurements from a population of
patterns.
• Classification: Map each pattern to a class.
E.G.M. Petrakis
Machine Vision (Introduction)
15
Structure of PR Systems
input
Sensor
Processing
Measurements
Classification
class
E.G.M. Petrakis
Machine Vision (Introduction)
16
Example of Statistical PR
•
•
Two classes:
I. W1 Basketball players
II. W2 jockeys
Description: X = (X1, X2) = (height, weight)
X1
W2
.. ..
. ..
. . .. .
+
W1
.. ……
. … ..
……
-
D(X) = AX1 + BX2 + C = 0
Decision function
X2
E.G.M. Petrakis
Machine Vision (Introduction)
17
Syntactic Pattern Recognition
• The structure is important
• Identify primitives
– E.g., Shape primitives
• Break down an image (shape) into a sequence of
such primitives.
• The way the primitives are related to each other to
form a shape is unique.
– Use a grammar/algorithm
– Parse the shape
E.G.M. Petrakis
Machine Vision (Introduction)
18
•Primitives
•G1,L(G1) : submedian Grammar
•G2,L(G2) : telocentric Grammar
E.G.M. Petrakis
Machine Vision (Introduction)
19
•Each digit is represented by a waveform representing
black/white, white/black transitions (scan the image from
Left to right.
E.G.M. Petrakis
Machine Vision (Introduction)
20
Computer Graphics (CG)
• Machine vision is the analysis of images
while CG is the decomposition of images:
– CG generates images from geometric primitives
(lines, circles, surfaces).
– Machine vision is the inverse: estimate the
geometric primitives from an image.
• Visualization and virtual reality bring these
two fields closer.
E.G.M. Petrakis
Machine Vision (Introduction)
21
Artificial Intelligence (AI)
• Machine vision is considered to be sub-field of AI.
• AI studies the computational aspects of
intelligence.
• CV is used to analyze scenes and compute
symbolic representations from them.
• AI: perception, cognition, action
– Perception translates signals to symbols;
– Cognition manipulates symbols;
– Action translates symbols to signals that effect the
world.
E.G.M. Petrakis
Machine Vision (Introduction)
22
Psychophysics
• Psychophysics and cognitive science have
studied human vision for a long time.
• Many techniques in machine vision are
related to what is known about human
vision.
E.G.M. Petrakis
Machine Vision (Introduction)
23
Neural Networks (NN)
• NNs are being increasingly applied to solve
many machine vision problems.
• NN techniques are usually applied to solve
PR tasks.
– Image recognition/classification.
• They have also applied to segmentation and
other machine vision tasks.
E.G.M. Petrakis
Machine Vision (Introduction)
24
Machine Vision Applications
•
•
•
•
•
•
•
Robotics
Medicine
Remote Sensing
Cartography
Meteorology
Quality inspection
Reconnaissance
E.G.M. Petrakis
Machine Vision (Introduction)
25
Robot Vision
• Machine vision can make a robot manipulator
much more versatile.
– Allow it to deal with variations in parts position and
orientation.
E.G.M. Petrakis
Machine Vision (Introduction)
26
Remote Sensing
• Take images from
high altitudes (from
aircrafts, satellites).
• Find ships in the aerial
image of the dock.
– Find if new ships have
arrived.
– What kind of ships?
E.G.M. Petrakis
Machine Vision (Introduction)
27
Remote Sensing (2)
• Analyze the image
– Generate a description
– Match this descriptions
with the descriptions of
empty docs
• There are four ships
– Marked by “+”
E.G.M. Petrakis
Machine Vision (Introduction)
28
Medical Applications
• Assist a physician to
reach a diagnosis.
• Construct 2D, 3D
anatomy models of the
human body.
– CG geometric models.
• Analyze the image to
extract useful features.
E.G.M. Petrakis
Machine Vision (Introduction)
29
Machine Vision Systems
• There is no universal machine vision system
– One system for each application
• Assumptions:
– Good lighting;
– Low noise;
– 2D images
• Passive - Active environment
– Changes in the environment call for different actions
(e.g., turn left, push the break etc).
E.G.M. Petrakis
Machine Vision (Introduction)
30
Vision by Man and Machine
• What is the mechanism of human vision?
– Can a machine do the same thing?
– There are many studies;
– Most are empirical.
• Humans and machines have different
– Software
– Hardware
E.G.M. Petrakis
Machine Vision (Introduction)
31
Human “Hardware”
• Photoreceptors take measurements of light signals.
– About 106 Photoreceptors.
• Retinal ganglion cells transmit electric and
chemical signals to the brain
– Complex 3D interconnections;
– What the neurons do? In what sequence?
– Algorithms?
• Heavy Parallelism.
E.G.M. Petrakis
Machine Vision (Introduction)
32
Machine Vision Hardware
• PCs, workstations etc.
• Signals: 2D image arrays gray level/color values.
• Modules: low level processing, shape from
texture, motion, contours etc.
• Simple interconnections.
• No parallelism.
E.G.M. Petrakis
Machine Vision (Introduction)
33
Course Outline
• Introduction to machine vision, applications,
Image formation, color, reflectance, depth,
stereopsis.
• Basic image processing techniques (filtering,
digitization, restoration), Fourier transform.
• Binary image processing and analysis, Distance
transform, morphological operators.
E.G.M. Petrakis
Machine Vision (Introduction)
34
Course Outline (2)
• Image segmentation (region segmentation, edge
segmentation).
• Edge detection, edge enhancement and
linking. Thresholding, region growing, region
merging/splitting.
• Relaxation labeling, Hough transform.
• Image analysis, shape analysis. Polygonal
approximation, splines, skeletons. Shape features,
multi-resolution representations.
E.G.M. Petrakis
Machine Vision (Introduction)
35
Course Outline (3)
• Image representation, image - shape recognition
and classification. Attributed relational graphs,
semantic nets.
• Image - shape matching (Fourier descriptors,
moments, matching in scale space).
• Texture representation and recognition, statistical
and structural methods.
• Motion, motion detection, optical flow.
• Video
E.G.M. Petrakis
Machine Vision (Introduction)
36
Bibliography
• “Machine Vision”, Ramesh Jain, Rangachar
Kasturi, Brian G. Schunck, Mc Graw-Hill, 1995
(highly recommended!).
• "Image Processing, Analysis and Machine
Vision", Milan Sonka, Vaclav Hlavac,
Roger Boyle, PWS Publishing, Second
Edition.
• "Machine Vision, Theory, Algorithms,
Practicalities'', E. R. Davies, Academic Press,
1997.
E.G.M. Petrakis
Machine Vision (Introduction)
37
• "Practical Computer Vision Using C'', J.
R. Parker, John Wiley & Sons Inc., 1994.
• Selected articles from the literature.
• Lecture notes
(http://www.intelligence.tuc/~petrakis)
• Webcourses (http://courses.ece.tuc.gr)
E.G.M. Petrakis
Machine Vision (Introduction)
38
Grading Scheme
• Final Exam (F): 40%, min 5
• Assignments (Α): 40%
• Two assignments
– Obligatory
E.G.M. Petrakis
Machine Vision (Introduction)
39