Download スライド 1

Sensory Information
Processing
Introduction to imaging sensors
How the image is recorded electrically?
What is the artifact of CCD?
How to select the CCD?
Camera
CCD
CCD driver /
Signal processing
Digital
Capture
lens
What is principal parameter of the lens?
What is aberration?
What is the image degradation by lenses?
How to select the lenses?
the Synchronize and
CameraHow
Sampling is performed?
CCD
CCD driver /
Signal processing
Digital
Capture
lens
How to drive the CCD?
How the color value is recovered?
Measuring the light
• History of light measurement
– Exposure meter for cameras
(Human can not percept absolute amount of
the light)
Leica MC meter (1954)
Contaflex(1935) : world’s first camera with exposure meter
Before that..
• Rely on inspiration and experience..
Senoguide
Device to measure
the amount of light
• Transducing energy of light to electricity
– Photo diode
– Solar cell (similar to photo diode)
• Change of resistance according to the light
– CdS (sulfureted cadmium) cell
– Photo tube
• Light  displacement, speed, etc..
– So weak, very difficult
CdS cell
• Change of resistance of sulfureted
cadmium (compound semiconductor)
– Free electrons are emerged by the light, and
conducts electric current well
– Wide range of sensitivity
– Slow reaction
– cheap
Inadequate for
image sensor
CdS cell(2)
electrode
• Winding CdS area
Relative sensitivity
Electric current (mA)
– Wider width, shorter path
Wavelength(nm)
・very good linearity
Burn out
Voltage (V)
phototube(1)
• Photoelectric effect
– electron is kicked off from metal by photon
ephoton
metal
• Characteristics of photoelectric effect
– No electron for the light with longer wavelength
than threshold which depends on materials
– Number of electron is proportional to the
intensity of light
phototube(2)
• Capturing the electron out from the metal
– Material: Ag, Cs, Na,
Te, Ga, As
– Very high speed
of response :
up to 10ps
•Electron is attracted by
electric potential by high voltage
Phototube(3)
• Very high sensitivity
(photomultiplier)
•Dynodes (SbCs, AgHg etc.) emit lots of electron
by hitting the accelerated electron．
•http://en.wikipedia.org/wiki/Photomultiplier
photomultiplier(4)
• Super-kamiokaNDE (observing facility of
http://en.wikipedia.org/wiki/Kamiokande
neutrino)
• Broken photomultiplier of kamiokaNDE
photodiode(1)
• Major light sensing device
– Photoelectron
emerged between
p and n
semiconductor
– No vaccum!
history of exposure meter
• Solar cell type : require no battery
– １９５０
Leica MC meter
• CdS type : high sensitivity (but slow)
– １９６０
Leica MR meter
• Photodiodes
– １９７０
Current products
2-D image sensor
• How to capture the 2-D distibution of light?
Array of sensors?
Connecting all sensors each?
2-D image sensors
• To reduce the
wiring cost,
scanning
mechanism is
necessary
Electric signal
Camera tube
• Magnetic field is used to deflection (scanning)
Camera and TV tubes
• Magnetic field for scanning
Image orthicon
CCD (charge coupled device)
• Electrostatic effect is
used to transfer the
electron
– Not limited to the imaging
device
– CCD is the term of the
way to transfer the
electron
– http://en.wikipedia.org/wiki
/Charge-coupled_device
Aside : delay line (1)
• How to delay temporal signal?
– “dead time component”
Aside : delay line (1)
Aside : delay line (1)
• Delay line is a memory
– Recording information for a short time
– No sampling, no quantization
Delay line (2)
• SAW (surface acoustic wave) filter
– http://en.wikipedia.org/wiki/Surface_acoustic_wave
Delay line (3) BBD
• BBD = Bucket Brigade delay
Predecessor of CCD
MN3004
Delay line (4) current standard
AD converter
CPU
DA converter
RAM
• Long time delay for sound, video is based
on digital technology
CCD types
•
•
•
•
Full frame
Interline transfer
Frame transfer
Frame interline transfer
• http://laser.physics.sunysb.edu/~jose/CCD
Info.html
• CCD university
– http://www.ccd.com/ccdu.html
Full frame
• While transfer, CCD
should be covered
• (mechanical shutter
is necessary)
• High aperture ratio
• Easy to manufacture
• Commonly used in
digital SLR
Interline transfer
• Most popular
• Mechanical
shutter is not
necessary
• Very short
exposure is
possible
• Used in most
video cameras
Frame transfer
• Image is
transferred to the
storage array
quickly
Image degradation in CCD
• smear
– Highlight spreading to the transfer direction
(vertical)
• blooming
– Highlight spreading to the peripherals
• Thermal noise
– Electron is emerged by the thermal
agitation of free electron
smear
Source of smear
mask mask mask mask
mask
• Imperfect mask
of transfer
CCD areas,
spilling out of
electron to the
transfer area
Anti-blooming
• Discarding
too much
electrons by
highlight
X-Y address image sensor
photodiodes are
connected by wire
and transistor
CMOS transistor
is used, therefore,
commonly called
as CMOS sensor
CMOS image sensor
JFET type
CMOS image sensor
• Low power consumption
– CCD needs high voltage to attract the
electron
• Other circuit can be embedded on a chip
(usual LSI process is used)
– AD converter, image processor can be
embedded
• Noise was more than CCD
– Now it is well improved
• Flexibility to access to each pixel
Rolling shutter
• For most CMOS sensors, exposure timing
is varied pixel by pixel
– Called “rolling shutter”
– If all pixels are exposed simultaneously, it is
called “global shutter”
Shutter types and image skew
Lens shutter
（interchanging the lens is not easy)
Focal plane shutter
（suit for interchanging lens)
SLR (single lens reflex) etc.
Skew of moving object
• Shape of the
object is skewed
by the varied
timing of
exposure
Ancient example of skew
• Lartigue “A.C.F grand prix”
World’s most famous example of skew
Color image
• Sequencial shots of R, G, B
– filter wheel
• Split R, G, B to three image sensors
optically
– ３ tubes, 3-CCD
• Each pixel is sensitive either R, G or B
– Primal color, complementary color
– Foveon
Filter wheel
３ tubes
prism
Camera tube
lens
３-CCDcamera
Each surface in the prism is
“interference filter” which
consist of transparent
materials with different
refractive index, and
interference of light reflect or
transmit the energy of the light
without loss.
• No loss of light energy, high sensitivity
• Color reproduction is good
Single CCD
R G R G R G R G
Cy Ye Cy Ye Cy Ye Cy Ye
G B G B G B G B
G Mg G Mg G Mg G Mg
R G R G R G R G
Cy Ye Cy Ye Cy Ye Cy Ye
G B G B G B G B
Mg G Mg G Mg G Mg G
R G R G R G R G
Cy Ye Cy Ye Cy Ye Cy Ye
G B G B G B G B
G Mg G Mg G Mg G Mg
Primal color
（Bayer pattern)
Complementary color
• Each pixel of CCD is colored
– Primal color
○natural color reproduction
– Complementary color
○high light efficiency
Color reproduction (2)
• Simple method (interpolation)
G
G
G
G
G
G
G
G
G
G
G
G
G
G
R
R
R
G
R
R
R
R
G
R
R
R
R
Low resolution
G
G
G
R
G
G
G
G
G
• See Fredo Durand and Bill Freeman’s slide
– http://groups.csail.mit.edu/graphics/classes/CompPho
to07/index2.html
– 03_colorInterpCP.ppt