Download CS 414

Multimedia Systems &
Interfaces
Karrie G. Karahalios
Spring 2007
Perception
•
•
•
•
Review
Expectations
Perception
Homework
Color and Visual System
• Color refers to how we
perceive a narrow band
of electromagnetic
energy
– source, object,
observer
• Visual system
transforms light energy
into sensory experience
of sight
Human Visual System
• Eyes, optic nerve,
parts of the brain
• Transforms
electromagnetic
energy
Human Visual System
• Formation
– cornea, sclera, pupil,
iris, lens, retina, fovea
• Transduction
– retina, rods, and cones
• Processing
– optic nerve, brain
Image Formation
•
•
•
•
•
•
Cornea and sclera
Pupil
Iris
Lens
Retina
Fovea
Sclera
Retina
Fovea
Lens
Cornea
Pupil
Iris
The Cornea
• Part of sclera – hard
white part of the eye
• Transparent part at
front of eye
• Allows light to enter,
refraction occurs
Sclera
Cornea
The Pupil and Iris
• Controls amount of
light passing through
– diameter varies in
response to light
Pupil
• Iris controls the
diameter of the pupil
– gives eye its color
Iris
The Lens
• Focuses light on the
retina using refraction
• Changes shape to
provide focus
– spherical for
closer objects
– flat for far objects
– accommodation
Lens
The Retina and Fovea
• Retina has photosensitive
receptors at back of eye
• Fovea is small, dense
region of receptors
– only cones (no rods)
– gives visual acuity
• Outside fovea
– fewer receptors overall
– larger proportion of rods
Retina
Fovea
The Transduction
• Transform light to
neural impulses
• Receptors signal
bipolar cells
• Bipolar cells signal
ganglion cells
• Axons in the ganglion
cells form optic nerve
Bipolar cells
Rods
Ganglion
Cones
Optic nerve
Rods and Cones
Cones
•
•
•
•
Contain photo-pigment
Respond to high energy
Enhance perception
Concentrated in fovea,
exist sparsely in retina
• Three types, sensitive to
different wavelengths
Rods
•
•
•
•
Contain photo-pigment
Respond to low energy
Enhance sensitivity
Concentrated in retina,
but outside of fovea
• One type, sensitive to
grayscale changes
Rod and Cone Destiny
120 million rods
6-7 million cones
From http://hyperphysics.phy-astr.gsu.edu/hbase/vision/rodcone.html
Tri-stimulus Theory
• 3 types of cones (6 to 7 million of them)
– Red (64%), Green (32%), Blue (2%)
• Each type most responsive to a narrow band
– red and green absorb most energy, blue the least
• Light stimulates each set of cones differently,
and the ratios produce sensation of color
Tri-stimulus Theory
Opponent-Color Theory
• Visual system contains two types of colorsensitive units
– red / green; blue / yellow
• Each component in a unit responds
opposite the other component
– e.g., if red-green responds to red, then
green is inhibited
• Explains concept of ‘after images’
Visual System Facts
• Distinguish hundreds of thousands of colors
– more sensitive to brightness
• Can distinguish about 28 fully saturated hues
– less sensitive to hue changes in less saturated colors
• Can distinguish about 23 levels of saturation for
fixed hue and lightness
• 10 times less sensitive to blue than red or green
– it absorbs less energy in the blue range
Physical Properties of Color
• Dominant wavelength
– electromagnetic waves
– 400nm (violet) to 700nm (red)
• Excitation purity
• Luminance
Spectral Distribution
• Dominant wavelength
• Excitation purity
– ratio between e2 / e1
Energy distribution
– spike in power (e2)
– white light is uniform
energy distribution
e2
e1
Wavelength
Perceptual Properties of Color
• Hue
– distinguishes named colors, e.g., RGB
– dominant wavelength of the light
• Saturation
– how far color is from a gray of equal intensity
• Brightness (lightness)
– perceived intensity
Color Perception
• Hue
– distinguishes named colors, e.g., RGB
– dominant wavelength of the light
• Saturation
– how far color is from a gray of equal intensity
• Brightness (lightness)
– perceived intensity
White
Tints
Pure
colors
Tones
Grays
Black
Shades
CIE
• Standard reference
Color Models
Additive (RGB)
Applies to light-emitting
sources (TVs, monitors, etc)
Subtractive (CMY)
Applies to reflected light
(printed images, paints, etc)
HSV (aka HSB)
• User-oriented, based on use of tints, shades,
and tones
RGB
• Based on the fact that the human visual system
maintains three types of cones (RGB cones)
• Different weightings produce different colors
Green
(0,1,0)
Yellow
(1,1, 0)
Cyan
(0,1,1)
White
(1,1,1)
Black
(0,0,0)
Blue
(0,0,1)
Red
(1,0,0)
Magenta
(1,0,1)
YIQ (aka YUV)
• Used in US television broadcasting
• Recoding of RGB for efficiency and
compatibility with black-and-white TV
• Y is luminance (not yellow!)
• I and Q is chromaticity
Y 
I 
 
Q 
=
.114 
.299 .587
.596  .275  .321


.212  .523 .311 
R
G 
 
 B 
CMYK
• Subtractive color model
– used for printing, painting, etc.
• CMY are the complements of RGB
– two complementary colors gives a primary
C 
M 
 
 Y 
=
1
1 _

1
R
G 
 
 B 
Gestalt
Auditory Perception
Waves
• Periodic disturbance that travels through
a medium (e.g. air or water)
• Transport energy
• Transverse or longitudinal
• Electromechanical or mechanical
Sound
• A longitudinal, mechanical wave
– caused by a vibrating source
• Pack molecules at different densities
– cause small changes in pressure
• Model pressure differences as sine waves
Volume and Pressure
Auditory System
• Ears, parts of brain,
and neural pathways
• Changes in pressure
move hair-like fibers
within the inner ear
• Movements result in
electrical impulses
sent to the brain
Process of Hearing (Transduction)
Frequency (temporal) Theory
• Periodic stimulation of membrane
matches frequency of sound
– one electrical impulse at every peak
– maps time differences of pulses to pitch
• Firing rate of neurons far below
frequencies that a person can hear
– Volley theory: groups of neurons fire in wellcoordinated sequence
Place Theory
• Waves move down basilar membrane
– stimulation increases, peaks, and quickly tapers
– location of peak depends on frequency of the
sound, lower frequencies being further away
Physical Dimensions
• Amplitude
– height of a cycle
– relates to loudness
• Wavelength (w)
– distance between peaks
• Frequency (  )
– cycles per second
– relates to pitch
– w = velocity
• Most sounds mix many
frequencies & amplitudes
Sound is repetitive changes
in air pressure over time
Psychological Dimensions
• Loudness
– higher amplitude results in louder sounds
– measured in decibels (db), 0 db represents
hearing threshold
• Pitch
– higher frequencies perceived as higher pitch
– hear sounds in 20 Hz to 20,000 Hz range
Psychological Dimensions
• Timbre (tam-bre)
– complex patterns added to the
lowest, or fundamental, frequency of
a sound, referred to as spectra
– spectra enable us to distinguish
musical instruments
• Multiples of fundamental
frequency give music
• Multiples of unrelated frequencies
give noise
Sound Intensity
• Intensity (I) of a wave is the rate at which sound
energy flows through a unit area (A)
perpendicular to the direction of travel
1 E P
I

A t
A
P measured in watts (W), A measured in m2
• Threshold of hearing is at 10-12 W/m2
• Threshold of pain is at 1 W/m2
Decibel Scale
• Describes intensity relative to threshold
of hearing based on multiples of 10
I
dB  10 log
I0
Decibels of Everyday Sounds
Sound
Decibels
Rustling leaves
10
Whisper
30
Ambient office noise
45
Conversation
60
Auto traffic
80
Concert
120
Jet motor
140
Spacecraft launch
180
Loudness from Multiple Sources
• Use energy combination equation
L  10 log( 10
L1
10
 10
L2
10
 ...  10
where L1, L2, …, Ln are in dB
LN
10
)
Exercises
• Show that the threshold of hearing is at 0 dB
• Show that the threshold of pain is at 120 dB
• Suppose an electric fan produces an intensity of 40 dB.
How many times more intense is the sound of a
conversation if it produces an intensity of 60 dB?
• One guitar produces 45 dB while another produces 50
dB. What is the dB reading when both are played?
• If you double the physical intensity of a sound, how many
more decibels is the resulting sound?
Loudness and Pitch
• More sensitive to loudness at mid
frequencies than at other frequencies
– intermediate frequencies at [500hz, 5000hz]
• Perceived loudness of a sound changes
based on the frequency of that sound
– basilar membrane reacts more to intermediate
frequencies than other frequencies
Fletcher-Munson Contours
Each contour represents an equal perceived sound
Masking
• Perception of one sound interferes with
another
• Frequency masking
• Temporal masking
Frequency Masking
• Louder, lower frequency sounds tend to
mask weaker, higher frequency sounds
From http://www.cs.sfu.ca/CourseCentral/365/
Frequency Masking
• Louder, lower frequency sounds tend to
mask weaker, higher frequency sounds
Frequency Masking
• Louder, lower frequency sounds tend to
mask weaker, higher frequency sounds
Temporal Masking
• When exposed to a loud sound, the human ear
contracts slightly to protect delicate structures
• Causes louder sounds to overpower weaker
sounds just before and just after it
From http://www.cs.sfu.ca/CourseCentral/365/
Combined Masking
From http://www.cs.sfu.ca/CourseCentral/365/
Localization of Sound
• Localization occurs because
– sound reaches one ear before the other
– the head creates a ‘sound shadow’ that
decreases intensity of the sound to far ear
Assignment
• Read pamphlets by Andries Van Dam:
Color
Illumination
Intensity Demo
Density Demonstration