Download Audio in Interactive Digital Media

10: Audio in Interactive Digital
Media
Introduction to Interactive Media
What is Sound?
• All sounds are produced
by the conversion of
energy into vibrations.
• Vibration becomes a
wave. When it reaches
the ear, it causes the
eardrum to vibrate at the
same frequency
Introduction to Interactive Media
Audible Spectrum
• Humans can
hear sounds in
range of 20 Hz –
20 kHz.
• Highest note on
piano is approx.
4kHz.
Introduction to Interactive Media
Sound Over Time
• Most sounds
change
frequency
over time.
• Represented
in waveform.
Introduction to Interactive Media
Music vs. Speech
Music waveform
Speech waveform
Waveform conveys gross character and dynamics of sound. Can provide
cues in syncing media.
Introduction to Interactive Media
The Nature of Sound
• Wave captures three features of sound:
– Amplitude
• Perceived as volume.
– Frequency
• Perceived as pitch.
– Duration
• Length of time sound lasts.
Introduction to Interactive Media
Getting Sound in and Out of
the Computer
• Analog to Digital Converter
captures separate
measures of sound
amplitude.
– Samples are recorded as
digital numbers.
• Digital values are used to
recreate the analog form
using a Digital to Analog
Converter.
Introduction to Interactive Media
Recording Sound
• Best to not do directly into computer as noise
from within the computer is picked up.
• Sample rate can be reduced when recording
speaking.
Introduction to Interactive Media
Recording Sound
• When capturing sound, adjust levels.
– If amplitude is too low, sound quality is reduced;
– If amplitude is too high, clipping occurs and
produces distortion
Introduction to Interactive Media
Sampling Sound
• Quality of the sampling
depends on:
– Sample resolution
– Sample rate.
Introduction to Interactive Media
Sample Resolution
• Number of bits to encode amplitude.
• Like images, more bits used to describe
information – more accurately it will be
represented.
• Two common sample resolutions are 8-bit and 16-bit.
– 8-bit resolution captures 256 different amplitude
levels.
– 16-bit sound has 65,000 different levels.
• CD quality sound.
• Inadequate sample resolution can distort the
sound.
Introduction to Interactive Media
Sample Rate
• Number of samples taken in a fixed interval of time.
– Stated in thousands of Hertz, or kilohertz.
– CD-quality sound captures 44.1kHz
to record frequencies as high as 22.05kHz.
(The highest frequency the human ear can detect is 20kHz.)
• Two measurements capture each cycle of the sound
wave:
• High value or peak
• Low value or trough.
Introduction to Interactive Media
Sound File Formats
• Common sampled sound file formats:
– WAV
– AIFF
– AU
• MP3 is most popular compressed
format (lossy)
– 10:1 file size ratio – WAV:MP3
Introduction to Interactive Media
Manipulating Sound
• Common applications that allow for
recording and generate effects.
– Apple’s GarageBand
– Adobe’s Soundbooth
– Sony’s Sound Forge
Introduction to Interactive Media
Common Audio Edits
• Hiss Removal
– sample the hiss and remove hiss based on
what you sampled.
• Normalization
– brings the average or peak amplitude to a
target level
• Time stretching
• Pitch alteration
Introduction to Interactive Media
Use of Audio in Interactive
Media
•
•
•
•
•
Ambience
Sound Effects
Auditory Feedback to aid Usability
Music
Speech
Introduction to Interactive Media
Sound on the Web
• Not always needed (can be annoying)
– Became popular on the Web when it
became possible (late 1990s – early
2000s)
– Fell out fashion on the Web
– Web has become more cinematic
Introduction to Interactive Media
How to Integrate Sound
• Authoring
applications
allow for
precise
syncing of
audio within
interactive
experience.
Introduction to Interactive Media
How to Integrate Sound
• HTML5
– Prior to HTML 5, there was not a standard
for playing audio files on a web.
– HTML 5 defines a new element which
specifies a standard way to embed an
audio file on a web page: the <audio>
element.
•
For more information re: format and browser support:
http://www.w3schools.com/html/html5_audio.asp
Introduction to Interactive Media
Beyond Sampled Sound
• Sampled sound
– All audio discussed so far: WAV, MP3,
AIFF, AU, etc…
– Sound is stored as a description of the
sound.
Introduction to Interactive Media
Synthesized Sound
• Sound is stored as a series of commands for the
computer to reproduce the sounds.
• Analogous to vector-based graphics.
• MIDI (Musical Instrument Digital Interface).
– Codes provided for:
•
•
•
•
•
Specific instruments
Notes
Force and duration of note
Routing commands to different instrument channels
Specialized control functions.
Introduction to Interactive Media
MIDI
• Simplest system contains:
– Digital musical instrument to create messages
– Sound synthesizer to interpret the messages
– Amplifier/speaker output system.
Introduction to Interactive Media
MIDI Videos
• Composing
– http://www.youtube.com/watch?v=pjPf24XQo1s
• Building Your MIDI Studio
– http://www.youtube.com/watch?v=JJLgwaIFePk&feature=fvw
Introduction to Interactive Media