Download Physically Based Sound

Physically Based Sound
UNC-CH 259 Physically Based
Class lecture, April 1st 2002
Vincent Scheib
Overview
 Summary & Introduction to Audio
 Case study of 3 recent papers
Audio Synthesis Overview:
 Adapted from the Siggraph 2000 Course Notes
by Perry R. Cook found here:
http://www.cs.princeton.edu/~prc/CookSig00.pdf
Audio Synthesis Overview:
Views of Sound
 Time Domain
x(t)
From physics
Related to production
 Frequency Domain
From math
Related to perception
x(f)
Audio Synthesis Overview:
Time Domain
 Forces cause
acceleration
> velocity
> > position changes over time
 Position changes cause sound waves over time
Audio Synthesis Overview:
Frequency Domain
 Physical systems have vibration modes
(damped oscillations)
Think of the modes on a string:
The frequency determines
the mode
 Specific Frequencies appear over time
 Solutions are sums of damped sines
Audio Synthesis Overview:
Spectra
 Plots over
frequency
and time of
Magnitude
Phase
Overview of not-so-physicallybased work

Magnitude
– Generate varying magnitudes over time

Stochastic
– Random methods, using statistics

Residual
– The difference between simulated and real instrument

Transients
– Swells in sound across all frequencies

Subtractive Synthesis (formants)
– Shape the global frequency envelope
Physically Based Methods
 Modal Synthesis
 Specific Models
String, Tube, Bar, Plate
Human head
Whistle, Maraca
What I’m not talking about
 Enviromental analysis/simulation
 How does sound propagate from a source to
your perception:
Direct transmission through media (air)
Reflected transmission (similar to global illumination)
Affected by shape of ears and your thick head
You have 2 ears, two sources
Recent Papers
 Synthesizing Sounds from Physically Based
Motion
James O’Brien, Cook, Essl – siggraph 2001
 FoleyAutomatic: Physically-based Sound Effects
for Interactive Simulation and Animation
Kees van den Doel, Kry, Pai – siggraph 2001
 Real-Time Modeling of Sound and Deformation
James O’Brien – GDC 2002
Synthesizing Sounds from P. B. M.
Basic Idea
 Deformable simulation with really tiny time
steps
 Compute sound from change in pressure of air
along object’s surface
Synthesizing Sounds from P. B. M.
Requirements of System
 Temporal Resolution
must capture 20,000Hz
 Deformation Modeling
Rigid body, and intertia-less, solutions not sufficient
 Surface Representation
Requires explicit surface, to solve for air vibration
 Physical Accuracy
Audio more sensitive than just animation
Synthesizing Sounds from P. B. M.
Getting Pressure
 For each triangle, get a normal and velocity
 Pressure from velocity dot normal
pressure = velocity . Normal
 Also compute area of that triangle
 Filter out in-audible frequencies
They cause unwanted ailiasing and DC components.
Synthesizing Sounds from P. B. M.
Hearing the Sound
 Simple version in this paper:
Record only direct “line of sight” sound
Diminish by “visible” area
Divide by distance from viewer
– Should be distance squared, but microphones and ears
respond to √(sound wave energy)
 Account for delay by computing distance to
camera and using speed of sound.
Synthesizing Sounds from P. B. M.
Results
 Movie
http://www.cs.berkeley.edu/~job/Projects/SoundGen/video.html
Several objects being bonked to make noise.
Takes a LONG TIME to compute.
– Hours!
– Days!
FoleyAutomatic
Basic Idea
 Real-time
 Uses modal models
 Special cases for:
Impact
Rolling
Sliding
FoleyAutomatic
Modal Resonance Models
 Modal model consists of three things
F = N Modal frequencies
D = N Decay rates
A = N by K gains,
– N = number of modal frequencies modeled
Decay
– K = number of
discrete locations on an object
Frequency
Gain
 Outputk(T) =
n=1..N( Ank e-Dn*Tsin(2pi FnT) )
FoleyAutomatic
Requirements of System:
 Multi-body Dynamics
Rolling, sliding contacts
 Smooth surfaces
Smooth continuous contact
 This paper used Loop subdivision surfaces
FoleyAutomatic
Impact!
 Two most distinguishing characteristics:
Energy transfer
– Force magnitude
Hardness
– Force duration (shorter == harder)
FoleyAutomatic
Scraping & Sliding
 Play back a recording of scraping noise at
variable rate based on velocity
recording from pre-simulation or acquired data
 Audio volume = (velocity * normal_force)
FoleyAutomatic
Sound Profile
 Fractal noise
amplitude of harmonics fit to real world data.
FoleyAutomatic
Rolling
 Very uncertain area
 Rolling has “softer contacts”, thus only use the
low frequecies of a sound profile?
Works okay, not great
 Rolling contact forces seem to be tied to the
modes of the objects – audio feedback into
forces – no longer linear – AAHHH!!
FoleyAutomatic
Results
 Movie
http://www.cs.ubc.ca/~pai/movies/foleyautomatic.mpg
Real-Time Modeling of Sound
and Deformation
 Slides & movies available at
http://www.cs.berkeley.edu/~job/Talks/