Download MPEG-4 - Computer Graphics at Stanford University

MPEG-4 Applications
•
•
•
•
•
Interactive TV (broadcast)
Streaming media on the web (internet)
Multimedia titles (CD-ROM)
Network games
Mobile multimedia (picture phone)
Technical Capabilities
• Define media objects
– Audio and visual
– Natural and synthetic
• Define composition architecture
– Scene graph extended/borrowed from VRML
• System level
– Streams: Multiplexing, synchronization, buffering
– Programmability?
• Simple interaction
Structured Video
• Natural representation: Exposure sheets
– Layers and elements, alpha compositing
– Chroma keying and blue-screen mattes
• Compression
– Standard video compression
• Trouble with motion and object boundaries
• Trouble with graphics: text, graphics
– Object coding/Computer graphics models
• Compact format: e.g. cels
• Resolution independent
• But beware! Be clever!
audiovisual objects
voice
hierarchically multiplexed
downstream control / data
sprite
hierarchically multiplexed
upstream control / data
2D background
audiovisual
presentation
y
3D objects
scene
coordinate
system
x
z
user events
video
compositor
projection
plane
audio
compositor
hypothetical viewer
speaker
display
user input
Display and
User
Interaction
Audiovisual Interactive
Scene
Composition and Rendering
...
Scene
Description
Information
Object
Descriptor
Return
Channel
Coding
Primitive
AV Objects
Elementary Streams
AL
...
AL
AL
Elementary Stream Interface
...
AL
AL
AL-Packetized Streams
FlexMux FlexMux
FlexMux
(RTP)
UDP
IP
AAL2
ATM
FlexMux
H223
PSTN
DAB
Mux
TransMux Streams
Transmission/Storage Medium
AccessUnit
Layer
Stream Multiplex Interface
FlexMux Streams
(PES)
MPEG-2
TS
Compression
Layer
FlexMux
Layer
TransMux Interface
...
...
TransMux
Layer
Video Stream
• VS > VO > VOL > GOV > VOP
• Video object
– Shape + Motion + Texture
• Sprites
– Texture + Projective transformation
Shape Coding
• Bounding box + shape-adaptive macroblocks
• Alpha / Shape
– Binary
• Transparent or opaque
• Context-based arithmetic encoder
• Motion predicted
– Grey (coded as textures)
• Texture
– Shape-based DCT
– Padding algorithm
Scalable Video
• Layered video
– Spatial scalability: base + enhancement
– Temporal scalability: IPI + B = IBPBI
• Scalable content
– Quality improves with decoder technology
– Inherent resolution independence (text,midi)
2D Meshes
• Type
– Uniform rectangular
– Triangular (Delauney)
• Geometry
– Intra: Predict position compression
– Inter: Compressed motion vector
• Texture
– Piecewise affine texture map
– Wavelet compression ala mipmaps
Face
• Facial Definition Parameter or user supplied
model/texture
• Facial Animation Parameter plus
Amplification and Filters
References