Download Binaural Recording, November 2012 Binaural Recording

Binaural Recording, November 2012
Crystal Wong
Binaural Recording: Method to Creating Auditory Space
Biography: Crystal Wong is currently a fourth year undergraduate student pursuing a Bachelor of
Architecture degree at the University of Southern California (USC), School of
Architecture as well as a Musical Studies in Piano Performance minor at the USC
Thornton School of Music. She hopes to pursue a career that ties together her passion for
both architecture and music to positively impact communities through using her unique
sense for both visual and audio spaces.
Abstract: Over the past few decades, the society has been growing technologically, especially in
the field of virtual reality. There has been a large push in the development of capturing
sounds so that it imitates the way human ears perceive sounds. Binaural recording utilizes
two separate microphones that have specific locations. The distance between the two
microphones is the average space between human ears. By recording the sound at two
points in space, the microphones will be able to capture and thus reproduce more realistic
sounds.
Keywords: binaural recording, binaural reproduction, auditory space
Multimedia: Clip of a binaural recording
There has been a recent push for more realistic experiences created by technology over the past
few years. In fact, enhancing virtual reality is one of the fourteen Engineering Grand Challenges
which were determined by the National Academy of Engineering because of the seemingly
limitless possibilities it suggests [B1]. Because of the increase of computers and smart phones
per household, there has been a growth in the demand for better visuals, faster processors, and
higher quality acoustics. Although acoustics were not the first of the three to improve, recent
developments in audio have significantly enhanced the virtual experience. The first
developments of binaural recordings dates back to 1881, but because of the lack of interest in
this area during that time, it did not progress as quickly as it has in the past few years. Binaural
recording requires two microphones that are spaced at a distance, just as a human’s ears are at a
distance, so it can capture sounds at two different points in space. This allows for the playback to
sound exactly as a human would perceive because it take into account the direction and distance
of the origin of the sound. As society continues to shift towards enhancing virtual experiences in
¹The numbers in brackets correspond to those of the bibliography in Annex B.
1
Binaural Recording, November 2012
the computer gaming industry, binaural recording is growing in popularity because of its ability
to capture sounds and replay them so that it imitates a real environment.
The uniqueness of binaural recording lies within the fact that engineers use two speakers that are
separated at a distance of about 18 centimeters, which is the average distance between two ears
for the human head. This allows for the different sound pressures located at two specific points
in space to be captured accurately, as shown in Figure 1.
Figure 1. The two separate microphones are place to the ears of each manikin. Source:
http://www.tutorial5.com/images/stories/binaural-recording.jpg.
According to Stephan Paul, a number of nineteenth and twentieth century researchers, such as
“Steinhauser, Thompson and J.W. Strutt (Lord Rayleigh)…largely agreed that the presence of
two sound receivers, the ears, is responsible for a large portion of the sophistication of human
sound perception, enabling localization and distance perception of sound sources” [B2]. Clement
Ader, a French inventor who is known for being the first person to discover binaural audio,
connected multiple phone transmitters from the Paris Opera House stage to the audience located
about two kilometers away [B3]. Through this technique, he discovered that by listening through
2
Binaural Recording, November 2012
two receivers, it allowed the audience to have an enhanced auditory experience. Although this
technology, called the Théâtrophone, was commercialized, the company was out of business by
1932 because of the cost of which prohibited the general public to enjoy and appreciate the
technology [B3]. The beginning of binaural recording dates back to centuries ago, but it has only
been through recent technological advances that this technique has become more widespread and
accepted.
Since the time Ader and researchers have solidified the theory that having two sound receivers
was the key to capturing and reproducing these nearly accurate sounds, there have been
numerous developments, from Harvey Fletcher’s binaural hearing aid to Doolittle’s binaural
broadcasting to the development of manikins and artificial heads that have contributed to
replicating the exact audio environment [B2]. By 1927, W. Barlett Jones filed a patent that
showed the basic design of an artificial head [B2]. He began to investigate more possibilities
such as placing the microphones where the pinnae, or the visible part of the human ear, were
located. Originally, the microphones were placed “in the cheeks rather than in the ears [which]
was justified by the size (1.4 inch) of the special type 4 moving coil microphones” [B2]. In
1933, AT&T made a debut of “Oscar,” a robot that had microphones at the location of the ears,
at the Chicago World Fair [B3]. Since then, there have been continual improvements in the
dimensions as well as the materials of the manikins which have allowed for the capturing and
reproduction of realistic sounds. The developments in the materials and manikins allow for an
exact duplicate of the sounds because sounds are influenced by the masses which can block,
absorb, or reflect sounds. Throughout history, there have been developments in binaural audio,
3
Binaural Recording, November 2012
but only recently has this technology for sound capturing and reproduction become a more
popularized and economical product to the general public to enjoy.
Although the mechanical principles behind binaural recording are complex, the overarching
purpose is to use two separate microphones to capture the exact sounds that human ears hear.
SENSE Lab notes that in order to “localize a sound, the auditory system uses multiple cues,
including binaural differences in timing and level that arise from the separation of the ears by the
solid mass of the head” [B4]. Because the “solid mass of the head” is a major factor in how a
person perceives sound, the artificial head must include the correct massing as well as the form
of the head and pinna. This leads to the investigation of sound transmission in the ear canal and
HRTFs, or head-related transfer functions. Through multiple experiments, Hammershoi and
Moller state that “the transverse modes are not transmitted to the eardrum [which means] the
centerline measurements (or recordings) give a true representation of the input to the hearing”
[B5]. Thus, the microphone can be placed at the ear canal because the error from recording it at
the eardrum versus the entrance of the ear canal is “not related to the spatial properties” [B5].
These minor adjustments can be modified through the computer, but it is sufficient for the
speakers to be placed at the ear canal for the microphone to receive the necessary information. In
addition, it is important to note that the microphone, which covers the whole ear canal, will still
pick some transverse modes at the entrance of the ear canal. Through these studies of HRTFS,
researchers have stated that artificial dummies are able to record sounds at the ear canal rather
than the eardrum.
4
Binaural Recording, November 2012
In addition to the mechanics of the location of the microphones, one must also understand how
the sounds can be reproduced to create realistic auditory spaces. Because the microphones record
the exact pressure and direction of the sounds at the ear canal, it is necessary for those who listen
to the playback to listen with headphones. Headphones allow for the playback to directly enter
the listener’s ear canal. This control is essential to recreating the exact recording because the
distance between the two ears as well as the head shadow, or mass of the head, affects the sound
pressure, time, and direction. The delay between each recording allows for another aural
dimension, which results in the user experiencing auditory 3D space rather than a 2D space.
David C. Apps gives an example of how “one experiences a feeling of expanded coordinates and
a truly satisfying feeling of presence and realism” by describing that in “a baritone solo part, the
soloist is easily localized at one’s right and, by interchanging signals to the left and right phones,
the soloist ‘moves’ to one’s left” [B6]. Although the most effective reproduction of the
recording is through the use of headphones, technology is rapidly advancing. In fact, there are
already some studies being done to maintain the quality of the binaural recording while using
speakers.
Binaural recording is strongly modeled by how human ears hear and process the information that
enters through the eardrums. In 1952, Hirsch “showed that cutting down the listener’s ability to
localize, either by preventing head movement or by preventing binaural listening, reduced his
ability to separate the sources of sounds in space and reduced his ability to understand speech”
[B7]. This explains the importance of binaural hearing because it assists in locating the origin of
the sound. Without the ability to hear through both ears, one would not be able to localize the
sound source. This is the result of ITD, or inter-aural time difference. The difference in “the
5
Binaural Recording, November 2012
arrival times of the sound wave emitted from a single source are usually not exactly the same at
the left and right eardrums – due to the different path-lengths to both ears” [B8]. The difference
in time is affected by the location as well as the head shadow effects. The head shadow effect is a
time delay in when each ear drum receives the sound because of the mass of the head which can
absorb, reflect, or block sounds. It is important to be aware that every part of the ear, whether it
is the shape of the pinna or material of the head, affects the final sound that the ear canal
receives. Specific tasks such as “detection tasks, require a more detailed simulation of the haircell functioning, namely modeling of the stochastic firing rate of the nerve fibers and the
adaptive, time variant hair-cell response” [B8]. Through analyzing the details down to the haircell, researchers can uncover the method in which the human ear receives sound information.
Only through understanding the purpose and function of the details can they succeed in
designing a successful binaural recording microphone and playback.
Because adding another dimension to the audio recording can enhance the auditory experience of
the listener, many industries are now very interested in this technology. The recent shift of
demands in better technology has grown which has allowed for the development of technology
such as binaural recording. Now that computers and smart phones are easily available to the
public, the users constantly want the next best thing that would allow for faster processors, better
resolution, and higher quality of audio so that their virtual experience could be as realistic as
possible. Jonas Braasch, Anthony Parks, Torben Pastore, and Samuel W. Clapp, all from the
School of Architecture at Rensselaer Polytechnic, explored how “a binaural manikin with a
motorized head, a technique to simulate head movements from impulse responses recorded with
a higher-order spherical microphone, and a binaural model that can process head movements”
6
Binaural Recording, November 2012
could assist in enhancing realistic virtual experiences [B8]. There has been a strong push in the
gaming industry from consumers which has allowed for great developments in binaural
recording and reproduction.
Although the technology of binaural recording was discovered centuries ago, it has only recently
become a focus in recent years because of the interest from many industries [B9]. It has
progressed drastically because it can effectively enhance the experience of virtual reality. This
proves “that in the next future this technology will be high-quality and cheap enough to become
common good” [B10]. It is essential that this technology will be available to the general public.
By allowing users to virtually experience acoustics that sound realistic, binaural recording and
reproduction has and continues to impact the society as a whole through various industries.
Although it is currently most accessible for the public to hear binaural recordings through
headphones as shown in Figure 2, there have been developments in producing the same quality
of sound via loudspeakers.
Figure 2. It is most effective to listen to binaural recording through headphones so that each
sound can be controlled and localized through the use of the right and left headphones. Source:
http://www.kallbinauralaudio.com/wp-content/uploads/2012/03/BinauralBeats.jpg
7
Binaural Recording, November 2012
For example, through recording impulse responses, these responses or images “can be applied to
any compact array and used in conjunction with variety of current spatial loudspeaker
reproduction systems to create convolution reverb-type spatial sound reproduction [B11]. Not
only will this enhance virtual reality in the computer gaming industry, but this could also
improve the lives of the hearing-impaired. For example, binaural technology is used in a way
“that the retrieved clock signal and the second sampling clock signal in the second hearing
prosthesis are synchronous to the coding clock signal generated by the first clock generator in the
first hearing prosthesis” [B12]. By using a clock signal that is programmed to delay sounds, the
hearing aids will allow the users to be able to perceive sounds accurately. Because of the delay,
this will also allow the user to locate the source of the sound. This technological discovery is a
breakthrough that proves that the advancement of binaural recording and reproduction is highly
beneficial and can improve the livelihoods of humans, especially for those who are disabled. As
the binaural system continues to rapidly improve, a growing number of industries are seeing the
benefits of localization. The growth in consumer demands of binaural recording and
reproduction has jumped significantly and this will ultimately improve the society.
8
Binaural Recording, November 2012
Works Cited
[B1] “Enhance virtual reality.” (2012). National Academy of Engineering of the National
Academies. [Online]. Available: http://www.engineeringchallenges.org.
[B2] S. Paul, “Binaural Recording Technology: A Historical Review and Possible Future
Developments,” ACTA Acustica United with Acustica, vol. 95, pp. 767 – 788, 2009.
[B3] A Brief History of Binaural Audio. (2011). Kall Binaural Audio. [Online]. Available:
http://www.kallbinauralaudio.com.
[B4] G. Frisk, “Architectural Acoustics: Auralization, Modeling, and Computational Methods,”
The Journal of the Acoustical Society of America, vol. 128, no. 4, pp. 2410-2463, 2010.
[B5] D. Hammershoi, and H. Moller, “Methods for Binaural Recording and Reproduction,” The
Journal of the Acoustical Society of America, vol. 88, pp. 303-311, 2002.
[B6] D. Apps, “The Use of Binaural Tape Recording in Automotive Noise Problems,” The
Journal of the Acoustical Society of America, vol. 24, no. 6, pp. 660-661, Nov. 1952.
[B7] E. Whetnall, “Binaural Hearing,” The Journal of Laryngology and Otology, vol. 78, no 12,
pp. 1079-1089, Dec. 2007.
[B8] J. Braasch, “Modelling of Binaural Hearing,” CIRMMT, pp. 75-108, 2002.
[B9] M. Rovaglio, and T. Scheele, “Virtual reality improves training in process industries,”
INTECH, pp. 32-36, 2011.
[B10] D. Lacrama, and D. Fera, “Virtual Reality,” Anale, Seria Informatica, vol. 5, no. 1, 2007.
[B11] N. Xiang, and D. Griesinger, “Architectural Acoustics, Psychological and Physiological
Acoustics, Engineering Acoustics, and Noise: Binaural and Spatial Evaluation of
Acoustics in Performance Venues,” The Journal of the Acoustical Society of America,
vol. 132, no. 3, pp. 1911-1976, Sept. 2012.
9
Binaural Recording, November 2012
[B12] P. Nielsen, and J. Melanson, “Synchronized binaural hearing system,” USPTO, 4 Jan.
2005.
10