Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Binaural Recording, November 2012 Crystal Wong Binaural Recording: Method to Creating Auditory Space Biography: Crystal Wong is currently a fourth year undergraduate student pursuing a Bachelor of Architecture degree at the University of Southern California (USC), School of Architecture as well as a Musical Studies in Piano Performance minor at the USC Thornton School of Music. She hopes to pursue a career that ties together her passion for both architecture and music to positively impact communities through using her unique sense for both visual and audio spaces. Abstract: Over the past few decades, the society has been growing technologically, especially in the field of virtual reality. There has been a large push in the development of capturing sounds so that it imitates the way human ears perceive sounds. Binaural recording utilizes two separate microphones that have specific locations. The distance between the two microphones is the average space between human ears. By recording the sound at two points in space, the microphones will be able to capture and thus reproduce more realistic sounds. Keywords: binaural recording, binaural reproduction, auditory space Multimedia: Clip of a binaural recording There has been a recent push for more realistic experiences created by technology over the past few years. In fact, enhancing virtual reality is one of the fourteen Engineering Grand Challenges which were determined by the National Academy of Engineering because of the seemingly limitless possibilities it suggests [B1]. Because of the increase of computers and smart phones per household, there has been a growth in the demand for better visuals, faster processors, and higher quality acoustics. Although acoustics were not the first of the three to improve, recent developments in audio have significantly enhanced the virtual experience. The first developments of binaural recordings dates back to 1881, but because of the lack of interest in this area during that time, it did not progress as quickly as it has in the past few years. Binaural recording requires two microphones that are spaced at a distance, just as a human’s ears are at a distance, so it can capture sounds at two different points in space. This allows for the playback to sound exactly as a human would perceive because it take into account the direction and distance of the origin of the sound. As society continues to shift towards enhancing virtual experiences in ¹The numbers in brackets correspond to those of the bibliography in Annex B. 1 Binaural Recording, November 2012 the computer gaming industry, binaural recording is growing in popularity because of its ability to capture sounds and replay them so that it imitates a real environment. The uniqueness of binaural recording lies within the fact that engineers use two speakers that are separated at a distance of about 18 centimeters, which is the average distance between two ears for the human head. This allows for the different sound pressures located at two specific points in space to be captured accurately, as shown in Figure 1. Figure 1. The two separate microphones are place to the ears of each manikin. Source: http://www.tutorial5.com/images/stories/binaural-recording.jpg. According to Stephan Paul, a number of nineteenth and twentieth century researchers, such as “Steinhauser, Thompson and J.W. Strutt (Lord Rayleigh)…largely agreed that the presence of two sound receivers, the ears, is responsible for a large portion of the sophistication of human sound perception, enabling localization and distance perception of sound sources” [B2]. Clement Ader, a French inventor who is known for being the first person to discover binaural audio, connected multiple phone transmitters from the Paris Opera House stage to the audience located about two kilometers away [B3]. Through this technique, he discovered that by listening through 2 Binaural Recording, November 2012 two receivers, it allowed the audience to have an enhanced auditory experience. Although this technology, called the Théâtrophone, was commercialized, the company was out of business by 1932 because of the cost of which prohibited the general public to enjoy and appreciate the technology [B3]. The beginning of binaural recording dates back to centuries ago, but it has only been through recent technological advances that this technique has become more widespread and accepted. Since the time Ader and researchers have solidified the theory that having two sound receivers was the key to capturing and reproducing these nearly accurate sounds, there have been numerous developments, from Harvey Fletcher’s binaural hearing aid to Doolittle’s binaural broadcasting to the development of manikins and artificial heads that have contributed to replicating the exact audio environment [B2]. By 1927, W. Barlett Jones filed a patent that showed the basic design of an artificial head [B2]. He began to investigate more possibilities such as placing the microphones where the pinnae, or the visible part of the human ear, were located. Originally, the microphones were placed “in the cheeks rather than in the ears [which] was justified by the size (1.4 inch) of the special type 4 moving coil microphones” [B2]. In 1933, AT&T made a debut of “Oscar,” a robot that had microphones at the location of the ears, at the Chicago World Fair [B3]. Since then, there have been continual improvements in the dimensions as well as the materials of the manikins which have allowed for the capturing and reproduction of realistic sounds. The developments in the materials and manikins allow for an exact duplicate of the sounds because sounds are influenced by the masses which can block, absorb, or reflect sounds. Throughout history, there have been developments in binaural audio, 3 Binaural Recording, November 2012 but only recently has this technology for sound capturing and reproduction become a more popularized and economical product to the general public to enjoy. Although the mechanical principles behind binaural recording are complex, the overarching purpose is to use two separate microphones to capture the exact sounds that human ears hear. SENSE Lab notes that in order to “localize a sound, the auditory system uses multiple cues, including binaural differences in timing and level that arise from the separation of the ears by the solid mass of the head” [B4]. Because the “solid mass of the head” is a major factor in how a person perceives sound, the artificial head must include the correct massing as well as the form of the head and pinna. This leads to the investigation of sound transmission in the ear canal and HRTFs, or head-related transfer functions. Through multiple experiments, Hammershoi and Moller state that “the transverse modes are not transmitted to the eardrum [which means] the centerline measurements (or recordings) give a true representation of the input to the hearing” [B5]. Thus, the microphone can be placed at the ear canal because the error from recording it at the eardrum versus the entrance of the ear canal is “not related to the spatial properties” [B5]. These minor adjustments can be modified through the computer, but it is sufficient for the speakers to be placed at the ear canal for the microphone to receive the necessary information. In addition, it is important to note that the microphone, which covers the whole ear canal, will still pick some transverse modes at the entrance of the ear canal. Through these studies of HRTFS, researchers have stated that artificial dummies are able to record sounds at the ear canal rather than the eardrum. 4 Binaural Recording, November 2012 In addition to the mechanics of the location of the microphones, one must also understand how the sounds can be reproduced to create realistic auditory spaces. Because the microphones record the exact pressure and direction of the sounds at the ear canal, it is necessary for those who listen to the playback to listen with headphones. Headphones allow for the playback to directly enter the listener’s ear canal. This control is essential to recreating the exact recording because the distance between the two ears as well as the head shadow, or mass of the head, affects the sound pressure, time, and direction. The delay between each recording allows for another aural dimension, which results in the user experiencing auditory 3D space rather than a 2D space. David C. Apps gives an example of how “one experiences a feeling of expanded coordinates and a truly satisfying feeling of presence and realism” by describing that in “a baritone solo part, the soloist is easily localized at one’s right and, by interchanging signals to the left and right phones, the soloist ‘moves’ to one’s left” [B6]. Although the most effective reproduction of the recording is through the use of headphones, technology is rapidly advancing. In fact, there are already some studies being done to maintain the quality of the binaural recording while using speakers. Binaural recording is strongly modeled by how human ears hear and process the information that enters through the eardrums. In 1952, Hirsch “showed that cutting down the listener’s ability to localize, either by preventing head movement or by preventing binaural listening, reduced his ability to separate the sources of sounds in space and reduced his ability to understand speech” [B7]. This explains the importance of binaural hearing because it assists in locating the origin of the sound. Without the ability to hear through both ears, one would not be able to localize the sound source. This is the result of ITD, or inter-aural time difference. The difference in “the 5 Binaural Recording, November 2012 arrival times of the sound wave emitted from a single source are usually not exactly the same at the left and right eardrums – due to the different path-lengths to both ears” [B8]. The difference in time is affected by the location as well as the head shadow effects. The head shadow effect is a time delay in when each ear drum receives the sound because of the mass of the head which can absorb, reflect, or block sounds. It is important to be aware that every part of the ear, whether it is the shape of the pinna or material of the head, affects the final sound that the ear canal receives. Specific tasks such as “detection tasks, require a more detailed simulation of the haircell functioning, namely modeling of the stochastic firing rate of the nerve fibers and the adaptive, time variant hair-cell response” [B8]. Through analyzing the details down to the haircell, researchers can uncover the method in which the human ear receives sound information. Only through understanding the purpose and function of the details can they succeed in designing a successful binaural recording microphone and playback. Because adding another dimension to the audio recording can enhance the auditory experience of the listener, many industries are now very interested in this technology. The recent shift of demands in better technology has grown which has allowed for the development of technology such as binaural recording. Now that computers and smart phones are easily available to the public, the users constantly want the next best thing that would allow for faster processors, better resolution, and higher quality of audio so that their virtual experience could be as realistic as possible. Jonas Braasch, Anthony Parks, Torben Pastore, and Samuel W. Clapp, all from the School of Architecture at Rensselaer Polytechnic, explored how “a binaural manikin with a motorized head, a technique to simulate head movements from impulse responses recorded with a higher-order spherical microphone, and a binaural model that can process head movements” 6 Binaural Recording, November 2012 could assist in enhancing realistic virtual experiences [B8]. There has been a strong push in the gaming industry from consumers which has allowed for great developments in binaural recording and reproduction. Although the technology of binaural recording was discovered centuries ago, it has only recently become a focus in recent years because of the interest from many industries [B9]. It has progressed drastically because it can effectively enhance the experience of virtual reality. This proves “that in the next future this technology will be high-quality and cheap enough to become common good” [B10]. It is essential that this technology will be available to the general public. By allowing users to virtually experience acoustics that sound realistic, binaural recording and reproduction has and continues to impact the society as a whole through various industries. Although it is currently most accessible for the public to hear binaural recordings through headphones as shown in Figure 2, there have been developments in producing the same quality of sound via loudspeakers. Figure 2. It is most effective to listen to binaural recording through headphones so that each sound can be controlled and localized through the use of the right and left headphones. Source: http://www.kallbinauralaudio.com/wp-content/uploads/2012/03/BinauralBeats.jpg 7 Binaural Recording, November 2012 For example, through recording impulse responses, these responses or images “can be applied to any compact array and used in conjunction with variety of current spatial loudspeaker reproduction systems to create convolution reverb-type spatial sound reproduction [B11]. Not only will this enhance virtual reality in the computer gaming industry, but this could also improve the lives of the hearing-impaired. For example, binaural technology is used in a way “that the retrieved clock signal and the second sampling clock signal in the second hearing prosthesis are synchronous to the coding clock signal generated by the first clock generator in the first hearing prosthesis” [B12]. By using a clock signal that is programmed to delay sounds, the hearing aids will allow the users to be able to perceive sounds accurately. Because of the delay, this will also allow the user to locate the source of the sound. This technological discovery is a breakthrough that proves that the advancement of binaural recording and reproduction is highly beneficial and can improve the livelihoods of humans, especially for those who are disabled. As the binaural system continues to rapidly improve, a growing number of industries are seeing the benefits of localization. The growth in consumer demands of binaural recording and reproduction has jumped significantly and this will ultimately improve the society. 8 Binaural Recording, November 2012 Works Cited [B1] “Enhance virtual reality.” (2012). National Academy of Engineering of the National Academies. [Online]. Available: http://www.engineeringchallenges.org. [B2] S. Paul, “Binaural Recording Technology: A Historical Review and Possible Future Developments,” ACTA Acustica United with Acustica, vol. 95, pp. 767 – 788, 2009. [B3] A Brief History of Binaural Audio. (2011). Kall Binaural Audio. [Online]. Available: http://www.kallbinauralaudio.com. [B4] G. Frisk, “Architectural Acoustics: Auralization, Modeling, and Computational Methods,” The Journal of the Acoustical Society of America, vol. 128, no. 4, pp. 2410-2463, 2010. [B5] D. Hammershoi, and H. Moller, “Methods for Binaural Recording and Reproduction,” The Journal of the Acoustical Society of America, vol. 88, pp. 303-311, 2002. [B6] D. Apps, “The Use of Binaural Tape Recording in Automotive Noise Problems,” The Journal of the Acoustical Society of America, vol. 24, no. 6, pp. 660-661, Nov. 1952. [B7] E. Whetnall, “Binaural Hearing,” The Journal of Laryngology and Otology, vol. 78, no 12, pp. 1079-1089, Dec. 2007. [B8] J. Braasch, “Modelling of Binaural Hearing,” CIRMMT, pp. 75-108, 2002. [B9] M. Rovaglio, and T. Scheele, “Virtual reality improves training in process industries,” INTECH, pp. 32-36, 2011. [B10] D. Lacrama, and D. Fera, “Virtual Reality,” Anale, Seria Informatica, vol. 5, no. 1, 2007. [B11] N. Xiang, and D. Griesinger, “Architectural Acoustics, Psychological and Physiological Acoustics, Engineering Acoustics, and Noise: Binaural and Spatial Evaluation of Acoustics in Performance Venues,” The Journal of the Acoustical Society of America, vol. 132, no. 3, pp. 1911-1976, Sept. 2012. 9 Binaural Recording, November 2012 [B12] P. Nielsen, and J. Melanson, “Synchronized binaural hearing system,” USPTO, 4 Jan. 2005. 10