Download Chapter 143 - Sensory Augmentation and Substitution

Kaczmarek, K. A. “Sensory Augmentation and Substitution.”
The Biomedical Engineering Handbook: Second Edition.
Ed. Joseph D. Bronzino
Boca Raton: CRC Press LLC, 2000
143
Sensory Augmentation
and Substitution
143.1
Visual System
Visual Augmentation • Tactual Vision
Substitution • Auditory Vision Substitution
143.2
Auditory System
Auditory Augmentation • Visual Auditory
Substitution • Tactual Auditory Substitution
Kurt A. Kaczmarek
143.3
University of Wisconsin at Madison
Tactual System
Tactual Augmentation • Tactual Substitution
This chapter will consider methods and devices used to present visual, auditory, and tactual (touch)
information to persons with sensory deficits. Sensory augmentation systems such as eyeglasses and hearing
aids enhance the existing capabilities of a functional human sensory system. Sensory substitution is the
use of one human sense to receive information normally received by another sense. Braille and speech
synthesizers are examples of systems that substitute touch and hearing, respectively, for information that
is normally visual (printed or displayed text).
The following three sections will provide theory and examples for aiding the visual, auditory, and
tactual systems. Because capitalizing on an existing sensory capability is usually superior to substitution,
each section will consider first augmentation and then substitution, as shown below:
Human Sensory Systems
Visual
Visual augmentation
Tactual vision substitution
Auditory vision substitution
Auditory
Tactual
Auditory augmentation
Visual auditory substitution
Tactual auditory substitution
Tactual augmentation
Tactual substitution
143.1 Visual System
With a large number of receptive channels, the human visual system processes information in a parallel
fashion. A single glimpse acquires a wealth of information; the field of view for two eyes is 180 degrees
horizontally and 120 degrees vertically [Mehr & Shindell, 1990]. The spatial resolution in the central
(foveal) part of the visual field is approximately 0.5 to 1.0 minute of arc [Shlaer, 1937], although Vernier
acuity, the specialized task of detecting a misalignment of two lines placed end to end, is much finer,
approximately 2 seconds of arc [Stigmar, 1970]. Low-contrast presentations substantially reduce visual
acuity.
© 2000 by CRC Press LLC
The former resolution figure is the basis for the standard method of testing visual acuity, the Snellen
chart. Letters are considered to be “readable” if they subtend approximately 5 minutes of arc and have
details one-fifth this size. Snellen’s 1862 method of reporting visual performance is still used today. The
ratio 20/40, for instance, indicates that a test was conducted at 20 ft and that the letters that were
recognizable at that distance would subtend 5 minutes of arc of 40 ft (the distance at which a normally
sighted, or “20/20,” subject could read them). Although the standard testing distance is 20 ft, 10 ft and
even 5 ft may be used, under certain conditions, for more severe visual impairments [Fonda, 1981].
Of the approximately 6 to 11.4 million people in the United States who have visual impairments, 90%
have some useful vision [NIDRR, 1993]. In the United States, severe visual impairment is defined to be
20/70 vision in the better eye with best refractive correction (see below). Legal blindness means that the
best corrected acuity is 20/200 or that the field of view is very narrow (<20 degrees). People over 65 years
of age account for 46% of the legally blind and 68% of the severely visually impaired. For those with
some useful vision, a number of useful techniques and devices for visual augmentation can allow
performance of many everyday activities.
Visual Augmentation
People with certain eye disorders see better with higher- or lower-than-normal light levels; an illuminance
from 100 to 4000 lux may promote comfortable reading [Fonda, 1981]. Ideal illumination is diffuse and
directed from the side at a 45-degree angle to prevent glare. The surrounding room is preferably 20% to
50% darker than the object of interest.
Refractive errors cause difficulties in focusing on an object at a given distance from the eye [Mountcastle, 1980]. Myopia (near-sightedness), hyperopia (far-sightedness), astigmatism (focus depth that
varies with radial orientation), and presbyopia (loss of ability to adjust focus, manifested as far-sightedness) are the most common vision defects. These normally can be corrected with appropriate eyeglasses
or contact lenses and are rarely the cause of a disability.
Magnification is the most useful form of image processing for vision defects that do not respond to
refractive correction. The simplest form of image magnification is getting closer; halving the distance to
an object doubles its size. Magnifications up to 20 times are possible with minimal loss of field of view.
At very close range, eyeglasses or a loupe may be required to maintain focus [Fonda, 1981]. Hand or
stand magnifiers held 18 to 40 cm (not critical) from the eye create a virtual image that increases rapidly
in size as the object-to-lens distance approaches the focal length of the lens. Lenses are rated in diopters
(D = 1/f, where f is the focal length of the lens in centimeters). The useful range is approximately 4 to
20 D; more powerful lenses are generally held close to the eye as a loupe, as just mentioned, to enhance
field of view. For distance viewing, magnification of 2 to 10 times can be achieved with hand-held
telescopes at the expense of a reduced field of view.
Closed-circuit television (CCTV) systems magnify print and small objects up to 60 times, with higher
effective magnifications possible by close viewing. Users with vision as poor as 1/400 (20/8000) may be
able to read ordinary print with CCTV [Fonda, 1981]. Some recent units are portable and contain
black/white image reversal and contrast enhancement features.
Electrical (or, more recently, magnetic) stimulation of the visual cortex produces perceived spots of
light called phosphenes. Some attempts, summarized in Webster et al. [1985], have been made to map
these sensations and display identifiable patterns, but the phosphenes often do not correspond spatially
with the specific location on the visual cortex. Although the risk and cost of this technique do not yet
justify the minimal “vision” obtained, future use cannot be ruled out.
Tactual Vision Substitution
With sufficient training, people without useful vision can acquire sufficient information via the tactile
sense for many activities of daily living, such as walking independently and reading. The traditional long
cane, for example, allows navigation by transmitting surface profile, roughness, and elasticity to the hand.
Interestingly, these features are perceived to originate at the tip of the cane, not the hand where they are
© 2000 by CRC Press LLC
transduced; this is a simple example of distal attribution [Loomis, 1992]. Simple electronic aids such as
the hand-held Mowat sonar sensor provide a tactile indication of range to the nearest object.
Braille reading material substitutes raised-dot patterns on 2.3-mm centers for visual letters, enabling
reading rates up to 30 to 40 words per minute (wpm). Contracted Braille uses symbols for common
words and affixes, enabling reading at up to 200 wpm (125 wpm is more typical).
More sophisticated instrumentation also capitalizes on the spatial capabilities of the tactile sense. The
Optacon (optical-to-tactile converter) by TeleSensory, Inc. (Mountain View, Calif.) converts the outline
of printed letters recorded by a small, hand-held camera to enlarged vibrotactile letter outlines on the
user’s fingerpad. The camera’s field of view is divided into 100 or 144 pixels (depending on the model),
and the reflected light intensity at each pixel determines whether a corresponding vibrating pin on the
fingertip is active or not. Ordinary printed text can be read at 28 (typical) or 90 (exceptional) wpm.
Spatial orientation and recognition of objects beyond the reach of a hand or long cane are the objective
of experimental systems that convert an image from a television-type camera to a matrix of electrotactile
or vibrotactile stimulators on the abdomen, forehead, or fingertip. With training, the user can interpret
the patterns of tingling or buzzing pints to identify simple, high-contrast objects in front of the camera,
as well as experience visual phenomena such as looming, perspective, parallax, and distal attribution
[Bach-y-Rita, 1972; Collins, 1985].
Access to graphic or spatial information that cannot be converted into text is virtually impossible for
blind computer users. Several prototype devices have been built to display computer graphics to the
fingers via vibrating or stationary pins. A fingertip-scanned display tablet with embedded electrodes,
under development in our laboratory [Kaczmarek et al., 1997], eliminates all moving parts; ongoing tests
will determine if the spatial performance and reliability are adequate.
Auditory Vision Substitution
Electronic speech synthesizers allow access to electronic forms of text storage and manipulation. Until
the arrival of graphic user interfaces such as those in the Apple Macintosh® and Microsoft Windows®
computer operating systems, information displayed on computer screens was largely text-based. A number of products appeared that converted the screen information to speech at rates of up to 500 wpm,
thereby giving blind computer users rapid access to the information revolution. Fortunately, much of
the displayed information in graphic operating systems is not essentially pictorial; the dozen or so
common graphic features (e.g., icons, scroll bars, buttons) can be converted to a limited set of words,
which can then be spoken. Because of the way information is stored in these systems, however, the screento-text conversion process is much more complex, and the use of essentially spatial control features such
as the mouse await true spatial display methods [Boyd et al., 1990].
Automated optical character recognition (OCR) combined with speech synthesis grants access to the
most common printed materials (letters, office memorandums, bills), which are seldom available in
Braille or narrated-tape format. First popularized in the Kurzweil reading machine, this marriage of
technologies is combined with a complex set of lexical, phonetic, and syntactic rules to produce understandable speech from a wide variety of, but not all, print styles.
Mobility of blind individuals is complicated, especially in unfamiliar territory, by hazards that cannot be
easily sensed with a long cane, such as overhanging tree limbs. A few devices have appeared that convert
the output of sonar-like ultrasonic ranging sensors to discriminable audio displays. For example, the
Wormald Sonicguide uses interaural intensity differences to indicate the azimuth of an object and frequency
to indicate distance [Cook, 1982]; subtle information such as texture can sometimes also be discriminated.
143.2 Auditory System
The human auditory system processes information primarily serially; having at best two receptive channels, spatial information must be built up by integration over time. This later capability, however, is
profound. Out of a full orchestra, a seasoned conductor can pinpoint an errant violinist by sound alone.
© 2000 by CRC Press LLC
Human hearing is sensitive to sound frequencies from approximately 16 to 20,000 Hz and is most
sensitive at 1000 Hz. At this frequency, a threshold root-mean-square pressure of 20 Pa (200 µbar) can
be perceived by normally hearing young adults under laboratory conditions. Sound pressure level (SPL)
is measured in decibels relative to this threshold. Some approximate benchmarks for sound intensity are
a whisper at 1 m (30 dB), normal conversion at 1 m (60 dB), and a subway train at 6 m (90 dB). Sounds
increasing from 100 to 140 dB become uncomfortable and painful, and short exposures to a 160-dB level
can cause permanent hearing impairment, while continuous exposure to sound levels over 90 dB can
cause slow, cumulative damage [Sataloff et al., 1980].
Because hearing sensitivity falls off drastically at lower frequencies, clinical audiometric testing uses
somewhat different scales. With the DIN/ANSI reference threshold of 6.5 dB SPL at 1 kHz, the threshold
rises to 24.5 dB at 250 Hz and 45.5 dB at 125 Hz [Sataloff et al., 1980]. Hearing loss is then specified in
decibels relative to the reference threshold, rather than the SPL directly, so that a normal audiogram
would have a flat threshold curve at approximately 0 dB.
Auditory Augmentation
Loss of speech comprehension, and hence interpersonal communication, bears the greatest effect on daily
life and is the main reason people seek medical attention for hearing impairment. Functional impairment
begins with 21- to 35-dB loss in average sensitivity, causing difficulty in understanding faint speech
[Smeltzer, 1993]. Losses of 36 to 50 dB and 51 to 70 dB cause problems with normal and loud speech.
Losses greater than 90 dB are termed profound or extreme and cannot be remedied with any kind of
hearing aid; these individuals require auditory substitution rather than augmentation.
Hearing loss can be caused by conduction defects in the middle ear (tympanic membrane and ossicles)
or by sensorineural defects in the inner ear (cochlear transduction mechanisms and auditory nerve).
Conduction problems often can be corrected medically or surgically. If not, hearing aids are often of
benefit because the hearing threshold is elevated uniformly over all frequencies, causing little distortion
of the signal. Sensorineural impairments differentially affect different frequencies and also cause other
forms of distortion that cannot be helped by amplification or filtering. The dynamic range is also reduced,
because while loud sounds (> 100 dB) are often still perceived as loud, slightly softer sounds are lost.
Looked at from this perspective, it is easy to understand why the amplification and automatic gain control
of conventional hearing aids do not succeed in presenting the 30-dB or so dynamic range of speech to
persons with 70+ dB of sensorineural impairment.
Most hearing aids perform three basic functions. (1) Amplification compensates for the reduced
sensitivity of the damaged ear. (2) Frequency-domain filtering compensates for hearing loss that is not
spectrally uniform. For example, most sensorineural loss disproportionately affects frequencies over
1 kHz or so, so high-frequency preemphasis may be indicated. (3) Automatic gain control (ACG) compresses the amplitude range of desired sounds to the dynamic range of the damaged ear. Typical AGC
systems respond to loud transients in 2 to 5 ms (attack time) and reduce their effect in 100 to 300 ms
(recovery time).
Sophisticated multiband AGC systems have attempted to normalize the ear’s amplitude/frequency
response, with the goal of preserving intact the usual intensity relationships among speech elements.
However, recent research has shown that only certain speech features are important for intelligibility
[Moore, 1990]. The fundamental frequency (due to vocal cord vibration), the first and second formants
(the spectral peaks of speech that characterize different vowels), and place of articulation are crucial to
speech recognition. In contrast, the overall speech envelope (the contour connecting the individual peaks
in the pressure wave) is not very important; articulation information is carried in second-formant and
high-frequency spectral information and is not well-represented in the envelope [Van Tasell, 1993].
Therefore, the primary design goal for hearing aids should be to preserve and make audible the individual
spectral components of speech (formants and high-frequency consonant information).
The cochlear implant could properly be termed an auditory augmentation device because it utilizes
the higher neural centers normally used for audition. Simply stated, the implant replaces the function
© 2000 by CRC Press LLC
of the (damaged) inner ear by electrically stimulating the auditory nerve in response to sound collected
by an external microphone. Although the auditory percepts produced are extremely distorted and noiselike due to the inadequate coding strategy, many users gain sufficient information to improve their
lipreading and speech-production skills. The introductory chapter in this section provides a detailed
discussion of this technology.
Visual Auditory Substitution
Lipreading is the most natural form of auditory substitution, requiring no instrumentation and no
training on the part of the speaker. However, only about one-third to one-half of the 36 or so phonemes
(primary sounds of human speech) can be reliably discriminated by this method. The result is that 30%
to 50% of the words used in conversational English look just like, or very similar to, other words
(homophenes) [Becker, 1972]. Therefore, word pairs such as buried/married must be discriminated by
grammar, syntax, and context.
Lipreading does not provide information on voice fundamental frequency or formants. With an
appropriate hearing aid, any residual hearing (less than 90-dB loss) often can supply some of this missing
information, improving lipreading accuracy. For the profoundly deaf, technological devices are available
to supply some or all of the information. For example, the Upton eyeglasses, an example of a cued-speech
device, provide discrete visual signals for certain speech sounds such as fricatives (letters like f of s,
containing primarily high-frequency information) that cannot be readily identified by sight.
Fingerspelling, a transliteration of English alphabet into hand symbols, can convey everyday words at
up to 2 syllables per second, limited by the rate of manual symbol production [Reed et al., 1990]. American
Sign Language uses a variety of upper body movements to convey words and concepts rather than just
individual letters, at the same effective rate as ordinary speech, 4 to 5 syllables per second.
Closed captioning encodes the full text of spoken words on television shows and transmits the data
in a nonvisible part of the video signal (the vertical blanking interval). Since July of 1993, all new television
sets sold in the United States with screens larger than 33 cm diagonal have been required to have builtin decoders that can optionally display the encoded text on the screen. Over 1000 hours per week of
programming is closed captioned [National Captioning Institute, 1994].
Automatic speech-recognition technology may soon be capable of translating ordinary spoken discourse accurately into visually displayed text, at least in quiet environments; this may eventually be a
major boon for the profoundly hearing impaired. Presently, such systems must be carefully trained on
individual speakers and/or must have a limited vocabulary [Ramesh et al., 1992]. Because there is much
commercial interest in speech command of computers and vehicle subsystems, this field is advancing
rapidly.
Tactual Auditory Substitution
Tadoma is a method of communication used by a few people in the deaf-blind community and is of
theoretical importance for the development of tactual auditory substitution devices. While sign language
requires training by both sender and receiver, in Tadoma, the sender speaks normally. The trained receiver
places his or her hands on the face and neck of the sender to monitor lip and jaw movements, airflow
at the lips, and vibration of the neck [Reed et al., 1992]. Experienced users achieve 80% keyword
recognition of everyday speech at a rate of 3 syllables per second. Using no instrumentation, this the
highest speech communication rate recorded for any tactual-only communication system.
Alternatively, tactile vocoders perform a frequency analysis of incoming sounds, similarly to the ear’s
cochlea [Békésy, 1955], and adjust the stimulation intensity of typically 8 to 32 tactile stimulators
(vibrotactile or electrotactile) to present a linear spectral display to the user’s abdominal or forehead
skin. Several investigators [Blamey & Clark, 1985; Boothroyd & Hnath-Chisolm, 1988; Brooks & Frost,
1986; Saunders et al., 1981] have developed laboratory and commercial vocoders. Although vocoder users
cannot recognize speech as well as Tadoma users, research has shown that vocoders can provide enough
© 2000 by CRC Press LLC
“auditory” feedback to improve the speech clarity of deaf children and to improve auditory discrimination
and comprehension in some older patients [Szeto & Riso, 1990] and aid in discrimination of phonemes
by lipreading [Hughes, 1989; Rakowski et al., 1989]. An excellent review of earlier vocoders appears in
Reed et al. [1982]. The most useful information provided by vocoders appears to be the second-formant
frequency (important for distinguishing vowels) and position of the high-frequency plosive and fricative
sounds that often delineate syllables [Bernstein et al., 1991].
143.3 Tactual System
Humans receive and combine two types of perceptual information when touching and manipulating
objects. Kinesthetic information describes the relative positions and movements of body parts as well as
muscular effort. Muscle and skin receptors are primarily responsible for kinesthesis; joint receptors serve
primarily as protective limit switches [Rabischong, 1981]. Tactile information describes spatial pressure
patterns on the skin given a fixed body position. Everyday touch perception combines tactile and kinesthetic information; this combination is called tactual or haptic perception. Loomis and Lederman [1986]
provide an excellent review of these perceptual mechanisms.
Geldard [1960] and Sherrick [1973] lamented that as a communication channel, the tactile sense is
often considered inferior to sight and hearing. However, the tactile system possess some of the same
spatial and temporal attributes as both of the “primary” senses [Bach-y-Rita, 1972]. With over 10,000
parallel channels (receptors) [Collins & Saunders, 1970], the tactile system is capable of processing a
great deal of information if it is properly presented.
The human kinesthetic and tactile senses are very robust and, in the case of tactile, very redundant.
This is fortunate, considering their necessity for the simplest of tasks. Control of movement depends on
kinesthetic information; tremors and involuntary movements can result from disruption of this feedback
control system. Surgically repaired fingers may not have tactile sensation for a long period or at all,
depending on the severity of nerve injuries; it is known that insensate digits are rarely used by patients
[Tubiana, 1988]. Insensate fingers and toes (due to advanced Hansen’s disease or diabetes) are often
injured inadvertently, sometimes requiring amputation. Anyone who has had a finger numbed by cold
realizes that it can be next to useless, even if the range of motion is normal.
The normal sensitivity to touch varies markedly over the body surface. The threshold forces in dynes
for men (women) are lips, 9 (5); fingertips, 62 (25); belly 62 (7); and sole of foot, 343 (79) [Weinstein,
1968]. The fingertip threshold corresponds to 10-µm indentation. Sensitivity to vibration is much higher
and is frequency-and area-dependent [Verillo, 1985]. A 5-cm2 patch of skin on the palm vibrating at
250 Hz can be felt at 0.16-µm amplitude; smaller areas and lower frequencies require more displacement.
The minimal separation for two nonvibrating points to be distinguished is 2 to 3 mm on the fingertips,
17 mm on the forehead, and 30 to 50 mm on many other locations. However, size and localization
judgments are considerably better than these standard figures might suggest [Vierck & Jones, 1969].
Tactual Augmentation
Although we do not often think about it, kinesthetic information is reflected to the user in many types
of human-controlled tools and machines, and lack of this feedback can make control difficult. For
example, an automobile with power steering always includes some degree of “road feel” to allow the
driver to respond reflexively to minor bumps and irregularities without relying on vision. Remote-control
robots (telerobots) used underwater or in chemical- or radiation-contaminated environments are slow
and cumbersome to operate, partly because most do not provide force feedback to the operator; such
feedback enhances task performance [Hannaford & Wood, 1992].
Tactile display of spatial patterns on the skin uses three main types of transducers [Kaczmarek & Bachy-Rita, 1995; Kaczmarek et al., 1991]. Static tactile displays use solenoids, shape-memory alloy actuators,
and scanned air or water jets to indent the skin. Vibrotactile displays encode stimulation intensity as the
© 2000 by CRC Press LLC
amplitude of a vibrating skin displacement (10 to 500 Hz); both solenoids and piezoelectric transducers
have been used. Electrotactile stimulation uses 1- to 100-mm2-area surface electrodes and careful waveform control to electrically stimulate the afferent nerves responsible for touch, producing a vibrating or
tingling sensation.
Tactile rehabilitation has received minimal attention in the literature or medical community. One
research device sensed pressure information normally received by the fingertips and displayed it on the
forehead using electrotactile stimulation [Collins & Madey, 1974]. Subjects were able to estimate surface
roughness and hardness and detect edges and corners with only one sensor per fingertip. Phillips [1988]
reviews prototype tactile feedback systems that use the intact tactile sense to convey hand and foot
pressure and elbow angle to users of powered prosthetic limbs, often with the result of more precise
control of these devices.
Slightly more attention has been given to tactile augmentation in special environments. Astronauts,
for example, wear pressurized gloves that greatly diminish tactile sensation, complicating extravehicular
repair and maintenance tasks. Efforts to improve the situation range from mobile tactile pins in the
fingertips to electrotactile stimulation on the abdomen of the information gathered from fingertip sensors
[Bach-y-Rita et al., 1987].
Tactual Substitution
Because of a paucity of adequate tactual display technology, spatial pressure information from a robot
or remote manipulator is usually displayed to the operator visually. A three-dimensional bar graph, for
example, could show the two-dimensional pressure pattern on the gripper. While easy to implement,
this method suffers from two disadvantages: (1) the visual channel is required to process more information (it is often already heavily burdened), and (2) reaction time is lengthened, because the normal human
tactual reflex systems are inhibited. An advantage of visual display is that accurate measurements of force
and pressure may be displayed numerically or graphically.
Auditory display of tactual information is largely limited to warning systems, such as excessive force
on a machine. Sometimes such feedback is even inadvertent. The engine of a bulldozer will audibly slow
down when a heavy load is lifted; by the auditory and vibratory feedback, the operator can literally “feel”
the strain.
The ubiquity of such tactual feedback systems suggests that the human-machine interface on many
devices could benefit from intentionally placed tactual feedback systems. Of much current interest is the
virtual environment, a means by which someone can interact with a mathematic model of a place that
may or may not physically exist. The user normally controls the environment by hand, head, and body
movements; these are sensed by the system, which correspondingly adjusts the information presented
on a wide-angle visual display and sometimes also on a spatially localized sound display. The user often
describes the experience as “being there,” a phenomenon known as telepresence [Loomis, 1992]. One can
only imagine how much the experience could be enhanced by adding kinesthetic and tactile feedback
[Shimoga, 1993], quite literally putting the user in touch with the virtual world.
Defining Terms
Distal attribution: The phenomenon whereby events are normally perceived as occurring external to
our sense organs—but also see Loomis’ [1992] engaging article on this topic. The environment or
transduction mechanism need not be artificial; for example, we visually perceive objects as distant
from our eyes.
Electrotactile: Stimulation that evokes tactile (touch) sensations within the skin at the location of the
electrode by passing a pulsatile, localized electric current through the skin. Information is delivered
by varying the amplitude, frequency, etc. of the stimulation waveform. Also called electrocutaneous
stimulation.
© 2000 by CRC Press LLC
Illuminance: The density of light falling on a surface, measured in lux. One lux is equivalent to 0.0929
foot-candles, an earlier measure. Illuminance is inversely proportional to the square of the distance
form a point light source. A 100-W incandescent lamp provides approximately 1280 lux at a
distance of 1 ft (30.5 cm). Brightness is a different measure, depending also on the reflectance of
the surrounding area.
Kinesthetic perception: Information about the relative positions of and forces on body parts, possibly
including efference copy (internal knowledge of muscular effort).
Sensory augmentation: The use of devices that assist a functional human sense; eyeglasses are one
example.
Sensory substitution: The use of one human sense to receive information normally received by another
sense. For example, Braille substitutes touch for vision.
Sound pressure level (SPL): The root-mean-square pressure difference from atmospheric pressure
(≈100 kPa) that characterizes the intensity of sound. The conversion SPL = 20 log (P/P0) expresses
SPL in decibels, where P0 is the threshold pressure of approximately 20 Pa at 1 kHz.
Static tactile: Stimulation that is a slow local mechanical deformation of the skin. It varies the deformation amplitude directly rather than the amplitude of vibration. This is “normal touch” for
grasping objects, ect.
Tactile perception: Information about spatial pressure patterns on the skin with a fixed kinesthetic
position.
Tactual (haptic) perception: The seamless, usually unconscious combination of tactile and kinesthetic
information; this is “normal touch.”
Vibrotactile: Stimulation that evokes tactile sensations using mechanical vibration of the skin, typically
at frequencies of 10 to 500 Hz. Information is delivered by varying the amplitude, frequency, etc.
of the vibration.
Virtual environment: A real-time interactive computer model that attempts to display visual, auditory,
and tactual information to a human user as if he or she were present at the simulated location.
The user controls the environment with head, hand, and body motions. A airplane cockpit simulator is one example.
References
Bach-y-Rita P. 1972. Brain Mechanisms in Sensory Substitution. New York, Academic Press.
Bach-y-Rita P., Kaczmarek, KA, Tyler, M. and Garcia-Lara, M. 1998. Form perception with a 49-point
electrotactile stimulus array on the tongue. J. Rehab. Res. Dev. 35:427–430.
Bach-y-Rita P, Webster JG, Tompkins WJ, Crabb T. 1987. Sensory substitution for space gloves and for
space robots. In Proceedings of the Workshop on Space Telerobotics, Jet Propulsion Laboratory,
Publication 87–13, pp 51–57.
Barfield, W., Hendrix, C., Bjorneseth, O., Kaczmarek, KA and Lotens, W. 1996. Comparison of human
sensory capabilities with technical specifications of virtual environment equipment. Presence
4:329–356.
Becker KW. 1972. Speechreading: Principles and Methods. Baltimore, National Educational Press.
Békésy GV. 1955. Human skin perception of traveling waves similar to those of the cochlea. J Acoust Soc
AM 27:830.
Bernstein LE, Demorest ME, Coulter DC, O’Connell MP. 1991. Lipreading sentences with vibrotactile
vocoders: Performance of normal-hearing and hearing-impaired subjects. J Acoust Soc Am 90:2971.
Blamey PJ, Clark GM. 1985. A wearable multiple-electrode electrotactile speech processor for the profoundly deaf. J Acoust Soc Am 77:1619.
Boothroyd A, Hnath-Chisolm T. 1988. Spatial, tactile presentation of voice fundamental frequency as a
supplement to lipreading: Results of extended training with a single subject. J Rehabil Res Dev 25(3):51.
Boyd LH, Boyd WL, Vanderheiden GC. 1990. The Graphical User Interface Crisis: Danger and Opportunity. September, Trace R&D Center, University of Wisconsin-Madison.
© 2000 by CRC Press LLC
Brooks PL, Frost BJ. 1986. The development and evaluation of a tactile vocoder for the profoundly deaf.
Can J Public Health 77:108.
Collins CC. 1985. On mobility aids for the blind. In DH Warren, ER Strelow (eds), Electronic Spatial
Sensing for the Blind, pp 35–64. Dordrecht, The Netherlands, Matinus Nijhoff.
Collins CC, Madey JMJ. 1974. Tactile sensory replacement. In Proceedings of the San Diego Biomedical
Symposium, pp 15–26.
Collins CC, Saunders FA. 1970. Pictorial display by direct electrical stimulation of the skin. J Biomed
Syst 1:3–16.
Cook AM. 1982. Sensory and communication aids. In AM Cook, JG Webster (eds), Therapeutic Medical
Devices: Application and Design, pp 152–201. Englewood Cliffs, NJ, Prentice-Hall.
Fonda GE. 1981. Management of Low Vision. New York, Thieme-Stratton.
Geldard FA. 1960. Some neglected possibilities of communication. Science 131:1583.
Hannaford B, Wood L. 1992. Evaluation of performance of a telerobot. NASA Tech Briefs 16(2): item 62.
Hughes BG. 1989. A New Electrotactile System for the Hearing Impaired. National Science Foundation
final project report, ISI-8860727, Sevrain-Tech, Inc.
Kaczmarek KA, Bach-y-Rita P. 1995. Tactile displays. In W Barfield, T Furness (eds), Virtual Environments
and Advanced Interface Design. New York, Oxford University Press.
Kaczmarek, KA, Tyler, ME and Bach-y-Rita, P. 1997. Pattern identification on a fingertip-scanned electrotactile display. Proc. 19th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. pp. 1694–1697.
Kaczmarek KA, Webster JG, Bach-y-Rita, Tompkins WJ. 1991. Electrotactile and vibrotactile displays for
sensory substitution systems. IEEE Trans Biomed Eng 38:1.
Loomis JM. 1992. Distal attribution and presence. Presence: Teleoperators Virtual Environ 1(1):113.
Loomis JM, Lederman SJ. 1986. Tactual perception. In KR Boff et al (eds), Handbook of Perception and
Human Performance, vol II: Cognitive Processes and Performance, pp 31.1–31.41. New York, Wiley.
Mehr E, Shindell S. 1990. Advances in low vision and blind rehabilitation. In MG Eisenberg, RC Grzesiak
(eds), Advances in Clinical Rehabilitation, vol 3, pp 121–147. New York, Springer.
Moore BCJ. 1990. How much do we gain by gain control in hearing aids? Acta Otolaryngol (Stockh)
Suppl. 469:250.
Mountcastle VB (ed). 1980. Medical Physiology. St. Louis, Mosby.
National Captioning Institute, Falls Church, Va, 1994. Personal communication.
NIDRR. 1993. Protocols for choosing low vision devices. U.S. Department of Education. Consensus
Statement 1(1–28).
Phillips CA. 1988. Sensory feedback control of upper- and lower-extremity motor prostheses. CRC Crit
Rev Biomed Eng 16:105.
Rabischong P. 1981. Physiology of sensation. In R Tubiana (ed), The Hand, pp 441–467. Philadelphia,
Saunders.
Rakowski K, Brenner C, Weisenberger JM. 1989. Evaluation of a 32-channel electrotactile vocoder
(abstract). J Acoust Soc Am 86(suppl 1):S83.
Ramesh P, Wilpon JG, McGee MA, et al. 1992. Speaker independent recognition of spontaneously spoken
connected digits. Speech Commun 11:229.
Reed CM, Delhorne LA, Durlach NI, Fischer SD. 1990. A study of the tactual and visual reception of
fingerspelling. J Speech Hear Res 33:786.
Reed CM, Durlach NI, Bradia LD. 1982. Research on tactile communication of speech: A review. AHSA
Monogr 20:1.
Reed CM, Rabinowitz WM, Durlach NI, et al. 1992. Analytic study of the Tadoma method: Improving
performance through the use of supplementary tactile displays. J Speech Hear Res 35:450.
Sataloff J, Sataloff RT, Vassallo LA. 1980. Hearing Loss, 2d ed. Philadelphia, Lippincott.
Saunders FA, Hill WA, Franklin B. 1981. A wearable tactile sensory aid for profoundly deaf children.
J Med Syst 5:265.
Sherrick CE. 1973. Current prospects for cutaneous communication. In Proceedings of the Conference
on Cutaneous Communication System Development, pp 106–109.
© 2000 by CRC Press LLC
Shimoga KB. 1993. A survey of perceptual feedback issues in dextrous telemanipulation: II. Finger touch
feedback. In IEEE Virtual Reality Annual International Symposium, pp 271–279.
Shlaer S. 1937. The relation between visual acuity and illumination. J Gen Physiol 21:165.
Smeltzer CD. 1993. Primary care screening and evaluation of hearing loss. Nurse Pract 18:50.
Stigmar G. 1970. Observation on vernier and stereo acuity with special reference to their relationship.
Acta Ophthalmol 48:979.
Szeto AYJ, Riso RR. 1990. Sensory feedback using electrical stimulation of the tactile sense. In RV Smith,
JH Leslie Jr (eds), Rehabilitation Engineering, pp 29–78. Boca Raton, Fla, CRC Press.
Tubiana R. 1988. Fingertip injuries. In R Tubiana (ed), The Hand, pp 1034–1054. Philadelphia, Saunders.
Van Tasell DJ. 1993. Hearing loss, speech, and hearing aids, J Speech Hear Res 36:228.
Verrillo RT. 1985. Psychophysics of vibrotactile stimulation. J Acoust Soc Am 77:225.
Vierck CJ, Jones MB. 1969. Size discrimination on the skin. Science 163:488.
Webster JG, Cook AM, Tompkins WJ, Vanderheiden GC (eds). 1985. Electronic Devices for Rehabilitation.
New York, Wiley.
Weinstein S. 1968. Intensive and extensive aspects of tactile sensitivity as a function of body part, sex
and laterality. In DR Kenshalo (ed), The Skin Senses, pp 195–218. Springfield, Ill, Charles C
Thomas.
Further Information
Presence: Teleoperators and Virtual Environments is a bimonthly journal focusing on advanced humanmachine interface issues. In an effort to develop tactile displays without moving parts, our laboratory
has demonstrated simple pattern recognition on the fingertip [Kaczmarek et al., 1997] and tongue [Bachy-Rita et al., 1998] using electrotactile stimulation.
The Trace Research and Development Center, Madison, Wisc., publishes a comprehensive resource
book on commercially available assistive devices, organizations, etc. for communication, control, and
computer access for individuals with physical and sensory impairments.
Electronic Devices for Rehabilitation, edited by J. G. Webster (Wiley, 1985), summarizes the technologic
principles of electronic assistive devices for people with physical and sensory impairments.
© 2000 by CRC Press LLC