Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
A portable wireless eye movement-controlled Human-Computer Interface for the Disabled ABSTRACT A portable wireless eye movementcontrolled Human-Computer Interface which can be used for the disabled who have motor paralysis and who cannot speak in multiple applications (such as communication aid and smart home applications) is described here. This Interface consists of four major parts: (1) Surface electrodes (2) A two-channel amplifier (3) A laptop (or a micro-processor) (4) A ZigBee wireless module. Horizontal and vertical ElectroOculography (EOG) signals are measured using five surface electrodes placed on the head .The vertical electrodes are placed about 1.0 cm above the right eyebrow and 2.0 cm below the lower lid of the right eye, the horizontal electrodes are placed 2.0 cm lateral to the each side of outer canthi and the last electrode is placed on userโs forehead to serve as a ground. The two-channel amplifier is comprised of instrumentation amplifiers, band-pass filters and shift circuits. The EOG signals are sampled at the rate of 250Hz and then sent to a laptop or a micro-processor for signal processing which is based on the method of mathematical morphology to recognize the direction of eye movements and voluntary eye blink. The ZigBee wireless communication technology, which is proved to be reliable, low-power and cost-efficient, is used in the portable interface. This interface provides a flexible method for the disabled to improve the life quality. I. INTRODUCTION Persons with severe diseases, such as amyotrophic lateral sclerosis (ALS), brainstem stroke, brain or spinal cord injury, cerebral palsy, muscular dystrophies, multiple sclerosis, etc., have difficulty conveying their intentions and communicating with other people in daily life. With the development of HumanComputer Interface (HCI), methods have been developed to help these people for communication. Unlike traditional HCIs (a keyboard, or a mouse, etc.), modern HCIs have played an important role in the area of rehabilitation. HCIs can be divided into cortical (all interfaces that exploit information collected from the human brain cortical relays) and non-cortical (all interfaces that do not access the signals generated by the human cortex directly). In the present study, we describe a novel portable wireless eye movementcontrolled HCI for the disabled. This interface is a real-time communication control system based on EOG signals. There are two main differences between our system and others mentioned above: (1) Designing and implementing a mathematical morphology method to preprocess original EOG signals. (2) Including a wireless module based on the ZigBee protocol to increase the scope of applications (communication aid, smart home. applications, etc.) of this system. II. SYSTEM OVERVIEW The system we have developed consists of four major parts: 1. Five surface electrodes 2. A two-channel amplifier, 3. A laptop (or a micro-processor) 4. A ZigBee wireless module. Fig 1: Overview of the EOG-based wireless Human-Computer Interface. Fig 1 is the schematic diagram of this system and the whole system adopts the star topology which is also used. In this system, horizontal and vertical EOG signals are measured by five surface electrodes placed around eyes. After a two-channel amplifier, the EOG signals are sampled at the rate of 250 Hz and then sent to a coordinator node which is connected with a laptop or a microprocessor through ZigBee wireless communication technology. The software on the laptop or micro-processor recognizes the direction of eye movement and voluntary eye blinking. Programs (typewriter, patient assistant software, etc.) in laptop or remote devices (TV, lamps, telephone, etc.) can be controlled by the recognized results. III.ELECTRODES PRINCIPLE AND THE The cornea of the eye is electrically positive relative to the retina of the eye and the potential is slowly varying when eyes move. The standing potential can be measured by electrodes placed around the eyes. The EOG value varies from 0.053.5 mV with a frequency range of about 0100 Hz. In this paper, there are five electrodes in all which are classified as horizontal, vertical and reference (ground) electrodes. As showed in Fig 1, the vertical electrodes are placed about 1.0 cm above the right eyebrow and 2.0 cm below the lower lid of the right eye, the horizontal electrodes are placed 2.0 cm lateral to the each side of outer canthi. And the last electrode is placed on userโs forehead to serve as a ground. If the eyes move left, horizontal EOG (HEOG) signal which is the difference between signals collected by electrode HEOL and HEOR acquires a positive voltage value. If the eyes turn right, HEOG signal changes into a negative voltage value. Identically, if the eyes move from the central position towards upside, vertical EOG (VEOG) signal which is the difference between signal collected by electrode VEOU and VEOL acquires a positive voltage value. If the eyes move downside, VEOG signal changes into a negative voltage value. An eye blinking can be described by EOG signals as a peak in VEOG but a flat in HEOG. We can distinguish the voluntary and involuntary blinking by the value and duration of the peak mentioned above. IV.AMPLIFIER The horizontal and vertical eye movement signals captured by the electrodes were then transmitted to a two-channel amplifier which consists of (1) preamplifiers (2) band-pass filters (3) shift circuits (4) right-leg driven circuits (5) power supply. The preamplifier is a micro-power instrumentation amplifier for accurate and low noise differential signal acquisition. The gain of the preamplifier is set to be 21 with a single external resistor. The band-pass filter (0.01-41 Hz) is provided with two Sallen-Key filters (One second-order high-pass filter and one fourth-order low-pass filters). The following circuits are secondary amplifier with variable gain and shift circuit to transform the signal level into the range Fig 1: Overview of the EOG-based wireless Human-Computer Interface. Fig 2: EOG signals during eye movement and blanking. (a) HEOG signals. (b) VEOG signals. of 0 V to 3 V for adapting the following analog-to-digital converter (ADC). Right-leg driven circuit connected with the reference electrode is used to reduce the common-mode components in the signal. Power for the board is supplied by one common 6V battery, which is then transformed into ± 3.3 V with AMS1117 and MAX828 respectively. V. WIRELESS MODULE The Wireless module takes responsibility for transmitting two-channel EOG signals from one node attached to the userโs body to the coordinator node connected with the laptop. Meanwhile, the coordinator can send messages to other remote controllers (TV, lamp, telephone, etc). The ZigBee wireless communication technology, which is proved to be reliable, low-power and cost-efficient, is used in this system. The module is established using CC2430, which is a true System-on-Chip solution specifically tailored for IEEE 802.15.4 and ZigBee applications. The CC2430 combines RF transceiver with an industry-standard enhanced 8051 MCU, 32/64/128 KB flash memory, 8 KB RAM and many other powerful features. At the transmission node, analog EOG signals from amplifiers are sampled at the rate of 250Hz and transmitted. At the reception node, EOG signals are transported to laptop with RS232-USB interface for signal processing. In the prototype software, the protocol is based on a ZigBee stack called MSSTATE_LRWPAN which implements a ZigBee subset wireless stack. The program in CC2430 is based on this protocol completely. VI. EOG SIGNAL PROCESSING The method is based on the mathematical morphology (MM), differential and integral algorithms to recognize the direction of eye movement and voluntary blinking. VEOG signals are used to detect up/down movement and voluntary eye blinking, while HEOG signals are used to detect left/right movement. Fig 3: The flowchart of EOG signal processing. MM Algorithm: The method of MM is widely used in ECG signal processing and other fields . It provides a good way to remove drift and magnify feature of the signal. The operators of MM are dilation, erosion, opening and closing. Opening generally smoothes the contour of an object, breaks narrow isthmuses, and eliminate thin protrusions. Closing also tends to smooth sections of contours and which generally fuses narrow breaks and long thin gulfs, eliminates small holes, and fills gaps in the contour. In our work, we only used symmetrical sequences as the structuring element. The result of VEOG signals after MM filter. Differential Algorithm: The VEOG signals, after MM filter, are feed into the differential module implemented. 9 19 Y(n)= โ ๐=0 x(n + i + 10) โ โ๐=0 x(n + i)/20 ---------------> (1) Where x (n) is the VEOG signals after MM filter, and y (n) is the result after the differential module. Integral Algorithm: The difference of original VEOG signals (delay 2N points, N is the length of the structuring element in MM algorithm) and signals after MM filter can be used for eye blinking recognition. Because the peak value of voluntary blinking is much larger than involuntary blinking, we can distinguish those two kinds of blinking by the integral module using (2) and the threshold. 19 y(n) = โ x(n + i)/20 ๐=0 ------------> (2) Where x(n) is the difference of original VEOG signals (delay 2N points) and signals after MM filter, y(n) is the result after the integral module. Decision Module: In Fig. 4, S1, S2 and S3 are the results by the methods mentioned above. Threshold1 is the voluntary eye blinking threshold, Threshold2 is the involuntary eye blinking threshold, Threshold3 is the movements (up/down) threshold, and Threshold4 is the movements (left/right) threshold. We can distinguish eye blinking (voluntary and involuntary) and eight-direction movement through these thresholds. Fig 4:Example of VEOG signal processing. VII.APPLICATION SOFTWARE TEST We have developed two application programs to test this system: the typewriter and the patient assistant software. The typewriter user interface is showed in Fig. 7a. Users make the cursor move up, down, left and right to select a letter in the table. The letters selected (by voluntary eye blinking) are showed above the table. The patient assistant software is showed in Fig. 7b. In this application, users move the cursor by eightdirectional eye movements, and the size of icon selected is enhanced. At the same time, the LED which indicates the direction of eye movement is lighted by the controlling of the remote ZigBee module. These two applications indicate that the portable wireless eye movementcontrolled we developed is doable. The performance (bit rate and latency) of this system can be calculated as the following paragraphs. Where T is the delay time, N is the number of sampled points delayed, and Fs are the sampling frequency. VIII. BIT RATE Ttotal=T1+max (T2, T3) ----------> (5) The bit rate of this system is calculated. Where B is the bit rate per selection, N is the possible choices, and P is the accuracy. Bit rate per minute of the system can be obtained from B multiplied by the number of selections in a minute. The sampling frequency is 250 Hz, and 100 sampled points are delayed during the method of mathematical morphology. 20 sampled points are delayed both in the processing of differential and integral algorithms. Thus, the total time delay Ttotal calculated by using (5) is 0.48 s. B=log2 โ ๐+Plog2 โp+(1-p)log2 (1-p)/(n-1) ----------> (3) Several subjects were asked to test the patient assistant software (N in this application is 8). They got accuracies all above 80% and could make at least 10 selections per minute. Therefore, the bit rate of this system is above 17.17 bits per minute. IX. LATENCY The Latency of this system is related with the method of real-time signal processing. The delay time of mathematical morphology, differential, and integral algorithms is T1, T2 and T3, respectively. The delay time of each algorithm is calculated by using (4). Thus, T1 = 0.4 s, T2 = T3 = 0.08 s. T = N / Fs ---------> (4) X.FACTORS INFLUENCING PERFORMANCE The EOG signals are mostly concentrated on the low frequency, especially near the DC component where lots of useful information locates. Therefore, the cut-off frequency of the high-pass filter should be set as low as possible (0.01 Hz in this system) otherwise the eye movement signals would decline rapidly rather than be hold for a long time. Because of the MM method, the influences of the drift and other noise were reduced. However, slow involuntary blinking (duration above 0.8 s) would be recognized as eye movement (Up), which is a mistake. So when using this system, slow involuntary blinking should be performed as few as possible. XI. IMPROVEMENT OF THE SYSTEM IN FUTURE Four thresholds which are measured in advance were set manually at the initialization stage of the software. Clearly, it is time-consuming and may result in unnecessary errors. Later evelopment should make the thresholds set automatically through a test program before use. Meanwhile, the thresholds can be updated during test period autoadaptively according to the userโs current state. For instance, the amplitude of the EOG signals would change slowly in the latter stage because of fatigue. The current signal process program is implemented on a laptop which also provides user interface on the screen. If we do not need user interfaces (e.g. controlling remote devices), the process program can be carried out only in a micro-processor which is integrated in the coordinator node. Then the processed results were sent wirelessly to the remote device which is attached with a ZigBee reception node.