Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
A Non-Invasive Technique for Monitoring Miotic Eye Responses Ahmed M. El-Sherbeeny Thesis submitted to the College of Engineering and Mineral Resources at West Virginia University in partial fulfillment of the requirements for the degree of Master of Science in Mechanical Engineering James E. Smith, Ph.D., Chair James V. Odom, Ph.D. Gregory J. Thompson, Ph.D. Robert R. Craven, BSChE Department of Mechanical and Aerospace Engineering Morgantown, West Virginia 2001 Keywords: pupil, pupillary. miosis, constriction, dilation ABSTRACT A Non-Invasive Technique for Monitoring Miotic Eye Responses Ahmed M. El-Sherbeeny The eye is a very sensitive organ of the human body, and is subject to many hazards that may cause disorders such as miosis. Thus non-costly, non-invasive, safe methods for pupil assessment and monitoring are of the essence. Proposed is a simple eye scanning mechanism that periodically reports whether the pupil has experienced miosis or dilatation due to an external illumination source. The process uses a Clanguage program to control the image-grabbing process, as well as determining the pupil constriction/dilation conditions. The program focuses on dynamically isolating the pupil by producing a threshold for it that varies with the measured light level, producing a threshold-illumination (TI) curve. Various tests were conducted using a fixed setup to study the pupillary size with different pupils, over different periods, and while changing the pupil's position. Using a standard monochrome camera and lens, monitoring the pupil was made possible using safe levels of a self-designed infrared light-emitting diode flashlight. It was concluded that linear TI curves produced the most efficient results after accounting for a consistent precision correction-factor in the data. Proposed is an algorithm and apparatus for further investigation and implementation of this pupillary monitoring mechanism. A Non-Invasive Technique for Monitoring Miotic Eye Responses Ahmed M. El-Sherbeeny Thesis submitted to the College of Engineering and Mineral Resources at West Virginia University in partial fulfillment of the requirements for the degree of Master of Science in Mechanical Engineering Approval of the Examining Committee _________________________ James E. Smith, Ph.D., Chair _________________________ James V. Odom, Ph.D. _________________________ Gregory J. Thompson, Ph.D. ______________ Date _________________________ Robert R. Craven, BSchE To my brother Khaled whom I will always be grateful to. You brought me to WVU, guided me all along, and made sure I completed this work. May Allah bless you always and keep you steadfast on the right path. This is also dedicated to father (Mohammed), mother (Zeinab), and sister (Heba). Allah has (AHL) answered your prayers. v Acknowledgement I would like to show appreciation and gratitude to my brother Khaled, my father (Mohammed), mother (Zeinab), and sister (Heba) for their physical support, and relentless prayers. Dr. Smith has been my advisor, and the closest to a second father as it can get. I would also like to express warm thanks to Mr. Robert Craven who has been there for me every step of the way. Special thanks also go to Dr. Odom who has assisted me extensively. Dr. Thompson has assured I produce a professional document. I would finally like to show my appreciation to all my brothers here in Morgantown who have encouraged me and pushed me further until this work was completed satisfactorily. vi Table of Contents Title Page ___________________________________________________________ i Abstract ____________________________________________________________ ii Acknowledgement ___________________________________________________ iv Table of Contents _____________________________________________________v List of Tables ______________________________________________________ vii List of Figures _____________________________________________________ viii List of Symbols ______________________________________________________ xi Introduction ________________________________________________________ 1 The Pupil _______________________________________________________________ 1 Miosis __________________________________________________________________ 3 Testing the Pupil _________________________________________________________ 6 Review of Literature __________________________________________________ 9 Pupillary Assessment and Monitoring _______________________________________ 9 1. Classical Pupillometry _______________________________________________________ 10 2. Early Assessment Trials ______________________________________________________ 11 3. Videopupillometry __________________________________________________________ 12 4. Pupillary Monitoring Techniques _______________________________________________ 14 Miosis and Other Ocular Signs of Organophosphates and Carbamates __________ 16 1. 2. Organophosphates (OP) ____________________________________________________ 17 Nerve gases______________________________________________________________ 25 Text of Investigation ________________________________________________ 29 Apparatus and Experimental Setup ________________________________________ 30 1. 2. 3. 4. 5. 6. 7. Illumination Source _______________________________________________________ 32 Eye ____________________________________________________________________ 36 B/W Camera and Lens _____________________________________________________ 37 Connections and Display ___________________________________________________ 38 Digitizer ________________________________________________________________ 40 Computer Code___________________________________________________________ 43 Data and Image Processing Software __________________________________________ 59 Results and Discussion ___________________________________________________ 64 1. Direct Pupil-Size Method (Test 1) ______________________________________________ 65 2. Threshold-Illumination Curves (Test 2) __________________________________________ 68 3. Illumination Experiment (Test 3) _______________________________________________ 80 4. Eye-Movement Experiment (Test 4) ____________________________________________ 90 Conclusions and Recommendations ____________________________________ 98 Bibliography ______________________________________________________ 101 vii Appendix _________________________________________________________ 107 Appendix 1 Listing of the Code ___________________________________________ 107 Appendix 2 Test 3 and Test 4 Results ______________________________________ 124 Appendix 3 Proposed Pupil Edge-Finder Routine (Algorithm) _________________ 132 Appendix 4 A proposed hardware implementation (Outline) __________________ 140 viii List of Tables Table 1 A list of some of the agents that cause miosis [7]....................................................................... 4 Table 2 Suggested price analysis for the eye-monitoring technique. .....................................................62 Table 3 Results of the direct pupil-size method for cases A, B, and C. ..................................................67 Table 4 Summary of the TI experimental data for each of the three subjects.........................................76 Table 5 Tabulated data for the Illumination Experiment conducted on subject A. ................................84 Table 6 Tabulated data for the Eye-Movement Experiment conducted on subject B. ............................94 ix List of Figures Figure 1 Front view of the eyelids, pupil, and other visible parts of the eye [3: Pg. 18]. ....................... 2 Figure 2 Horner’s syndrome of the right eye; notice the smaller pupil in the right eye, as well as ptosis [8: Pg. 107]............................................................................................................................... 6 Figure 3 A schematic of the swinging flashlight test [4: Pg. 159]. ......................................................... 7 Figure 4 Schematics of the infrared-sensitive camera (Left) and the superimposed image of the eye from the computer screen (Right); note the ellipse measurement tool [24: Pg. 46]. ...............13 Figure 5 Acute angle-closure glaucoma; severely high intraocular pressure resulted in eye inflammation, in addition to corneal edema, and a partially dilated pupil (note also the disturbed light reflex) [49: Pg. 142]. .......................................................................................21 Figure 6 Using atropine to treat miosis. Left: Miosis due to corneal foreign body in a dog; Right: Mydriasis after treatment with atropine [65: Pg. 405]. ...........................................................24 Figure 7 A simplified schematic of the experimental setup used for grabbing and processing images of the eye. The big arrows going towards the eye indicate orders sent from the program, and the small arrows going from the eye indicate feedback going into the program. ..........................30 Figure 8 Side views of the final employed experimental setup, showing the individual components. Left: Note that monitoring the experiment was made possible by both a PC-display (right), as well as a color monitor (left). Right: subject during one of the employed tests. ......................31 Figure 9 Partially designed infrared flashlight used to light the pupil during the experiment. Left: front view of flashlight showing the commercial diodes used (one is not functional). Right: internal view of the flashlight, showing the parallel connection from the diodes to two AAA, easily replaceable batteries. ...............................................................................................................32 Figure 10 Impact of using infrared vs. regular room abiment light solely. Left: photo of the right eye of a subject taken at regular ambient light. Right: photo of the same eye, under identical conditions with the added infrared light (note how the diodes are creating a reflection at the right of the pupil). ....................................................................................................................34 Figure 11 Artificial light-sources used in the experiment. Left: IR flashlight close to the chin rest and close to the position of the subject's right eye; the faulty diode has been marked. Right: The Xenon Lightsource kept away at a considerably large distance. .............................................35 Figure 12 Assembly of physical components in the experiment, the chin rest, IR flashlight, camera, and relevant fixtures. Left: the Burle, IR sensitive camera has been positioned as to be completely stable during all experiments; Right: a clearer view of the image-capturing setup. ...............38 Figure 13 Connections and channels used in the experimental setup. Left: the four rightmost BNC pins represent the R, G, B, and Sync channels. The lowermost pin represents the terminal end of x the 9-ft coaxial cable; the white connection (on the left) represents the line going from the camera to the TV monitor. Right: Meteor connection to the PC. Note the (4) individual channels. ..................................................................................................................................39 Figure 14 On-line display is possible using the TV monitor (for the subject pupil, left), and PC monitor (for data and images, right). ....................................................................................................40 Figure 15 Matrox Meteor/RGB digitizer-board flow diagram [78: Pg. 220]. .......................................41 Figure 16 Adopted flowchart for a simplified version of the program [103]. ........................................58 Figure 17 The image processing involved in creating a TI curve. Left: an actual grab from the program, with dim light level. Right: a screen grab (to the same scale as the one to the left) from Scion Image after the threshold process has been performed; note how the pupil is accurately represented and isolated in the image. ...................................................................62 Figure 18 Direct pupil-size method (subject B). Left: maximum dilatation method. Right: maximum constriction method. Note how the divisions are still visible with both dim and bright lights. 66 Figure 19 Screen capture of a text file containing 10 different image readings at a fixed light value and their average (subject A). This capture was taken at the high illumination value. ..................69 Figure 20 Screen capture as in Figure 19 for the same subject at a lower illumination value. .............69 Figure 21 Screen capture of the "result.txt" file that automatically gathers the indexed, average illumination values (two of which are in Figure 19, Figure 20) and lists them for future manipulation (subject A). .........................................................................................................70 Figure 22 Saved images from the TI program (left) and screen captures showing the corresponding threshold images as obtained from Scion Image (right). The images are listed from top to bottom with decreased camera illumination (subject A). Note how the images get darker, yet the pupillary size in the threshold images is almost constant. .................................................72 Figure 23 The next set of three photos for the same experiment as in Figure 22. Up to the middle image (fifth as a whole) the threshold pupil is still clear and of size almost identical to that obtained earlier. The bottom image shows some traces of reflection, but the pupil size is still regular. ....................................................................................................................................73 Figure 24 The last three photos for the same experiment (subject A, Test 2a) as in Figure 22 and Figure 23. Note, at this very little lighting, the iris and other surrounding structures start to appear in the threshold image, and the pupil size starts to vary, and become discontinuous in shape. The data from these photos has been discarded. ..........................................................74 Figure 25 Linear regression used to produce characteristic TI curves for subject A. ...........................78 Figure 26 Polynomial and linear regression analysis used to produce TI curves for subject B. ...........78 Figure 27 Power regression analysis used to produce both TI curves for subject C. ............................79 Figure 28 Summary of the characteristic TI curves for all three subjects. ............................................79 xi Figure 29 Screen captures of the "test2.txt" file for the Illumination experiment (subject A). ...............81 Figure 30 Some of the photographed images from the Illumination Experiment (left) and the corresponding foot-printed threshold images obtained from the generated program (right). The images are listed from top to bottom with decreased camera illumination (subject A). The threshold pupil is identified by the blue spot in the middle of the screen. ................................83 Figure 31 Scatter of diameter readings for the Illumination Experiment: subject A. ............................88 Figure 32 Scatter of diameter readings for the Illumination Experiment: subject B. ............................88 Figure 33 Scatter of diameter readings for the Illumination Experiment: subject C. ............................89 Figure 34 Screen captures of the "test2.txt" file for the Eye-Movement experiment (subject B)............91 Figure 35 Images from the Eye-Movement experiment (left) and the corresponding threshold images obtained from the generated program (right). The images are listed from top to bottom with eye-movement positions: forward (move 1), up (move 2), left (move 3), down (move 4), and right (move 5) (subject B). .......................................................................................................93 Figure 36 Scatter of diameter readings for the Eye-Movement Experiment: subject A. ........................96 Figure 37 Scatter of diameter readings for the Illumination Experiment: subject B. ............................96 Figure 38 Scatter of diameter readings for the Illumination Experiment: subject C. ............................97 Figure 39 Proposed strategy for determining the left, right, upper, and lower boundaries of the pupil. ...............................................................................................................................................133 Figure 40 Sketch of the Tangent-Perpendicular Method for determining as many points as possible on the edge of the pupil. ..............................................................................................................134 xii List of Symbols ACGIH antiChEs CB CF CNS CV EEG ERG FT IR LED MD MIL MPA mWF MWF N NBPF OP PAM PSP PTR RAPD RD RGBS SF ST TAT TEPP TI TLV TNBPF TPT TRT VC++ VER American Conference of Governmental Industrial Hygienists Anticholinesterases Carbamate Correction Factor Central Nervous System Contraction Velocity Electroencephalogram Electroretinogram Finishing Time Infrared Light Emitting Diode Measured Distance Matrox Imaging Library Maximum Pupillary Area Minor Warning Flag Major Warning Flag Number of Runs Number of Black Pixels Per Frame Organophosphate Pralidoxime Paint Shop Pro Processing Time Per Run Relative Afferent Pupillary Defect Real Distance/ Real Diameter Red/Green/Blue/Sync. Channels Scale Factor Starting Time Total Actual Time Tetraethylpyrophosphate Threshold-Illumination/Time Increment Threshold Limit Value Threshold NBPF Total Processing Time Total Required Time Visual C++ Visually Evoked Response 1 Introduction The main objective of this thesis is to describe a new imaging technique by which to monitor the eye and detect miosis. Design criteria include a non-costly, safe, fixed, easily assembled apparatus by which to assess the pupillary condition of the studied subject. The grabbed images and results should be repeatable, and easily reproducible. The setup should also account for time-critical, or dynamic, monitoring and inspection of the pupil, and of the imaging operations as a whole. Such would be required in a hazardous environment (such as a chemical spill area), where the eye could serve as an early biomarker. The method should, additionally, take into consideration involuntary eye-movement, and blinking in the results, and allow for their adjustment. Future implementation of the technique into a physical model should also be provided. The thesis covers a background about miosis, the currently applied methods for depicting this disorder, and a proposed technique set forth to capture and analyze miosis. In this section, the pupil is described, along with its most common disorders, miosis. The Pupil The external part of the eye consists of three layers. The top layer is the sclera, attached at the front of the eye to the transparent cornea. The middle layer is the choroid, which carries the blood vessels. The retina forms the innermost layer, and is 2 responsible for sensing light. The cornea, in turn, consists of five layers, and serves to light into the internal part of the eye. Behind the cornea lies the lens, separated by a watery fluid, the aqueous humor [1]. Figure 1 Front view of the eyelids, pupil, and other visible parts of the eye [2: Pg. 18]. The lens is responsible for focusing the vision. The pupil lies in the center of the colored iris, which lies between the cornea and the lens (see Figure 1). The pupil 3 has a muscle towards its edge that functions to control the amount of light permitted to enter the eye. The iris thus opens and contracts allowing the pupil to transmit light in adequate amounts, forming an image on the retina [3]. The size of the pupil changes with several factors like age, the color of the iris, and the refractive power of the lens. Young children with blue eyes and myopia are likely to have large pupils. The pupil is likely to smaller, however, with infants and adults. In all cases, the normal pupils should be (both) almost identical in size, round, regularly shaped, and should have a regular response to light stimulation [4]. The diameter of each pupil at usual lighting conditions for a healthy adult should range between 3 to 5 mm. Discrepancies in the size and/or shape of the pupil can result from congenital abnormalities, lesions of the iris, eye defects, or inflicted by a surgical process [4]. The next section covers such eye disorders. Miosis When the pupil contracts, miosis is said to have occurred (or the eye has become miotic), usually with the pupil having a diameter less than or equal to 2 mm [2,5]. A similar disorder is anisocoria, where the pupils of each eye are of unequal sizes [2]. 4 Table 1 A list of some of the agents that cause miosis [7]. Medicinal Agents Antihistamines Bethanechol Chloride Caffeine Chloral Hydrate Clonidine Codeine Dextromethorphan Hydrobromide Edrophonium Chloride Fentanyl Citrate Guanabenz Acetate Hydromorphone Hydrochloride Levomethadyl Acetate Hydrochloride Levorphanol Tartrate Lidocaine Hydrochloride Meperidine Hydrochloride Meprobamate Methadone Hydrochloride Methyprylon Morphine Sulfate Nalbuphine Hydrochloride Nicotine Opium Alkaloids (Hydrochlorides) Opium Tincture Oxycodone and Acetaminophen Oxycodone Hydrochloride Oxymorphone Hydrochloride Paregoric Pentazocine Phencyclidine Hydrochloride Procaine Hydrochloride Propoxyphene Rauwolfia Serpentina Reserpine Valproic Acid and Derivatives Zolpidem Tartrate Non-Medicinal Agents Aldicarb Bromophos Carbaryl Chenopdium Oils Chlorfenvinphos Diazinon Dicrotophos Dioxathion 2, 4D Fensulfothion Fenthion Malathion Methidathion Methiocarb Methomyl Parathion Profenofos Propoxur Terbufos Tetraethyl Pyrophosphate Types of miosis include accommodative, spastic, paralytic, and spinal miosis [5]. Accommodative miosis is one occurring as a result of the pupil’s adjustment to 5 changes in illumination. Faint light tends to dilate the pupil, whereas bright light constricts it. The pupil is also constricted during sleep. Psychological factors such as surprise, fear, and pain can cause constriction of the pupil [4]. An Adie’s or myotonic pupil is one where the pupil almost undergoes no contraction with sudden exposure to light, while constricting to accommodation [2]. This pupillary disorder is often mistaken to that being due to cranial paralysis, or paralytic miosis [6]. Also termed irritative miosis, spastic miosis is one occurring due to severe contraction of the pupil’s sphincter muscle. The hippus, for instance, is an irregular dilation and constriction of the pupil that occurs without a light stimulus. It is initiated by the sudden drop in intraocular pressure, foreign bodies to the eye, hysteria, or drug action. Such drug-induced miosis is caused either by topical medication (e.g. treatment against glaucoma such as pilocarpine), by systemic drugs like narcotics (such as morphine), or by barbiturates [4,6]. Table 1 lists some of the miosis-causing agents commonly found in various industries. They are classified as being either medicinal or non-medicinal agents. Many of the non-medicinal chemicals are neurotoxins and will be reviewed more extensively in the Review of Literature section. Last is paralytic miosis, which is triggered by the paralysis of the pupil’s dilator pupillae muscle, due to lesions in the sympathetic nervous system. With syphilis disease or diabetes, an accompanying pupil disorder occurs, known as Argyll Robertson pupil, where the simultaneous reaction of both pupils is not functional, 6 although accommodation for close targets is slightly active [2,6]. Spinal miosis is similar to paralytic miosis, and it is initiated by lesions in the cervical sympathetic chain [5]. Figure 2 Horner’s syndrome of the right eye; notice the smaller pupil in the right eye, as well as ptosis [8: Pg. 107]. An example of paralytic miosis is Horner’s syndrome (as shown in Figure 2). Paralysis to the sympathetic nerve causes miosis, as well as ptosis (slight sag of one of the eyelids), in addition to lack of sweating on one side of the face [2]. Testing the Pupil Pupillary response can practically be examined using a simple light beam. In this process, the patient is positioned in a faintly lit room, with evenly distributed light. Three factors are studied, how well the eye converges to the target, accommodation, and pupillary constriction. In one test, the light is directly pointed at the pupil, causing the pupil to constrict. The examiner then looks to see if direct constriction of the lit eye was achieved. The unilluminated eye is additionally said to 7 be normal if it also undergoes constriction, where both pupils then have a consensual light reflex [4]. Figure 3 A schematic of the swinging flashlight test [4: Pg. 159]. Another technique is called the swinging flashlight test, where both the direct and consensual reflexes of the same eye can be examined. The process is demonstrated in Figure 3. Light is shone on one eye, until it constricts to the 8 minimum size at which point the magnitude of the constriction of both eyes is noted. The light is then alternated to the second eye. If the unlit eye has less constriction than the lit eye, then the former one is said to have a relative “afferent” pupillary defect (RAPD), also known as the Marcus Gunn pupil [4,9]. When the light source is crossed from the normal to the defective eye, the RAPD pupil may also be characterized by an instantaneous dilation instead of the regular constriction of the eye. 9 Review of Literature This section covers a summary of currently applied techniques for monitoring and acquiring images of the exterior eye. Also included is a review of some of the literature on neurotoxins that cause miosis; these are included to show the hazards of such a short-termed disorder. Information about such cases (where spastic miosis occurs) is required in applications like battlefields or chemical sites where monitoring is essential for the safety of individuals. Pupillary Assessment and Monitoring There are various reasons why monitoring the eye is significant. Such a process serves to supply the researcher with data concerning eye dynamics (or oculomotor data) [10]. It also aids in analyzing information on eye behavior and discarding irrelevant data or grouping related ones. Most of all, in this application, it assists in displaying and detecting eye changes, and in producing an indication of discrepancies, as with the unexpected change of the pupil in some miosis cases [11]. Eye monitoring also has, furthermore, theoretically proven to be important in a variety of applications. These include reading [12], music reading [13], X-ray examination [14], map design [15], learning mathematics [16], playing chess [17], the fine arts [18], mental stress [19], and emotional expression [20]. This can yet be generalized to any field of interest where the eye and vision need to be examined, or 10 even indirectly where mental or emotional expressions have some effect on the eye’s position or the pupillary size [11]. This section covers some of the literature on systems used to generally detect and/or monitor eye dynamics and image. A considerable amount of work has also been focused on assessing and conducting computer-based monitoring of the pupil. Some of these techniques are described next. 1. Classical Pupillometry A pupillometer (also called coreometer) is a device that serves in calculating the diameter of the pupil. The terms pupillography and pupillometry refer, respectively, to the photography of the pupil (or documenting its reactions), and the process used to measure its size. Various types of pupillometers exist, and are briefly described here. These include Broca’s, Haab’s, Laurence’s, Morton’s, Priestley Smith’s, Schlösser’s, and the projection pupillometer [5]. Broca’s pupillometer projections on the retina are produced using two small light beams placed at an eye’s exterior; the distance between the two lights is used as a direct correlation to the pupil’s diameter. Haab’s and Morton pupillometers are two similar methods where a succession of graduated circles are used compared against the pupil to determine its size [5]. Schlösser’s pupillometer also utilizes direct pupillary measurement, where a series of parallel lines on a transparently graded instrument are compared against the 11 pupil. Priestley Smith’s and projection pupillometers use a similar millimeter scale, where in the former the witness would stand with a graduated convex lens of the subject pupil. In the latter, a millimeter scale of the pupil is cast into the plane of the pupil. A caliper-type device is used with Laurence’s pupillometer [5]. 2. Early Assessment Trials The reaction of the pupil has been among the many clinical evaluation techniques for eye disorders such as optic neuritis. Other methods include observations made to the optic disc, visual field defects, or via visually evoked responses. Fison et al. describe a pupillography method of recording pupillary diameters [21]. The readings were acquired in the dark using a Whittaker Corporation Series 1800 binocular infrared TV pupillometer. Light was provided using Sylvania 2-watt tungsten arc lamps, having a 2-mm beam diameter focused on the subject pupil (thus normally would be smaller than the pupillary diameter). The visual threshold of each tested patient was initially obtained using the pupillometer light. The eye of each person was then dark-adapted by wearing dark-red goggles for a 30-minute duration. Afferent pupillary defects were then determined for a group of healthy volunteers, as well as for patients with defective sight using a modified version of the swinging light test (described in the previous section). Here each eye was subjected to 12 a one-second continuous light exposure, totaling to 12 separate stimuli for each pupil. Thus, the test could be termed a unilateral swinging flashlight test in this case. Tests of inducing a virtual pupillary defect on healthy patients as well as with defective subjects yielded expected measurements, as correlated with visual acuity and threshold. However, this procedure has been criticized for its limitation as being efficient in cases where only one eye has a RAPD. Instances of bilateral lesions as in optic nerve diseases are well not simulated due to this restriction as commented by others in this field [22,23]. 3. Videopupillometry Straub et al. describe a computer-based imaging-analyzing system referred to as videopupillomotery [24]. This method is applied mainly to detect the ocular manifestations of neuropathy experienced with diabetic patients [25]. The authors first describe a number of systems already used in industry like infrared receiver and transmitter techniques, as well as video techniques [26,27,28]. Lowenstein et al. were considered the first to use videopupillometry while using infrared LED’s and sensitive infrared cameras [28]. These methods, however, were found rather expensive and were optimally invented to serve pupillometry purposes. The setup manipulates a Genlock device, which is used to present the RGB signal from the computer onto a VHS video device. The arrangement also takes darkness into consideration using an infrared sensitive video camera. The images 13 produced containing the patient’s data (taken before imaging starts) and the eye information are videotaped for later examination. Figure 4 Schematics of the infrared-sensitive camera (Left) and the superimposed image of the eye from the computer screen (Right); note the ellipse measurement tool [24: Pg. 46]. The pupillometry technique as well as the screen representation is shown in Figure 4. The light box consists of a white light produced by a 120-W lamp that is reflected 45º by a mirror. The light then passes by a beamsplitter also at 45º to light up one eye directly. There are also 20 LED’s positioned on a rectangular plate at the eye’s view to illuminate the eye in the case of darkness. The objects at the eye’s target are kept away at a distance of 16 cm, with an illuminated area of 5 cm 6 cm. The computer is set with a BASIC program for analyzing the images, as well as determining the circular or elliptic area of the pupil as seen in Figure 4. The device also allows the measurement of irregularly shaped pupils. The video images are grabbed at the rate of 25 images per second, and thus the pupil size is only measured at separate constant intervals. A screen resolution of 640 by 512 pixels is used. When the device was studied on 20 healthy subjects the device proved to give reliable 14 maximum pupillary area (MPA), as well contraction velocity (CV) readings. The output data for each patient are then stored on VHS videotape for a later analysis. The total cost of the system is about $5,740 (in 1993). 4. Pupillary Monitoring Techniques One of the most restricting factors to date with pupillary monitoring has been the software, or rather the means of processing the acquired images and analyzing them. In most cases, it is the software that is being adapted to the present research equipment available for the application. The continuous growth of the IBM series and other fast personal computer systems has provided computer packages used specifically to assist with pupillometers [11]. Every emerging integrated monitoring system has served its purpose but has its advantages and disadvantages [29]. Nowadays, a typical pupil tracking monitor is one that uses a video-camera system, is relatively user-friendly, provides reliable pupil size information, and takes into consideration undesirable yet inevitable experimental conditions such as blinking. In the past, some of this equipment has proven not to provide adequately rapid sampling rates in some applications, even at a rate as fast as 60 samples per second. This is becoming a lesser problem today, and is being optimized along with the cost factor. McConkie et al. describe a series of programs developed as an interface for a Stoelting Wide Angle Eye Movement Tracker and TV Pupillometer System [11]. 15 This oculomotor/pupillometer monitoring combination is composed of a video camera that is used to grab eye images and assess pupillary movement and size. It has also been designed to suit other eye monitoring equipment in the industry. In addition, Microsoft C language software has been developed as an interface with the computer, and its pupillary size aspects are described next. The program consists of four routines, one for data collection, two for data reduction (one for eye gaze and the other for pupillary size or EEG readings), and one for data acquisition and future editing of the gathered information. The data collection subprogram collects the eye’s position and pupil size data. It also inquires the user of certain personal information such as name and identification. It then handles with the hardware in an active process, whereby the person focuses on a display grid on the screen and this is used to generate the threshold pupillary measurements; this also considers factors such as blinks. The data reduction routine, called the Stream Data Reduction Program, produces an ASCII data matrix of the person’s different pupillary readings. The fourth program finally allows the examiner to go over the acquired data and permits him/her to edit or change any erroneous reading, especially in very obvious cases where the reading is remarkably off the number trend. 16 Miosis and Other Ocular Signs of Organophosphates and Carbamates Organophosphates (OPs) and carbamates (CBs) are two types of anticholinesterases, substances that restrain the action of cholinesterase enzyme by combining with it chemically [30]. Cholinesterase exists in the blood, and is used to hydrolyze (i.e. break down using hydrogen and hydroxide ions) choline esters, which is a base crucial for liver performance. Compounds used to treat anticholinesterases include the directly applied atropine, or systemic cholinesterase reactivators like Obidoxime. Anticholinesterases (antiChEs), beside adverse effects, are used as miotics, i.e. drugs causing miosis, the abnormal contraction of the pupil. They are moreover applied in treating myasthenia gravis, a neuromuscular disease characterized by extensive muscle fatigue, and with ocular signs such as external ophthalmoplegia (paralysis of the extraocular muscles), ptosis, and diplopia. Other uses involve treating glaucoma, the increased intraocular pressure leading to diminished visual acuity, and damage to the optic disk. Neurotoxic agents having effects similar to anticholinesterases are divided into organophosphorus pesticides or insecticides like Malathion and Parathion, and nerve gases like Carbaryl and Sarin. The following analysis is a summary of some literature that treats the ocular signs due to these agents. 17 1. Organophosphates (OP) OPs are used mostly as pesticides. The most common forms include Malathion, Parathion, Tetraethylpyrophosphate, and Trichlorfon. Background Parathion is an antiChE pesticide that exists as a brown or yellow liquid, and has caused several fatalities. nitrophenylthiophosphate, Parathion Thiophos, or is also E605. denoted Malathion as (also diethyl-pDiethyl mercaptosuccinate, Malathon, and Carbophos) is found as a colorless liquid, but is an anticholinesterase of less toxicity than other organophosphates [30,31,32]. As mentioned, various antiChEs are used to treat certain diseases. Tetraethylpyrophosphate (TEPP) is another OP used formerly to treat glaucoma, myasthenia gravis, and pediculosis of eyelashes (the infection of head and body lice to the eyes and lashes, leading to reddening, and watering of the eyes) [31]. Trichlorfon, also known as Chlorofos or Dipterex, is also an anticholinesterase and insecticide applied as an anthelmintic (i.e. used for destroying intestinal worms), and applied mostly as a topical eye agent [31,33]. The following threshold limits are based on the ACGIH TLV designations (1987) [32]. Only 0.1 mg/m3 are permissible for parathion, while the less toxic malathion is allowed a level 10 mg/m3, both determined using skin absorption measures. Supporting evidence was shown in one report where workers exposed to 18 3.3 mg/m3 of Malathion for a duration of 5 hours showed blood malathion levels that did not indicate a decreased cholinesterase level [32]. Exposure to OPs can take place through inhalation, the skin, via ingestion (e.g. Malathion), or by direct exposure of the eye (e.g. Parathion) [31,32]. These chemicals cause restraining of cholinesterase enzymes, as previously mentioned [31,32]. This leads to accumulation of acetylcholine, a neurotransmitter formed enzymatically in the tissues from choline, since the choline enzyme is no longer broken down [32]. Acetylcholine leads to various nervous system, skeletal, and muscular disorders [34]. A common unique disturbance of OP’s is called “delayed distal axonopathy,” which has no ocular manifestations, and only affects the arms and legs. Experiment has shown that this affects mostly animals due to organophosphate poisoning [31,35]. For the eye, the cholinesterase restraining is produced due to both direct (topical) and systemic exposure [31]. Excretion is mostly achieved by liver esterases. For Malathion, this “aging” is done quickly within 24-48 hours [32]. Eye Symptoms One of the most extensive surveys for the ocular manifestations of OP’s and carbamates on humans was made by Erickson-Lamy and Grant in 1992 [36]. Eye signs due to OP exposure are mostly due to systemic exposure, but some local eye toxicity is treated here too. 19 “Saku Disease” A series of poisoning cases known as the “Saku disease” occurred in Japan due to OP exposure. This is regarded as a third type of action due to OP’s (in addition to axonopathy and cholinesterase inhibition), because OP’s have been found here to systemically affect the eyes, but have directly targeted vision [31]. A lot of question, and controversy has been raised about such findings of many poisoning cases in Japan [37], and other parts of the world [38]. Chronic toxicity eye signs were found with school children, as caused by Malathion and parathion [37,39]. Myopia was frequently noted, as well as miosis in some cases. Corneal astigmatism also occurred; this is described as astigmatism, due to change in the shape of the cornea’s curvature. Other adverse eye effects included disorder of eye pursuit movements, constriction of peripheral visual fields, irregular ERG readings, and optic atrophy. Extensive treatment with pralidoxime resulted in full recovery, supporting that ocular signs could have indeed been triggered by pesticide poisoning [38]. Here are some reports summarized by Grant [31]. Workers spraying OP insecticides including Baytex, Temophos, and Paris Green encountered declined visual acuity and pigmentary maculopathy (i.e. pigmentary disorder of the macula lutae) in 14 out of 64 of the cases [40]. Such disorders of the retina were due to extended exposure to OP’s. Macular disorders were also discernible in a similar case with 15 out of 79 others spraying fenthion, another OP [41]. 20 In a suicide attempt using diazinon, miosis appeared without optic nerve damage, associated with cranial palsies [42]. A controversial case is also recorded of a patient exposed to Bromophos, where 15 days after exposure cease, iridocyclitis (inflammation of the iris and ciliary body) appeared [31]. About one-third of 232 people exposed to an organophosphorus pesticide experienced blurred vision, occurring almost immediately after onset, and lasting in one patient to about two days after exposure ceased [43]. Claims are that vision effects could have been due to poor circulation at the basilar arteries (located towards the optic chiasm) [31]. In an attempt to review the symptoms associated with this strange phenomenon, scientists have tried to experiment with Saku OPs on animals. Reviewers, however, have debated the authenticity of these tests. Also Saku OPs, ethylthiometon, fenitrothion were chronically induced to Beagle dogs over a period of two years [44]. Chronic signs included myopia to 1.25 of recorded values with controls, and degeneration of ciliary muscle cells those on the annular ring of the lens. Partial necrosis of retinal pigment epithelial cells (known as Muller cells) also occurred. Furthermore, the number of nerve fibers in the optic nerve decreased, with a sudden increase of glial cells, while the count of ganglion cells remained normal [45]. Tests on other animals were also conducted. Rats injected with fenthion showed deteriorated retinal pigments and irregular ERG results [46], a symptom that has also occurred with mice given doses of parathion, mevinphos, and malathion [47]. Chlorfenvinphos administered to cats similarly affected ERG readings and VER’s [31,48]. 21 Figure 5 Acute angle-closure glaucoma; severely high intraocular pressure resulted in eye inflammation, in addition to corneal edema and a partially dilated pupil (note also the disturbed light reflex) [49: Pg. 142]. Acute Exposure Eye disorders of acute exposure to OP include miosis, upbeat nystagmus, deteriorated macula [50], and ocular bobbing, the downward periodic movement of the eyeballs, as caused by Diazinon, for instance [51]. For more severe cases, acute angle-closure (also closed angle) glaucoma usually occurs (Figure 5) [52]. This type of glaucoma can be induced either topically or systemically, and is very rare. It is characterized by an unbalanced increase in intraocular pressure associated with 22 blockage of the anterior chamber, and the result is an inhibited passage of aqueous humor out of the anterior chamber. This more commonly takes place with patients having liability to getting it (i.e. with a thin anterior chamber), and one having a small anterior angle. In an unusual case of ten humans exposed to OP’s, the aforementioned acute signs occurred prematurely, while paralytic neuropathy appeared 2-4 weeks later in the extremities [53]. Other Effects Here is a brief account of the eye symptoms for some of the previously mentioned OP’s. Malathion is known for causing miosis, lacrimation, and disturbance of far vision [32]. Miosis has been among the most pronounced systemic symptoms, also occurring with parathion. Eye adjustment difficulty occurs also for close distances, accompanied by an irritation of the external eye, and haziness of far images [31]. In addition, pupils systemically become extremely narrow and sharp, with hazy eyesight, although there are cases where they have not been constricted [54,55]. Acute systemic exposure to parathion causes the pupils to become wide, expanded, and unresponsive. In cases, the pupils have expanded from extreme constriction to severe dilation, possibly occurring with extended fainting [31,56,57]. In one incident of arguable authenticity, a farmer appears to have been exposed to parathion, and consequently shown to have optic neuritis [31]. Also arguable, a child poisoned with parathion showed constricted visions and scotomas. However, histological aspects as well as vision appeared to be normal. 23 TEPP and trichlorfon have similarly caused miosis with eye irritation. However, miosis lasts with trichlorfon for only a brief duration [31]. Eye adjustment difficulty is also noted with TEPP for close distances, accompanied with eye irritation and haziness of far images [58]. This is normally associated with neurological, muscular convulsions, and gastrointestinal disorders. Other Symptoms Some of the general symptoms experienced with OP’s involve muscular and abdominal convulsions, diarrhea, and hallucinations, all chronic signs of OP exposure [43,59]. Acute signs are well demonstrated by the formerly mentioned case study where ten humans were poisoned, whereby paralytic neuropathy occurred in the limbs. This is a degeneration of the nervous system eventually leading to paralysis [5,31]. Prevention and Treatment Prevention of OP intoxication is achieved by monitoring the level of cholinesterase in the blood. This is a currently applied method, used on an annual basis. Initially, a record is kept for each worker indicating a pre-exposure level of this enzyme. This is watched so as not to drop below 40% as compared to that before initial exposure. Patients who drop below this are removed away from the exposure source until the level of cholinesterase returns to about 80% of the initial value [32]. 24 Pralidoxime is a popular antidote, also named 2-PAM, Protopam, or 2pyridine aldoxime. Applied in its is chloride, iodide, mesylate salts, it is induced systemically, and acts as a cholinesterase stimulant to treat intoxication caused by organophosphates, nerve gases, and miotics [31,32]. PAM is administered orally, intravenously, or topically to the eye. Orally, one g of pralidoxime mesylate plus 2 mg of atropine, and 10-mg metaraminol have been found to cause no visual or other side effects [60]. For intravenous induction, a 15-30 mg/kg concentration of pralidoxime iodide is used, with minor effects like distorted vision lasting up to about half an hour, as well as nausea [61]. These side effects have not been reported as being too serious, or enough to discontinue using the antidote [62,63]. Topical application is used to treat miosis and cyclotonia triggered by anticholinesterase action; this takes a few hours to act, with no irritation found to the eye at 5% concentration when used as eyedrops [64]. Figure 6 Using atropine to treat miosis. Left: Miosis due to corneal foreign body in a dog; Right: Mydriasis after treatment with atropine [65: Pg. 405]. Directly applied antitoxins used to cure OP symptoms include anticholinergic eye drops (i.e. those inhibiting the action of acetylcholine) and atropine (referred to as 25 dl-hyoscyamine). Atropine is similarly an anticholinergic agent used to dilate the pupil and treat the eye’s adjustment for close distances. Figure 6 shows a rare photo of an animal suffering miosis that was relieved into mydriasis using atropine. In a case with TEPP intoxication, constricted pupils treated with excessive amounts of atropine returned to their normal size gradually as the rest of the body systems recovered [66]. 2. Nerve gases Two of the most common forms of nerve gases are carbamates and sarin gas. Their eye symptoms are briefly treated in the following analysis. Background Carbaryl is also named Sevin, Arylam, and 1-Naphthyl-Methylcarbamate. Natural carbaryl is frequently produced in the bodies of warm-blooded vertebrates [31,32,67]. It is generally not one of the most hazardous antiChE to the eye, although it is usually grouped among cholinesterase inhibitors and pesticides. Sarin, also known as methylfluorophosphonic acid isopropyl ester, is another anticholinesterase inhibitor and nerve gas [31]. Carbaryl has a TLV (based on ACGIH 1987 documentation) of five mg/m3. Carbaryl’s anticholinesterase action occurs as a rapid, reversible process, accumulating acetylcholine consequently, as mentioned with organophosphates. Both 26 carbaryl and sarin act to limit the size of the pupil, leading to various visual defects [31,32]. Eye Symptoms Miosis, lacrimation, and diminished vision are usually encountered with systemic carbaryl poisoning [32,67]. The ingestion of carbaryl at a concentration of 0.5 g per kg, led to acute eye symptoms such as miosis, and disturbance of oculomotor movement including rapid spontaneous eye movements, and rapid eyelid movement [68]. Other symptoms included the loss of sensation and salivation. Ocular symptoms appeared within three hours of administration in this accident, while muscular convulsions were delayed up to 12 hours. Peripheral neuropathy appeared after about two days, with loss of feeling in the arms and legs, while no ocular signs occurred within this period. The patient recovered partially after 9 months. Such symptoms are not common to carbamate poisoning. A brief study by Farago describes a suicide case where vision disorders occurred, but the method of disturbance was not demonstrated [69]. With mice, the induction of carbaryl through the abdomen resulted in distorted ERG (Electroretinogram)1 readings. Topical carbaryl also induces some eye symptoms. In one event, a mixture of carbaryl and dimethoate (not an eye irritant) splashed on a person’s eyes [31]. Damage to the corneal epithelium was noted, as well as inflammation of the eyelids. Full recovery was attained shortly. The intermediate 1 ERG is a graphic electrical record of the electrical activity recorded from the surface of the eyeball. 27 compound produced during the production of dimethoate, also called bis (dimethoxythiophosphoryl) disulfide, another nerve gas, has been tested on the eyes of rabbits. Signs have included corneal opacities, in addition to swelling and edema of the conjunctiva, and lids [31]. Sarin also causes miosis, but one that usually does not persist for a long time. It also causes refined, constant eye accommodation for short distances (or cyclotonia), as well as restricted visual accuracy in faint light [5,31]. This is due to the reduction in the size of the pupil, thus allowing less light to enter the eye [70]. Some research by Rubin argues that the inhibited vision with decreased illumination is perhaps not attributed to miosis, but could be due to some other CNS disorder [71,72,73]. Sarin gas, however, has in some occasions produced extreme miosis as an immediate reaction to acute exposure [74]. The pupillary stimulus started 11 days following exposure in one such incident, and recovery took place after 30 to 45 days later. The byproduct of Sarin production, Difluoro, has been noted to cause eye irritation, and corneal opacity, following direct exposure [31]. Other Symptoms Some of the common symptoms accompanying exposure to nerve gases include dizziness and loss of balance. Sweating, salivation, are often noted, as well as a unique change in the color of the nails [67]. 28 Prevention and Treatment Blood levels are also used to determine exposure levels associated with nerve gases, with similar exposure indices as with OP’s. In addition, measuring urinary levels of 1-naphthol (a metabolite of the reaction involving carbaryl excretion) are commonly used to monitor carbaryl levels [32]. Atropine is generally applied to treat carbaryl, instead of the regularly used cholinesterase activators [69,75]. PAM, for instance, is usually not utilized due to its inherent toxic action [32]. For sarin gas, miosis and cyclotonia are treated topically with cyclopentolate eyedrops, responsible for adjusting eye accommodation, although with some doubt about its efficiency [76]. 29 Text of Investigation The aim of this research, as outlined earlier in the report, is to design and test a practical experimental setup for scanning eye images (or grabs), and reporting abnormal pupillary reactions, with emphasis on miotic changes. The photos have been taken using a standard B/W (i.e. black and white) Burle camera. Additionally, a commercial Matrox Meteor/RGB hardware board has been used, utilizing an applied C-language package called MIL-Lite® 6.0. The C programming medium used for this process is Microsoft Visual C++® 5.0. The code acquires images to the PC for later processing. The C program takes advantage of the various MIL-Lite commands in order to process the grabs, electronically monitor the size and status of the pupil, and report minor and/or major problems to the user. The aspects and adopted procedure of this setup are described in the following section. Figure 7 shows a simple representation of the system used for capturing and processing eye images, with the primary aim of detecting changes in the pupillary size. The figure has been adapted from a proposal of a pending patent, which has a more general concept of monitoring the dynamics of the eye [77]. The arrows indicate either output orders produced by the code or data input to the program through the various flow-components. To summarize, the flow starts at an illumination source situated near a stationery eye. An infrared (IR) source has been designed for this purpose in such a way that it lights up the eye without affecting the pupil's size, i.e. without causing the 30 pupil the react to the light by either contracting or dilating. This is all recorded by an IR-sensitive camera, which captures a sequence of frames and sends them to the PC. Apparatus and Experimental Setup 1. Illumination Source 2. Eye 7. Data Processing Software 6. Computer Code 5. Digitizer 3. B/W Camera and Lens 4. Camera to P.C. Connection Figure 7 A simplified schematic of the experimental setup used for grabbing and processing images of the eye. The big arrows going towards the eye indicate orders sent from the program, and the small arrows going from the eye indicate feedback going into the program. The grabbed images are then transmitted (via a special hardware connection) to the digitizer board, whose main function is to convert the data coming through the different channels to a format that the PC can manipulate or process. This imaging 31 board computer-chip also comes with additional software to enable on-screen tracing of the grabbing process. In addition, the manufacturers have also created an imaging library, with advanced C-language image-processing commands that enable the programmer to interface with the camera and the other grabbing procedures. Once a single grab is transmitted to the program, the code sends an order (as indicated by the larger arrows) for a new image to be grabbed. The cycle repeats for a user-specified time. This control code is then used to save data to file. Once all the images have been grabbed and processed, the data file is finally fed into data- and image-processing software, where the information can be illustrated and analyzed through various charts. Following is a more detailed description of each of the components in Figure 7. A general view of the experimental components is shown in Figure 8. Figure 8 Side views of the final employed experimental setup, showing the individual components. Left: Note that monitoring the experiment was made possible by both a PC-display (right), as well as a color monitor (left). Right: subject during one of the employed tests. 32 1. Illumination Source Illumination used in this research has been possible using two forms of illumination, ambient room and artificial light. Infrared light is used primarily to illuminate the eye in dark conditions. In most applications several IR diodes are gathered on one plate and used to light the eye; usually this plate is positioned to indirectly face the eye using reflective optics, as was shown in the sketch in Figure 4. Figure 9 Partially designed infrared flashlight used to light the pupil during the experiment. Left: front view of flashlight showing the commercial diodes used (one is not functional). Right: internal view of the flashlight, showing the parallel connection from the diodes to two AAA, easily replaceable batteries. Such low-frequency radiation has proven in most current practical situations to be efficient and below an acceptable degree of hazard to the eye and the human safety in general. In addition, increased or decreased intensity of IR light does not cause the pupil to dilate and contract as visible light would [24,78]. This causes it to be a useful control factor in assessing other changes in the pupil, such as change in 33 size due to movement, while maintaining an unchanged size due to increased luminance. Figure 9 shows an approximately real-size photo of the partially designed IR flashlight that was utilized specifically for the experiment. The flashlight consists of 12 IR diodes clustered together to produce the most amount of focused light. The diodes have been chosen to be all of the highest viewing angle2, and maximum output voltage. At the opposite side of the flashlight case is the digital switch for the device (not visible in the figure), as well as a red-colored light-emitting diode to ascertain whether the device is on or off. The infrared diodes have been assembled in a parallel fashion so that if one becomes dysfunctional, the rest will still be operational. Indeed, one diode has become impaired by time (as marked in Figure 11), while the rest are still working. The diodes have been powered by two AAA batteries, and placed in a battery-case in order to be easily replaceable. The importance of using infrared light is represented in Figure 10. First, an image of a subject's eye was taken using regular ambient light 3. At this light, notice how the eye is hardly visible, and the pupil is hard to distinguish from the iris and the rest of the eye. The image to the right of that is that taken with the IR flashlight. The 2 Beam angle: 70; VF Max: 2 V @ 20 mW/cm2, and 100 mA. All the experiments were performed indoors, and the room ambient light used has been set to a (fixed, uncontrolled) medium level in all the studies. 3 34 pupil is clearly distinguishable, and appears as the darkest part of the image, which is a one of the goals of the experiment. Figure 10 Impact of using infrared vs. regular ambient room light solely. Left: photo of the right eye of a subject taken at regular ambeint light. Right: photo of the same eye, under identical conditions with the added infrared light (note how the diodes are creating a reflection at the right of the pupil). The image also shows some minor reflection of the infrared light on the eye, as marked by the white spots to the right of the pupil. This drawback has proven difficult to be altogether avoided since most of the diffusers experimented have faded the infrared light significantly. However, the flashlight has been placed at such a distance and angle to the eye so as to minimize the size of the reflection, and to avoid it from being in the course of movement of the pupil. The reflection in Figure 10, for instance, has been adjusted to be outside the boundaries of the pupil in its forward or normal position. Extra illumination has also been manipulated in this experiment in order to determine the minimum size of the pupil of each subject, as a response to very bright light (on the visual spectrum), as will be shown later. The device used for this 35 purpose is a ILC 402 Xenon High Intensity Lightsource. Due to the apparent hazard involved, the Lightsource has been kept from the target eye at a considerably large distance of about 50 cm from the subject. In addition, the duration of exposure has been minimized, i.e. long enough for only a single reading to be achieved. Figure 11. Artificial light-sources used in the experiment. Left: IR flashlight close to the chin rest and close to the position of the subject's right eye; the faulty diode has been marked. Right: The Xenon Lightsource kept away at a considerably large distance. The positioning of both the main light-source (the IR flashlight), as well as the artificial light (the Xenon ring light) is displayed in Figure 11. The flashlight has been setup to provide light to the eye at a close distance, without interfering with the subject's viewing, or with the camera's field of vision, since it has been zoomed-in to solely grab photos of the eye. Note how the "bad" diode has been marked in the image, as mentioned previously. The artificial light, is not a major component of the experiment, and so has been kept at some distance. The large control box behind it is designed for manipulating such factors as the intensity and magnitude of the shone light. 36 2. Eye One of the aspects of this research is that the subject eye is not treated with any chemical (e.g. Mydriatics like atropine) and is not put in pre-conditions or accommodations; the eye is just monitored in its normal state. The right eye –a random choice– has been used in all experiments. In order to unify the direction and accommodation of the pupil throughout the experiment, a sheet of white paper has been placed in front of the subject eye at an adjustable distance. The paper has a black spot (as seen in Figure 12) at which the subject is asked to concentrate on throughout the examination. Doing so helps to get the person to feel relaxed, and minimize his eye's gaze (looking for a point on which to focus). This also aids in capturing a clear, stationery image of the eye on the screen, and has been found to decrease blinking rate. Blinking is indeed an inevitable obstacle in such tests, but the speed at which it occurs has helped to overcome the problem as will be discussed in the programming section. Furthermore, most of the subjects have been chosen as ones wearing eyeglasses. This choice has come about after realizing that for those who look directly into the camera, a reflection of the camera on pupil is visible on the grabbed image. Thus, the concavity of the lens on the eyeglasses has served in removing this reflection, without the lens itself appearing on the grabbed images. Some trials have proven successful even without the use of eyeglasses. 37 In order to fix the position of the eye all through the tests, the head of the subject is positioned on a chin rest, as the one shown in Figure 12. This component is a professional device used in ophthalmology, and has the asset that the height, and angle, and other degrees of freedom can simply be adjusted for different people. Hence, the camera and flashlight and other components can be still in the process. 3. B/W Camera and Lens The camera used is a B/W (or monochrome) TC651 Burle camera that works at specifications of 120 VAC, 60 Hz, and 4.5 Watts. Figure 12 shows a close look at the camera, its fixtures, as well as a general view of the grabbing setup, including the flashlight, and chin-rest. The lens attached to that is a Japan Lens 18-108/2.5 Zoom Lens4. Both the camera and lens have proven to be highly sensitive to IR light. This and their high zooming and focusing capabilities, as well as clear black and white images (thus well suited for the pupil), have made this equipment a highly recommended selection for this application. All results shown are those produced using this camera/lens combination. Although not clearly visible, the focus and zooming settings have been taped down (at the bottom of the zoom/focus locations) to prevent an accidental distortion of the readings during the experiments. Actually, the camera is stationery positionwise and adjustment-wise, except for minor changes at the beginning of each 4 This signifies a close focus zoom lens of focal length of 18-108 mm, with a corresponding 6:1 zoom ratio; the aperture range is F2.5-16C (C means closed). 38 experiment, depending on the size of the eye of each subject. The positioning adjustments are taken care of using the chin-rest, as described earlier. Figure 12 Assembly of physical components in the experiment, the chin rest, IR flashlight, camera, and relevant fixtures. Left: the Burle, IR sensitive camera has been positioned as to be completely stable during all experiments; Right: a clearer view of the image-capturing setup. 4. Connections and Display The Video Out port of the camera has been connected to the computer digitizer via a special DB9 (from the digitizer end) to a 4BNC (at the video or camera end) coaxial cable. The signal from the camera is output as one continuous monochrome signal. Therefore, the individual red (R), green (G), blue (B), and synchronization (Sync) channel lines have all been hooked together using coaxial cable pins. The various channels and connections have been photographed in Figure 13. Since the utilized camera is B/W, each of the four cable lines carries the same monochromatic signal to each of the respective BNC pins in the digitizer, as will be demonstrated in Figure 15. 39 Figure 13. Connections and channels used in the experimental setup. Left: the four rightmost BNC pins represent the R, G, B, and Sync channels. The lowermost pin represents the terminal end of the 9-ft coaxial cable; the white connection (on the left) represents the line going from the camera to the TV monitor. Right: Meteor connection to the PC. Note the (4) individual channels. An extra 9-ft coaxial cable has been hooked from one side to the camera's Video Out terminal, and from the other to the bundle of coaxial pins going to the digitizer input. The purpose of this additional connection length is to allow the camera freedom to be positioned remotely from the computer as much as the application dictates. Also attached to the pin-bundle is a connection, drawing the camera's output to the TV monitor. Thus the grabbed images are viewable both using the imaging board and the TV (as was shown in Figure 8). It is probably relevant to mention here these utilized display screens, shown in Figure 14. The main aim of using a TV monitor is being able to continuously monitor the input to the camera, the subject's eye. The TV is a 14-inch Panasonic®, Color Pilot; it produces clear images, and avoids the processing lag experienced in the digitizer display (i.e. similar to the continuous mode described next). 40 Figure 14 On-line display is possible using the TV monitor (for the subject pupil, left), and PC monitor (for data and images, right). Display on the PC is possible using a 20-inch (displayable size) Apple Multiple Scan monitor. When running the program, two display windows are viewable, an MS-DOS output window, and a MIL buffer display screen. The DOS screen is the usual VC++ output window for displaying data and results. The MIL display screen is a screen affiliated with the MIL package specifically intended for displaying graphics, although showing text is also possible. In any case, both display screens have been concealed from the subject's visual fields so as not to affect his attention, which could change the pupil's response. 5. Digitizer The digitizer is called Matrox® Meteor/RGB. It is designed to work on a Windows® (e.g. Windows 95® and Windows NT®) and MS-DOS® computer operating systems/ mediums. The Meteor board has a frame grabber for both monochrome (which is the case in this study) and colored (PCI) images. A simplified diagram of the connection into and inside the digitizer is shown in Figure 15. 41 Figure 15 Matrox Meteor/RGB digitizer-board flow diagram [79: Pg. 220]. Meteor does not contain any on-board memory or display, but all grabbed images are automatically handled by a Host memory buffer (for saving the data), or a VGA frame buffer (for displaying the grab on the screen) [79]. Since a B/W camera is used, the images are grabbed in RS-170 format, which is the standard monochrome video format in the U.S. Frame scan cameras (such as the Burle camera) can be controlled in the following modes: continuous mode, pseudo-continuous mode, trigger mode, asynchronous reset mode, control mode, and long exposure (integration) mode. In continuous mode, the camera continuously outputs images at 30 frames or 60 fields per second (North American timings), where each frame consists of an even and an odd field [80]. That is, if the camera were to transmit pictures “non-stop," a maximum of 30 consecutive grabs per second would be permissible to match with the 42 processing time of the digitizer (i.e. about 0.03 seconds of processing time per image). The trigger mode is used to grab a separate image or a sequence of images. Both the continuous and trigger modes have been used in the program, with more emphasis on the latter mode, as will be described in the following section. The MIL (Matrox Imaging Library) package is the library of the different image-processing functions and commands, designed especially for the Meteor board and the series of other digitizer boards manufactured by the same company (Matrox®). The terms MIL and MIL-Lite are used synonymously throughout the thesis report, since the latter (which is used here) is just a simplified version of MIL, and has most of its capabilities. MIL manipulates the grabbing of eight-bit grayscale or color images through the Meteor board. Generally, the color format consists of black-and-white information (luminance) plus color information (referred to as chrominance or color subcarrier). The magnitude of each pixel value is indicated using an n-bit system, where a value between 0 and 2n represents the magnitude of the brightness of the pixel [80,81]. With the increase in the value of n, more information is possible. In the case of the addressed eight-bit system (since a monochrome camera is used), pixels are allowed to have 256 (28) different values, ranging from 0 to 255. Thus if n is increased, more shades of gray would be available, which results in a more accurate representation of the grabbed image. 43 More information about the grabbing mechanism, the Meteor digitizer, and other image-processing details are available in the various manuals that come with the Meteor board [79,80,81]. 6. Computer Code Following is a description of how the various described features of the hardware/software have been put to use into a control program for monitoring the pupil. The program is ideally visualized as one that determines beforehand the predicted normal (or threshold) size of the pupil for different lighting conditions, and for a certain duration “keep an eye on” changes in the size of the pupil. It should be acceptable that usual eye accommodation changes, blinking, and normal eye movements take place, and these should not be reported. However, if the pupil constricts or dilates above a certain percentage of the initial threshold, the program should identify and report the error. It should then prompt the user or whoever is supervising the monitoring that means should be taken to correct this situation. MIL Package When the problem at stake was first examined, three perspectives were thought to be viable to preparing this program. MIL-Lite is designed to be compatible with the programming environments Microsoft Visual C++®, and Microsoft Visual Basic®, while it seems the developers have made most of the applications to more suit Visual C++. The three options were to either use a VC++ code to directly acquire 44 images through the Meteor digitizer. Another was to use a C code written in the Visual C++ environment. The third alternative was to manipulate Visual C++ with a code written in the C format as well as the built-in MIL library to simplify the programming task. The latter solution was adopted. The MIL package has the benefit that it is specifically designed for “image acquisition, data manipulation, graphics, and display control” [81: Pg. 14]. It can work with either grayscale or color images, as described. It deals mainly with buffer, or active data-storage locations. It has built-in image buffers, which handle the images to be grabbed, displayed, and analyzed. The display controller, for instance, is another displayable buffer that shows manipulated data that one may access in a program. Once these and other system buffers are allocated (or declared) in the code, the MIL environment is activated or initialized so that the various commands can be manipulated. The defaults must be released at the end of the code. Most of the MIL commands are accessed by calling a header file called mil.h which must be linked in the code, just as (say) the standard input-output header file is linked in a regular C program. Next is a list of the some of the most important commands and functions that have been utilized in the program. The remaining are either default settings or regular C commands. Perhaps the most important MIL command used in the code is the MdigGrab(MilDigitizer, MilImage). This tells the digitizer about a MIL command that Grabs a single image and the parameters or arguments between parentheses (as 45 with the any other C function) are the digitizer, and image buffer identifiers, respectively. Images can be grabbed in continuous mode using a similar command, MdigGrabContinuous(). The MbufGetColor() function accesses and assigns to an array the contents of an input channel (e.g. the red one in this program). In this case, a double array has been used to access all rows and columns of the 640 by 480 pixelresolution images. MbufPutColor(), on the other hand, is responsible for placing the contents of a certain channel onto the image buffer window. The current time is indicated using the MappTimer() function. Images can be saved while the program is being executed using the MbufExport() command, more commonly in the tiff format. Another command is the MgraText() which writes text to the chosen buffer. More information about these commands, the manipulation of buffers, and other relative information can be found in the manual, MIL-Lite Version 6.0. User Guide and Command Reference [81]. Program Algorithm The listing of the code is included in Appendix 1. The program has been developed using menu-like structured programming (in C language) where the user is provided a menu of different options. It has been tested for various running conditions, and has been thoroughly checked for errors. Ample comments have been included to show the meanings of the various statements. Three types of comments have been added to the program. 46 Comments in the beginning of the program give a general view of the purpose of the code, as well as how to use it in a most efficient manner. Remarks have also been attached to the start of each subsection, or function in the program, serving as a tool for those intending to glimpse the main structure of the program. This type of comment is thus not highly concerned with the significance of the individual statements. Comments have also been implemented on a near command-to-command basis, where each briefly describes the purpose of the statement, and perhaps the meaning of the used variables. Variables names and pointers have been chosen to reflect the intended use of each. The variable lumin_sum, for instance, is a double-accuracy float variable whose purpose is clearly to carry the sum of the luminance values for a certain image. Program Objectives This program is ultimately intended for observing and reporting the size of a person's pupil. The aim is to first determine a threshold-illumination (TI) curve for each subject. The concept behind this is that the darker the grabbed becomes, the lower the threshold lighting has to be in order to view only the pupil. In other words, the pupil is expected to be the object of the lowest luminance in the image (since it absorbs light). Thus a darker image dictates that the lighting level drop to a lower level than normal in order to isolate the pupil from the rest of the objects. An analogy to that is the normal accommodation of the eye when a person moves from a highly lit environment to a darker one (e.g. a movie-theatre). When the 47 person enters the dark medium, everything appears dark at first, but once the pupil thresholds down to the appropriate light level, the clearest objects will then become visible. In a comparable (but negative) fashion, the image is inspected for the 'blackest' region using this threshold technique. For a higher light level, it is similarly required for the pupil to adapt to the higher luminance in order to cut it off from the rest of the eye's regions. Some commercial cameras indeed achieve this function automatically, but this has been attempted manually to attain the most feedback for each subject. The TI fit is one in which every point on the line tells the current average lighting (illumination), and the corresponding pixel value of the light required to isolate (or threshold) the pupil from the rest of the eye. The relation is expected to be a direct varying correspondence between the threshold and luminance, but not necessarily a linear one. This curve should then be implemented back into the program. The TI curve is to be tested in a number of ways. For the same person, the threshold test (described in the preceding analysis) is applied for different times, in each producing a separate TI curve. Then the size of the pupil for different lighting conditions is examined for consistency (light test). Another test is the eye-movement test. Here, the subject is asked to move his/her pupil is different directions, and the program tests to see if the pupil is still being isolated, and again determines the size of 48 the pupil in each position. Finally, the program is tested for the variability of the TI curve for different people. These tests are to be described in detail in the next section. Threshold-Illumination Routine The program first cross-references the MIL library file and other files to be accessed by the code. It declares and assigns values to certain variables and pointers (like the PC clock and monitoring flags) and arrays (a double array to store the red channel input luminance values, for instance). The program then allocates MIL defaults; i.e. it sets the MIL environment and gets it ready for processing. The following is a more detailed description of the actual statements and function of the program. An abbreviated version of this analysis is offered at the introductory remarks section of the actual code (Appendix 1). The first two options are provided from the menu of choices to the user as a test for the general pre-grabbing conditions. The first menu option is designated to grabbing a single image, while the second is for grabbing images continuously. Such is used to see if the pupil is in the required position, is sufficiently stationery, or at the proper lighting, before starting the actual grabbing process. Options 6, 7, and 8 are concerned with developing the TI curve, the first phase of the monitoring process. Thus, this should be accomplished and tested for before the technique is actually put to use. For each illumination, a series of images are grabbed using option 6. Each time, the average of the pixel lighting-values is 49 determined. The user is provided with the number of averaged points in the image, and their global average illumination. Normally, the total number of points for such a 640-by-480 resolution would be exactly 307,200 points. However, it has been observed that the grabbing mechanism can erroneously5 leave the image with partially black or distorted boundaries, a phenomenon that can seriously influence the data. In accord, the first five pixels have been excluded from the image. Thus only pixels 5 through 635 (in the horizontal direction), and pixels 5 through 475 (in the vertical direction) have been used for reading from and writing to the various channels. This reduces the number of pixels to 296,100, all of which have been used. Still in option 6, the user is then asked as to whether the grab has been successful or not. If not, the data values are erased, and if acceptable, the data is written to a text file. After the user has taken a number of grabs that he finds are sufficient to estimate the current lighting, option 7 can be used to view the number of grabs and the corresponding average lighting. At this point, the program performs a further averaging for all these (average-illumination) values and provides the result to the user. The user here can easily determine whether one of the points was too high or too low, since the values are displayed in a tabular fashion. Errors can naturally occur in cases where the subject has blinked, or the eye has wandered off from the target image. Note, option 6 can be returned to at this stage for further grabs. 50 At this stage, the data should be saved to file for further examination. Once all images have been grabbed, option 8 is chosen so as to prompt the user for the name of the file to be saved, representing the last grabbed image (in .tif or .tiff formats). In addition, the user is also asked for the name of the (.txt) file in which to save the summary list described in option 7. Finally, the universal average of the current lighting (plus index) is automatically written to a file called "result.txt." The user now chooses Quit from the choice menu. After a new adjustment to the lighting condition, the user now reruns the program, and repeats all the steps described so far (i.e. options 1, 2, 6, 7, and 8), as required. This is repeated for as many luminance settings as the user requires. The user at this point should access the individual luminance-reading files and make sure that all the data is consistent. If a point is off that wasn't adjusted in option 6, then it should be removed, and the new global average should be manually calculated and written to the file, which should be saved for changes. Furthermore, the "result.txt" file, carrying all the global averages, is accessed to make corresponding adjustments. At this point, the TI curve should be created. The user should open the result.txt file using spreadsheet software (like Microsoft Excel®). Meanwhile, the saved images in tagged-format are to be converted to bitmap files (.bmp). Imageprocessing software (like Scion Image®) should now be used to determine the current threshold of the pupil. The latter step should be done for each of the images saved 5 The source of the error has not been determined, but it is apparent that the cause has been some kind 51 from the program. Each of these threshold values should be substituted in a corresponding column for each of the illumination values. A scatter of these pair of values is now plotted, with the threshold on the abscissa, and illumination on the ordinate. Excel has a regression tool for curve-fitting a line through the points and providing the corresponding equation (T as a function of I). The user copies the formula and simply pastes it in option 9, the monitoring option. This uses a method referred to as the Black-Pixel technique that will be described next. Every time a new equation is added, the existing formula is commented (for possible future use) and the new formula is added, in addition to the subject's name, and date of the experiment. The curve-fit can initially be tested for using option 4 (where a copy of the equation must also be provided). After a single image of the eye is grabbed (now at any illumination) using option 6 in the menu, the user chooses option 4 to threshold the image. Accordingly, the program calculates the average illumination and automatically substitutes in the TI equation to determine the required threshold. It then thresholds the image (as would be achieved with Scion Image) and shows the pupil in blue to the user, in the MIL buffer display window. This image can be saved by choosing option 3 from the menu, which saves the current displayed image to file (again in .tif format). Note, that option 3 is not associated with the threshold option, and can be used to save images grabbed by of malfunction in the digitizer. 52 either of options 1, 2, 4, or 6. Option 5 has been used solely for testing purposes. It was used to experiment with MIL's abilities to access individual pixels on the screen, and to change the color of a pixel or group of pixels from one color to another. This option could prove very useful with a pupil edge-follower algorithm. It is apparent from the previous analysis that the program is not fully selfsufficient. The code also relies on the use of additional pre-processing software such as Scion Image (for threshold images), Paint Shop Pro (or PSP for file conversions), and Microsoft Excel for plotting graphs, and further data analysis. Nevertheless, the program is autonomous throughout the entire monitoring process, allows flexible, menu choices, and performs some minor image and data analysis. Reliance on other takes place either before starting the monitoring (for calibration purposes), or finally for plotting the results. Both PSP and Scion are free downloads available on the Internet. Thus, the extra cost involved is that of purchasing Excel, an essential dataprocessing tool, and one of the most commonly used spreadsheets. Processing Time Option 9 is now used to monitor the pupil size. The user is prompted for the total time that he/she wishes to monitor and measure the pupillary size and the required increment of time (or delay times) between each grab. The processing time has been taken into account here, and has been added to the time for each grab. The purpose of doing so is to give the user the “feeling” that the processing time is zero, while the total time is actually increased for this lag. 53 Delay time is not exactly equal to that required by the user, because there is an inevitable processing time. This includes the time required for the order to be received from the user, sent from the digitizer to the camera, the grab to be taken, and the image to be transferred from the camera to the program. The frame rate of the camera, as mentioned, is equal 30 grabs per second; it thus requires about 0.03 seconds to grab one image. In addition, if required, time is essential for the image to be displayed and saved to file. Note the time for the processor to execute the code, and display the image is a parameter that is processor-dependent and will decrease with an increased processor speed. The processing time has been calculated using experimentation. A run of the program execution was attempted for a total required time (TRT) of 100 sec, with a 0.1-sec time increment (TI) is considered here as a sample of this test. The program has been set with a stopping criterion as the total number of runs (i.e. total time divided by the time increment = 1000 runs). The starting time (ST) was determined by noting the processor time printed on the MS-DOS screen, as well as the finishing time (FT). The difference between these two represents the total actual time (TAT) required to execute the monitoring process (336 seconds). The difference between the TRT and TAT clearly quantifies the total processing time (TPT), 236 seconds. In effect, the processing time per run (PTR) is equal to the TPT divided by the total required number of grabs (N), 0.236 seconds. 54 Various trials, with different TRT values and TI have yielded PTR's ranging from 0.20 to 0.29 seconds. Note the values obtained for the PTR is associated with option 9 only, and would vary from menu option to another. The processing time per run can be generally summarized in Equation (1): PTR = TPT / N = [TAT – TRT] / N = [(FT – ST) – TRT] / N (1) The program had also been designed to consider time-critical operations. Such are cases when it is more desirable to execute the monitoring process in a limited time, without the need for acquiring additional data and illustrations. Factors like foot printing the image using the aforementioned continuous threshold method, as well as the periodic saving of images (mentioned in the next section) during the experiment cause the program not to be dynamic. In effect, option 9 has been supplemented by the giving the user the option of removing such time-consuming code. The result of this has been a considerable decrease in the processing time. The application of Equation (1) in this case resulted in a PTR of 0.22 seconds. Thus, an approximate 7 percent of the time was saved. For the tests executed with a total duration of one minute, about one second could be saved per execution. Strictly speaking, it is possible to achieve the monitoring process using three different methods. The time-critical method and the footprint method are both techniques used to monitor the pupil automatically. The third method is the manual 55 method, where the user can achieve almost all the facilities offered by the other two by using the previously described menu options. Pupil Pixel-Count Routine On the MS-DOS screen, the user is regularly updated with information such as the grab index, current illumination, the threshold area (i.e. the number of black pixels representing the pupil), and PC time. In some cases, the pupil status (i.e. constriction, or dilation) is also recorded. The MIL display window is used to show the pupil and the corresponding threshold (as with option 4). All the conditions and data written to the MS-DOS screen during the test are reported to a text file by the name of "test2.txt." In addition, each tenth grab (up to 8 grabs, i.e. 80 images), starting from the first image, is saved to file, with the names image1.tif, image2.tif, ..., image8.tif. Additional grabs or threshold images are possible using option 4 as described. The test can be repeated as many times as desired, but it is preferable that the program be rerun to clear the memory. The monitoring block is designed so as repeat while the program has not exceeded the total required time, and so long as not too many warnings (described next) have been encountered. First, an image is grabbed, and the threshold is calculated, as well as the Number of Black Pixels per Frame (NBPF). Accordingly, the current image is subjected to the threshold value, and the blue channel is displayed on the MIL buffer display window. The pupil should thus appear as a blue 56 spot on the screen. This obviously consumes a significant portion of the processing time, especially that this is repeated for every cycle of the loop. The first five NBPF’s are obtained and averaged out. The resulting NBPF is compared to each of the initial individual NBPF’s, and should be found within 10% of the average. This sensitivity was chosen at random, and could be changed according to the application. If a certain NBPF is found outside this region, the user is prompted of a termination error because the threshold could not be set and the user then has to restart the program. This could occur when the user doesn’t have the camera turned on, moves, or blinks to significantly change the reading from very dark to very bright. In other words, this is sort of an error analysis to test for bad images in the data. The average value obtained from this process is also very important because the viability of the calculations in the rest of the code depend on its accuracy. If the code then goes on, it then performs either of two tests, a dilation or constriction test, depending on the magnitude of the rest of the NBPF’s. The current NBPF value is obtained and compared with the described threshold NBPF (TNBPF), and the loop is continued as before if the former is within 10% of the latter. If the NBPF is greater than 10% of the threshold, then the dilation test has failed and a minor dilation flag (minor warning flag or mWF) is incremented. If this flag has been incremented five consecutive times, then a minor warning (flashing on the buffer display window) is printed. 57 Once the image has reasonably (i.e. within the calculated |10%|) returned to an initial state, the flag is reset to its initial zero value. However, if by the time the minor warning is printed, the pupil has not returned to its initial state, the flag is also zeroed but a flag for a possible major warning is incremented (major warning flag or MWF). If five consecutive minor warnings are printed out (i.e. 25 successive dilation readings), the loop flow is interrupted and the program execution is halted. A major error message is then printed to the screen and all counters and flags are reset to zero. In addition, the MIL defaults are released. The constriction test is very similar to the dilation test. In this case, the NBPF is compared to see if it is below 10% of the TNBPF, which would indicate too few black pixels and thus a possible constricted pupil. The rest of the procedure is then almost identical to the one described earlier. Program Flowchart The logic of the program is summarized in the flow diagram shown in Figure 16. The flow of the diagram is self-explanatory; however, there are a few points to note. This flowchart comes as part of a similar study to monitor miotic eye responses only, but the proposed program accounts for both miotic and mydriatic (or dilative) eye responses. 58 Start (zero all counts) Acquire image Analyze image No Yes Pupil Found? Too Small? Measure Size Zero bad image count No Yes A May have blinked. Increment bad image count Indicate minor warning STOP Acquire image No Too many Bad images? Analyze image Yes Indicate problem A Zero small pupil count Probably blinked indicating bright light, etc. No Pupil Found? Yes Measure Size No Zero small pupil count Too Small? Yes Increment small pupil count. A INDICATE MAJOR PROBLEM (Zero small pupil count) Yes Too many small pupils? Figure 16 Adopted flowchart for a simplified version of the program [77]. No 59 Another aspect is that the method of assessing a “bad” or faulty image is based on the average or on the integration of several grabs, which is not shown in the flowchart. The TI technique used in the program is also an additional tool. Finally, the last block of the flowchart in Figure 16 (or set of conditions and loops) is almost identical to the one in this study. The sole difference lies in the fact that unlike the program, minor and major warning flags are not incremented until five consecutive readings for each respective error have been encountered. After the program is executed, the various NBPF’s and times in which they were grabbed (truncated to the nearest second) are written to a file. This actually occurs while the program is in process, and this text file is accessed later for further processing. One of the powerful tools of C is that the file need not be created before running the program, and it is automatically created and the data is transferred into it consecutively (using the fprintf command). One caution upon running the program is that the name of the destination file must be altered or moved to another folder for every execution. Otherwise, the file is deleted and replaced with a new set of data. 7. Data and Image Processing Software The following is a summary of the various software and data- and image processing packages utilized in this study. Most of these have previously been mentioned, but are repeated here for completeness. 60 The program has been developed with C language on Microsoft VC++ medium. C language is famous for its built-in power in numerous scientific, graphics, and engineering applications. The language is also distinguished by using a compiler, whereby all the commands are translated into machine language before executing the program. This factor saves a significant part of the processing time as compared to a language like BASIC. The latter uses an interpreter, and thus the program is being executed as it is being translated into a low-level language. The MIL commands have been encoded in a C-like set of commands and have many built-in graphics tools that have been very useful in the analysis. As it is being executed, the program writes to text files (on Microsoft Notepad® v. 4.0) as well as image files. This tool allows for data accumulated and displayed during the program's execution to be written to file for post-processing and analysis. Thus, data is either of displayed on the MS-DOS screen, the MIL buffer display screen, or written to file directly from the program. Image files that the program best handles are .tif files. MIL also has the capability of two other image formats known as .mim (MIL image) and .raw. These formats, however, have not proven to be as compatible with the image-processing software as much as the .tif format. The same applies to the images acquired with Intellicam, which uses an incorporated interpreter (called MIL-INTER). This serves as a quick reference and testing to the grabbing process. It has also been found that images grabbed using Intellicam have an enhanced resolution and 61 quality that cannot (unfortunately) be incorporated into the program. The reason for actually not using Intellicam for the monitoring technique is that it's not as readily designed for programming with C as VC++ is, and the fact that it uses an interpreter instead of a compiler. Images in the .tif format are in a viewable state, but are not yet configured for image post-processing. Paint Shop Pro® v. 5.03 software is used to open .tif files and convert saved images (normal or threshold) from .tif to .bmp (bitmap) format. BMP files are in such a format where the pixels are more readily accessible for analysis. Scion Image® (v. 4.0.02, Beta Version) is an image-processing software that is equipped with many imaging features. It is specifically designed for a Scion framegrabber, i.e. digitizer, like Meteor. The Scion software is not compatible with the Meteor digitizer, but has been utilized here to process grabbed images. Scion is used to determine the threshold to generate the TI curve for each subject. A sample of this process is shown in Figure 17. Using the threshold tool, the image is converted into a B/W negative version and a gage is manually used to measure the level at which the pupil is isolated (or the blackest level), as previously described. Note that the program itself produces automatically-threshold images of the pupil, however, this is performed at a phase (specifically option 4) after the TI curve has already been determined. The efficiency of such threshold images from the program is to be discussed in the Results section. 62 Figure 17 The image processing involved in creating a TI curve. Left: an actual grab from the program, with dim light level. Right: a screen grab (to the same scale as the one to the left) from Scion Image after the threshold process has been performed; note how the pupil is accurately represented and isolated in the image. All the threshold data, as well as the .txt files from the program are then fed into the data-processing spreadsheet, Microsoft Excel 97® (SR-2). Excel easily reads and identifies all the data from the file and creates clear, powerful plots of the TI curve. Here, both scatters of the individual threshold-illumination values are permissible, as well as a superimposed line of best fit. This regression tool has many powerful features. Many factors such as the correlation factor and standard deviation are automatically provided to the user (for linear-regression curves). In addition, the equation of fit is displayed on the screen, ready to be copied and pasted as the characteristic TI equation in the program. Table 2 shows a summary of the components associated with the proposed imaging technique and a cost analysis for the hardware and software used. Some of the hardware is that actually used in the aforementioned apparatus, while some is an equivalent or cheaper counterpart found nowadays in the market. InfranView®, for 63 example, is free imaging software, and has been found to perform all the features in Paint Shop Pro required for this application. The total price of the system is about $2,900, almost half that mentioned in the Literature Review. It can be further seen that most of the cost is intensified in the hardware components (95%), although the code is not self-sufficient, and uses many specialized software packages. The next section shows how the apparatus was put to use for the various investigated tests and the corresponding results. Table 2 Suggested price analysis for the eye-monitoring technique. Component Description Infrared light source 12 (high viewing angle) IR diodes, plus circuitry and black box Fiber optics back light Approximate Price 12 * $0.5 (IR), $15 (circuitry) $257 Special ophthalmology equipment Camera holder, table Burle camera (TC651) Japan lens (18-108/2.5) $300 $225 $327 $551 13" RCA color TV 64 MB (RAM), 20 GB (Hard Drive), 700 MHz (Speed), HP Computer; 15" HP Monitor; 648C DeskJet HP printer Matrox Meteor II MIL, Intellicam, Scion, IrfanView Microsoft Visual C++ (6.0) Microsoft Excel (i.e. Office 97) Total Cost $90 $400 Visible high-intensity light Chin rest Supports and fixtures B/W camera Close focus, zoom lens (IR sensitive) TV (external) monitor P.C. + monitor + printer Digitizer (imaging) board Imaging software Programming medium Data-processing medium $600 FREE $90 $70 $2931 64 Results and Discussion Figure 7 (page 30) illustrates and summarizes the means by which the images and data were acquired and processed. Four different test or methods were conducted to evaluate the proposed pupil monitoring technique. The first two are procedures that were performed for every subject, specifically measuring the pupil size and threshold under different lighting conditions. The other methods are a mere testing of the second test and the monitoring technique as a whole. The first method is that of determining the maximum constriction and dilatation sizes of the subject pupil. The second is the procedure used to generate the TI curve. The last two experiments are designed, respectively, for testing the accuracy of the system under different lighting and eye-movement conditions. The relevant experimental preparations, factors, and constraints have all been previously referred to in the Apparatus and Experimental Setup section. The following is a brief explanation of each of the four procedures, and results obtained after their application to three different subjects. They have been referred to here as subject A, subject B, and subject C. In some cases, the experiments have been repeated for assessing a higher precision, and providing grounds of comparison. The results have been displayed using figures, tables, and charts where applicable. Additional data has been placed in the Appendix section. 65 1. Direct Pupil-Size Method (Test 1) Test 1 is a procedure used to directly measure the extreme constriction and dilatation pupil sizes of each subject. The concept behind the experiment is that the eye should exhibit a maximum dilation state when subjected to a minimal light, and conversely a maximum contraction when exposed to a very bright light. In both cases, the exposure to the illumination condition has been a gradual process, especially when bright light was involved, due to the obvious hazard. In addition, the eye was allowed to reach its normal state by allowing the pupil to accommodate to the environment for about one minute. The apparatus has been designed for a fixed distance throughout the experiments as mentioned. Minor changes in the orientation of the pupil from one subject to the other have been achieved solely using the chin-rest. In general, a constant position of the pupil's image on the camera has been decently obtained. The right pupil has been placed at about 50 cm from the Xenon Lightsource, and 7 cm from the IR flashlight. The camera lies at approximately 16 cm from the target. This data has been the same for the other three experiments as well. First, all the room ambient lights were shut off, and IR light was lit, to which the pupil has almost no constriction. The subject was then asked to place a ruler right in front of the pupil, in order for it to be visible in the camera while showing the ruler's divisions (centimeter-graded). The ruler used was a transparent one for most of the cases, but a solid one proved more useful with bright light. The subject was asked 66 to keep the ruler at a near horizontal level, and to place it just below or above the pupil, in order to facilitate reading the diameter's length. The resulting image of the eye was then photographed using Intellicam, and the image was saved to file. Two results were possible from this simple technique, an estimate of the maximum and minimum sizes of the pupil, as well as the degree of magnification attained by the camera. The diameter, in mm, could easily be read off the screen, assuming that the curvature of the monitor is sufficiently small as not to cause a significant source of error. In addition, the enlargement factor could similarly be obtained by placing the same ruler horizontally on the monitor and determining the ratio of the manually measured distance (MD), to the real distance (RD) obtained by inspection from the captured image. Figure 18 Direct pupil-size method (subject B). Left: maximum dilatation method. Right: maximum constriction method. Note how the divisions are still visible with both dim and bright lights. The procedure is demonstrated in Figure 18, showing the constriction/dilatation test. Note that the enlargement factor should not vary from one subject to the other, since the monitor screen has a constant size, and the camera's 67 scale factor (SF) was kept almost fixed. However, it has been calculated as a confidence factor. Equation (2) was used to calculate the (SF) in all cases: Scale Factor = Measured Distance / Real Distance (2) After conducting the experiment on all three subjects and implementing Equation (2), a table of results was obtained (Table 3). The figures have been rounded to the nearest half-millimeter since they have been obtained by inspection. The data serves in showing the maximum and minimum pupil limits from one person to the other, the size-range within different pupils, and the scale factor. All these factors are to be used to compare the reliability of the data for the subsequent experiments. Table 3 Results of the direct pupil-size method for cases A, B, and C. MD (mm) Subject A B C RD (mm) SF Constriction Dilatation Constriction Dilatation Constriction Dilatation 41.5 70.0 36.0 75.0 100.0 49.0 3.0 5.0 3.0 5.5 7.0 4.0 13.8 14.0 12.0 13.6 14.3 12.3 The following can be deduced from Table 3. The discrepancy in the SF values between the constriction and dilatation of each subject is about 0.2 to 0.3, less than the predetermined accuracy of half a digit. Thus, it would be safe to take the SF as the average of the two values for every case, i.e. 13.7, 14.2, and 12.2 for subjects A, B, and C, respectively. This variation is present on a larger scale from subject to the other (variation from 12.2 to 14.2) and, therefore, a separate (mean) SF is to be used in each case. 68 Pupillary diameters were previously mentioned to normally range between three and five mm. This has been consistent with all cases except subject B, whose maximum and minimum sizes ranged from 5 to 7 mm. Observe how, additionally, the variance in pupil diameter is different from person to the other, even for the same lighting and experimental conditions. Subjects A and C, for instance, both have the same constricted sizes, but C only expands an additional mm, while A dilates 2.5 mm. 2. Threshold-Illumination Curves (Test 2) The remaining tests are more directly related to the programming part of the thesis, a lot of which has already been mentioned in previous sections. Different images were grabbed for each TI test, and the data was verified for occurrences of blinking and off-focus images, etc. The TI curve for each of the subjects was performed more than once, and the resulting data was investigated for the same subject, and between different subjects. The application of the TI experiment on subject A is treated next. The camera's aperture was set to the highest illumination, where the image is clearest, and thus the highest reliability of results. The program was then executed with minor adjustments to make sure that the image was focused and clear. However, no modifications were conducted during the test itself. 69 Figure 19 Screen capture of a text file containing 10 different image readings at a fixed light value and their average (subject A). This capture was taken at the high illumination value. Figure 20 Screen capture as in Figure 19 for the same subject at a lower illumination value. 70 Figure 21 Screen capture of the "result.txt" file that automatically gathers the indexed, average illumination values (two of which are in Figure 19, Figure 20) and lists them for future manipulation (subject A). Since the TI test is a manual one as previously described, all captures were taken while interfacing with the program's menu options. The set of data for an individual illumination value was each written to a separate text file, as the ones shown in Figure 19 and Figure 20. Each of these shows what the program writes to file while the data is being gathered and calculated. However, accessing and editing these files can only be achieved after the program has been exited. For that reason, it is a requirement to end the program before acquiring a new list of data at a different illumination. This process was repeated for 8-11 different light values (from high to low) with the global average in each case automatically calculated and written to the "result.txt" file. A screen capture of the resulting file for subject A is demonstrated in Figure 21. The data in each "set" file as well as in the summary file is then edited, and all values that visually reflect large changes in the average are discarded. 71 Finally, the user should take all the data files and place them in a different directory and save them to disk for post-processing. The reason this movement process has to take place is that for every execution of the program, the user is prompted as to whether a new experiment is required. So long as the same experiment is being conducted (i.e. for the same subject), the answer should be no, in order to keep the previous data-set files until the end of the experiment. If the user chooses yes, the program assumes that a new experiment is being performed and all the previous data and image files are deleted or replaced with new ones. Image files are saved while conducting the TI test. The last grabbed image is chosen as the one to correspond to all the data files. One such image is considered sufficient, since all the grabs would be almost identical (by inspection) unless the subject had blinked or moved away his/her eye. In the latter case, an image is grabbed separately (after the run) using option 3. The TI test was carried out two to three times for each subject. This was mostly restricted by the time-convenience of each person. Each test was separated by significant time duration to make sure that the results were consistent, although all the tests took place in doors. For subject A, for instance, the test was performed once at 9 p.m. of a certain day, repeated after about an hour of the same day, then a third time in the next day's evening. A similar process was possible for subject B. Two tests only, however, were permissible for subject C, both at the same session. 72 Figure 22 Saved images from the TI program (left) and screen captures showing the corresponding threshold images as obtained from Scion Image (right). The images are listed from top to bottom with decreased camera illumination (subject A). Note how the images get darker, yet the pupillary size in the threshold images is almost constant. 73 Figure 23 The next set of three photos for the same experiment as in Figure 22. Up to the middle image (fifth as a whole) the threshold pupil is still clear and of size almost identical to that obtained earlier. The bottom image shows some traces of reflection, but the pupil size is still regular. 74 Figure 24 The last three photos for the same experiment (subject A, Test 2a) as in Figure 22 and Figure 23. Note, at this very little lighting, the iris and other surrounding structures start to appear in the threshold image, and the pupil size starts to vary, and become discontinuous in shape. The data from these photos has been discarded. 75 The image files were then transformed into the bmp format, viewable by Scion Image software, whereby each image was footprint using the Threshold option. Here, a scroll bar is selected to manually select the level of light at which only the pupil would be visible. The results of the original (left images), and each of the corresponding threshold images (right images) using Scion for subject A, Test 2a are presented in Figure 22 through Figure 24. The images have been listed by decreasing illumination. It is interesting to note how the threshold pupil is almost the same size for differently lit images. This is particularly important in Figure 22 and Figure 23, where the pupil appears as a distinct black spot on the screen, with almost no sign of black spots in the background. However, Figure 24 shows that beyond a certain degree of darkness, it becomes difficult to isolate the pupil from the very dim surrounding environment. This causes one to conclude that the program would fail to accurately predict the size of the pupil at this lighting value even with IR light. Thus, the fifth image (i.e. the middle figure in Figure 23) has been considered as the cut line between valid and invalid threshold images. The valid threshold data is recorded and then manually written to an Excel file. In turn, the indexed, result data from the corresponding experiment is fed to the same Excel file. This process was repeated for every subject, and for each of the different experiments that were conducted at different timings. A summary of the tabulated data obtained is shown in Table 4. 76 Table 4 Summary of the TI experimental data for each of the three subjects. Index 1 2 3 4 5 6 7 8 9 Test 2a Test 2: Subject A Test 2b Test 2c Test 2a Test 2: Subject B Test 2b Test 2c Aperture Luminance Thresh. 2a Luminance Thresh. 2b Luminance Thresh. 2c Reading 73.08 35 73.44 31 70.05 29 2.5 67.51 30 67.51 26 67.66 26 3 58.55 21 59.30 21 54.23 20 3.5 44.64 17 41.54 16 41.69 15 4 40.64 14 30.85 14 35.88 13.5 4.8 26.78 12 25.28 12 25.97 12 5.6 23.20 11.5 18.66 11.5 20.41 11.5 6.8 16.74 11 15.52 11 16.77 10.5 8 12.51 10.5 11.86 9.5 11.99 9.5 11 Aperture Index Luminance Thresh. 2a Luminance Thresh. 2b Luminance Thresh. 2c Reading 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Index 1 2 3 4 5 6 7 8 43.1 36.61 34.78 29.98 20 18 17 14 26.4 14 21.24 17.07 15.08 14.87 12.47 12.36 12.01 11.33 10.94 10.62 10.37 13 12 11 13 11 11 11 10 10 10 9 75.1 38.36 39 16 66.99 36 53.28 20.61 23 12 43.72 18 29.8 13.73 14 11 23.79 13 18.6 12.5 16.19 11 13.65 11.5 12.26 10 11.69 10.5 10.73 9.5 Test 2: Subject C Test 2a Test 2b Luminance 75.83 62.15 44.25 34.28 24.34 16.37 12.39 10.80 Thresh. 2a 47 32 24 17 14 12 11 10 Luminance 76.11 65.15 47.55 37.34 24.98 15.75 Thresh. 2b 49 37 23 18 14.5 11.5 Aperture Reading 2.5 3.3 4 4.8 5.6 8 11 16 2.5 2.5 3.3 4 4 4.8 5.6 5.6 6.8 8 9.5 11 13.5 16 2.5 4 5.6 8 77 The data in Table 4 is the list of all the data that was obtained in the TI experiments. Discarded data of the nature mentioned above has also been included here, but highlighted. Note that in some cases, a reading at a specified lighting was repeated due to some doubt in the results and this has only served to enhance the resulting curve. The table heading for the test producing the most desirable results for each subject has been italicized. The data was then used to generate the characteristic threshold curve as a function of the illumination curves for subjects A, B, and C, as shown in Figure 25 through Figure 27, respectively. The corresponding equation (obtained directly from the Excel menu) as well as a scatter of the original points is also shown. Care has been taken to generate plots with the same grid size and scales for the different subjects. Different regression curves have been attempted to compare the various types of possible fits. The TI curves in Figure 25 for subject A have been selected of the linear type, since only five data points were evaluated to be effective. Note how the individual points lie close to the curve, although the data range is limited as compared to Figure 26. In the latter, a second-order polynomial was attempted for two cases, as well as a third linear curve. Cubic curves showed to quickly diverge over a large range from the data, and were thus concluded undesirable. Power curves were the choice for subject C and proved to closely resemble linear fits. 78 Characteristic Threshold Curve (Test 2): Subject A 50 45 40 Threshold 35 30 20 2 .379 .=0 15 .) (Lum + 0.9 126 0.6 .= esh Thr 25 237 m.) (Lu 8 .44 - 11 sh Thre 10 .439 .=0 esh r h T 5 0 0 5 10 15 20 4 m.) (Lu 25 - 2.9 872 30 35 40 45 50 55 60 65 70 75 80 75 80 Luminance Thresh. 2a Linear (Thresh. 2a) Thresh. 2b Linear (Thresh. 2b) Thresh. 2c Linear (Thresh. 2c) Figure 25 Linear regression used to produce characteristic TI curves for subject A. Characteristic Threshold Curve (Test 2): Subject B 50 0. 90 96 (L ) + 3. 11 62 45 + 40 2 04 (L 3 ) -0 .0 29 (L ) 21 T= 0 4(L 00 = 0. 25 3 20 T= 4(L .441 )+ 3. 47 2 35 8(L 14 )+ 9.5 .1 00 30 T Threshold 35 ) 5(L E-0 0 2 ) + .0 -0 6 15 10 5 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 Luminance Thresh. 2a Thresh. 2b Thresh. 2c Poly. (Thresh. 2a) Poly. (Thresh. 2b) Linear (Thresh. 2c) Figure 26 Polynomial and linear regression analysis used to produce TI curves for subject B. 79 Characteristic Threshold Curve (Test 2): Subject C 50 45 40 Threshold 35 30 994 0.8 25 h. = re s Th 20 30 0.8 .) um 9(L 747 1.1 15 10 . esh Thr 5 .) um 4(L 3 9 .2 =0 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 Luminance Threshold 2a Threshold 2b Power (Threshold 2a) Power (Threshold 2b) Figure 27 Power regression analysis used to produce both TI curves for subject C. Characteristic TI Curves for Subjects A, B, and C 50 45 747 1.1 40 Threshold 35 .= esh Thr 30 0.2 m.) (Lu 934 1 448 .472 11. )+3 . m ) m. 4(Lu Lu .441 6( 2 .=0 1 h s .6 e Thr =0 h. res h T 25 20 15 10 5 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 Luminance Test 2 (A) Linear (Test 2 (A)) Test 2 (B) Linear (Test 2 (B)) Test 2 (C) Power (Test 2 (C)) Figure 28 Summary of the characteristic TI curves for all three subjects. 70 75 80 80 Figure 28 shows comprehensive curves used to closely compare the data from Figure 25, Figure 26, and Figure 27. Linear and power fits have proven to produce the best curve-fits, since they have the highest correlation factors. Also, note how the slopes in the three curves are very similar, especially for the higher luminance values. Thus, for a certain lighting value, the threshold is just displaced vertically as the trend increases. One explanation for this could be the varying pupillary sizes among each subject, thus counting as the sole distinction factor between each subject and the other, with sufficient light. The characteristic equation for each subject was copied and pasted into the program to perform the final two tests, the Illumination and EyeMovement Experiments discussed next. 3. Illumination Experiment (Test 3) The threshold-illumination experiment is not to be confused with the illumination experiment, since the latter is a test to examine the effectiveness of the curve generated from the TI experiment. The monitoring experiment was executed for each of the subjects for a total duration of one minute, and at a time increment of one second. This was figured as a sufficient time to examine the whole range of illumination values. The foot-printing option was selected, in view of the fact that that the grabbed images would be required for post-processing and evaluation. A series of combined screen captures of the resulting data file for subject A is shown in Figure 29. 81 Figure 29 Screen captures of the "test2.txt" file for the Illumination experiment (subject A). 82 The data written to file (as in Figure 29) includes the image index, average luminance calculated, threshold calculated from the characteristic equation and the corresponding threshold area (pupil size in pixels), and the time when the grab was taken. The file also includes instances where a minor constriction of the pupil was detected. It is also possible to have minor dilation warnings in addition to that, but these did not occur in this case. Constriction warnings set a clear example of instances when the curve has failed to detect the pupil, or part thereof. Dilatation warnings dictate that the pupil as well as other parts has also been detected, producing an error in the calculation. A sample of the captured images by the program for subject A is illustrated in Figure 30. The figure shows the original images (as seen live) and the corresponding threshold images6. Unlike the previous test, these images are all processed, captured, and saved to file automatically by the program. The pupil in each case is located and exposed in a distinct color (blue). Notice how as the light decreases in the images, the pupil still maintains approximately the same shape and size. The middle image shows a considerably smaller pupil, but on quick reference, one notices that the pupil has actually contracted in the left image due to natural eye accommodation. 6 The threshold images are those of photos of the pupil taken at nearly identical experimental conditions to the original ones on the left. 83 Figure 30 Some of the photographed images from the Illumination Experiment (left) and the corresponding foot-printed threshold images obtained from the generated program (right). The images are listed from top to bottom with decreased camera illumination (subject A). The threshold pupil is identified by the blue spot in the middle of the screen. 84 Table 5 Tabulated data for the Illumination Experiment conducted on subject A. Test 3 (Light Test): Subject A Program Output Calculations Required Time: 60 sec.; Time Increment: 1 sec. Acceptable Data Grab PC Time Ave. Calc. (HH:MM:SS) Lumin. Thresh. 1 20:51:46 70.14 31.52 2 20:51:47 70.01 31.44 3 20:51:48 70.06 31.47 4 20:51:49 69.62 31.20 5 20:51:50 69.13 30.90 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 31 33 34 35 36 42 43 44 Ave.: 20:51:51 20:51:52 20:51:53 20:51:54 20:51:56 20:51:57 20:51:58 20:51:59 20:52:01 20:52:02 20:52:03 20:52:04 20:52:05 20:52:06 20:52:08 20:52:09 20:52:10 20:52:11 20:52:12 20:52:13 20:52:14 20:52:16 20:52:17 20:52:18 20:52:20 20:52:23 20:52:24 20:52:25 20:52:26 20:52:33 20:52:34 20:52:35 69.79 68.63 68.53 68.56 68.33 68.36 68.46 59.33 59.36 59.25 59.07 59.19 59.20 58.94 59.03 58.71 57.91 46.72 46.46 44.21 44.50 44.31 44.70 44.83 44.69 44.58 35.52 35.59 35.79 35.52 66.49 66.57 66.78 31.31 30.59 30.53 30.55 30.41 30.43 30.49 24.90 24.91 24.85 24.74 24.81 24.81 24.66 24.71 24.52 24.02 17.17 17.01 15.63 15.81 15.69 15.93 16.02 15.93 15.86 10.31 10.35 10.48 10.31 29.28 29.33 29.46 RD (mm): 4.3; SF: 13.7 Thresh. Pixels 14590 16509 15685 15347 16888 Diameter (Pixels) 136.30 144.98 141.32 139.79 146.64 Diameter Error (mm) (%) 9.95 10.58 10.32 10.20 10.70 15804 19664 18811 17553 17532 18377 18860 18483 16655 16241 17326 16843 16376 16745 15283 15629 16370 14281 14991 14801 12156 13777 12852 13199 13280 12144 10086 10694 12067 12174 18516 17474 16345 141.80 158.23 154.76 149.50 149.41 152.97 154.96 153.41 145.62 143.80 148.53 146.44 144.40 146.02 139.50 141.07 144.37 134.84 138.16 137.28 124.41 132.44 127.92 129.64 130.03 124.35 113.32 116.69 123.95 124.50 153.54 149.16 144.26 10.35 4.80 4.69 4.53 4.53 4.64 4.70 4.65 4.42 4.36 4.50 4.44 4.38 4.43 4.23 4.28 4.38 4.09 4.19 4.16 3.77 4.02 3.88 3.93 3.94 3.77 3.44 3.54 3.76 3.78 4.66 4.52 4.37 CF: 2.41 11.6 9.1 5.4 5.4 7.9 9.3 8.2 2.7 1.4 4.7 3.3 1.8 3.0 1.6 0.5 1.8 4.9 2.6 3.2 12.3 6.6 9.8 8.6 8.3 12.3 20.1 17.7 12.6 12.2 8.3 5.2 1.7 85 45 46 56 58 59 60 61 20:52:37 20:52:38 20:52:49 20:52:52 20:52:53 20:52:54 20:52:55 Rejected Data Grab PC Time (HH:MM:SS) 32 37 38 39 40 41 47 48 49 50 51 52 53 54 55 57 20:52:22 20:52:27 20:52:28 20:52:29 20:52:31 20:52:32 20:52:39 20:52:40 20:52:41 20:52:42 20:52:44 20:52:45 20:52:46 20:52:47 20:52:48 20:52:50 67.07 67.79 35.15 35.10 52.13 51.57 51.45 Ave. Lumin. 35.00 25.49 24.01 23.94 23.47 33.68 24.81 19.11 19.20 19.53 24.63 24.79 24.77 34.48 34.92 34.88 29.64 30.08 10.09 10.06 20.49 20.15 20.07 Average: Std_Dev: 14041 13455 12036 13228 18478 18466 18764 15489 2620 133.71 130.89 123.79 129.78 153.38 153.34 154.57 139.92 12.11 4.05 3.97 3.75 3.94 4.65 4.65 4.69 5.7 7.7 12.7 8.5 8.2 8.1 9.0 4.24 0.37 7.3 4.5 Calc. Thresh. 10.00 4.17 3.26 3.22 2.93 9.19 3.75 0.26 0.32 0.52 3.64 3.74 3.73 9.68 9.94 9.92 Average: Std_Dev: Thresh. Pixels 1631 3 9 7 0 2307 13 9 10 5 7 12 17 3986 2188 2402 788 1278 Diameter (Pixels) 45.57 1.95 3.39 2.99 0.00 54.20 4.07 3.39 3.57 2.52 2.99 3.91 4.65 71.24 52.78 55.30 19.53 25.75 Diameter (mm) 1.38 0.06 0.10 0.09 0.00 1.64 0.12 0.10 0.11 0.08 0.09 0.12 0.14 2.16 1.60 1.68 0.59 0.78 Error (%) 67.9 98.6 97.6 97.9 100.0 61.8 97.1 97.6 97.5 98.2 97.9 97.2 96.7 49.8 62.8 61.0 86.2 18.2 The data from the test2.txt file was introduced to Excel in order to evaluate the results. Table 5 shows the data obtained for subject A. Table headings include all the data from the program (test2.txt) as well as the equivalent diameter in pixels and in millimeters. Another important factor included is the deviation of each of the calculated diameters from the mean of the values listed in Table 3. In this case, the maximum and minimum measured diameters were 3.0 and 5.5 mm, respectively, and so the mean is about 4.23 mm. 86 A correction factor (CF) is also calculated, arising due to the fact that other regions of the eye may be subject to threshold, like the dark ring that may be found in people with dark irises (as shown in Figure 35). Although the presence of an error is not a desirable matter, if the error were to exist, however, it should merely shift the data off their course. Thus, the CF would help to bring the data back to their correct shape. In other words, the CF acts to fix the accuracy (i.e. the exactness or nearness to the actual value) of the data rather than their precision (or the repeatability of the successive measurements in the experiment) [82]. This is evidenced by the plots of scattered data for both Test 3 and Test 4. Following is a description of the calculations performed in Table 5. First, the initial five values in the Thresh. Pixels columns of data are each converted into diameter values. Thus, an assumption was made here that the original shape of the pupil is that of a circle. Another postulation is that even if the pixels did not assume a circular shape when they were counted, they could be reshaped to fit the shape of a circle. This is probably a valid assumption since an individual pixel can be used to form a uniform structure. The result of this division produces the diameter pixel value of these first five data points. This is similar to the process done in the code, where the first five points are used as a basis for producing a boundary limit to the constriction and dilation of the pupil. Each of the readings is then converted into the mm equivalent by dividing it by the scale factor mentioned in Experiment 1. The average of these resulting mm 87 diameters is compared against the mean of the real diameter values, and the ratio of these two is considered the correction factor. The same analysis is repeated for the remaining data values in the experiment. The following equation summarizes the above description: Diameter (mm) = Diameter (pixels) / (CF * SF) = 2 * [thresh_area(pixels) / ] / (CF * SF) (3) After Equation (3) was applied to the data, the numbers were inspected for values below and those exceeding the nominal diameter values listed in Table 3. The accuracy flaw in the data of Experiment 1 was taken here into consideration; the accuracy to the nearest 0.5 diameter meant that any value within 0.25 of the boundary limit was still considered valid. Thus for subject A, any diameter calculated and found to be outside the range 2.75-5.75 was regarded as a rejected data point, as shown at the bottom of Table 5. The same calculations were performed for the Illumination experiment for each of the other two subjects. The results for subjects B and C are tabulated in Appendix 2. Notice the proximity of the correction factor in cases A, B, and C (2.41, 2.37, and 2.58, respectively). Thus, the range of error in all three cases is only about 5% from the average. One can conclude from this that although the error does exist in the experimental technique, the overall procedure can be adjusted using the CF to make the data fit the desired range better. 88 Illumination Experiment (Test 3): Subject A 8.00 Pupillary Diameter (mm) 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 20:51:48 20:51:56 20:52:05 20:52:13 20:52:22 20:52:31 20:52:39 20:52:48 20:52:57 20:53:05 Time (HH:MM:SS ) Acceptable Data Rejected Data Figure 31 Scatter of diameter readings for the Illumination Experiment: subject A. Illumination Experiment (Test 3): Subject B 8.00 Pupillary Diameter (mm) 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 21:08:47 21:08:56 21:09:04 21:09:13 21:09:22 21:09:30 21:09:39 21:09:48 21:09:56 Time (HH:MM:SS ) Acceptable Data Rejected Data Figure 32 Scatter of diameter readings for the Illumination Experiment: subject B. 21:10:05 89 Illumination Experiment (Test 3): Subject C 8.00 Pupillary Diameter (mm) 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 19:18:37 19:18:46 19:18:55 19:19:03 19:19:12 19:19:21 19:19:29 19:19:38 19:19:47 19:19:55 Time (HH:MM:SS ) Acceptable Data Rejected Data Figure 33 Scatter of diameter readings for the Illumination Experiment: subject C. Plots of the pupillary diameter (mm), versus time for each of the experiments are shown in Figure 31 through Figure 33. The boundary limits have been drawn so that anything lying in-between is considered valid. The rejected data lying outside the boundary lines were also included to show the quantity of off-values, as well as how far off they were. The fact that the subject may have blinked should not be thought off on a digital basis, i.e. that such would produce a zero-diameter reading. The reason is that the eyelid may be half-closed at times, or the pupil may have moved so that the pupil remains partially in the camera range only. The slight continuous variation in the size of the diameter even in the valid data range could well be due to the natural eye- 90 accommodation process. In general, most of the data (about 70%) lie inside the limits, which is a desirable aspect. In addition, it is clear that the program has successfully been able to produce a consistent reading of the diameter with varying light values. 4. Eye-Movement Experiment (Test 4) This final experiment is very similar in nature to the Illumination experiment. The major difference lies in the fact that the monitoring process for each subject is conducted at a constant lighting while the pupil is moved in several different positions. Here a closer eye was put on subject B, for which the most efficient results were achieved. A screen-capture of the experimental values (Figure 34) shows from the extracted data the test to have lasted for a minute, with a one-second increment. The subject was asked to move his eye in a clockwise fashion, with a time interval of about 10 seconds in each fixed position, referred to as "moves." Of course, the head and chin would remain stationary throughout the test. As soon as the experiment started, the subject was asked to look forward (move one, as with Test 3), then look up (move two), left (move three), down (move four), and finally right (move five). Having the eye fixed in these positions each for a few seconds would give the program sufficient time to automatically capture the necessary images. 91 Figure 34 Screen captures of the "test2.txt" file for the Eye-Movement experiment (subject B). 92 93 Figure 35 Images from the Eye-Movement experiment (left) and the corresponding threshold images obtained from the generated program (right). The images are listed from top to bottom with eye-movement positions: forward (move 1), up (move 2), left (move 3), down (move 4), and right (move 5) (subject B). The resulting original and footprint images of the experiment (subject B) for the five moves are shown in Figure 35. Note that although the lighting conditions are identical in all the moves, the pupil size and shape varies from position to the other. Also, notice that the fact that the pupil is seeking the extremities of the eye causes there to be some source of reflection (as with move 5) and some minor footprint parts outside the pupil. Such is, however, too small to be considered a significant source of error, especially since the eye is in an abnormal position and would soon be expected to return to its original location, move 1. Calculations are shown in Table 6. 94 Table 6 Tabulated data for the Eye-Movement Experiment conducted on subject B. Test 4 (Movement Test): Subject B Program Output Calculations Required Time: 60 sec.; Time Increment: 1 sec. Acceptable Data Grab PC Time Ave. Calc. (HH:MM:SS) Lumin. Thresh. 1 21:13:07 60.50 30.18 2 21:13:08 60.56 30.20 3 21:13:09 60.26 30.07 4 21:13:10 59.66 29.81 5 21:13:12 59.07 29.55 6 7 8 9 10 11 14 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 34 35 36 37 38 40 42 45 RD (mm): 6.0; SF: 14.2 Thresh. Pixels 19466 18881 17831 16962 15622 Diameter (Pixels) 157.43 155.05 150.68 146.96 141.04 Diameter Error (%) (mm) 11.09 10.92 10.61 10.35 9.93 CF: Ave.: 60.01 29.96 17753 150.23 10.58 2.46 21:13:13 59.02 29.53 25066 178.65 5.11 18.9 21:13:14 58.95 29.49 24102 175.18 5.01 16.6 21:13:15 58.77 29.41 24000 174.81 5.00 16.4 21:13:16 58.81 29.43 23579 173.27 4.96 15.3 21:13:17 58.96 29.50 23235 172.00 4.92 14.5 21:13:18 58.86 29.45 23787 174.03 4.98 15.8 21:13:22 58.15 29.14 23915 174.50 4.99 16.2 21:13:24 64.28 31.84 32716 204.10 5.84 35.9 21:13:25 75.00 36.57 31566 200.48 5.74 33.4 21:13:26 74.96 36.56 32301 202.80 5.80 35.0 21:13:27 66.51 32.83 30650 197.55 5.65 31.5 21:13:28 65.05 32.18 25227 179.22 5.13 19.3 21:13:30 62.55 31.08 25796 181.23 5.19 20.6 21:13:31 62.46 31.04 24864 177.93 5.09 18.4 21:13:32 61.96 30.82 25935 181.72 5.20 21.0 21:13:33 64.09 31.76 26870 184.97 5.29 23.1 21:13:34 64.47 31.93 28156 189.34 5.42 26.0 21:13:35 63.68 31.58 29645 194.28 5.56 29.3 21:13:36 55.57 28.00 33363 206.11 5.90 37.2 21:13:37 55.51 27.97 29846 194.94 5.58 29.8 21:13:38 70.41 34.55 32638 203.85 5.83 35.7 21:13:39 67.12 33.10 45634 241.05 6.90 60.5 21:13:40 66.44 32.80 47918 247.00 7.07 64.4 21:13:42 66.19 32.69 43014 234.02 6.70 55.8 21:13:44 56.46 28.39 27101 185.76 5.32 23.6 21:13:45 62.01 30.85 23528 173.08 4.95 15.2 21:13:46 58.98 29.51 27176 186.01 5.32 23.8 21:13:47 58.55 29.32 30240 196.22 5.62 30.6 21:13:48 65.50 32.38 39732 224.92 6.44 49.7 21:13:51 54.88 27.70 33789 207.42 5.94 38.1 21:13:53 59.26 29.63 47423 245.72 7.03 63.6 21:13:56 57.36 28.79 42194 231.78 6.63 54.3 95 46 47 49 50 52 53 54 55 57 58 59 61 21:13:57 21:13:59 21:14:01 21:14:02 21:14:04 21:14:05 21:14:07 21:14:08 21:14:10 21:14:11 21:14:12 21:14:14 Rejected Data 12 21:13:19 13 21:13:21 15 21:13:23 33 21:13:43 39 21:13:50 41 21:13:52 43 21:13:54 44 21:13:55 48 21:14:00 51 21:14:03 56 21:14:09 60 21:14:13 57.58 55.88 66.10 57.23 59.51 61.15 64.53 63.16 55.79 56.81 56.91 57.52 69.64 58.58 58.58 65.56 63.52 59.55 58.62 58.50 57.71 57.55 60.26 57.55 28.89 28.14 32.65 28.73 29.74 30.46 31.96 31.35 28.10 28.55 28.59 28.86 Average: Std_Dev: 47090 40679 28205 24952 28962 23984 31907 24576 25306 24798 32472 25806 30540 7326 244.86 227.58 189.50 178.24 192.03 174.75 201.56 176.89 179.50 177.69 203.33 181.27 195.93 22.48 7.01 6.51 5.42 5.10 5.50 5.00 5.77 5.06 5.14 5.09 5.82 5.19 34.21 29.33 29.33 32.41 31.51 29.76 29.35 29.29 28.95 28.87 30.07 28.87 Average: Std_Dev: 213 19847 19714 54579 52635 51624 51343 53390 20670 18745 20333 20486 31965 19150 16.48 158.97 158.43 263.61 258.88 256.38 255.68 260.73 162.23 154.49 160.90 161.51 189.02 73.63 0.47 4.55 4.53 7.55 7.41 7.34 7.32 7.46 4.64 4.42 4.61 4.62 63.0 51.5 26.1 18.6 27.8 16.3 34.2 17.7 19.5 18.3 35.3 20.7 5.61 0.64 30.4 15.0 89.0 5.8 5.5 75.5 72.3 70.7 70.2 73.6 8.0 2.8 7.1 7.5 5.41 2.11 40.7 36.4 As with the illumination experiment, the diameter values for each of the data values in the experiment were calculated using Equation (3). The calculations performed to subjects A and C have been listed in Appendix 2. The correction factors here were 2.47 (subject A), 2.46 (subject B), and 2.57 (subject C). The resulting error is about 2.5% from the average, half of what it was in the illumination experiment. In addition, the mean of the CF's is very close in both tests (2.45 as compared to 2.50). The scatter of the values is shown in Figure 36, Figure 37, and Figure 38. 96 Eye-Movement Experiment (Test 4): Subject A 8.00 Pupillary Diameter (mm) 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 20:57:24 20:57:33 20:57:42 20:57:50 20:57:59 20:58:08 20:58:16 20:58:25 20:58:34 20:58:42 Time (HH:MM:SS ) Acceptable Data Rejected Data Figure 36 Scatter of diameter readings for the Eye-Movement Experiment: subject A. Eye-Movement Experiment (Test 4): Subject B 8.00 Pupillary Diameter (mm) 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 21:13:06 21:13:15 21:13:24 21:13:32 21:13:41 21:13:49 21:13:58 21:14:07 21:14:15 Time (HH:MM:SS) Acceptable Data Rejected Data Figure 37 Scatter of diameter readings for the Illumination Experiment: subject B. 21:14:24 97 Eye-Movement Experiment (Test 4): Subject C 8.00 Pupillary Diameter (mm) 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 19:24:49 19:24:58 19:25:06 19:25:15 19:25:24 19:25:32 19:25:41 19:25:49 19:25:58 19:26:07 Time (HH:MM:SS ) Acceptable Data Rejected Data Figure 38 Scatter of diameter readings for the Illumination Experiment: subject C. It is easy to realize by inspection that there is a great amount of variability in the plots. The reason for this is that although the magnitude of the diameter is fluctuating rapidly from one position to the other, the mean of the values remains within acceptable bounds. This is especially notable in subjects A and B with a considerably large pupil size. For subject C, many of the data points were outside the permissible boundary limits, particularly for the down position when the lower lid shielded the pupil (proven by the many data points underneath the lower limit). 98 Conclusions and Recommendations A few aspects can be concluded from the aforementioned study. The suggested experimental setup as well as the adopted programming technique seems to have in general met the objectives of this investigation, as well as to proving that the pupil is monitored for miotic and mydriatic pupillary changes. The concept behind the program, simple as it is, is that it does not actually identify the pupil as a single structure, but as a number of threshold, or black pixels, which are exclusive of the rest of the eyeball. Doing so has been possible by performing an initial experiment from which a characteristic threshold-illumination curve is produced for each different individual. This has been a unique feature not encountered in any of methods found in the literature. Using the TI curve, tests are performed to measure the pupillary size under different lighting conditions and eye-movement positions. The resulting data is compared to an experimental measure of the maximum and minimum pupil sizes and the rejected data is identified and discarded. A program has been developed in C language to perform all the previous functions, with the aid of several data and image processing software. The apparatus has been equipped for holding the camera, lights, and chin rest at a fixed, reproducible distance, and has produced repeatable images. The technique has taken advantage of safe levels of infrared-light, for which a flashlight with a 99 dozen IR LED's has been designed with an easy attachment to the setup and illumination for the pupil. In effect, hazardous, bright light has not been required in the experiments, except for short intervals to test the size limits of the eye. In addition to lighting, processing time has been a major concern in the experiment. The Meteor digitizer has proven to give a slow start to the experiment, and a lag when foot printing and image capturing is required. For this purpose, the program has been designed to give the user the option of performing time-critical (or dynamic) operations, where only data is gathered and saved to file for postprocessing. Saving images to file and calculating a threshold has also been possible. Experimental discrepancy has been mostly encountered when physically measuring the pupil size for comparison with the light and eye-movement tests. Thus, the correction factor was introduced, and implemented into the data. The result is that a consistent shift in the data was hence corrected by this factor, as demonstrated by the various tables and charts in the Results and Discussion section. The data generated by the various tests have generally been acceptable, especially with linear plots. Probably the major problem with such curves is that the range in which they are effective is rather limited, as compared with the studied power and polynomial curves. Recommendations are to try to eliminate the need for a correction factor by increasing the brightness and/or contrast of the grabbed image so that the pupil can be properly isolated from the rest of the eye. Other recommendations include the 100 implementation of a pupil edge-finder routine (as proposed in Appendix 3), and incorporation of a helmet in which to place the human head, so that the tests could be performed while the subject is in motion (Appendix 4). 101 Bibliography 1. Bartley, G. B., and T. J. Liesegang. Essentials of Ophthalmology. Philadelphia: Lippincott, 1992. 2. Vaughan, D. G., T. Asbury, P. Riordan-Eva. General Ophthalmology. 14th ed. Norwalk: Appleton, 1995. 3. Stein, H. A., B. J. Slatt, and R. M. Stein. Ophthalmic Terminology. Third ed. St. Louis: Mosby, 1992. 4. Stein, H. A., B. J. Slatt, and R. M. Stein. The Ophthalmic Assistant. Sixth ed. St. Louis: Mosby, 1994. 5. Cline, D., H. W. Hofstetter, and J. R. Griffin. Dictionary of Visual Science. Third ed. Radnor: Chilton, 1980. 6. Walsh, T. J. “Pupillary Abnormalities.” Neuro-Ophthalmology: Clinical Signs and Symptoms. Fourth ed. Ed. Thomas J. Walsh. Baltimore: Williams, 1997. 5582. 7. Leiken, J. B., F. P. Pauloucek. Poisoning & Toxicology Handbook. Hudson: Lexi-Comp, 1995-96. 8. Frith, P. The Eye in Clinical Practice. Oxford: Blackwell, 1994. 9. Wright, K. W. Textbook of Ophthalmology. Baltimore: Williams, 1997. 10. Zuber, B. L. Models of Oculomotor Behavior and Control. Boca Raton: CRC P, 1981. 11. McConkie, G. W., C. W. Scouten, P. K. Bryant, and J. H. Wilson. A Microcomputer-Based Software Package for Eye-Monitoring Research. Champaign: U of Illinois, 1988. 12. Rayner, K., ed. Eye Movements in Reading: Perceptual and Language Processes. New York: Academic P, 1983. 13. Sloboda, J. A. “The Psychology of Music Reading.” Psychology of Music 6 (1978): 3-20. 14. Nodine, C. F., and H. L. Kundel. “The Cognitive Side of Visual Research in Radiology.” Eye Movements: From Physiology to Cognition. Eds. J. K. O’Regan, and A. Levy-Schoen. Amsterdam: North Holland, 1987. 573-583. 15. Antes, J. R., K. Chang, and C. Mullis. “The Visual Effect of Map Design: An Eye-Movement Analysis.” The American Cartographer 12 (1985): 143-155. 16. Suppes, P., M. Cohen, R. Laddaga, J. Anliker, and R. Floyd. “Research on Eye Movements in Arithmetic Performance.” Cognition and Eye Movements. Eds. R. Groner, and P. Fraisso. Amsterdam: North Holland, 1982. 57-73. 102 17. Chase, W. G., and H. A. Simon. “The Mind’s Eye in Chess.” Visual Information Processing. Ed. W. G. Chase. New York: Academic P, 1973. 215281. 18. Lochegr, P., and C. Nodine. “Symmetry Catches the Eye.” Eye Movements: From Physiology to Cognition. Eds. J. K. O’Regan, and A. Levy-Schoen. Amsterdam: North Holland, 1987. 353-361. 19. Kahneman, D., and J. Beatty. “Pupil Diameter and Load on Memory.” Science 154 (1966): 1583-1585. 20. Hess, E. H. “Pupillometrics: A Method of Studying Mental, Emotional, and Sensory Processes.” Handbook of Psychophysiology. Eds. N. S. Greenfield, and R. A. Sternbach. New York: Holt, 1972. 491-531. 21. Fison, P. N., D. J. Garlick, and S. E. Smith. “Assessment of Unilateral Afferent Pupillary Defects by Pupillography.” Br. J. Ophthalmol. 63 (1979): 195-199. 22. Halliday, A. M., W. I. McDonald, and J. Mushin. “Delayed Visual Evoked Response in Optic Neuritis.” Lancet 1 (1972): 982-985. 23. Rushton, D. “Use of Pulfrich Pendulum for Detecting Abnormal Delay in the Visual Pathway in Multiple Sclerosis.” Brain 98 (1975): 283-296. 24. Straub, R. H., B. J. Arnolds, and L. Kerp. “Biometry with a Video-Genlock Interface and a Computer-Based Image-Analyzing: Use of a TVVideopupillometer.” Biomed. Instrum. Technol. 27 (1993): 43-48. 25. Straub, R. H., L. Kerp, and J. Schölmerich. “Gastrointestinal Motility Disorders in Diabetics Predicted by Pupillary and Neurocardiological Parameters.” Gastroenterology 100 (1991): A169. 26. Gunderson, H. J. G. “A New Photostimulator and Videopupillograph for Quantitative Neuroophthalmological Studies.” Ophthalmologica 172 (1976): 6268. 27. Ishikawa, S., M. Naito, and K. Inaba. “A New Videopupillography.” Ophthalmologica 160 (1970): 248-259. 28. Lowenstein, O., and I. E. Loewenfeld. “Electronic Pupillography.” Arch. Ophthalmol. 59 (1958): 352-363. 29. Young, L. R., and D. Sheena. “Survey of Eye Movements Recording Techniques.” Behavior Research Methods and Instrumentation 7 (1975): 397429. 30. Grant, W. M. Toxicology of the Eye. Second ed. Springfield: Thomas, 1974. 31. Grant, W. M., and J. S. Schuman. Toxicology of the Eye. Effects on the Eyes and Visual System from Chemicals, Drugs, Metals and Minerals, Plants, Toxins 103 and Venoms; also Systemic Side Effects from Eye Medications. Fourth ed. Springfield: Thomas, 1993. 32. Proctor, N. H, J. P. Hughes, and M. L. Fischman. Chemical Hazards of the Workplace. Second ed. Philadelphia: Lippincott, 1988. 33. WWWebster Dictionary. The World Wide Web. http://www.m-w.com/. 34. Taylor, P. “Anticholinesterase Agents.” Goodman and Gilman’s The Pharmacological Basis of Therapeutics. Seventh ed. Eds. A. G. Gilman, et al. New York: Macmillan, 1985. 110-129. 35. Davies, C. S., and R. J. Richardson. “Organophosphorus Compounds.” Experimental and Clinical Neurotoxicology. Eds. P. S. Spencer, and H. H. Schaumburg. Baltimore: Williams, 1980. 527-544. 36. Erickson-Lamy, K., and W. M. Grant. “Ophthalmic Toxicology of Anticholinesterases." Clinical and Experimental Toxicology. Eds. B. Ballantyne, and T. C. Marrs. Oxford: Butterworth, 1992. 180-194. 37. Ishikawa, S., and M. Miyata. “Development of Myopia Following Chronic Organophosphate Pesticide Intoxication.” Neurotoxicity of the Visual System. Eds. W. H. Merigan, and B. Weiss. New York: Raven, 1980. 233-254. 38. Plestina, R., and M. Piukovic-Plestina. “Effect of Anticholinesterase Pesticides on the Eye and on Vision.” CRC Crit. Rev. Toxicol. 6 (1978): 1-23. 39. Tamura, O. “Organophosphorus Pesticides as a Cause of Myopia in School Children.” Excerpta Media ICS 1 (1979): 202-206. 40. Bertrams, J. “Macular Degeneration Associated with Chronic Pesticide Exposure.” Lancet 1 (1982): 288. 41. Misra, U. K., D. Nag, N. K. Misra, M. K. Mehra, and P. K. Ray. “Some Observations on the Macula of Pesticide Workers.” Hum. Toxicol. 4 (1985): 135145. 42. Wadia, R. S., C. Sadagopan, R. B. Amin, and H. V. Sardesai. “Neurological Manifestations of Organophosphorus Insecticide Poisoning.” J. Neurol. Neurosurg. Psychiatry 37 (1974): 841-847. 43. Tabershaw, I. R., and W. C. Cooper. “Sequelae of Acute Organic Phosphate Poisoning.” J. Occup. Med. 8 (1966): 5-20. 44. Suzuki, H., and S. Ishikawa. “Ultrastructure of the Ciliary Muscle Treated by Organophosphate Pesticide in Beagle Dogs.” Br. J. Ophthalmol. 58 (1974): 931940. 45. Uga, S., S. Ishikawa, and K. Mukuno. “Histopathological Study of Canine Optic Nerve and Retina Treated by Organophosphate Pesticide.” Invest. Ophthalmol. 16 (1977): 877-881. 104 46. Imai, H., M. Miyata, S. Uga, and S. Ishikawa. “Retinal Degeneration in Rats Exposed to an Organophosphorus Pesticide (Fenthion).” Environ. Res. 30 (1983): 453-465. 47. Carricaburu, P., R. Lacroix, and J. Lacroix. “Electroretinographic Study of the White Mouse Intoxication by Organo-Phosphorus.” Toxicol. Eur. Res. 3 (1981): 87-91. 48. Taked, Y., I. Tsukahara, and S. Takaori. “Effects of Chlorfenvinphos, an Organophosphate Insecticide, on Afferent Transmission in the Central Visual System.” Jpn. J. Ophthalmol. 20 (1976): 195-203. 49. Beck, A. D. “10. Glaucoma.” Ophthalmology for the Primary Care Physician. Eds. David A. Palay, and Jay H. Krachmer. St. Louis: Mosby, 1997. 135-142. 50. Jay, W. M., R. W. Marcus, and M. S. Jay. “Primary Position Upbeat Nystagmus with Organophosphate Poisoning.” J. Pediatr. Ophthal. Strabismus 19 (1982): 318-319. 51. Hata, S., E. Bernstein, and L. E. Davis. “Atypical Ocular Bobbing in Acute Organophosphate Poisoning.” Arch. Neurol. 43 (1986): 185-186. 52. Chandler, P. A, and W. M. Grant. Glaucoma. Philadelphia: Lea, 1979. 53. Senanayake, N., and M. K. Johnson. “Acute Polyneuropathy after Poisoning by a New Organophosphate Insecticide.” N. Engl. J. Med. 306 (1982): 155-157. 54. Ganelin, R. S., G. A. Mail, and C. Cueto, Jr. “Hazards of Equipment Contaminated with Parathion.” Arch. Environ. Health 8 (1964): 826-828. 55. Simon, R. D. “Parathion Poisoning. A Case Report.” Am. J. Dis. Child 105 (1963): 527. 56. Bogolepov, N. K., S. L. Kaplan, and T. A. Luzhetskaia. “Apoplectiform and Epileptiform Syndrome in Thiophos Poisoning.” Sov. Med. 31 (1968): 132-133. 57. Eitzman, D. V., and S. L. Wolfson. “Acute Parathion Poisoning in Children.” Am. J. Dis. Child 114 (1967): 397-400. 58. Grant, W. M. “Miotic and Antiglaucoma Activity of Tetraethyl Pyrophosphate in Human Eyes.” Arch. Ophthalmol. 39 (1948): 579-586; Arch. Ophthalmol. 44 (1950): 362-364. 59. Ecobichon, D. J., D. R. Ozere, E. Reid, and J. F. S. Crocken. “Acute Fenitrothion Poisoning.” Can. Med. Assoc. J. 116 (1977): 377-379. 60. Taylor, W. J. R., E. Llewellyn-Thomas, G. C. Walker, and E. A. Sellers. “Effects of a Combination of Atropine, Metaraminol and Pyridine Aldoxime Methanesulfonate (AMP Therapy) on Normal Human Subjects.” Can. Med. Assoc. J. 93 (1965): 957-961. 105 61. Jagar, B. V., and G. N. Stagg. “Toxicity of Biacetyl Monoxime and of Pyridine-2-Aldoxime Methiodide in Man.” Bull. Johns Hopkins Hosp. 102 (1958): 203-211. 62. Quinby, G. E. “Further Therapeutic Experience with Pralidoxime in Organic Phosphorus Poisoning.” J. Am. Med. Assoc. 187 (1964): 202-206. 63. Quinby, G. E., T. A. Loomis, and H. W. Brown. “Oral Occupational Parathion Poisoning Treated with 2-PAM Iodide (2-Pyridine Aldoxime Methiodide).” N. Engl. J. Med. 268 (1963): 639-643. 64. Krishna, N., and I. H. Leopold. “Effect of Protopam on the Rabbit Pupil.” Am. J. Ophthalmol. 52 (1961): 566-567. 65. Collins, B. K. “Atropine– An Eye Opener.” Veterinary Technician 15 (94): 404407. 66. Reeder, D. H. “Organic Phosphate Insecticide Poisoning.” J. Occup. Med. 3 (1961): 129-130. 67. Block, J. B. The Signs and Symptoms of Chemical Exposure. Springfield: Thomas, 1980. 68. Dickoff, D. J., O. Gerber, and Z. Turovsky. “Delayed Neurotoxicity after Ingestion of Carbamate Pesticide.” Neurology 37 (1987): 1229-1231. 69. Farago, A. “Fatal, Suicidal Case of Sevin (1-Naphthyl-N-Methyl-Carbamate) Poisoning.” Arch. Toxikol. 24 (1969): 309-315. 70. Stewart, W. C., H. D. Madill, and A. M. Dyer. “Night Vision in the Miotic Eye.” Can. Med. Assoc. J. 99 (1968): 1145-48. 71. Rubin, L. S., and M. N. Goldberg. “Effect of Tertiary and Quaternary Atropine Salts on Absolute Scotopic Threshold Changes Produced by an Anticholinesterase (Sarin).” J. Appl. Physiol. 12 (1958): 305. 72. Rubin, L. S., and M. N. Goldberg. “Effects of Sarin on Dark Adaptation in Man.” J. Appl. Physiol. 11 (1957): 439. 73. Rubin, L. S., S. Krop, and M. N. Goldberg. “Effect of Dark Adaptation in Man.” J. Appl. Physiol. 11 (1957): 445. 74. Rengstorff, R. H. “Accidental Exposure to Sarin: Vision Effects.” Arch. Toxicol. 56 (1984): 201-203. 75. Best, E. M., and B. L. Murray. “Observations on Workers Exposed to Sevin Insecticide.” J. Occup. Med. 4 (1962): 507-517. 76. Moylan-Jones, R. J., and D. P. Thomas. Cyclopentolate in Treatment of Sarin Miosis.” Br. J. Pharmacol. 48 (1973): 309-313. 106 77. Odom, J. V., J. E. Smith, R. P. Craven, and A. M. El-Sherbeeny. NonInvasive Ocular Toxicity Assessment Instrument (NOTAI): Number Three, Dynamic Monitoring. Pending Patent Disclosure. West Virginia University: Morgantown, 1999. 78. McGregor, G. Personal Interview. Jul. 23 1999. 79. MIL / MIL-Lite version 6.0 Board-Specific Notes. Manual no. 10515-MF-0600. February 24, 1999. 80. Matrox Intellicam. User Guide Version 2.05. Manual no. 10521-MU-0301. January 15, 1999. 81. MIL-Lite Version 6.0. User Guide and Command Reference. Manual no. 10514MU-0600. February 24, 1999. 82. Eide, A. D., J. D. Roland, L. H. Mashaw, and L. L. Northup. Engineering Fundamentals and Problem Solving. Third ed. New York: McGraw, 1997. 83. Edmund Industrial Optics. 2000 Optics and Optical Instruments Catalog. N001A (1999): 58. 84. Meyer, M. The Little, Brown Guide to Writing Research Papers. Second ed. U of Connecticut: HarperCollins, 1985. 107 Appendix Appendix 1 Listing of the Code /* File name: mgrab.c Synopsis: This program monitors the size of a subject's pupil. The aim is to initially determine a threshold-illumination (TI) curve for each subject; every point on this line tells the current average lighting (illumination), and the corresponding pixel value of the light required to isolate (or threshold) from the rest of the eye. The next step is then to implement this curve back in to the program, and test the size of the pupil for different lighting conditions to see if it's the same (light test) Another test, the movement test, whereby the subject is asked to move his/her pupil is different directions, and the program tests to see is the pupil is still still being isolated, and again determines the size of the pupil in each position. The program has been developed using menu-like structured programming (in C language) where the user is provided a menu of different options. The first two options are primarily a pre-test, just as with Intellicam. The first menu option is designated to grabbing a single image, while the second is for grabbing images continuously. Such is used to see if the pupil is in the required position before starting the actual grabbing process. Options 6, 7, and 8 are concerned with the TI curve. For each illumination, a series of images are grabbed using option 6. Each time, the average of the pixel lightingvalues is determined. The user is provided with the number of averaged points, and their average illumination. The user is also asked whether the grab has been successful or not. If not, the data values are erased, and if acceptable, the data is written to a text file. After the user has taken a number of grabs that he finds are sufficient to estimate the current lighting, option 7 can be used to view the number of grabs and the corresponding average lighting. At this point, the program performs a further averaging for all these (average-illum.) values and provides the result to the user. The user here can easily determine whether one of the points was too high or too low. This can naturally occur in cases where the subject has blinked, or his eye wandered off from the target image. Note, option 6 can be returned to at this stage for further grabs. Once all images have been grabbed, option 8 is chosen so as to prompt the user for the name of the file to be saved, representing the last grabbed image (in .tif or .tiff formats). In addition, the user is also asked for the name of the (.txt) file in which to save the summary list described in option 7. Finally, the universal average of the current lighting (plus index) is automatically written to a file called "result.txt." The user now chooses Quit from the choice menu. After a new lighting condition has been adjustment, the user now reruns the program, and repeats all the steps described so far (i.e. options 1, 2, 6, 7, and 8). This is repeated for as many luminance settings as the user requires. The user at this point should access the individual luminance-reading files and make sure that the data is 108 all consistent. If a point is off, then is should be removed, and the new global average should be manually calculated, and written to the file, which should be saved for changes. Furthermore, the "result.txt" file, carrying all the global averages, is accessed to make corresponding adjustments. At this point, the TI curve should be determined. The user should open the result.txt file using a spread-sheet software (like Excel). In addition, the saved images are viewed converted to bitmap files (.bmp). An image-processing software (like Scion) should now be used to determined to determine the current threshold of the pupil. The latter step should be done for each of the images saved from the program. Each of these threshold values should be substituted in a corresponding column for each of the illumination values. A scatter of these pair of values is now plotted, with the threshold on the abscissa, and illumination on the ordinate. Excel has a regression tool for curve-fitting a line through the points and providing the corresponding equation (T as a function of I). The user copies the formula and simply pastes it in option 9, the monitoring option, using a method referred to as the Black-Pixel technique. Every time a new equation is added, the existing formula is commented (for possible future use) and the new formula is added , with the addition of the subject's name and date of the experiment. The curve-fit can initially be test for using option 4 (note a copy of the equation must be provided in option 4, too). After a single image of the eye is grabbed (now at any illumination) using option 6 in the menu, the user chooses 4 to threshold the image. The program determines the average illumination and automatically substitutes in the TI equation to determine the required threshold. It then thresholds the image (as would be achieved with Scion Image) and shows the pupil in blue to the user in the MIL buffer display window. This image can be saved by choosing option 3 from the menu, which saves the current displayed image to file (again in .tif format). Note, that option 3 is not associated with the threshold option, and can be used to save images grabbed by either of options 1, 2, 4, or 6. Option 5 has been used solely for testing purposes. It was used to experiment with MIL's abilities to access individual pixels on the screen, and to change the color of a pixel or group of pixels from one color to another. This option could prove very useful with a pupil edge-follower algorithm. Option 9 is now used to monitor the pupil size. The user is prompted for the total time that he/she wishes to monitor and measure the pupillary size, and the required increment of time between each grab. The processing time has been taken into account here, and has been added to the time for each grab. On the MS-DOS screen the user is regularly updated with information such as the grab index, current illumination, the threshold area (i.e. the number of black pixels, representing the pupil), in addition to pupil status (i.e. constriction, or dilation). The MIL display window is used to show the pupil and the corresponding threshold (as with option 4). The mean of the first five images is used to reference the ones to follow. If a grab were found to have threshold area lower than 10% of the set size than a warning indication of a possible pupil constriction is reported. This is done when 5 consecutive such failures have been found, since the cause of the error could be a sudden flash of light or a regular accommodation of the pupil. Five such warnings reported consecutively (i.e. a total of 25 failing images) cause the program to terminate. If the readings return to within the 'normal' range, the flags for 109 the various warnings are reset and the process is repeated. A similar monitoring process is done for dilation (or dilatation) failures of the pupil (higher than 10%) of the reference values. All the conditions and data written to the MS-DOS screen during the test is reported to a text file by the name of "test2.txt." In addition, each tenth grab (up to 8 grabs, i.e. 80 images) is saved to file, with the names image1.tif, image2.tif, ..., image8.tif. Additional grabs or threshold images are possible using option 4 as described. The test can be repeated as many times as desired, but it is preferable that the program be rerun to clear the memory. The preceding was a summary of the aim and procedure used to approach the described. Additional language-specific commands and details are included nearly each of the program's statements as well to follow. Comments are also included as subheadings for further clarity. Note that separate or a groups of statements and commands have been commented in some cases; these indicate instances where the commands are no required but kept for possible future applications, possible testing for errors, or just for clarity of the current statements. */ #include <stdio.h> //standard input output library #include <mil.h> //special mil library #include <math.h> //math library, for absolute function #include <time.h> //time library, for finding the system time #define image_no "image_no.txt" // MAIN BLOCK OF THE PROGRAM void main(void) { MIL_ID MilApplication, /* Application identifier. */ MilSystem, /* System identifier. */ MilDisplay, /* Display identifier. */ MilDigitizer, /* Digitizer identifier. */ MilImage; /* Image buffer identifier. */ char tmpbuf[128]; // save the system time in this variable unsigned char red[480][640]; // the array in which "mono" red channel will be stored unsigned char blue[480][640]; // the array in which "mono" blue channel will be stored //unsigned char green[480][640]; // the array in which "mono" green channel will be stored int method, // store the number for the required method black[5001], //array to store the consecutive counts of black pixels per image ymin=0, ymax=0, xmin=0, xmax=0; double *TimePtr, //time pointer time, // defining the time between digitizer image grabs stoppingtime, // if pupil is in error for more than this time, terminate program processingtime // the time required by the system to process one loop (experimental value) 110 float test, //required time between grabs ave_f = 0, black_f[5001], zgrabsf; // required number of grabs to achieve input time long int x=0, y=0, // random use counters for x and y z = 0, // counter for the number of image grabs already processed ztime, // the program running time provided by the user in seconds zgrabs, // total calculated required number of grabs sum = 0, // counter for storing the number of black pixels per image frames = 5, // defines the number of grabs that determine the initial condition ave = 0, // the average number of black pixels in the first images cond = 5, // number of bad images that signal a warning cond2 = 5, // number of warnings that signal a termination error flag = 1, // signals an error in the initial values flag2 = 0, // holds the number of dilating images flag3 = 0, // holds the number of dilating warnings flag4 = 0, // holds the number of constricting images flag5 = 0; // holds the number of constricting warnings //defining variables for foot printing int j, //counter to clear the contents of ave_lumin array k, //counter for the number of grabbed images index1 = 0, //index for the number of points used to determine luminance thresh_count, //contains the number of thresholded pixels image_count, //index for the number of grabbed images time_critical = 0; //used for a time_critical (real-time) monitoring process double lumin_sum = 0, //sums up all the luminance values in the frame ave_lumin[5001], //array for the average luminance of each grab lum, //the current average luminance sum_lumin_sum = 0,//sums up the individual average luminance sums threshold, //the current threshold calculated by the experimental eqn. ave_ave_lumin = 0;//averages the variable sum_lumin_sum char choice, //determines from user if the data in a grab is valid or not //used also to see if user need a time_critical monitoring process wait[80], //waits for the user to hit enter to go to the main menu t_filename[25]; // the file containing thresholding data for the current luminance FILE * lumin_p = fopen("lumins.txt", "w"); //writes summary info in case 7 in this file FILE * rename_p = fopen("rename.txt", "w"); //copies lumins.txt here to be renamed after each program run FILE * result_p = fopen("result.txt", "a"); //stores the average luminance from each lighting int t_file_num = 0; //the image number FILE * thresh_file_number = fopen(image_no, "r+"); //this file carries the value of t_file_num //THIS FILE MUST NEVER BE DELETED !!! FILE * sums = fopen("test2.txt", "w"); // open file for data logging long RunFlag; char mySaveFile[50] ; 111 long int minX, maxX, minY, maxY; //defining variables for printing out the images to files //first declare pointers each of the images you want to save char * imag1, * i2, * imag3,* imag4,* imag5,* imag6,* imag7; char * images8,* imag9,* imag10, * other; //assign characters each having the name of the image to be saved char imag11='1', i22='2', imag33='3',imag44='4',imag55='5',imag66='6',imag77='7'; char images88='8', imag99='9', imag100='0', other2 = '8'; //assign the contents of the character names to the pointers imag1 = &imag11 ; i2 = &i22 ;imag3 = &imag33 ;imag4 = &imag44 ; imag5 = &imag55 ;imag6 = &imag66 ;imag7 = &imag77 ; images8 = &images88 ; imag9 = &imag99 ;imag10 = &imag100 ; other = &other2; MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, // Allocate defaults. &MilDisplay, &MilDigitizer, &MilImage); MappTimer(M_TIMER_RESET, M_NULL); // reset the mil timer for(j = 1; j < 50; j++) ave_lumin[j] = 0; //clear the contents of ave_lumin[] //the number of the file is reset if the user wants to run a new experiment printf("Run a new thresholding experiment? (y or n) "); scanf("%c", &choice); printf("\n\n"); if (choice == 'y' || choice == 'Y') { fprintf(thresh_file_number, "%i", t_file_num); fclose(result_p); fopen("result.txt", "w"); //clears the contents of this file and... fclose(result_p); fopen("result.txt", "a"); //reopens it for adding new data } // define which method to use RunFlag=1; while(RunFlag==1){ printf("What would you like to do? \n\n"); printf("1 - Grab a single Image\n"); printf("2 - Live image for focussing camera.\n"); printf("3 - Save image to file\n"); printf("4 - Thresholding the image (Must do 6 First)\n"); printf("5 - Print a stream of points on the buffer display\n"); printf("6 - Grab a single image and determine its average luminance\n"); printf("7 - Show the average of all the average luminances so far\n"); printf("8 - Save the data in case 7 to a new file\n"); printf("9 - Monitor the size of the pupil (Black-Pixel method)\n"); printf("10 - Quit\n"); scanf("%i", &method); 112 switch (method){ case 1: printf("selected choice 1\n\n"); MdigGrab(MilDigitizer, MilImage); break; // Grab an image. case 2: printf("selected choice 2\n\n"); MdigGrabContinuous(MilDigitizer, MilImage); // Grab images continuously break; case 3: printf("selected choice 3\n\n"); printf("Enter a filename.\n"); scanf("%s", mySaveFile); MbufExport(mySaveFile, M_TIFF, MilImage); // Saves the grabbed image in .tif format break; case 4: printf("selected choice 4\n\n"); //printf("Enter a threshold.\n"); //scanf("%i", &threshold); threshold = 11.84 + (0.0055 * lum * lum) + (0.0848 * lum); //this's 255 minus the experimentally obtained eqn (from Excel) printf("The current luminance is: %.2f\n", lum); printf("The threshold for the grabbed image is: %.2f\n", threshold); MbufGetColor(MilImage, M_SINGLE_BAND, M_RED, red); // assign the red channel to the array // count number of 'black' pixels for (y = minY; y < maxY; y++) for (x = minX; x < maxX; x++) { if (red[x][y] < threshold) { //i.e. if the current red-channel pixel's luminance // is less than the threshold... sum++; //count one more such pixel blue[x][y]=255; //and turn the corresponding blue-channel pixel // into total blue (i.e. 255 on the blue scale) } else blue[x][y]=0; //else turn it into 'no' blue (zero on blue scale) } MbufPutColor(MilImage, M_SINGLE_BAND, M_BLUE, blue); //show the blue channel on the image buffer window printf("The thresholded area contains %i pixels\n", sum); thresh_count = sum; if (image_count <= 1) { fprintf(result_p, "Luminance Threshold Thresholded Area\n"); fprintf(result_p, "_________ _________ ________________\n\n"); } fprintf(result_p, "%.2f %.2f %i\n", lum, threshold, thresh_count); 113 sum = 0; break; case 5: printf("selected choice 5\n\n"); for (y=minY;y<maxY;y++) for (x=minX;x<maxX;x++) {blue[x][y]=0; } x=100;y=100; for (i=1;i<3;i++) // // // // { blue[x+i][y+i]=255; blue[100][100]=255; blue[101][101]=255; blue[102][102]=255; blue[103][103]=255; blue[104][104]=255; blue[105][105]=255; MbufPutColor(MilImage,M_SINGLE_BAND,M_BLUE,blue); //printf("HitReturn\n"); //scanf("%s",mySaveFile); } break; // case 6: minX = 5; maxX = 475; minY = 5; maxY = 635; //reset these values; for some reason the program changes them! printf("selected choice 6\n\n"); MdigGrab(MilDigitizer, MilImage); // Grab an image. MbufGetColor(MilImage, M_SINGLE_BAND, M_RED, red); // assigns the red channel to the double array 'red' for(y = minY; y < maxY; y++) //i.e. the max. y value doesn't count for(x = minX; x < maxX; x++) //i.e. the max. x value doesn't count { lumin_sum += red[x][y]; //sum all the luminance value in the frame index1++; //counter for the number of averaged points } ave_lumin[image_count] = lumin_sum / index1; //calculate the average luminance for the current frame lum = ave_lumin[image_count]; //assigns the current luminance to lum printf("For grab number: %i,\n", (image_count+1)); // printf("The total luminance is: %.2f and\n", lumin_sum); printf("The average luminance of: %i points\n", index1); printf("is: %.2f\n", ave_lumin[image_count]); //This group of statements checks to see if the user is satisfied //with the luminance data in this image, and if not cancel them label1: printf("Is the data for this image acceptable (y/n)? "); scanf("%s", &choice); if (choice == 'Y' || choice == 'y') 114 { image_count++; //increment the number of grabs (for the next image) // printf("number of points = %i points\n", index1); } else if (choice == 'N' || choice == 'n') { printf("\no.k., the data has been discarded, and you may try again\n"); ave_lumin[image_count] = 0; // printf("number of points = %i points\n", index1); } else { printf("Invalid answer! Try again\n\n"); // printf("number of points = %i points\n", index1); goto label1; } lumin_sum = 0; index1 = 0; //reset the values of lumin_sum and index1 for the next grab printf("\n\n"); //leave an extra couple of blank lines before starting over break; case 7: //prints the individual average luminance(s) as well as their average printf("selected choice 7\n\n"); printf("The following is the contents of the array containing\n"); printf("The average luminance of all the grabbed images\n\n"); printf(" Image Number Average Luminance\n"); printf(" ____________ _________________\n\n"); fprintf(lumin_p, " Image Number Average Luminance\n"); fprintf(lumin_p, " ____________ _________________\n\n"); fprintf(rename_p, " Image Number Average Luminance\n"); fprintf(rename_p, " ____________ _________________\n\n"); for (k = 0; k < image_count; k++) { sum_lumin_sum += ave_lumin[k]; printf("%11i %29.2f\n", (k + 1), ave_lumin[k]); fprintf(lumin_p, "%11i %29.2f\n", (k + 1), ave_lumin[k]); fprintf(rename_p, "%11i %29.2f\n", (k + 1), ave_lumin[k]); } printf("\n\n"); ave_ave_lumin = sum_lumin_sum / image_count; //note how image_count was incremented in case 6 printf("\nThe average of the %i average luminances is: ", image_count); printf("%.2f\n\n", ave_ave_lumin); fprintf(lumin_p, "The average of the %i average luminances is: " , image_count); fprintf(lumin_p, "%.2f\n\n", ave_ave_lumin); fprintf(rename_p, "The average of the %i average luminances is: " , image_count); fprintf(rename_p, "%.2f\n\n", ave_ave_lumin); 115 printf("\nPlease press Enter to continue "); gets(wait); gets(wait); printf("\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"); sum_lumin_sum = 0; break; case 8: printf("selected choice 8\n\n"); //this increments the number of the file that represents // the data for a specific lighting //the current file number is first read from the file image_no.txt fscanf(thresh_file_number, "%i", &t_file_num); fclose(thresh_file_number); //the incremented (by 1) file number is now written again to image_no.txt thresh_file_number = fopen(image_no, "w"); t_file_num++; fprintf(thresh_file_number, "%i", t_file_num); fclose(thresh_file_number); //save the file containing the current luminance thresholding data fprintf(lumin_p, "Luminance number: %i\n\n", t_file_num); fprintf(rename_p, "Luminance number: %i\n\n", t_file_num); printf("Please enter the name of the file (.txt or .dat) "); printf("where you want to store \n"); printf(" the data for luminance number: %i\n", t_file_num); gets(t_filename); gets(t_filename); //changes the name of the file "rename.txt" to the input file name fprintf(lumin_p, "This file should be named: %s\n\n", t_filename); fprintf(rename_p, "This file should be named: %s\n\n", t_filename); fclose(rename_p); if (rename("rename.txt", t_filename) == 0) printf("Renamed rename.txt to %s.\n", t_filename); else perror("rename"); //Save the last saved image (.tif) for the current luminance number printf("\nPlease enter the name of the file (.tif or .tiff) "); printf("where you want to store\n the last saved image "); printf("for luminance number: %i\n", t_file_num); scanf("%s", mySaveFile); MbufExport(mySaveFile, M_TIFF, MilImage); //Writes the final average luminance values to the file "result.txt" //write the heading of the "results.txt" file containing the summary data if (t_file_num == 1) { fprintf(result_p, " Luminance Number Average Luminance\n"); fprintf(result_p, " ________________ _________________\n\n"); } 116 fprintf(result_p, " %i t_file_num, %.2f\n", ave_ave_lumin); printf("\n\n\n\n"); break; case 9: printf("selected choice 9\n\n"); printf("Provide the required monitoring time in seconds: ");//prompt input scanf ("%li", &ztime); // store input in variable ztime fprintf(sums, "required monitoring time, in seconds: %li\n", ztime); //write this information to the output file "test2.txt" printf("enter required time between grabs, in seconds: "); //prompt input scanf("%f", &test); // store input in test fprintf(sums, "required time between grabs, in seconds: %.2f\n\n", test); //write this information to the output file "test2.txt" printf("is time_critical monitoring required (y or n)? "); scanf("%c", &choice); scanf("%c", &choice); //determine if the user wants the monitoring in least possible time if (choice == 'y' || choice == 'Y') { processingtime = 0.220; //the lag is due to processor time and grabbing time time_critical = 1; } else { processingtime = 0.236; //the lag is due to the above time plus the foot printing time_critical = 0; } zgrabsf = (ztime / test); // calculate required number of grabs to achieve input time zgrabs = zgrabsf; // convert number of grabs from float to integer (truncate) if (zgrabsf - zgrabs >= 0.5) zgrabs++; //check if we should have rounded up or if truncation was o.k. stoppingtime = cond * cond2 * (test + processingtime); //terminate if pupil is in error for this time time = test - processingtime; TimePtr = &time; //calculate actual time between grabs //assign to the time pointer this actual increment while(z <= zgrabs && flag) //i.e. while these conditions are equal to one /* z defines number of grabs, flag makes sure that no termination errors occurred*/ { 117 MdigGrab(MilDigitizer, MilImage); // Grab an image. if (time_critical == 0) //this option is available only if foot printing is required { if (z==0) { MbufExport(imag1, M_TIFF, MilImage); rename("1", "image1.tif"); } if (z==10) { MbufExport(i2, M_TIFF, MilImage); rename("2", "image2.tif"); } if (z==20) { MbufExport(imag3, M_TIFF, MilImage); rename("3", "image3.tif"); } if (z==30) { MbufExport(imag4, M_TIFF, MilImage); rename("4", "image4.tif"); } if (z==40) { MbufExport(imag5, M_TIFF, MilImage); rename("5", "image5.tif"); } if (z==50) { MbufExport(imag6, M_TIFF, MilImage); rename("6", "image6.tif"); } if (z==60) { MbufExport(imag7, M_TIFF, MilImage); rename("7", "image7.tif"); } if (z==70) { MbufExport(other, M_TIFF, MilImage); rename("8", "image8.tif"); } /*if (z==80) { MbufExport(imag9, M_TIFF, MilImage); rename("9", "image9.tif"); } if (z==90) { 118 MbufExport(imag10, M_TIFF, MilImage); rename("0", "image10.tif"); }*/ } MbufGetColor(MilImage, M_SINGLE_BAND, M_RED, red); // count number of 'black' pixels minX = 5; maxX = 475; minY = 5; maxY = 635; for(y = minY; y < maxY; y++) for(x = minX; x < maxX; x++) { lumin_sum += red[x][y]; //sum all the luminance value in the frame index1++; //counter for the number of averaged points } ave_lumin[z] = lumin_sum / index1; //calculate the average luminance for the current frame lum = ave_lumin[z]; //assigns the current luminance to lum ///printf("For grab number: %i,\n", z+1); ///printf("The average luminance of: %i points\n", index1); ///printf("is: %.2f\n", ave_lumin[z]); threshold = 0.2934 * pow(lum, 1.1747);//Dr. Essam Dec. 18 ///threshold = (0.4414 * lum) + 3.4721; //Fahad Dec. 17 ///threshold = 11.84 + (0.0055 * lum * lum) + (0.0848 * lum); //this's 255 minus the experimentally obtained eqn. (from Excel) ///printf("The threshold for the grabbed image is: %.2f\n", threshold); for (y = minY; y < maxY; y++) for (x = minX; x < maxX; x++) { if (red[x][y] < threshold) { //i.e. if the current red-channel pixel's luminance // is less than the threshold... sum++; //count one more such pixel blue[x][y]=255; //and turn the corresponding blue-channel pixel // into total blue (i.e. 255 on the blue scale) } else blue[x][y]=0; //else turn it into 'no' blue (zero on blue scale) } if (time_critical == 0) //this option is available only if foot printing is required MbufPutColor(MilImage, M_SINGLE_BAND, M_BLUE, blue); //show the blue channel on the image buffer window thresh_count = sum; black[z] = sum; //assign number of black pixels to the storage array 119 black_f[z] = sum; //get current system time: _strtime( tmpbuf ); if (z==(frames-1)) //check if exactly the first (5) images have been grabbed { //calculate average of the first "frames" for (x = 0; x < frames; x++) { ave = ave + black[x]; ave_f = ave_f + black_f[x]; } ave = ave / frames; ave_f = ave_f / frames; //check if deviation from average among all these frames is within 10% for (x = 0; x < frames; x++) if ( abs ((ave - black[x]) / ave) > 0.1) flag = 0; } /* if the first "frames" grabbed had a large deviation from the average (>10%), then signal an error message and terminate the program*/ if (flag == 0) { //print this termination text to the MIL display screen: MgraText(M_DEFAULT, MilImage, 176L, 210L, " -------------------------------------- "); MgraText(M_DEFAULT, MilImage, 176L, 235L, " Error, Unable to set Inital Values !!! "); MgraText(M_DEFAULT, MilImage, 176L, 260L, " -------------------------------------- "); //print this text to the MS-DOS display screen: printf("Error, Unable to set Inital Values !!! \n"); printf("Please rerun the application. \n"); //print this text to the data file "test2.txt" fprintf(sums, "Error, Unable to set Inital Values !!! \n"); //provide a 10 second wait then terminate the program time = 10; TimePtr = &time; MappTimer(M_TIMER_WAIT, TimePtr); } if (z > (frames-1)) // check that the current image is past the initial grabs { //check if the pupil is constricting (<10% of initial value) if ((ave_f / black_f[z]) > 1.1) //since too much light could've constricted the pupil { flag2 = 0; //reset # of dilating images flag3 = 0; //reset # of dilating warnings 120 flag4++; //increment # of constricting images } //check if pupil is dilating (>10% of initial value) else if ((ave_f / black_f[z]) < 0.9) { flag2++; //increment # of dilating images flag4 = 0; //reset # of constricting images flag5 = 0; //reset # of constricting warnings } else //otherwise (i.e. within normal range), reset all counters { flag2 = 0; //these counters are reset... flag3 = 0; //since the pattern of dilation... flag4 = 0; //or constriction has been interrupted flag5 = 0; } } if (flag2 == cond) // check if number of consecutive dilating images... // is enough to signal a warning { flag2 = 0; //this's reset for a new set of dilation images flag3++; //increment # of dilation warnings if (flag3 < cond2) //check if number of dilation warnings... //are not enough to signal a termination { //print this warning text to the MIL display screen: MgraText(M_DEFAULT, MilImage, 176L, 210L, " ---------------------- "); MgraText(M_DEFAULT, MilImage, 176L, 235L, " WARNING: Pupil is dilating !!! "); MgraText(M_DEFAULT, MilImage, 176L, 285L, " ---------------------- "); //print this text to the MS-DOS display screen: printf("WARNING: Pupil is dilating !!! \n"); //print this warning to the file "test2.txt": fprintf(sums, "Warning, Pupil is dilating !!! \n"); } else //i.e. number of dilation warnings are... //enough to signal a termination { //print this (dilation) termination text to MIL display screen: MgraText(M_DEFAULT, MilImage, 176L, 210L, 121 " ---------------------- "); MgraText(M_DEFAULT, MilImage, 176L, 235L, " Pupil is dilating !!! "); MgraText(M_DEFAULT, MilImage, 176L, 260L, " TERMINATE ACTIVITY "); MgraText(M_DEFAULT, MilImage, 176L, 285L, " ---------------------- "); /* print this (dilation) termination text... to MS-DOS display screen: */ printf("Pupil is dilating !!! \n"); printf("TERMINATE ACTIVITY \n"); printf("your pupil has been dilating for %.1f seconds\n", stoppingtime); /* print this (dilation) termination text "test2.txt": */ fprintf(sums, "Pupil is dilating !!! \n"); fprintf(sums, "TERMINATE ACTIVITY !!! \n"); fprintf(sums, "your pupil has been dilating for %.1f seconds\n" , stoppingtime); flag = 0; // reset flag to terminate the program (normally 1) // provide a 10 second wait before terminating the program time = 10; TimePtr = &time; MappTimer(M_TIMER_WAIT, TimePtr); } } // same as dilation above, but for constriction if (flag4 == cond) { flag4 = 0; flag5++; if (flag5<cond2) { MgraText(M_DEFAULT, MilImage, 176L, 210L, " ---------------------- "); MgraText(M_DEFAULT, MilImage, 176L, 235L, " WARNING: Pupil is constricting !!! "); MgraText(M_DEFAULT, MilImage, 176L, 285L, " ---------------------- "); printf("WARNING: Pupil is constricting !!! \n"); fprintf(sums, "Warning, Pupil is constricting !!! \n"); } else { MgraText(M_DEFAULT, MilImage, 176L, 210L, " ---------------------- "); MgraText(M_DEFAULT, MilImage, 176L, 235L, " Pupil is constricting !!! "); MgraText(M_DEFAULT, MilImage, 176L, 260L, " TERMINATE ACTIVITY "); MgraText(M_DEFAULT, MilImage, 176L, 285L, " ---------------------- "); 122 printf("Pupil is constricting !!! \n"); printf("TERMINATE ACTIVITY \n"); printf("your pupil has been constricting for %.1f seconds\n", stoppingtime); fprintf(sums, "Pupil is constricting !!! \n"); fprintf(sums, "TERMINATE ACTIVITY !!! \n"); fprintf(sums, "your pupil has been constricting for %.1f seconds\n", stoppingtime); flag = 0; time = 10; TimePtr = &time; MappTimer(M_TIMER_WAIT, TimePtr); } } sum = 0; /* reset the number of black pixels per image... before executing a new loop*/ //Print-out to the MS-DOS screen: printf("Grab: %i, luminance: %.2f, ", z+1, ave_lumin[z]); printf("thresh.: %.2f, thresh_area: %.0f, ", threshold, black_f[z]); printf("time: %s", tmpbuf); fprintf(sums, "Grab: %i, luminance: %.2f, ", z+1, ave_lumin[z]); fprintf(sums, "thresh.: %.2f, thresh_area: %.0f, " , threshold, black_f[z]); fprintf(sums, "time: %s\n", tmpbuf); /* The following was an earlier of the printouts to the MS-DOS window printf("(z= %i)\n", z); printf("The average luminance of: %i points ", index1); printf("is: %.2f\n", ave_lumin[z]); printf("The threshold for the grabbed image is: %.2f\n", threshold); printf("number of black pixels = %.2f\n", black_f[z]); fprintf("For grab number: %i, ", z+1); fprintf("(z= %i)\n", z); fprintf("The average luminance of: %i points ", index1); fprintf("is: %.2f\n", ave_lumin[z]); fprintf("The threshold for the grabbed image is: %.2f\n", threshold); fprintf("black_f[z] = %.2f\n", black_f[z]); if (z >= 4) //i.e. for the sixth grab on { printf("The initial average is: %.2f\n", ave_f); //DELETE printf("ave_f / black[z]= %.3f\n", ave_f / black_f[z]); printf("flag2= %li, flag3= %li, flag4= %li, flag5= %li\n", flag2, flag3, flag4, flag5); } printf("%s \t", tmpbuf ); fprintf(sums, "%s \t", tmpbuf ); gets(wait); gets(wait); */ 123 printf("\n"); lumin_sum = 0; index1 = 0; //resets the values of lumin_sum and index1 for the next grab z++; //increment the number of grabbed images //wait the "TimePtr" seconds before grabbing the next image: MappTimer(M_TIMER_WAIT, TimePtr); } z = 0; break; //reset the number of grabbed images for a new experiment case 10: printf("bye bye\n\n"); RunFlag=0; break; default: printf ("Not a legal choice\n"); } } fclose(lumin_p); fclose(result_p); fclose(sums); // close the log file MappFreeDefault(MilApplication, MilSystem, MilDisplay, MilDigitizer, MilImage); // Release defaults. } 124 Appendix 2 Test 3 and Test 4 Results Test 3 (Light Test): Subject B Program Output Calculations Required Time: 60 sec.; Time Increment: 1 sec. Acceptable Data Grab PC Time Ave. Calc. (HH:MM:SS) Lumin. Thresh. 1 21:08:51 65.59 32.42 2 21:08:52 64.83 32.09 3 21:08:53 64.91 32.12 4 21:08:54 64.94 32.13 5 21:08:55 65.30 32.30 7 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 39 40 41 43 44 45 46 47 48 49 54 RD (mm): 6.0; SF: 14.2 Thresh. Pixels 16870 17875 15520 15917 16407 Diameter (Pixels) 146.56 150.86 140.57 142.36 144.53 Diameter Error (%) (mm) 10.32 10.62 9.90 10.03 10.18 CF: Ave.: 65.11 32.21 16518 144.98 10.21 2.37 21:08:57 64.21 31.81 20176 160.28 4.75 10.6 21:09:03 64.23 31.82 20304 160.79 4.77 10.9 21:09:04 63.19 31.37 21773 166.50 4.94 14.8 21:09:05 41.86 21.95 27584 187.41 5.56 29.3 21:09:07 37.58 20.06 31352 199.80 5.93 37.8 21:09:08 38.12 20.30 28927 191.91 5.69 32.4 21:09:09 38.55 20.49 23868 174.33 5.17 20.2 21:09:10 38.55 20.49 26514 183.74 5.45 26.7 21:09:11 38.10 20.29 28405 190.17 5.64 31.2 21:09:12 37.69 20.11 28725 191.24 5.67 31.9 21:09:14 23.46 13.83 26079 182.22 5.40 25.7 21:09:15 21.97 13.17 24680 177.27 5.26 22.3 21:09:16 22.98 13.62 28065 189.03 5.61 30.4 21:09:17 22.63 13.46 25006 178.43 5.29 23.1 21:09:18 22.83 13.55 21870 166.87 4.95 15.1 21:09:19 22.45 13.38 28365 190.04 5.64 31.1 21:09:21 22.33 13.33 27782 188.08 5.58 29.7 21:09:22 22.86 13.56 26150 182.47 5.41 25.9 21:09:23 22.53 13.42 28323 189.90 5.63 31.0 21:09:24 21.98 13.17 28629 190.92 5.66 31.7 21:09:35 19.33 12.00 28586 190.78 5.66 31.6 21:09:36 19.37 12.02 28738 191.29 5.67 31.9 21:09:37 29.66 16.56 26823 184.80 5.48 27.5 21:09:39 31.04 17.17 28811 191.53 5.68 32.1 21:09:41 30.63 16.99 27780 188.07 5.58 29.7 21:09:42 30.87 17.10 29987 195.40 5.80 34.8 21:09:43 30.76 17.05 29038 192.28 5.70 32.6 21:09:44 31.06 17.18 27856 188.33 5.59 29.9 21:09:45 31.73 17.48 26842 184.87 5.48 27.5 21:09:46 30.55 16.96 22288 168.46 5.00 16.2 21:09:52 19.36 12.02 21731 166.34 4.93 14.7 125 56 57 58 59 60 61 21:09:54 21:09:56 21:09:57 21:09:58 21:09:59 21:10:00 Rejected Data 6 21:08:56 8 21:08:58 9 21:09:00 10 21:09:01 11 21:09:02 31 21:09:25 32 21:09:27 33 21:09:28 34 21:09:29 35 21:09:30 36 21:09:31 37 21:09:32 38 21:09:33 42 21:09:38 50 21:09:47 51 21:09:49 52 21:09:50 53 21:09:51 55 21:09:53 33.55 33.69 33.58 33.50 33.82 34.05 64.38 64.72 64.41 64.26 64.16 14.34 14.18 14.50 14.22 14.06 11.76 11.72 11.66 36.03 18.75 18.81 19.26 19.25 19.05 18.28 18.34 18.29 18.26 18.40 18.50 Average: Std_Dev: 26762 27302 24764 28622 25652 27550 26533 2774 184.59 186.45 177.57 190.90 180.72 187.29 183.54 9.87 5.47 5.53 5.27 5.66 5.36 5.56 31.89 32.04 31.90 31.84 31.79 9.80 9.73 9.87 9.75 9.68 8.66 8.65 8.62 19.37 11.75 11.78 11.97 11.97 11.88 Average: Std_Dev: 19569 17869 19346 19103 18790 3266 2590 3274 2584 2617 14 12 18 14662 16037 16460 12637 12446 14865 10324 7780 157.85 150.84 156.95 155.96 154.67 64.49 57.43 64.56 57.36 57.72 4.22 3.91 4.79 136.63 142.89 144.77 126.85 125.88 137.57 100.28 57.10 4.68 4.47 4.65 4.63 4.59 1.91 1.70 1.91 1.70 1.71 0.13 0.12 0.14 4.05 4.24 4.29 3.76 3.73 4.08 27.3 28.6 22.5 31.7 24.7 29.2 5.44 0.29 26.6 6.8 8.9 4.0 8.3 7.6 6.7 55.5 60.4 55.5 60.4 60.2 97.1 97.3 96.7 5.8 1.4 0.1 12.5 13.2 5.1 2.97 1.69 34.6 36.0 126 Test 3 (Light Test): Subject C Program Output Calculations Required Time: 60 sec.; Time Increment: 1 sec. Acceptable Data Grab PC Time Ave. Calc. (HH:MM:SS) Lumin. Thresh. 1 19:18:40 77.04 48.28 2 19:18:41 76.84 48.13 3 19:18:42 77.38 48.53 4 19:18:43 76.75 48.07 5 19:18:44 76.45 47.85 6 7 8 9 10 11 13 14 15 16 17 20 22 23 24 25 26 28 29 34 35 36 37 38 39 40 41 42 43 44 46 47 RD (mm): 3.5; SF: 12.2 Thresh. Pixels 10011 9392 9853 9025 9436 Diameter (Pixels) 112.90 109.35 112.01 107.20 109.61 Diameter Error (%) (mm) 9.25 8.96 9.18 8.79 8.98 CF: Ave.: 76.89 48.17 9543 110.21 9.03 2.58 19:18:46 76.90 48.18 8915 106.54 3.38 3.3 19:18:47 76.72 48.05 9238 108.45 3.44 1.6 19:18:48 76.55 47.92 9211 108.30 3.44 1.7 19:18:49 76.48 47.87 9261 108.59 3.45 1.5 19:18:50 74.94 46.74 10099 113.40 3.60 2.9 19:18:51 62.67 37.88 12172 124.49 3.95 13.0 19:18:53 64.42 39.13 11992 123.57 3.92 12.1 19:18:54 64.25 39.01 12749 127.41 4.05 15.6 19:18:55 65.66 40.02 11652 121.80 3.87 10.5 19:18:56 64.64 39.29 11669 121.89 3.87 10.6 19:18:58 64.02 38.85 10973 118.20 3.75 7.2 19:19:01 43.06 24.38 13971 133.37 4.24 21.0 19:19:04 40.84 22.91 13445 130.84 4.15 18.7 19:19:05 24.99 12.86 6163 88.58 2.81 19.6 19:19:06 25.78 13.35 8560 104.40 3.32 5.3 19:19:07 25.07 12.92 6103 88.15 2.80 20.0 19:19:08 24.99 12.87 5943 86.99 2.76 21.1 19:19:10 25.76 13.33 7849 99.97 3.17 9.3 19:19:12 24.72 12.70 7292 96.36 3.06 12.6 19:19:18 64.44 39.14 12020 123.71 3.93 12.2 19:19:19 64.16 38.94 10524 115.76 3.68 5.0 19:19:20 64.29 39.04 11377 120.36 3.82 9.2 19:19:21 64.52 39.20 10877 117.68 3.74 6.8 19:19:22 65.58 39.96 9457 109.73 3.48 0.4 19:19:23 63.97 38.81 10581 116.07 3.69 5.3 19:19:24 62.86 38.02 11034 118.53 3.76 7.5 19:19:26 72.70 45.10 7611 98.44 3.13 10.7 19:19:27 73.58 45.75 7403 97.09 3.08 11.9 19:19:28 74.33 46.30 8132 101.75 3.23 7.7 19:19:29 51.81 30.30 13193 129.61 4.12 17.6 19:19:32 36.56 20.11 13845 132.77 4.22 20.5 19:19:33 34.87 19.03 13821 132.66 4.21 20.4 127 48 50 55 56 58 59 60 19:19:34 19:19:36 19:19:42 19:19:43 19:19:45 19:19:47 19:19:48 Rejected Data 12 19:18:52 18 19:18:59 19 19:19:00 21 19:19:02 27 19:19:09 30 19:19:13 31 19:19:14 32 19:19:15 33 19:19:16 45 19:19:30 49 19:19:35 51 19:19:37 52 19:19:39 53 19:19:40 54 19:19:41 57 19:19:44 61 19:19:49 23.86 23.85 54.33 54.22 57.39 57.08 56.92 78.55 57.67 43.51 42.58 25.17 21.02 17.97 21.32 17.15 51.41 33.28 23.21 17.44 17.26 53.34 53.99 54.65 12.18 12.18 32.04 31.96 34.16 33.95 33.84 Average: Std_Dev: 8476 7510 13365 12662 13683 11261 11723 10405 2387 103.88 97.79 130.45 126.97 131.99 119.74 122.17 114.32 13.59 3.30 3.11 4.14 4.03 4.19 3.80 3.88 49.39 34.36 24.68 24.06 12.98 10.50 8.73 10.68 8.26 30.02 18.01 11.80 8.43 8.33 31.35 31.80 32.25 Average: Std_Dev: 19083 14805 14193 14487 5723 162 0 3 0 14416 41441 1820 6 0 15669 16547 15556 10230 11006 155.88 137.30 134.43 135.81 85.36 14.36 0.00 1.95 0.00 135.48 229.70 48.14 2.76 0.00 141.25 145.15 140.74 88.72 74.00 4.95 4.36 4.27 4.31 2.71 0.46 0.00 0.06 0.00 4.30 7.29 1.53 0.09 0.00 4.49 4.61 4.47 5.7 11.3 18.4 15.2 19.8 8.6 10.9 3.63 0.43 11.1 6.3 41.4 24.6 22.0 23.2 22.5 87.0 100.0 98.2 100.0 22.9 108.4 56.3 97.5 100.0 28.2 31.7 27.7 2.82 2.35 58.3 36.0 128 Test 4 (Move Test): Subject A Program Output Calculations Required Time: 60 sec. Time Increment: 1 sec. Acceptable Data Grab PC Time Ave. (HH:MM:SS) Lumin. 1 20:57:29 54.02 2 20:57:30 54.07 3 20:57:31 53.80 4 20:57:33 53.27 5 20:57:34 52.74 6 7 8 9 10 11 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 34 35 36 37 40 48 49 50 51 52 RD(mm): 4.3 SF: 13.7 Calc. Thresh. 21.64 21.68 21.51 21.18 20.86 Thresh. Pixels 18249 17701 16717 15902 14646 Diameter (Pixels) 152.43 150.13 145.89 142.29 136.56 Diameter Error (%) (mm) 11.13 10.96 10.65 10.39 9.97 CF: Ave.: 53.58 21.37 16643 145.46 10.62 2.47 20:57:35 52.70 20.84 15666 141.23 4.18 2.9 20:57:36 52.63 20.79 15064 138.49 4.09 4.8 20:57:37 52.47 20.69 15000 138.20 4.09 5.0 20:57:38 52.51 20.72 14737 136.98 4.05 5.8 20:57:39 52.64 20.80 14522 135.98 4.02 6.5 20:57:41 52.55 20.75 14867 137.58 4.07 5.4 20:57:43 52.30 20.59 14217 134.54 3.98 7.5 20:57:44 51.92 20.36 14947 137.95 4.08 5.2 20:57:45 52.30 20.59 14196 134.44 3.97 7.6 20:57:46 57.39 23.71 30671 98.81 2.92 32.1 20:57:47 66.96 29.57 19729 158.49 4.69 9.0 20:57:49 66.93 29.55 20188 160.33 4.74 10.2 20:57:50 59.38 24.93 19156 156.17 4.62 7.4 20:57:51 58.08 24.13 23650 173.53 5.13 19.3 20:57:52 55.85 22.76 24184 175.48 5.19 20.6 20:57:53 55.77 22.71 23310 172.28 5.09 18.4 20:57:54 55.32 22.44 24314 175.95 5.20 21.0 20:57:56 57.22 23.61 25191 179.09 5.29 23.1 20:57:57 57.56 23.81 26396 183.33 5.42 26.0 20:57:58 56.86 23.38 27792 188.11 5.56 29.3 20:57:59 49.62 18.95 20852 162.94 4.82 12.0 20:58:00 49.56 18.91 18654 154.11 4.56 5.9 20:58:07 50.41 19.43 16938 146.85 4.34 1.0 20:58:08 55.37 22.47 14705 136.83 4.04 5.9 20:58:09 52.66 20.81 16985 147.06 4.35 1.1 20:58:11 52.28 20.58 18900 155.13 4.59 6.6 20:58:14 49.00 18.57 21118 163.98 4.85 12.7 20:58:23 51.53 20.12 12919 128.25 3.79 11.8 20:58:24 59.02 24.71 17628 149.82 4.43 3.0 20:58:26 51.10 19.85 15595 140.91 4.17 3.1 20:58:27 51.38 20.03 13778 132.45 3.92 8.9 20:58:28 53.13 21.10 18101 151.81 4.49 4.4 129 53 54 55 56 57 58 59 60 61 20:58:29 20:58:30 20:58:32 20:58:33 20:58:34 20:58:35 20:58:36 20:58:37 20:58:39 Rejected Data 12 20:57:42 29 20:58:01 30 20:58:02 31 20:58:04 32 20:58:05 33 20:58:06 38 20:58:12 39 20:58:13 41 20:58:15 42 20:58:16 43 20:58:18 44 20:58:19 45 20:58:20 46 20:58:21 47 20:58:22 54.60 57.62 56.39 53.80 49.81 50.72 50.81 51.38 51.36 62.18 62.87 59.93 59.32 59.10 58.54 58.48 56.71 53.17 52.91 52.34 52.23 51.21 51.41 49.89 22.00 23.85 23.10 21.51 19.07 19.62 19.68 20.03 20.02 Average: Std_Dev: 14990 19942 23040 13083 23724 23248 20295 13429 16129 18826 4544 138.15 159.35 171.28 129.07 173.80 172.05 160.75 130.76 143.30 151.36 18.84 4.08 4.71 5.06 3.82 5.14 5.09 4.75 3.87 4.24 26.65 27.06 25.26 24.89 24.75 24.41 24.38 23.30 21.12 20.96 20.62 20.55 19.92 20.04 19.11 Average: Std_Dev: 200 30598 42782 44923 40326 47311 37249 49345 45630 44459 45509 46303 39557 44147 38137 39765 11936 7.98 197.38 233.39 239.16 226.59 245.43 217.78 250.66 241.04 237.92 240.72 242.81 224.42 237.09 220.36 217.51 59.51 0.24 5.83 6.90 7.07 6.70 7.26 6.44 7.41 7.13 7.03 7.12 7.18 6.63 7.01 6.51 5.0 9.5 17.7 11.3 19.5 18.3 10.5 10.1 1.5 4.47 0.56 10.9 7.9 94.5 35.7 60.5 64.4 55.8 68.7 49.7 72.3 65.7 63.6 65.5 66.9 54.3 63.0 51.5 6.43 1.76 62.1 12.9 130 Test 4 (Move Test): Subject C Program Output Calculations Required Time: 60 sec.; Time Increment: 1 sec. Acceptable Data Grab PC Time Ave. Calc. (HH:MM:SS) Lumin. Thresh. 1 19:24:53 74.62 46.51 2 19:24:54 75.25 46.96 3 19:24:55 74.37 46.32 4 19:24:56 74.31 46.28 5 19:24:57 74.49 46.41 RD (mm): 3.5; SF: 12.2 Thresh. Pixels 9900 9407 9439 9714 8901 Diameter (Pixels) 112.27 109.44 109.63 111.21 106.46 Diameter Error (%) (mm) 9.20 8.97 8.99 9.12 8.73 CF: Ave.: 74.61 46.50 9472 109.80 9.00 2.57 7 19:24:59 75.31 47.01 7803 99.67 3.18 9.2 8 19:25:00 74.80 46.63 8155 101.90 3.25 7.2 9 19:25:01 74.72 46.58 8286 102.71 3.27 6.5 10 19:25:02 76.95 48.21 6633 91.90 2.93 16.3 11 19:25:03 77.16 48.37 7301 96.42 3.07 12.2 12 19:25:04 77.03 48.28 8704 105.27 3.36 4.1 13 19:25:05 81.05 51.25 7095 95.05 3.03 13.4 14 19:25:06 82.94 52.65 6664 92.11 2.94 16.1 20 19:25:13 73.28 45.52 8146 101.84 3.25 7.2 21 19:25:14 81.42 51.52 6479 90.83 2.90 17.3 23 19:25:17 77.82 48.85 6948 94.06 3.00 14.3 33 19:25:28 75.32 47.02 7298 96.40 3.07 12.2 34 19:25:30 72.84 45.21 9737 111.34 3.55 1.4 35 19:25:31 72.80 45.17 10525 115.76 3.69 5.4 36 19:25:32 73.21 45.47 11791 122.53 3.91 11.6 37 19:25:33 73.18 45.45 9928 112.43 3.58 2.4 41 19:25:38 75.59 47.22 7135 95.31 3.04 13.2 42 19:25:39 75.47 47.13 7912 100.37 3.20 8.6 43 19:25:40 76.85 48.14 7930 100.48 3.20 8.5 45 19:25:42 76.37 47.79 9535 110.18 3.51 0.3 46 19:25:44 76.12 47.60 7571 98.18 3.13 10.6 47 19:25:45 76.30 47.74 8685 105.16 3.35 4.2 48 19:25:46 74.53 46.44 7974 100.76 3.21 8.2 49 19:25:47 73.75 45.87 8591 104.59 3.33 4.7 54 19:25:53 74.72 46.58 7095 95.05 3.03 13.4 58 19:25:58 74.96 46.75 6505 91.01 2.90 17.1 61 19:26:01 74.92 46.72 6097 88.11 2.81 19.8 Average: 8019 100.72 3.21 9.8 Std_Dev: 1356 8.30 0.26 5.3 Rejected Data 6 19:24:58 76.65 48.00 5082 80.44 2.56 26.7 15 19:25:07 87.76 56.26 3710 68.73 2.19 37.4 131 16 17 18 19 22 24 25 26 27 28 29 30 31 32 38 39 40 44 50 51 52 53 55 56 57 59 60 19:25:09 19:25:10 19:25:11 19:25:12 19:25:16 19:25:18 19:25:19 19:25:20 19:25:21 19:25:22 19:25:24 19:25:25 19:25:26 19:25:27 19:25:34 19:25:35 19:25:36 19:25:41 19:25:48 19:25:50 19:25:51 19:25:52 19:25:54 19:25:55 19:25:57 19:25:59 19:26:00 88.51 88.99 76.33 79.39 75.46 77.55 76.20 76.05 76.10 76.03 75.94 77.52 78.24 75.92 83.13 84.64 77.54 81.71 77.06 77.27 76.34 76.18 77.40 78.06 76.32 76.21 76.26 56.83 57.19 47.76 50.02 47.12 48.66 47.66 47.55 47.59 47.54 47.47 48.64 49.17 47.46 52.79 53.92 48.65 51.74 48.30 48.45 47.76 47.65 48.55 49.03 47.75 47.67 47.70 Average: Std_Dev: 2725 3579 5364 1492 16231 3914 26654 22043 20216 19392 16902 26560 29545 5764 19678 17432 3345 16902 5140 4098 3462 3787 18641 2179 3188 4135 3817 10861 9021 58.90 67.50 82.64 43.59 143.76 70.59 184.22 167.53 160.44 157.13 146.70 183.89 193.95 85.67 158.29 148.98 65.26 146.70 80.90 72.23 66.39 69.44 154.06 52.67 63.71 72.56 69.71 107.47 48.59 1.88 2.15 2.63 1.39 4.58 2.25 5.87 5.34 5.11 5.01 4.68 5.86 6.18 2.73 5.05 4.75 2.08 4.68 2.58 2.30 2.12 2.21 4.91 1.68 2.03 2.31 2.22 46.4 38.5 24.7 60.3 30.9 35.7 67.8 52.6 46.1 43.1 33.6 67.5 76.6 22.0 44.2 35.7 40.6 33.6 26.3 34.2 39.5 36.8 40.3 52.0 42.0 33.9 36.5 3.43 1.55 41.6 13.2 132 Appendix 3 Proposed Pupil Edge-Finder Routine (Algorithm) 1) Preliminary considerations: a) Grab an image b) Get the current ave_lumin[i] (the average illumination of the current image) c) Get the current thresh[i] d) Include in the program a routine to ignore all sources of ‘blackness’ other the pupil; e.g. to ignore eyelids include a condition testing for a certain number of consecutive black pixels in the x-dir. or in the y-dir. 2) Tangent-Perpendicular Method: This approach is different from the pixel-count method in that although it may not give an exact screen pixel-count of the pupil, it approximates its actual (or original) shape: a) Use the vertical- and horizontal-line scanning techniques to locate the four borders of the pupil (Figure 39): i) Left edge of the pupil: Xmin ii) Right edge of the pupil: Xmax iii) Upper edge of the pupil: Ymax 133 iv) Lower edge of the pupil: Ymin v) Note, use the 5-consecutive pixel method when scanning (to make sure you’ve hit the pupil and don’t worry about the loss of accuracy that might be involved) ymax xmin xmax ymin Figure 39 Proposed strategy for determining the left, right, upper, and lower boundaries of the pupil. b) Start at Xmin (make that the average of the 5 consecutive pixels that you find) c) Travel up (vertically) say 10 pixels from Xmin (draw that line in another color say blue on the buffer display); let A = Xmin (Figure 40) d) Draw a perpendicular line to this vertical one (i.e. one going to the right) at this point [Xmin, (Ystart –10)]; and let B equal to this point e) Travel right until you hit a “black” point on the pupil’s ellipse, and let C equal to this new (x,y) location; (note A,B,C are all variables that vary for each loop) 134 Figure 40 Sketch of the Tangent-Perpendicular Method for determining as many points as possible on the edge of the pupil. f) Save the locations of A and C in an array called pupil_points[1000], and mark these points in another color on the screen, say red g) Draw a line through A and C, points 1 and 2 on the pupil_points[] array (use a green color for this connecting line); the equation is: y = [(yC – yA) / (xC – xA)] • x and have it travel a distance of 10 further pixels from C (i.e. extrapolation): 135 i) note since this is not a vertical or horizontal line anymore, you will have to round up or down all x or y values to an absolute pixel size (unless this is done automatically) ii) The slope is set to mAC = [(yC yA) / (xC xA)] is set to mAC, i.e. the slope of the line connecting A and C (and thus will be a variable throughout the program) iii) Now that we have moved to point 2, it is declared as A, and thus the slope mAC is declared as mAB iv) From trigonometry, tan = m (all units in radians); thus AB = arctan mAB (rad), where AB is the angle between the points A and the horizontal, and B xB = xA + 10•cos AB (rad) don’t forget to round yB = yA + 10•sin AB (rad), don’t forget to round h) Now we need to find the coordinates of the next point on the pupil (3 or now C), using the line AB (i.e. its slope, mAB) and B: i) the line connecting B and C (BC) is perpendicular () to line AB; thus its slope is mBC = 1/mAB ii) Now we draw a line y = mBC•x starting from point B until a new point on the ellipse (3 or C) is hit 136 iii) Remember this is any point found to have a luminance lower than that of the threshold iv) Since this distance is not known, like in the previous point, the distance is varied from an absolute value of 1 pixel distance up to say 20 pixels, and you have to test whether you have hit a black pixel or not (note how the same angle AB is used here from the property of triangle trigonometry since AB = 90 BC) v) Thus a loop of trials is conducted as follows One pixel distance: xC = Int [xB + (1) sin AB] the y-component: yC = Int [xB + (1) cos AB] … For the 20th pixel: xC = Int [xB + (20) sin AB] the y-component: yC = Int [xB + (20) cos AB] etc. vi) Once you have “hit” the pupil, record point 3 in the pupil_points[] array: (x3,y3) = (xC,yC), and reset the point A as A = C vii) Optional: produce an error or some kind of warning if the pupil is found at a greater distance than 20 pixels, since this implies something wrong with the curvature of the pupil, e.g. that there might be something wrong with the lighting causing the pupil not to have its elliptical shape 137 viii) Also, here’s another option in case a crescent or “bite” has been found: then start from (x1,y1) and go the opposite direction (ccw), this time taking a vertical step down; the rest of the technique is almost identical; when you have hit the other side of the bite then the program should stop (note this does not take into account that there might be multiple bites in the pupil) i) Repeat this loop (going around the pupil clockwise), every time resetting A, B, and C, storing C value into the pupil_points[] array j) Now test to see if you have gotten back to (i.e. less than) the starting ystart horizontal line or not: i) To do so, make a flag when y first becomes greater than ystart (since y increases as you go down); this occurs at half the way round; make this flag be called half_pupil_flag and set it to 1 ii) Now once this flag is ‘true’ and you have gone in addition to y < ystart then the program should stop because you have gone all the way around the pupil, and this point should not be recorded in the pupil_points[] array (since it might slightly affect the data); this flag is named full_pupil_flag and is now set to one 3) Determining the center and area of the pupil: 138 a) Now you have an array of all the hit positions on the ellipse (make sure, they’re all marked in red as stated before on the buffer display and that this is saved to a file) b) Finding the center of the pupil (the center of the major axis): i) Average all the x and y values (add all the x values and divide by the size of the array, then add all the y values and divide by the size of the array) ii) This (xcenter,ycenter) is an approximation to the ellipse’s center c) Calculating the area of the pupil: i) Now make another 1-D array (called radius[] and make its size equal to that of one of the dimensions of the pupil_points[] array) ii) Calculate for each point on the pupil_points[] array, its distance from the (xcenter,ycenter) point or ellipse’s center; e.g. for the first point, the distance: radius[1] = Int { [(xcenterx1)2 + (ycentery1)2]} here the value can also rounded since the distance should also be in pixels iii) Repeat for the rest of the points until you fill the array iv) Recommended: make another buffer display showing the pupil and have it appear in a grey color (by setting it to say 128), then mark all the points on 139 the pupil_points[] array in red, then have the line drawn from each point on the ellipse to the center in say blue v) Now search the radius[] array until you find the greatest value, and call it major_radius (or just a); this signifies the longest line from the center of the pupil to the pupil’s boundary, and is half the length of the major axis (the major axis would be called a diameter if it were a circle) vi) The area of the circle that the pupil’s ellipse represents in this image is thus: actual_pupil_area = • a2 vii) This area should now be compared with the area obtained from the pupil pixel-count method and it should be within say 20% threshold percentage (screen_pupil_area) d) The user should finally be prompted whether he wants to make another grab, save the processed image (either of the mentioned buffer displays to file), or quit the program 140 Appendix 4 A proposed hardware implementation (Outline) Computer software and hardware Digitizer card (input hardware) connected to the PC: Matrox Meteor/RGB Imaging interface between monitoring setup and computer hardware: Matrox Imaging Library (MIL) Products: Matrox Intellicam (Version 2.05) Computer code to process input images (see next section) Monitoring setup: This mechanism is constructed to Regularly capture pupillary responses Process these images (by means of predetermined time and size specifications) using a C code Produce an indication (using a warning signal or light) in case the system detects a clear case of miosis Design consists of: Head-mounted, light, flexible helmet (helmet should allow for extra components to be easily added or removed, such as camera, lenses or output signaler) A small, lightweight camera attachable to the helmet (with possible additional refractive lens configuration as not to cause distraction for the eye/head) The camera should regularly capture images of the eye The optimum time duration between each such “grab” is to be determined in the course of this research The process should account for such factors as initial (or normal) size of the pupil, natural pupil accommodation to changes in illumination, and natural blinking Connection to and from the PC hardware One cable should send the input image to the PC where it is processed 141 The same connection carries the PC output It sends the order for the regular (but not continuous) grabbing Sends the order to signal in case miosis has been detected Computer code C language is used (for its known built-in power in dealing with images The code uses the image file produced by MIL for each grab Processing the grab Image file should contain measures of light intensity (along with other image characteristics for each pixel) This intensity is processed by the code, the lowest being at the region of the pupil The code then determines if this ‘dark’ area is below a certain threshold percentage, and if it has remained in such a state past a threshold duration, and produce an output thereupon (as mentioned) Code should initialize the capturing process, regulate the number and timing between each grab, and send an order to stop grabbing when the process is over (or when miosis has been detected) Code should trigger the signaler in case of miosis