Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
University of Siena Robotics for Rehabilitation and Medical Applications Domenico Prattichizzo Robotics & Systems Lab University of Siena www.dii.unisi.it/prattichizzo [email protected] Centro per la ricerca scientifica e tecnologica ITC - IRST Trento, December 2003 0 University of Siena Outline Visuo-Haptic Interaction 1) Grasp Rehabilitation 2) the FeTouch Project Eye Tracking 3) ASTIDET 1 University of Siena Haptic Interaction ? 2 University of Siena Touch is very important in medicine HAPTIC interfaces from Greek haptesthai : to touch Increasing telepresence through video-haptic interfaces 3 University of Siena Anthromorphic Robot Arm (kinematics) • sensors to measure end-tip position • actuators to apply end-tip force 4 University of Siena Haptic Interface Principle • measure finger positions• compare with the virtual obj. (collision detectio • get finger position • if the hand-avatar is inside, apply a force … 5 University of Siena Ungrounded Haptic Interfaces joint angles are measured and forces applied relative to mount point on the body Cybergrasp by Immersion inc. 6 University of Siena Visio-Haptic Interaction bidirectional flow Haptic Interface VR Visual Interface 7 University of Siena Outline Visuo-Haptic Interaction 1) Grasp Rehabilitation 2) the FeTouch Project Eye Tracking 3) ASTIDET 8 University of Siena Grasp Rehabilitation University of Siena & University Vita-Salute San-Raffaele Other partners Service de Reeducation, Hopitaux Universitaires, Switzerland Institut National de la Sante et de la Recherche Medicale, France Institut Guttmann, Spain FCS Control Systems B.V, Netherlands 9 University of Siena Grasp rehabilitation goals 1) Rehabilitation units equipped with robotic systems helping patients with motor deficits to relearn basic manual skills such as to open doors, use forks and knives, drink a glass of water, or button a shirt. 2) Virtual Reality Environments that allow pre-surgery patients to face the expected deficits after a surgical intervention, by mimicking its effects on fine manual skills. 10 University of Siena Robotic upper-limb rehabilitation HapticMasterTM with the Endo-Ring (project GENTLE/S). 11 University of Siena Upper-limb rehabilitation @ MIT • MIT-MANUS (1990s) - The robot provided guidance, assistance or resistance while the patient executed a pointing movement. Robot-aided patients improved more than the control patients. The gains obtained with this technique remain significant after 3 years. {{{{{{{{{{{{{{{{{{{{{{ Burke Hospital-MIT group Palo Alto Veteran's Affairs Hospital / Stanford group Spaulding Rehabilitation Hospital (1999) Baltimore VA Hospital (2000) 12 University of Siena Upper-limb rehabilitation in Europe • MULOS Motorised Upper Limb Orthotic System is a 5 dofs robot mounted on a wheel chair that can move the joints of the upper limb under joystick control. • REHAROB This project used industrial robots to passively manipulate the arm of patients. • GENTLE project … 13 University of Siena Robotic rehabilitation @ Siena Grasp rehabilitation Simpler robotic devices 14 University of Siena Robotic rehabilitation @ Siena/Milan The Grip Analyzer. The Grip Analyzer is essentially a real object instrumented with several six degreeof-freedom (DOF) transducers for measuring contact forces. 15 University of Siena Robotic rehabilitation @ Reachin The Virtual Grasping Environment. Simulation of a double point contact interaction using two degrees of freedom devices. ReachinTM setup 16 University of Siena Robotic rehabilitation @ Siena Multi finger haptic interaction. Stability analysis Performance evaluation Deformable objects Stabilizing a multi contact haptic intercation: Problem overview 17 University of Siena References D.Prattichizzo, J.K. Salisbury, and A. Bicchi, “Contact and grasp robustness measures: Analysis and experiments.’’ in Experimental Robotics IV}, Lecture Notes in Control and Information Sciences, pages 83--90. Springer-Verlag, 1997. A.Bicchi and D.Prattichizzo, in \Analysis and optimization of tendinous actuation for biomorphically designed robotic systems\ in Robotica 18:23--31, 2000. Workshop ``Multi-point interaction: from robotic hands to haptic interfaces'‘ organized by F. Barbagli, D. Prattichizzo and K. Salisbury at the IEEE International Conference on Robotics and Automation}, 2004, (submitted). To be published on a Springer Verlag Serie. F. Barbagli, D. Prattichizzo, J.K. Salisbury, ``Multi-contact haptic interaction with deformable objects: amulti-rate approach'' in ISRR 2003, the 11th International Symposium of Robotics Research, October 19-22, 2003, Siena, Italy. To be published on a Springer Verlag Serie. 18 University of Siena Outline Visuo-Haptic Interaction 1) Grasp Rehabilitation 2) the FeTouch Project Eye Tracking 3) ASTIDET 19 University of Siena The FeTouch project @ Siena Robotics & Systems Lab Dip. di Pediatria, Ostetricia e Medicina della Riproduzione University of Siena www.fetouch.org University of Siena Medical Applications • training & simulation, for: – diagnosis – tissue palpation • Minimally Invasive Surgery (MIS) – training through simulation – the real thing: “fly by wire” to improve interaction • telesurgery demonstrated on a human in 2002 (at the EITS in Strasburg from New York) 21 University of Siena Confluence of Ultrasound and Haptic Interfaces real patient data Haptic Interface (1KHz) (tactile rendering) Siemens Sonoline Elegra Dipartimento di Pediatria, Ostetricia e Medicina della Riproduzione. Vision (25fps) Isosurface 22 University of Siena The FeTouch System 23 University of Siena Volume and Surface Rendering Direct Volume Rendering (Ray-Casting) NEW ! (geometric) Isosuphase Extraction (Marching Cubes) Marching Cubes (triangles) 24 University of Siena Segmentation and Surface Extraction Ultrasound Image Segmentation Marching Cubes (triangles) 25 University of Siena Haptic Principle 1. [Avatar] Measure hand position (encoders) 2. [Collision Detection] Localize the avatar in VR 3. [Force Rendering] Apply force to the operator(motor) 4. [Visual Feedback] Modify VR (deformable objects) F FR Problems fast force feedback loop 1KHz slow deformable object dynamics … Internal organ modeling 26 University of Siena Proxy and force rendering 27 University of Siena US3D & US3DTouch Session US3D: US3DTouch: Fetus face (baby.vtk) VIDEO Heartbeat to know more www.dii.unisi.it/fetouch www.fetouch.org 28 University of Siena Neonatology in cooperation with 29 University of Siena References B. La Torre, D. Prattichizzo, F. Barbagli, and A. Vicino.``The fetouch project, a tool for augmenting the fetus perception.\ in The Inter. Journal of Medical Robotics and Compurer Assisted Surgery, Jan.2004. B.~la~Torre, D.~Prattichizzo, F.~Barbagli, and A.~Vicino. “The fetouch project.\ in Proc. IEEE Int. Conf. on Robotics and Automation, 2003. B. La Torre, D. Prattichizzo, F. Barbagli, and A.Vicino, ``The fetouch system: Visual-haptic rendering of fetuses.\ in Proc. Euro{Haptics 2003, Dublin, Ireland, 6-9 July 2003. 30 University of Siena Outline Visuo-Haptic Interaction 1) Grasp Rehabilitation 2) the FeTouch Project Eye Tracking 3) ASTIDET 31 University of Siena ASTIDET, Advanced STImuli Design for Eye Tracking Easy Applicability for Ocular Movement Study in Neurology Robotics & Systems Lab Dep. of Neurological and Behavioural Sciences University of Siena University of Siena Eye Tracking PC Eye movement data 33 University of Siena Eye-Tracking: Setup Eye Movements Pan/Tilt Infrared camera (ASL) observed by controlled by Pupil, CR, Line of gaze Control Unit 34 University of Siena Infrared Video-Based Eye-Tracker •CCD camera-IR emitting diodes with light source coaxial with optical axis •Detection of light reflections from cornea (CR) and bright pupil. •Real time processing of eye image sequences and extraction of pupil and CR centroids. •Line of gaze computed using pupil and CR centroids. 35 University of Siena ASTIDET © Stimulus generator 36 University of Siena Experimental Results: Saccades Study Tg=20cs Ta=180cs 12 ° 37 University of Siena vertical movement horizontal movement 38 University of Siena Eye tracking for medical diagnosis • Disease causes abnormal oscillations of the eyes • Images of stationary objects move excessively on the retina (blurring of vision and oscillopsia) 39 University of Siena Ocular rehabilitation Generated scene can change with respect to the subject’s eye movements! 40 Scene Viewing-Fixation 41 University of Siena References A. Rufa, D. Prattichizzo, G.L. Mariottini, A. Vicino, and A. Federico, ``Advanced STImuli Design for Eye Tracking - ASTIDET: a software tool for scene generation, data acquisition and analysis in ocular movements study,'‘ in Proc. of the American Academy of Neurology Conference, San Francisco, USA, April 2004, A. Rufa, D. Prattichizzo, G.L Mariottini, A. Patrichi, A. Vicino, and A. Federico, ``Advanced STImuli Design for Eye Tracking (ASTIDET): easy applicability for ocular movement study in neurology,'' in Atti del Simposio Italiano di Neurologia, Rome, Italy, October 2003. 42 University of Siena Concluding Visuo-Haptic Interaction Grasp Rehabilitation the FeTouch Project Eye Tracking ASTIDET Task Driven Robotic Devices Towards Simplicity (home tools) Modularity rslab.dii.unisi.it www.dii.unisi.it/prattichizzo 43