Download iii. system organisation for hand gesture recognition

Hand Gesture Recognition System for
Differently-able using LabVIEW
Gunjeet Kaur, Rashpinder Kaur

Abstract— Gestures are natural and intuitive form of
interaction and communication which are used to convey
messages using hand shapes, its movement and orientation.
Usually differently-able people
find difficulty in
interacting with others who don’t understand sign language
as they use sign language for communication. The present
work is based on the need of developing an electronic device
that can translate sign language into commands in order to
make the communication. A prototype has been designed
and developed to cater the needs of a significant group of
disabled and/or elderly by trying to give advancement to
assistance to people who cannot securely operate
conventional services. An input data glove is used and
connected with flex sensors along the length of all fingers
and the thumb. The disabled people can use these gloves to
perform hand gesture and it will be displayed on screen
using LabVIEW software. Flex sensors plays the key role as
they change in resistance depending on the amount of bend
of the sensor. Therefore, this work focuses at designing a
virtual instrumentation system which can be used to provide
communication aid for deaf and dumb.
Index Terms—Differently-able people, hand gesture recognition,
Virtual Instrument, flex sensors.
many approximation processes and Appearance modeling has
low computational complexity that uses real time processing.
[3] Human-machine interface would be more useful if it made
greater use of gestures as humans generally express
themselves through sound, vision and gestures . Therefore
this project designs and builds a man-machine interface using
a flex sensor glove to interpret the sign language. Information
of the hand is required to be collected in order to detect hand
gestures. A judgement has to be made as to the nature and
source of the data. In the current work, flex sensors are used
which outputs a stream of data that changes with degree of
bend. Then the analog outputs from the sensors are fed to the
ADC which performs analog to digital conversion and is fed
to microcontroller. The tool is able to recognize in real time
six different hand gestures, captured using a glove fitted with
flex sensors.
II. OVERALL ARCITECRURE
The proposed system which is designed using LabVIEW was
interfaced with the hardware for Hand Gesture Recognition
System for interpreting the gestures for real time data input.
DESIGN AND DEVELOPMENT
I. INTRODUCTION
Sign language is a communication skill that uses facial
expressions to express gracefully a speaker’s thoughts and
gestures instead of sound to convey meaning which blends
hand shapes, its movement and orientation. We use our hands
constantly to interact with things, pick them up, move them,
activate them in some way or transform their shape. Similarly
in unconscious way, we gesticulate in expressing fundamental
ideas like ‘goodbye’, ‘stop’, ‘go, ‘come’, ‘no’, ‘agreed’, and
so on. Hand Gesture Recognition is a method in which
gestures produced by the user are decoded by the System [1].
Gestures are expressive, meaningful body motions i.e.
physical movements of the fingers, arms, hands, head, face, or
body used for communication. The primary motivation of the
research is to provide an alternative communication tool for
people whose motor abilities are limited.
Vision-based
method which consists of 3D hand modeling and
Appearance modeling and Glove-based method which uses
special glove-based device to extract hand posture are two
common techniques for Hand gesture recognition.[2] 3D
hand modeling has high computational complexity that uses
Hardware for Hand Gesture Detection system
Interfacing between hardware and software
Virtual instrumentation System using LABVIEW
Software
Figure 1: Overall Block Diagram
III. SYSTEM ORGANISATION FOR HAND GESTURE
RECOGNITION
For making this system work, Flex sensors were used. These
sensors are analog resistors which work as variable voltage
analog dividers. Inside the flex sensor are carbon resistive
elements within a thin flexible substrate. When the substrate is
bent it produces a resistance output relative to bend. When the
flex sensor is bent, the resistance gradually increases. [4]
They are usually in the form of thin strip from 1” to 5” long
and the resistance varies between few ohms and kilo ohms. [5]
The flex sensor’s operating temperature is -30 to 80 degree
Celsius. As the flex sensor is bent, the resistance increases to
30- 40 kilo ohms at 90 degrees [6]. There are two types of flex
sensors namely, Bidirectional flex sensor and Unidirectional
Flex Sensor.
developed to take the serial bits data and display the output on
the screen. Depending on the gesture made, different
commands were displayed on the front panel of the software
such as: “NO REQUIREMENT”, “FOOD REQUIRED”,
“WATER REQUIRED” etc.
FLEX SENSOR GLOVE
FOR HAND GESTURE
DETECTION
ANALOG TO DIGITAL
CONVERTOR ADC 0809
MICROCONTROLLER
P89V51RD2
Figure 2: Image of Flex Sensor [7]
MAX 232 CIRCUITIRY
In this project work five sensors are connected serially and
the output from the sensors is inputted to the analog to digital
converter. The outputs from the flex sensors are inputted into
the microcontroller.
USB TO SERIAL
CONVERTOR
VIRTUAL
INSTRUMENTATION
SYSTEM USING
LABVIEW SOFTWARE
5
5
2
3
3
Graphical Output
Figure 4: Block Diagram Representation of Hand Gesture
Recognition System
IV. HARDWARE CONNECTION CIRCUIT
Figure 3: Picture of Gloves with Flex sensors
In Figure 4, Block Diagram Representation of gesture system
is shown. Flex sensor analog input is given to the ADC block
for digital conversion which gives output to Microcontroller
P89V51RD2 [7]. In order to adjust signal to voltage levels
present on the microcontroller pins (TTL standard), it is
necessary to use a voltage level converter. The MAX232 is
used to perform necessary adjustment. It is used to convert a
serial signal from TTL to RS232C standard and vice versa by
means of a built in voltage generator. The female connector
DB9 enables connection with devices that use RS232
standard whereas the 6 pin connector enables connection with
the microcontroller pins intended for serial communication.
USB to serial convertor then sends the serial data to the
computer port. Here, A Virtual Instrumentation System was
In Figure 5, the hardware circuit of the hand gesture system is
shown. Five flex sensors are connected in series. Any change
in sensor’s movement causes change in resistance which in
turn is used to change the input data to the ADC. Hence the
microcontroller takes this data from the ADC and performs
the function as per the code. The interfacing between the
controller section and the LabVIEW software is done using
MAX232 circuit followed by DB9 connector. This output is
readable by any person trying to communicate with the
disabled person or the patient who cannot speak under critical
conditions. The output can be given to voice section also
which can play out through speaker if the sign is matched. [8]
2
Figure 6: VI for Hand Gesture Recognition (1)
Figure 5: Picture of designed Hardware Section of Hand
gesture Recognition System.
V.
SOFTWARE SECTION
LabVIEW software is used to show the results on screen
graphically. LabVIEW programs are known as virtual
instruments, or VIs as their appearance and operation
resembles physical instruments, such as multimeters and
oscilloscopes. Every VI uses functions that takes the input
from the user interface or any other sources and display that
information to other files. A VI consists of the following
three components:
A case structure is created which has one case named “A”
which is a default value set to notify the user that the hardware
device is ready to perform the function. Then a series of
sequential stacked sequence are used that executes
sequentially. For each finger, there are 3 stacked sequences
used as each finger uses 3 bits to denote serial data. Hence for
five fingers total 15 stacked sequences are used. Inside the
stack, visa read tool is used to ready the specified bytes from
the device and a decimal string to number convertor is used to
convert the numeric character to the decimal integer. A case
structure is created which has one case named “A” which is a
default value set to notify the user that the hardware device is
ready to perform the function. Then a series of sequential
stacked sequence are used that executes sequentially. For
each finger, there are 3 stacked sequences used as each finger
uses 3 bits to denote serial data. Hence for five fingers total 15
stacked sequences are used. Inside the stack, visa read tool is
used to ready the specified bytes from the device and a
decimal string to number convertor is used to convert the
numeric character to the decimal integer.
• Front panel — it serves as the user interface.
•Block diagram — it contains the graphical source code that
defines the functionality of the VI.
• Icon and connector pane— it identifies the VI so that it can
be used in another VI. A VI within another VI is known as a
subVI. It is same as a subroutine in text-based programming
languages.
VI for Hand gesture recognition system
First of all a VISA serial configure Port is created which
initializes the serial port specified by VISA resource name to
the specified settings. Visa resource name specifies the
resource to be opened. The Visa resource named
“Communication Port” here specifies the session and class.
Baud rate is the rate of transmission whose default value is
9600. Data bits are the number of bits in the incoming data
whose default value is 8. The value of data bits is between five
and eight. VISA resource name out is a copy of the VISA
resource name that VISA functions return. Communication
Port through which serial data is received is selected from the
front Panel window. VISA Read Function reads the specified
number of bytes from the device or interface specified by
Communication port and returns the data in read buffer. Read
Buffer contains the data read from the device and the return
Count contains the number of bytes actually read.
Figure 7: VI for Hand Gesture Recognition (2)
Figure 8: VI for Hand Gesture Recognition (3)
In the end, again a stacked sequence is used to assemble the
values of each finger comprising of 3 stacked sequences each
and display it on front panel. Local variables are used to read
the value of the control from the front panel and are sent to
input variables of formula node. Total 5 formula nodes are
used for five fingers. The output variable is connected to tank
which is displayed in front panel.
VI.
RESULTS
The following results were obtained using LabVIEW
software. Total 6 gestures were designed for various
requirements. If all fingers were high then no requirement was
decoded. When first and second fingers were high and rest
were low then Water required was decoded. Similarly when
first finger and thumb was low then “suffering from pain” was
decoded. This model was tested particular sample of people
and proved to be an efficient model. Hence this real time
hand gesture system is very useful in hospitals for supervision
of patients.
Figure 11: Gesture made for “water required” by the patient
Figure 12: Front panel output for “water required” displayed
using LabVIEW Software
VII.
Figure 9: Gesture made for “No requirements” by the patient
CONCLUSION
This work was done to check feasibility of recognizing sign
language using flex sensor gloves and displaying the data
using LabVIEW software which proved to be an efficient
system. The main feature of this research work is that the
gesture recognizer is a standalone system which is applicable
in daily life and for biomedical purposes. The overall system
gives the design and its implementation for particular needs of
a significant group of disabled or elderly people by trying to
give improvement to assistance to people who cannot
securely operate conventional services.
VIII. APPLICATIONS
Figure 10: Front panel output for “No requirement”
displayed using LabVIEW Software
- Hospitals that need several measurement systems which can
investigate physiological parameters of the patients.
- It can act as Communication aid for deaf and dumb.
-Hand Gesture recognition System can be used in robotics,
Desktop and Tablet PC Applications and gaming.
-Hand gesture recognition system with commonly used
gestures can also be used by uneducated people.
- Military based on hand gestures which can be used for squad
communication. [9]
4
REFRENCES:
[1] Laura Dipietro, Angelo M. Sabatini, Paolo Dario, “A
Survey of Glove-Based Systems and Their Applications”,
IEEE transactions on systems on systems, man, and
cybernetics, Vol. 38, No. 4, July 2008.
[2] Popa M., “Hand Gesture recognition based on
accelerometer sensors”, IEEE 7th International conference on
Networked Computing and Advanced Information
Management (NCM),pp- 115-120,june 2011.
[3] Megha Kolhekar,Swati Khandelwal, Priyanka Chavan,
Mohit Goyal, Chetan Pokale, “A reliable hand gesture
recognition system using multiple schemes”, International
Journal of Research in Engineering, IT and Social Sciences,
IJREISS, Vol. 2. Iss - 11, November 2012.
[4] Prapat Parab, Sanika Kinalekar, Rohit Chavan, Deep
Sharan, Shubhadha Deshpande, “ Hand Gesture Recognition
using Microcontroller & Flex Sensor”, IJSAE,Vol. 2, Iss. – 3,
pp- 518-522, ISSN (e): 2321-7545, 2014.
[5] K. Aarthy, T.S.Keerthana,S. Menaga, S. Monisha, K.C.
Sriharipriya, “ Flex sensor based nonspecific user hand
recognition system”, IJIRS,
Vol. 2,Iss-5, ISSN:
2319-9725,May 2013.
[6] Shoaib Ahmed. V, “Magic gloves - Hand Gesture
Recognition and Voice Conversion System for Differentially
Able Dumb People”, Tech Expo-The Global Summit, London
2012.
[7] Solanki Krunal M., “Indian Sign Languages using Flex
Sensor Glove”, IJETT, Vol. 4, Iss-6, June 2013.
[8] Praveenkumar S Havalagi, Shruthi Urf Nivedita, “The
amazing digital gloves that give voice to the voiceless”,
IJAET, Vol. 6, Iss. 1, pp. 471-480, March 2013.
[9]Tongrod,N. ,Lokavee,S. , Kerdcharoen,T. , Watthanawisu
th, N. ,Tuantranont, A., “ Gestural system based on
mutlti-functional sensors and zigbee networks for Squad
Communication”, IEEE conference on Defense Science
Research Conference and Expo (DSR), pp- 1-4, August 2011.
Gunjeet Kaur is pursuing M.E. Fellowship in Electronics and
Communication (2011-2014) from Chitkara University, Rajpura. She has
passed B.Tech in Electronics and Communication from Punjabi University,
Patiala and has been teaching in Chitkara University since 2011. Her field of
interest is Digital Electronics, Digital Image Processing, Embedded System
and LabVIEW programming.
Rashpinder Kaur is working as an assistant professor in Chitkara
University, Rajpura since 2008. She graduated and post graduated from
Punjab technical university, Jalandhar. Her field of interest is wireless
communication, digital communication and LabVIEW programming. She
has published 9 papers in reputed journal and IEEE conference.