Download Predictive Data Mining for Diagnosis of Thyroid Disease using

International Journal of Research in Management, Science & Technology (E-ISSN: 2321-3264)
Vol. 3, No. 2, April 2015
Available at www.ijrmst.org
Predictive Data Mining for Diagnosis of
Thyroid Disease using Neural Network
Prerana1, Parveen Sehgal2, Khushboo Taneja3
1
Research Scholar, CSE Department, Prannath Parnami Institute of Management and Technology, Hisar, India
2
Research Scholar, NIMS University, Jaipur, Rajasthan, India
3
Assistant Prof., Sharda University, Greater Noida, India
1
[email protected]
[email protected]
3
[email protected]
2
Abstract—This paper presents a systematic approach
for earlier diagnosis of Thyroid disease using back
propagation algorithm used in neural network. Back
propagation algorithm is a widely used algorithm in this
field. ANN has been developed based on back
propagation of error used for earlier prediction of
disease. ANN was subsequently trained with
experimental data and testing is carried out using data
that was not used during training process. Results show
that outcome of ANN is in good agreement with
experimental data; this indicates that developed neural
network can be used as an alternative for earlier
prediction of a disease.
Keywords—Back propagation, decision tree, gradient
descent, prediction, Supervised Learning
I. INTRODUCTION
Disease diagnosis is a very complex and tedious
task; as it requires lots of experience and knowledge.
One of the traditional ways for diagnosis is doctor’s
examination or a number of blood tests. The main task
is to provide disease diagnosis at early stages with
higher accuracy. Data mining plays a vital role in
medical field for disease diagnosis. This paper
presents a study of various disease diagnosis
techniques used in data mining. Data mining is a
process of analysing large data sets to find some
patterns. These patterns can be helpful for prediction
modelling. Hospitals and clinics accumulate a large
amount of patient data over the years. These data
provide a basis for the analysis of risk factors for
many diseases [1].
Generally disease diagnosis is done according to
and depends upon doctor’s experience and
knowledge. Diagnosis is based on signs, symptoms
and physical examination of patient. Predictive data
mining plays a vital role in disease diagnosis.
Artificial neural network can be employed for the
disease diagnosis, so as to improve the quality of
diagnosis. Various data mining approaches exist for
prediction modeling but here we employ trained
2321-3264/Copyright©2015, IJRMST, April 2015
artificial neural networks to develop a predictive
system for disease diagnosis.
Thyroid gland secretes thyroid hormones to control
the body’s metabolic rate. The malfunction of thyroid
hormone will leads to thyroid disorders. The thyroid
or the thyroid gland is an endocrine gland. The
thyroid gland releases thyroxine (T4) and
triiodothyronine (T3) into the blood stream as the
principal hormones. The functions of the thyroid
hormones are to regulate the rate of metabolism and
affect the growth. There are two most common
problems of thyroid disorder or thyroid disease. They
are Hyperthyroidism – releases too much thyroid
hormone into the blood due to over active of thyroid
and Hypothyroidism - when the thyroid is not active
and releases too low thyroid hormone into the blood
[2].
II. LITERATURE SURVEY
Jacqulin Margret et al. proposed the diagnosis of
thyroid disease using decision tree splitting rules [3].
Various splitting rule for decision tree attribute
selection had been analysed and compared. This helps
to diagnosis the thyroid diseases through the extracted
rules. From this work, it is clear, that normalized
based splitting rules have high accuracy and
sensitivity or true positive rate. This work can be
extended for any medical datasets. Further
enhancement can be made by using various
optimization algorithms or rule extraction algorithms.
Anurag Upadhayay et. al. performed the empirical
comparison study by data mining classification
algorithms (C 4.5, C5.0) for thyroid cancer set [4].
Farhad Soleimanian Gharehchopogh et al.
performed a case study in diagnosis of thyroid disease
using artificial neural network [5]. The importance of
using ANNs to diagnose disease is to increase the
accuracy of performance. The appropriate selection of
ANN architecture affects the network performance
effectively to reach the high accuracy. By selecting a
hidden layer and log-sigmoid activation function for
hidden layer and 6 neurons in the hidden layer, we
can reach the classification accuracy for Thyroid
75
International Journal of Research in Management, Science & Technology (E-ISSN: 2321-3264)
Vol. 3, No. 2, April 2015
Available at www.ijrmst.org
disease to 98.6%. The proposed method in this paper
can be a solution to increase the performance of ANN.
So, it can be generalized to the other disease
diagnoses systems of ANN.
S. B. Patel worked to predict the diagnosis of heart
disease patients using classification mining techniques
[6]. Three classification function techniques in data
mining are compared for predicting heart disease with
reduced number of attributes. These are Naïve Bayes,
decision tree and classification by clustering. Genetic
algorithm are also employed to determine the
attributes which contribute more towards the
diagnosis of heart ailments which indirectly reduces
the number of tests which are needed to be taken by a
patient. Fourteen attributes are reduced to 6 attributes
using genetic search. Also, the observations exhibit
that the decision tree data mining technique
outperforms other two data mining techniques after
incorporating feature subset selection with relatively
high model construction time. Naïve Bayes performs
consistently before and after reduction of attributes
with the same model construction time. Classification
via clustering performs poor compared to other two
methods. Inconsistencies and missing values were
resolved before model construction.
III. PROBLEM STATEMENT
A bulk amount of historical data is stored in
hospital information system databases. This
information system can be used for decision making;
but most of the systems are designed to support
inventory, patient billing and generation of simple
statistics. These systems can answer the queries like
“what is the average age of patient having thyroid
disease?”, “Identify the female patients who are above
40 years old and have been treated for cancer” and
“Finding the average number of female or male
patients affected by thyroid disease”. However these
systems cannot answer complex queries like “Given
patient records, predict the probability of patients
getting thyroid disease” and “Given patient records on
cancer, should treatment include surgery, only
medicines ,or both surgery and medicines?”. There
can be some more queries that can only be answered
by a doctor. Answer of all these queries are based on
doctor knowledge or past experience rather than rich
data hidden in database.
IV. DATASET DESCRIPTION
Dataset is taken from UCI machine learning
repository [7]. Dataset is given from Garavan institute
and documentation is given by Ross Quinlan.
Database consists of patients records. Each record is
having 29 attributes. Attribute can be Boolean or
continuous valued.
Table I. Data Description
SN
1
Attribute Name
age
Value Type
continuous
2321-3264/Copyright©2015, IJRMST, April 2015
2
3
4
5
M,F
F,T
F,T
F,T
6
7
8
9
sex
on thyroxine
query on thyroxine
on antithyroid
medication
sick
pregnant
thyroid surgery
i131treatment
10
query hypothyroid
F,T
11
query hyperthyroid
F,T
12
lithium
F,T
13
goitre
F,T
14
tumor
F,T
15
hypopituitary
F,T
16
psych
F,T
17
TSH measured
F,T
18
19
20
21
22
23
24
25
26
27
28
29
TSH
T3 measured
T3
TT4 measured
TT4
T4U measured
T4U
FTI measured
FTI
TBG measured
TBG
referral source
continuous
F,T
continuous
F,T
continuous
F,T
continuous
F,T
continuous
F,T
continuous
WEST, STMW,
SVHC, SVI,
SVHD, other
F,T
F,T
F,T
F,T
Data pre-processing is applied on collected data so
as to remove duplicates. Missing values are filled
manually so as to improve the quality.
V. ANN ALGORITHMS EMPLOYED
Neural network have ability to derive meaning
from complicated or imprecise data. It can be used to
extract patterns and detect trends that are too complex
to be noticed by human beings or other computer
techniques. A neural network is a parallel, distributed
information processing structure that consists of
multiple number of processing elements. These
processing elements are called as nodes. Nodes are
connected via unidirectional signal channels called
connections. Each node has a single output connection
that branches into many connections. NN can be used
to train the system. Training can be done in two ways:
76
International Journal of Research in Management, Science & Technology (E-ISSN: 2321-3264)
Vol. 3, No. 2, April 2015
Available at www.ijrmst.org
Phase I: Propagation
1) Supervised learning
2) Unsupervised learning
The proposed work selects the back propagation
algorithm for prediction analysis of Thyroid disease.
Back propagation is a neural network learning
algorithm. Neural network is a set of connected
input/output units in which each connection has a
weight associated with it. There are many different
kinds of neural networks and neural network
algorithms. The most popular neural network
algorithm is back propagation. The back propagation
algorithm performs learning on a multilayer feedforward neural network.
Every propagation involves:
1) Forward propagation of a training pattern's input
through the neural network in order to generate the
propagation's output activations.
2) Back propagation of the propagation's output
activations through the neural network using the
training pattern's target in order to generate the deltas
of all output and hidden neurons.
Fig. 1. A Multilayer Feed-Forward Neural Network
For many years there has been no rule available for
updating the weight of a multi-layer network
undergoing supervised learning. The weight
adaptation rule is known as back propagation. Neural
networks are mathematical models composed by
several neurons arranged in different layers, linked
through the variable weights. These weights are
calculated by an iterative method during the training
process when the network is fed with a large amount
of training data, input and output pairs that represent
the pattern attempting to be modelled [8]. The back
propagation algorithm works in two phases:
propagation and weight update [9].
Fig.3 General Process diagram
Fig. 2. Back Propagation network
2321-3264/Copyright©2015, IJRMST, April 2015
77
International Journal of Research in Management, Science & Technology (E-ISSN: 2321-3264)
Vol. 3, No. 2, April 2015
Available at www.ijrmst.org
Phase II: Weight update
For each weight-synapse:
1) Multiply its output delta and input activation to
get the gradient of the weight.
2) Bring the weight in the opposite direction of the
gradient by subtracting a ratio of it from the weight.
Repeat the phase I and II until the performance of
the network is good enough.
The work process in diagnosing the disease
includes three basicstages. The first stage is the data
collecting and classifying. The second include
architecture selection and ANN learning and the third
stage is to compare network performance and
reaching to the best answer as indicated in Fig 3. [5].
Fig. 5. Training performance plot when employing gradient
descent method
Fig. 6. Training performance plot when employing
Levenberg Marquardt method
Fig. 7. Gradient plot when employing gradient descent method
Fig. 4. Training of Predictive Neural Network Model implemented
in MATLAB
2321-3264/Copyright©2015, IJRMST, April 2015
78
International Journal of Research in Management, Science & Technology (E-ISSN: 2321-3264)
Vol. 3, No. 2, April 2015
Available at www.ijrmst.org
etc. Also, we can extend our research to finding
theoretical formulations for optimal values of these
changed parameters.
REFERENCES
Fig. 8. Gradient plot when employing Levenberg Marquardt
method
VI. EXPERIMENTAL RESULTS
We have implemented the predictive neural model
to classify the thyroid disease in MATLAB Neural
Network Toolbox software. FTI values have been
taken as input to classify in three different classes
from 1-3 values. Training performance plots for
gradient descent training algorithm and Levenberg
algorithm, presenting variation of MSE verses
numbers of epochs and plots for the variation of error
gradient values during training process are shown in
Fig. 4 to Fig. 8.
The gradient plots depict the variation of error
gradient verses number of epochs. These plots also
present the initial and final values of the gradient
values. It has been observed that Levenberg
Marquardt method has shown a better training
performance for achieving the set target in 59 epochs
and gradient decent is showing a poor performance as
it is unable to achieve the set target value of 0.0001 in
1000epochs.
VII.
CONCLUSION & FUTURE SCOPE
While training the neural network with error back
propagation in conjunction with gradient based
training methods, from our experiments we conclude
that Levenberg Marquardt method has shown a better
performance in comparison with simple gradient
descent algorithm.
Also gradient descent which is first order method
has shown a poor convergence in comparison to
Levenberg Marquardt algorithm. In addition, our
observations conclude that error accuracy limit
achieved by Levenberg Marquardt method is superior
and it trains the models to accuracy level of the order
of 10-5 for the applied data set. Research can be
extended at different angles like analysis and effects
of varying network parameters like number of layers
and number of neurons in hidden layers, learning rate,
adaptive learning rate and other network parameters
2321-3264/Copyright©2015, IJRMST, April 2015
[1]
M. R. Nazari Kousarrizi, F.Seiti, and M. Teshnehlab,” An
Experimental Comparative Study on Thyroid Disease
Diagnosis Based on Feature Subset Selection and
classification”, International Journal of Electrical &
Computer Sciences IJECS-IJENS Vol: 12 No: 01
[2]
Dr. Sahni BS, Thyroid Disorders [online]. Available :
http://www.homoeopathyclinic.com/articles/diseases/thyroid.
pdf
[3]
J.Margret
,B.
Lakshmipathi
and
S.A.Kumar
,
“Diagnosis of Thyroid Disorders using Decision Tree
Splitting Rules”,International Journal of Computer
Applications (0975 – 8887) Volume 44– No.8, April 2012.
[4]
A.Upadhayay, S. Shukla, S.Kumar,” Empirical Comparison
by data mining Classification algorithms (C 4.5 & C 5.0) for
thyroid cancer data set”, International Journal of Computer
Science & Communication Networks,Vol 3(1), 64-68,
ISSN:2249-5789.
[5]
F. S. Gharehchopogh, M. Molany and F. D.Mokri,”USING
ARTIFICIAL NEURAL NETWORK IN DIAGNOSIS OF
THYROID DISEASE: A CASE STUDY”, International
Journal on Computational Sciences & Applications (IJCSA)
Vol.3, No.4, August 2013.
[6]
S.B Patel , P. K Yadav, Dr. D. P.Shukla,” Predict the
Diagnosis of Heart Disease Patients Using Classification
Mining Techniques”,(IOSR-JAVS) , e-ISSN: 2319-2380, pISSN: 2319-2372. Volume 4, Issue 2 (Jul. - Aug. 2013), PP
61-64, (www.iosrjournals.org).
[7]
Thyroid
Data
Set
[online].
Available:
https://archive.ics.uci.edu/ml/datasets/Thyroid+Disease
[8]
J. Han and M. Kamber, Data Mining Concepts and Techniques, Elsevier, Australia, 2006.
[9]
Rumelhart, Hinton and Williams (1986), Learning internal
representations by error propagation
[10]
http://www.emsl.pnl.gov:2080/docs/cie/techbrief/NN.techbri
ef.ht
[11]
A.K. Sen, S.B Patel, Dr. D. P. Shukla,”A Data Mining
Technique for Prediction of Coronary Heart Disease Using
Neuro-Fuzzy Integrated Approach Two Level”, ISSN:23197242Volume 2 Issue 9 Sept., 2013 Page No. 26632671,(www.ijecs.in)
[12]
K. Manikya, Kanti, P, Srinivasa Rao “Prediction of bead
geometry in pulsed GMA welding using back propagation
neural network” J of Mater Process Technol 200 (2008)
,300–305.
K.Sudhakar,Dr. M. Manimekalai, “Study of Heart Disease
Prediction using Data Mining”, Volume 4, Issue 1, January
2014 ISSN: 2277 128X.
[13]
[14]
V. Chaurasia, et al, “Early Prediction of Heart Diseases
Using
Data
Mining
Techniques”,
Carib.j.SciTech,2013,Vol.1,208-217.
[15]
T. P. Pradeep,1S.Venkataraman, “ Literature Review On
History And Screening Models For Parkinsonism”, IJPSM,
e-ISSN 2249 - 7749 ,Print ISSN 2249 – 7757,
www.ijpsmjournal.com.
79
International Journal of Research in Management, Science & Technology (E-ISSN: 2321-3264)
Vol. 3, No. 2, April 2015
Available at www.ijrmst.org
[16]
N. D. Ghetiya, K.M. Patel, “Prediction of tensile strength in
friction stir welded aluminium alloy using artificial neural
network”, ICIAME, Procedia Technology 14 ( 2014 ) 274 –
281
2321-3264/Copyright©2015, IJRMST, April 2015
80