Download 2 data description

Mine Planning and Equipment Selection 2002
MODELING CONDITION AND PERFORMANCE OF MINING EQUIPMENT
Tad S. Golosinski and Hui Hu
Department of Mining Engineering, University of Missouri-Rolla, Rolla, MO 65409-0450, USA
ABSTRACT: The paper follows on the earlier MPES paper that suggested use of data mining techniques to model
operation of mining equipment. It reports on the new developments that concentrated on modeling of performance and
condition of mining trucks based on the analysis of digital data collected in the field by truck vital sign information and
management system. The modeling was done using several algorithms as provided in the Intelligent Miner software
package of IBM. The paper presents approaches with the models built using decision tree algorithm. The models
developed as the result of this work allow for projection of truck condition and performance into future with reasonably
high accuracy. As such they allow for better control of mining operation and are expected to find numerous applications in
mines worldwide.
1 INTRODUCTION
Modern mining equipment equipped with
numerous sensors monitors its condition and
performance. Data collected by these sensors is used
to alert the operator to existence of abnormal
operating conditions and to perform emergency
shutdown if the pre-set upper or lower limits of the
monitoring parameters are reached. This data is also
used for post-failure diagnostics and for reporting
and analysis of equipment performance.
It is believed that availability of this voluminous
data, together with availability of sophisticated data
processing methods and tools, may allow for
extraction of additional information contained in the
data. One method that may permit this is data
mining (Golosinski, T. S. 2001) (Golosinski, T. S.
and Hu, H. 2001).
The research presented in this paper investigates
use of the data collected from various sensors
installed on the mining truck for construction of a
truck model, which may allow for reliable prediction
of both the truck performance and its condition into
the future. Subject to research was data collected by
a variety of sensors installed on off-highway mining
trucks that together constitute the VIMS (Vital
Information Monitoring System) system of
Caterpillar (Caterpillar 1999). The data mining tool
was the IBM Intelligent Miner for Data (IBM 2000).
2 DATA DESCRIPTION
The data used in this research consists of
snapshot (event recorder) and datalogger records,
each containing values of 70 truck parameters
measured over a period of time. The data was
collected from 6 Caterpillar 789B trucks during their
operation in a surface mine.
The snapshot stores a segment of truck history
that contains values of all 70 monitored parameters
recorded during the period of six minutes, each
parameter value recorded once per second. The
snapshot recording is triggered by one of a set of
predefined events, usually occurrence of an
abnormal situation where a specific parameter
reaches a critical value. A snapshot record describes
truck conditions from five minutes before the event
to one minute after the event (Caterpillar 1999). In this paper,
every snapshot record is called “event” for simplicity.
Unlike snapshot, the data logger records values of all truck
parameters that are monitored by VIMS over varying periods
of time, also at one-second intervals (Caterpillar 1999). The
recording and its end are triggered manually, with individual
records covering periods of up to 30 minutes of truck
operation. Datalogger records do not have to be associated
with any events.
Of the 70 truck parameters used in this research, values of
26 were recorded as categorical and the remaining 44 as
numeric values. The examples of basic statistical description
of both the categorical and numerical parameter values are
presented in Table 1 and 2. Previously research investigated
the data mining directly on the second data (Hu, H. and
Golosinski, T. S. 2002). The new approach uses the statistical
data calculated from one to three minute intervals. The major
statistical parameters include:









Minimum
Maximum
Range
Average
Standard Deviation
Variance
Regression Intercept
Regression Slope
Regression Sum of Square
Table 1. Example of categorical parameter values
Parameter Name
Modal Value
ACTUAL_GEAR_352
Neutral
Modal Frequency
(%)
41.55
AFTRCLR_LVL_137
OK
98.95
BODY_LVR_727
Not Moving
95.4
BODY_POS_726
Down
93.7
Table 2. Example of numerical parameter values
Parameter Name
AFTRCLR_TEMP_110
Minimum Maximum
Value
Value
0
95
Mean
Value
41.8
Standard
Deviation
12.8
AMB_AIR_TEMP_791
0
38.5
21.9
7.0
ATMOS_PRES_790
0
93
89.4
9.1
BOOST_PRES_105
0
164
31.0
50.1
Figure 1 explains how the model predicts one of the VIMS
events, “high engine speed”. In this illustration, the VIMS data
is aggregated into statistical data of every three minutes and
“high engine speed” is the predicted VIMS event. The
Mine Planning and Equipment Selection 2002
predicted label is the name of the class defined in
Intelligent Miner for classification. If one threeminute data is classified as the “Eng_1” by the
model, it recognizes this data has similar statistical
characteristics with the first three minutes of the
“high engine speed” snapshot. Because the event
actually takes place at the end of the fifth minute of
the snapshot (total of six minutes), it is possible that
high engine speed will take place after two minutes
according to the interpretation.
VIMS Event Prediction
Normal Engine Speed
High Engine Speed
Normal Engine Speed
Snapshot
VIMS
Data
0 0 0 0 0 0 1 2 3 4 5 6 0 0 0 0 0
High Eng
Event_ID
Predicted
Label
Other
Other
767_1
767_2
Eng_1
Eng_2
Other
Other
Figure 1. VIMS Event Prediction
As described in Figure 1, the one-minute model
can provide 1-4 minutes early prediction; and the
two-minute model can provide 1 and 3 minutes early
prediction; and the three-minute model can only
provide two minutes early prediction.
3 MODELING DESIGN
3.1. Objective
Modeling was designed to evaluate and quantify
the pattern of changes in parameter values as
associated with various events. As the sensors
installed on the truck activate the snapshot recorder
when the predefined limit of a parameter is reached,
the objective was to identify any patterns in
parameter values that may allow for early failure
recognition. These patterns were then used for
prediction of future events by building a decision
tree classification model of the truck. The model
was to predict an occurrence of a selected event
based on the pattern of changes in values of other
parameters.
The “high engine speed” events recorded most
frequently in the available VIMS data set was
selected as the main targets of analysis. In addition
other events, the data collected during normal
operation was selected as class “other” as well. The
“engine speed” is defined as the actual rotational
speed of the crankshaft. For the modeled truck this
event is activated when the engine speed reaches
2250 rpm and deactivated when the speed drops to
1900 rpm.
3.2. Data Mining Tools
IBM Intelligent Miner software package was
used as the data mining tool. The basic algorithm
used was SPRINT, a modified CART (Classification
and Regression Tree). It was chosen in preference to
the neural network classification algorithm as the Decision
Tree approach is easier to interpret and understand by
engineers, thus facilitating easy analysis of the truck failure
pattern (IBM 1999).
The workings of SPRINT are similar to that of most
popular decision tree algorithms, such as C4.5 (Quinlan J.R.
1993); the major distinction is that SPRINT induces strictly
binary trees and uses re-sampling techniques for error
estimation and tree pruning, while C4.5 partitions according to
attribute values (Jang, J. and Sun, C. 1997). The GINI index is
used to measure the misclassification for the point split by
SPRINT algorithm. For a data set S containing examples from
n classes, the gini(s) is defined as shown in Eq.(1) where pj is
the relative frequency of class j in S. If a split divides s into
two subsets s1 and s2, the index of the divided data ginisplit(s) is
given by Eq.(2). The advantage of this approach is that the
index calculation requires only the knowledge of distribution
of the class values in each of the partitions (Breiman, L. and
Friedman, J. 1984).
(1)
gini (s)  1   p 2j
ginisplit( s ) 
n1
n
gini ( s1 )  2 gini ( s2 )
n
n
(2)
The tree accuracy is estimated by testing the classifier on
the subsequent cases whose correct classification has been
observed (Quinlan J.R. 1993). The v-fold cross-validation
technique estimates the tree error rate. This estimation of error
rate is used to prune the tree and choose the best classifier.
More detail about this algorithm can be found elsewhere
(Shafer, J. 1996).
3.3. Modeling Procedures
The two main procedures of data mining are training
called also model construction, and testing called also model
validation. In training mode, the function builds a model based
on the selected input data. This model is later used as a
classifier. In test mode, the function uses a set of data to verify
that the model created in the training mode produces results
with satisfactory precision.
In this work all available data was split into two parts.
Bulk of the data, 86.4%, was used for model training. The
remainder, 13.6% of available data, was used for model
testing. The test data includes dataset #1 (random selection)
and dataset #2 (whole snapshot and datalogger).
After three models are built on the one, two, and three
minute statistical data, the error rate was defined and used for
evaluating the performance of the training and the testing
processes.
4 RESULTS AND DISCUSSIONS
The model built on three-minute statistical data has less
than 5% training error rate, 19% error rate on test #1, and 14%
error rate on test #2. The model shows better performance on
unseen VIMS event prediction than one- and two- minute
models (Table 3). However, the tradeoff is that this model can
only provide two-minute early prediction with three classes,
“Eng1”, “Eng2” and “Other”.
Mine Planning and Equipment Selection 2002
#2 respectively, which means the model is more robust than
using statistical data at one and two minute interval.
Table 3. Model Performance Comparison
Error Rate %
training
test #1
test #2
one-minute
6.7
24
17.9
two-minute
three-minute
5.7
4.9
29.8
19
15.3
14
o n e - m in u te
tw o - m in u te
th r e e - m in u te
A v e ra g e
E rro r R a te
%
0 .4 5
0 .4 0
Representative three-minute model output is
shown in Figure 1-3 for training, test #1 and test #2
respectively as the confusion matrix. A confusion
matrix for the pruned tree shows the distribution of
the misclassifications. In every matrix, the number
on the diagonal is the correct classification; others
are the number of misclassification.
0 .3 5
0 .3 0
0 .2 5
0 .2 0
0 .1 5
0 .1 0
0 .0 5
0 .0 0
tr a in in g
Total Errors = 28 (4.878%)
Predicted Class --> | OTHER | ENG1
| ENG2
|
---------------------------------------------------OTHER
|
411 |
23 |
4|
total = 438
ENG1
|
1|
65 |
0|
total = 66
ENG2
|
0|
0|
70 |
total = 70
Figure 5. Average Error Rate Comparison
88 |
74 |
o n e -m in u te
E rro r R a te
S ta n d a rd
D e v ia tio n
---------------------------------------------------412 |
te s t
total = 574
tw o -m in u te
th re e -m in u te
0 .4 0
Figure 2. Training with Three-Minute Statistical
Data
0 .3 5
0 .3 0
0 .2 5
Total Errors = 12 (19.05%)
0 .2 0
Predicted Class --> | OTHER
| ENG1
| ENG2
|
0 .1 5
---------------------------------------------------OTHER
|
42 |
9|
0|
total = 51
0 .1 0
ENG1
|
3|
5|
0|
total = 8
0 .0 5
ENG2
|
0|
0|
4|
total = 4
0 .0 0
---------------------------------------------------45 |
14 |
4|
Figure 3. Test#1 with Three-Minute Statistical Data
Total Errors = 9 (14.06%)
Predicted Class --> | OTHER
| ENG1
| ENG2
|
---------------------------------------------------OTHER
|
47 |
5|
0|
total = 52
ENG1
|
4|
2|
0|
total = 6
ENG2
|
0|
0|
6|
total = 6
---------------------------------------------------51 |
7|
6|
tra in in g
total = 63
Figure 6. Error Rate Standard Deviation Comparison
In addition, the error rate, as well as the related error rate
mean and standard deviation, is calculated on every class. As
Figure 5 and 6, the three-minute model presents the best
prediction performance in terms of the 3% and 21% average
error rates and 3% and 26% standard deviation for training
and tests.
Table 4. Error Rate Statistical Calculation
total = 64
Training
Figure 4. Test#2 with Three-Minute Statistical Data
4.1. Model Performance Analysis
After the VIMS data is aggregated into
statistical data at minute interval, the number of
rows is dramatically reduced and the data mining
process is much quicker than mining second data.
Error rates on test data (unseen events) are reduced
into 19% and 14% (Figure 2, 3) for test #1 and test
te s t
Total Test
correct
total
Error Rate %
correct
total
Error Rate %
Other
411
438
0.06
89
103
0.14
Eng1
65
66
0.02
7
14
0.50
Eng2
Average
(Error Rate)
Standard
Deviation
(Error Rate)
70
70
0.00
10
10
0.00
0.03
0.21
0.03
0.26
The three-minute model has only 3% and 21% average
prediction error rate, but the high error rate standard deviation
Mine Planning and Equipment Selection 2002
on tests makes the prediction unstable. The class
“Other” is defined as the operation with normal
engine speed. In three-minute model, 14% error rate
of “Other” (Table 4) on tests statistically implies the
14% probability of the false alarm of the high
engine speed. The class “Eng1” is defined as the two
minutes before high engine speed and the 50% error
rate (Table 4) might implies the 50% probability of
the missing alarm about the high engine speed.
Because the class “Eng2” is defined as a period
overlapping with the snapshot triggered time, it
doesn’t provide any prediction before the event. The
model still has high probability of missing alarm.
4.2. Three-Minute Decision Tree Classification
Model
The approach shows more knowledgeable
decision tree (simpler) that may be presented as a
binary decision tree (figure 7). Each interior node of
the binary decision tree tests an attribute of a record.
If the attribute value satisfies the test, the record is
sent down the left branch of the node. If the attribute
value does not meet the requirements, the record is
sent down the right branch of the node. Three
classes are marked with different colors at upper left
corner. The solid circles are the decision nodes. The
binary decision tree consists of the root node on top,
followed by non-leaf nodes and leaf nodes.
Branches connect a node to two other nodes. Root
and non-leaf nodes are represented as pie charts.
Leaf nodes are represented as rectangles.
Figure 7. Decision Tree Structure on Three-Minute
Statistical Data
From this decision tree (as Figure 7), the root
node, named “ENG_SPD_MAX”, classifies nearly
all “Eng2” events into right leaf (69 out of 70)
displaying as the yellow rectangle. The reason is
that the “high engine speed” event is activated and
recorded when the engine speed reaches the
predefined limit at the second three-minute of the snapshot.
The rule for this classification is:
If (ENG_SPD_MAX>=2184.25)
then class=ENG
The activation value of “high engine speed” event defined
by VIMS is 2250 rpm. This difference with the value
discovered by the decision tree model makes four events,
which are not “Eng2” events, misclassified into “Eng2”.
The rest of events are further classified according to more
complex rules. One of major rules classifies the “Eng1” event
as follows (circled leaf in Figure 7):
If (ENG_SPD_MAX<2184.25)
and (TRBO_IN_PRES_MIN<87.75)
and (ENG_SPD_REGR_SYY<44991960)
and (RTR_LTR_SUSPCYL_REGR_INTERCEPT<-1688.9)
and (GEAR_SELECT_RANGE>=100.5)
then class=ENG1
This rule classified 14 “Eng1” events correctly and only
one is misclassification. It uncovers the three-minute statistical
characteristics of the two minutes before high engine speed.
5 CONCLUSIONS
This approach compresses the information into statistical
table and provides the prediction with certain accuracy. It also
gives the possibility to predict the event for two minutes
earlier. However, the prediction accuracy needs further
improvement and results need verified by more test data. The
possible approach to improve the prediction accuracy is to add
more statistical parameters and use more VIMS data.
REFERENCE
Breiman, L. and Friedman, J. 1984, Classification and
regression tree . Wadsworth International Group.
Caterpillar, Inc. 1999, Vital information management System
(VIMS): system operation testing and adjusting. Company
publication.
Golosinski, T. S. 2001, data mining uses in mining.
Proceedings, Computer Applications in the Minerals
Industries (APCOM), Beijing, China, 2001, pp. 763-766.
Golosinski, T. S. and Hu, H 2001, Intelligent Miner-data
mining application for modeling VIMS condition
monitoring data. Proceedings of the Artificial Neural
Networks in Engineering Conference (ANNIE 2001), St.
Louis, Missouri, U.S.A.
Hu, H. and Golosinski, T. S. 2002, Failure pattern recognition
of a mining truck with a decision tree algorithm. Mineral
Resources Engineering (in print).
IBM (International Business Machines Corporation) 2000,
Manual: “Using the Intelligent Miner for Data”. Company
publication.
IBM (International Business Machines Corporation) 1999,
Intelligent Miner for Data: enhance your business
intelligence. Company publication.
Jang, J. and Sun, C. 1997, Neuro-fuzzy and soft computing.
Prentice-Hall, Inc.
Mine Planning and Equipment Selection 2002
Quinlan, J. R. 1993, C4.5: Programs for machine
learning. Morgan Kaufmann Publishers, Inc.
Shafer, J. 1996, SPRINT: a scalable parallel
classifier for data mining. Proceedings of the
22nd VLDB Conference, Mumbai (Bombay),
India, 1996.