Download International Journal of Innovative

A Bayesian Approach for the Recognition of Control Chart Patterns
M. Kabiri Naeini, M. S. Owlia*, M. S. Fallahnezhad
Mehdi Kabiri Naeini is PhD student at the Department of Industrial Engineering, Yazd University, Yazd, Iran
Mohammad Saleh Owlia is Associate Professor at the Department of Industrial Engineering, Yazd University, Yazd, Iran
Mohammad Saber Fallahnezhad is Assistant Professor at the Department of Industrial Engineering, Yazd University, Yazd, Iran
ABSTRACT
In this research, an iterative approach is employed to recognize and classify control chart patterns. To do this, by taking new
observations on the quality characteristic under consideration, the Maximum Likelihood Estimator of pattern parameters is
first obtained and then the probability of each pattern is determined. Then using Bayes’ rule, probabilities are updated
recursively. Finally, when one of the updated derived statistics falls outside the calculated control interval a pattern
recognition signal is issued. The advantage of this approach comparing with other existing CCP recognition methods is that it
has no need for training. Simulation results show the effectiveness and accuracy of the new method to detect the abnormal
patterns as well as satisfactory results in the estimation of pattern parameters.
KEYWORDS
Control Chart
Pattern Recognition
Bayes' Rule
Maximum Likelihood Estimation
1. Introduction
Control chart pattern (CCP) recognition is one of the important
tools in statistical process control (SPC) to identify process
problems. The observed variation of quality characteristics
generally results from either common causes or assignable causes.
Common causes are considered to be due to the inherent nature of
normal process and assignable causes occur when the process has
been changed in materials, machines, operators etc. Assignable
causes result in the unnatural variation to the process, which should
be identified and eliminated as soon as possible. A normal (NOR)
pattern indicates that the process is operating under natural variation.
Unnatural variations are signaled mainly by exhibition of five types
of unnatural patterns, i.e., cyclic (CYC), increasing trend (IT),
(a) Normal
(b) Cycle
(c) Increasing
Trend
decreasing trend (DT), upward shift (US) and downward shift (DS)
[1], as shown in Fig. 1. There are also other types of unnatural
patterns which are either special forms of basic CCPs or mixed
forms of two or more basic CCPs. Recognition of unnatural patterns
is a crucial task in SPC for identifying underlying root causes.
Traditionally, control chart patterns have been analyzed and
interpreted manually. Over the years, many supplementary rules,
like zone test or run rules have been developed to detect the CCPs
[1, 2]. Artificial intelligence approaches, such as expert system,
fuzzy logic and neural network have been introduced in some
studies. Various kinds of expert systems have been proposed [3-6].
Some researchers [7, 8] have applied the concept of fuzzy sets and
membership functions to detect unnatural patterns. Artificial neural
network is the most popular pattern classifier [9, 10].
(d) Decreasing
Trend
(e) Upward
Shift
(f) Downward
Shift
Fig. 1. Six types of basic CCPs
Some researchers used statistical methods based on correlation
analysis for the problem of CCP recognition [11, 12]. In this
research we introduce a new approach for CCP recognition. In this
approach, in each iteration of data gathering process, using
observation data, the parameters of each pattern are estimated and
the probability of existence of each pattern is calculated and
updated. Updating of probabilities is using a recursive equation
based on Bayes’ Rule. We name the probabilities as belief. When
one of the updated beliefs fell out of its calculated control limits, a
pattern recognition signal i`s issued.
*Email: [email protected]
The rest of the paper is organized as follows: Section 2, introduces
the procedure to estimate patterns parameters. In section 3, the
probabilities and the recursive method to their improvement are first
defined. Then we design the new CCP recognition method. Section
4 contains the performance evaluation of the proposed scheme using
simulation
experiments.
Finally,
the
conclusions
and
recommendation for future research appear in section 5.
For the sake of simplicity, assume only one single observation
(n=1) is gathered on the quality characteristic of interest in each
iteration of the data gathering process. At the tth iteration, let
3. The Proposed Method
3.1. Estimation of Patterns Parameters
In the first step of presented method we obtain the Maximum
Likelihood Estimator (MLE) of patterns parameters. Assume that
the quality characteristic of interest follows a normal distribution
with mean µ and variance σ2. The equations along with the
corresponding parameters of the six basic CCPs are given in Tab. 1.
Normal
Cycle
Trend (IT,
DT)




CCP parameter(s)
Mean (μ)
Standard deviation (σ)
Amplitude (A)
Period (T)

Slope (k)


beliefs based on the observation vector O t 1 and the new
After taking a new observation, x t , let B i ( x t , O t 1 ) , denotes the
CCP equation
probability of existing pattern i in the process.
xt = rt
Now let B i ( x t 1 , O t - 2 ) denotes the prior probabilities of existing
xt = rt+Asin(2πt/T) A,
T >0
pattern i in the process, the posterior probabilities B i (O t ) can be
determined by applying Bayes' rule, thus we have
xt= rt+kt
xt= rt+su(t)
u(t)=1 if i≥P, else
u(t)=0
Note: t = discrete time point at which the pattern is sampled (t = 1, 2, 3, …),
rt = random value of a normal distribution with µ and σ2 at tth time point, and
xt = sample value at tth time point.
Shift (US,
DS)
iteration t, after taking a new observation x t , our aim is to improve
observation x t .
Tab. 1. Equations and parameters for control chart
pattern simulation
CCP
O t  ( x 1 , x 2 , ..., x t ) be a vector of observations on the quality
characteristic of the current and the previous t-1 iterations. In
Magnitude (s)
Position (P)
B i (O t )  B i ( x t , O t 1 )  Pr{Pattern i x t , O t 1 }

Pr{Pattern i , x t O t 1 }
Pr{x t O t 1 }

Pr{Pattern i O t 1 } Pr{x t Pattern i , O t 1 }
Pr{x t O t 1 }
Using equations mentioned in Tab. 1, in case of each pattern, by
subtracting the pattern part from the observation data, the remainder
value is a normal random variable with known parameters µ and σ2,
as presented in Eq. (1-4):
rt = xt
(1)
rt = xt-kt
(2)
rt = xt-s
(3)
rt = xt-Asin(2πt/T)
(4)
By knowing the distribution of rt, parameters k, s, A and T can be
estimated using a point estimation technique like maximum
likelihood estimation. MLE of parameters k, s and A has been
determined in Eq. (5-7) as follows:
kˆ 
2
( n  1)
(x   )
(5)
sˆ  x  


(6)
sin(2 t / Tˆ )( x t )
t 1
n
Aˆ 
recursive equation.
B i ( x t , O t 1 ) 


In
2
To determine Aˆ , first we need to calculate Tˆ by solving the Eq.
(8).
t cos(2 t / Tˆ )( x t 
t 1
n
j 1
Pr{Pattern j O t 1 } Pr{x t Pattern j }
B i ( x t 1 , O t  2 ) Pr{x t Pattern i }
4
j 1
 sin(2 t / Tˆ )(x ) sin(2 t / Tˆ ))  0
 sin (2 t / Tˆ )
t 1
n
t
(8)
2
t 1
A numerical search procedure by testing the values of 4, 5, …, 20
for Tˆ is used to solve Eq. (8). The value of Tˆ that makes the right
hand side of Eq. (8) positive minimum is the solution of this
equation.
3.2. Probabilities and the approach to their
improvement
In the second step of presented method, it is needed to calculate the
probability of existence for each pattern. The pattern that has the
greatest probability of existence, belief, is candidate to be selected.
In each iteration of data gathering process beliefs are calculated and
updated using a recursive equation based on Bayes’ Rule.
(9)
B j ( x t 1 , O j ,t  2 ) Pr{x t Pattern j }
Eq.
(9),
we
need
to
evaluate
the
probability
Pr{x t Pattern i , O t 1 }  Pr{x t Pattern i } . When pattern i exists in
the process then rik
are independent variables with standard
normal distribution. Hence by assuming the existence of pattern i in
the process, we have:
( rit ) / 2
2
Pr{rit Pattern i }=(1/ 2 )e
n


Pr{Pattern i O t 1 } Pr{x t Pattern i }
4
(7)
sin (2 t / Tˆ )
t 1
n
Thus the beliefs B ( x t , O t -1 ) can be determined by the following
(10)
3.3. Pattern recognition (stopping condition)
In each iteration, beliefs are updated and when one of the updated
beliefs falls outside the calculated control interval a pattern
recognition signal is issued. To calculate the control limits, assume
two biggest values of beliefs are B g 1 (O t ) and B g 2 (O t ) . Thus we
have
 ( rg 1,t ) / 2
2
Pr{rg 1,t Pattern g 1 } 
(1/ 2 )e
Pr{rg 2 ,t Pattern g 2 } 
(1/ 2 )e
 ( rg 2 ,t ) / 2
2
Then by defining the statistic Z t
B g 1 (Ot )
Zt 
(11)
B g 2 (O t )
The recursive equation will be
Zt  e
 ( rg 1,t ) / 2  ( rg 2 ,t ) / 2
2
e
should be in the interval (  2c ,  2c ) to be
2t
accepted as an in control variable. Hence we have
Z t 1
2 ln Z t
 ( rg 1,t ) / 2  ( rg 2 ,t ) / 2
2
2
If normal pattern existed in the process then estimations of all
pattern parameters are approximately equal to zero and therefore all
rit approximately
follow standard normal distribution hence we
conclude that random variables ( rg 1,t ) and ( rg 2 ,t )
2
2
follow a
chi-square distribution with one degree of freedom. Thus we have

1
1
 ln Z t  ln Z t 1  
Z t 1
2
( rg 1,t ) 
2
1
2
( rg 2 ,t )
2
2

tc
)
1
B g 1 (O 0 )
2
2
2
one and it never falls out of the control limits. Hence we assume
that if for a predetermined value like q of sample observations, they
all fall in the control limits, we conclude that no pattern exists in the
process and it is in state of normal. The parameters c and q are
adjusted through experiment.
1
Thus we have
1
2
( 1 (t )   2 (t ))
2
(12)
2
1 (t )  t  2 (t )  t
follow standard normal distribution. Now
2t
2t
from Eq. (12) we have

4
j 1
 ( rit )2 / 2
2 )e
B j ,t 1 (1 /
, the belief statistic
 ( r jt )2 / 2
,will be updated.
statistic of Z t is obtained.
Step 6: If t is smaller than q and the statistic of Z t is in the control
1 1 (t )  t  2 (t )  t
(

). 2t
2
2t
2t
2
2
1 (t )  t
2
(
B i ,t 
2 )e
B i ,t 1 (1 /
rit
Step 5: By selecting two biggest beliefs ( B g 1,t and B g 2 ,t ), the
2
,
Step 4: For each pattern, using
2
variances equal to 2t thus for sufficiently large t according to the
Central Limit Theorem it is concluded that random variables
2 ln Z t
( B 1, 0  B 2 , 0  B 3, 0  B 4 , 0  0.25 ).
determine the residual, rit (Eq. (1-4)).
2
Since random variables  1 (t ),  2 (t ) have mean equal to t and
2
Steps of the proposed method are as follow:
Step 0: Set the probabilities of each patterns as equal
Step 1: In iteration t get the new observation.
Step 2: Using t observations and according to Eq. (5-8), estimate the
patterns parameters.
Step 3: For each pattern, using estimated parameters in step 2,
B g 2 (O 0 )
ln Z t  
approximately the same and the values of Z t will be approximately
Z t falls out of the derived control limits, a pattern signal is issued.
( 1 (t )   2 (t ))
have equal values 0.25 hence
ln Z t  
Consequently the values of B g 1 (Ot ) and B g 2 (Ot ) will be
In the proposed approach, a recursive equation is used to update the
probability of each pattern. Then, when the defined statistic of
For the initial values of B i (O0 ) i=1,2,3,4 it is assumed that they
2t
 2 (t )  t
1 (t )  t
interval (e
tc
,e

tc
) , go to step 1.
Step 7: If t is smaller than q and the statistic of Z t is not in the

 2 (t )  t
)
13
2t
2


2
2t
2
Since (
,e
pattern gr exists in the process.
When no pattern exists in the process, the estimated values for
parameters of each pattern will be approximately zero thus the
values of random part for each pattern will be approximately equal.
2
ln Z t  ln Z 0  

tc
3.4. The Algorithm of proposed method
( 1 (1)   2 (1))
2
So we have
Z0 

If statistic Z t falls out of these control limits, we conclude that
In other words
Zt
1
1
2
2
ln
 ln Z t  ln Z t 1   ( rgr ,t )  ( rsm ,t )
Z t 1
2
2
Zt
 (  2c ,  2c )  Z t  (e
2t
Z t 1
ln
2 ln Z t
variables
2
Hence
Zt
random variables, will be (  2c ,  2c ) .Where c is the parameter
of control chart. Now from Eq. (13) it is concluded that random
) follows a normal distribution with
2t
2t
mean 0 and variance 2, the standard control charting limits for these

tc

tc
,e
) , existence of pattern g1 is signaled
control interval (e
and decision making process ends.
Step 8: If t is greater than q, the process is in state of normal. (c and
q are constants and the best combination of them could generally be
obtained by simulation experiments.)
4. The Testing Procedure
Start (t=0)
B1,0=B2,0=B3,0=B4,0=0.25
t=t+1
t>q
Yes
Process is in state of
normal
No
Receive new
observation xt
Estimate the
patterns
parameters
Calculate the rit
Since using real pattern data is difficult and economically
unjustifiable, a common approach adopted by most researchers is
using simulated data for developing and validating a CCP
recognizer. Using MATLAB 2008, we generated 60000 (10000×6)
pattern data for testing the presented method.
Recognition accuracy of presented method is introduced through
confusion matrix in Tab. 2. It can be seen that there is a tendency for
the trend patterns to be mostly confused with shift. The presented
recognizer misclassifies about 1.6% of trend patterns as shift patterns.
Shift patterns are the hardest to be classified (97.69%).
Results for classification of normal patterns (98.80%) suggest that
the type I error probability for proposed recognizer does not seem to
be very good.
Recognition accuracy is governed by the slope for the trend, the
magnitude for the shift pattern and the amplitude for the cycle. There
is a much higher probability for the proposed method to recognize
cycle patterns with small amplitude as normal patterns.
Small standard deviation of correct recognition percentage implies
that presented recognizer has a good predictive consistency.
Pattern average run length (PARL) is another performance criteria
used to evaluate the recognition speed of presented method. PARL
is the average number of observations that must be taken before an
a recognition signal.
To get PARL of each pattern, the mean of 10000 run length was
calculated.
Compared with the results from Guh 2002 [13] and Gauri 2009 [14],
it can be seen that presented method obviously has better RA for six
patterns all above 0.97. Also from the experiment results of the PARL
calculation, it can be concluded that presented pattern recognizer has
a faster recognition speed than the traditional Shewhart control chart.
PARLs of method are introduced in Tab. 3.
Tab. 2. Confusion matrix
For each pattern i,
and using rit update
Bit
Find Bgr,t and Bsm,t
and calculate Zt
e-ct˄0.5 <Zt<e-ct˄0.5
Normal
Trend
Shift
Cycle
Normal
0.9880
0
0.0001
0.0119
t=0.1
Trend
0.0001
0.9773
0.0158
0.0068
s=1.5
Shift
0.0074
0.0026
0.9769
0.0131
A=1.5, T=8
Cycle
0.0002
0
0
0.9998
c=1.5, q=50
Tab. 3. Comparison of recognition accuracy
Pattern
RA of presented
method
RA from R.S.
Guh [13]
RA from S.K.
Gauri [14]
Normal
Trend
Shift
Cycle
0.9880
0.9773
0.9769
0.9998
-
0.86
0.90
0.92
0.9362
0.8948
0.8442
0.9638
No
Tab. 4. PARL and standard error
Process is in state of
pattern gr.
c=1.5, q=50
Stop
Fig. 2. Flowchart of proposed method
PARL
Standard Error
50.7
Normal
50.4
t=0.1
Trend
27.3
4.8
s=1.5
Shift
16.3
18.3
A=1.5, T=8
Cycle
22.4
75.6
5. Discussion
Recognition accuracy of method is governed by the pattern
parameters (slope for the trend pattern, magnitude for the shift
pattern and the amplitude and the period for the cycle). For example
there is a much higher probability for the proposed method to
recognize cycle patterns with small amplitude as normal.
It is assumed that the process follows normal distribution. In
practice it is possible that observations follow any other distribution.
In this case we have two alternative solutions:
1. We can use subgroups for data gathering and then calculate the
average value of observations in each subgroup that follows
normal distribution according to the Central Limit Theorem.
2. We can estimate the pattern parameters by applying the
maximum likelihood method on the regarding probability
distribution.
[4]
[5]
[6]
[7]
[8]
6. Conclusions
This paper described a new statistical approach for recognizing
control chart patterns and estimating the patterns parameters. The
proposed methodology employs an iterative approach based on
Maximum Likelihood Estimator and Bayesian inference. This
method starts out with defining measures called beliefs, and
subsequently the beliefs in the process to be in state of each patterns,
are updated by taking new observations on the quality characteristic
under study. This method employs Bayes’ rule for continuously
updating the knowledge about the state of the process. Then by
means of a control charting method when the stopping condition is
met, the process is determined to be in state of signaled pattern.
The obtained results from simulation experiments have shown the
effectiveness of proposed approach. The proposed framework gives
highly accurate results to detect the abnormal patterns and
satisfactory results in estimation of patterns parameters. Compared
with neural-based methods, recognition results reached by this
system are not based on training samples and will be more reliable
than that reached by neural networks. Second, when compared with
Bayesian techniques, this approach requires no assumptions about
probability distributions functions of the unnatural patterns classes.
There are several directions for future research:

Determining the optimum control threshold of the beliefs is an
interesting subject for research.

Only six types of control chart patterns were studied in this
research. In case of the other patterns such as systematic,
stratification, the performance of method could be studied.

The effect of patterns parameters magnitude, in the
performance of method is another research direction.

Some techniques (e.g. wavelet analysis) could be joined to this
method to improve its discriminating ability.
Acknowledgment
The authors gratefully acknowledge the helpful comments and
suggestions of the reviewers, which have improved the presentation.
References
[1]
[2]
[3]
Western Electric, Statistical Quality Control Handbook,
Indianapolis: Western Electric Company, 1958.
L. S. Nelson, “The Shewhart control chart – test for special
causes,” Journal of Quality Technology, Vol. 16, 1984, PP.
237-239.
J. R. Evans, W. M. Lindsay, “A framework for expert system
[9]
[10]
[11]
[12]
[13]
[14]
development in statistical quality control,” Computers and
Industrial Engineering, Vol. 14, 1988, PP. 335-343.
D. T Pham, E. Oztemel, “XPC: an on-line expert system for
statistical quality control,” International Journal of
Production Research, Vol. 30, 1992, PP. 2857-2872.
C. S. Cheng, N. F. Hubele, “Design of Knowledge-Based
Expert System for Statistical Process Control,” Computers
and Industrial Engineering, Vol. 22, No. 4, 1992, PP. 501-517.
J. A. Swift, J. H. Mize, “Out-of-control pattern recognition
and analysis for quality control charts using LISP-based
systems,” Computers and Industrial Engineering, Vol. 28,
1995, PP. 81-91.
M. S. Yang, J. H. Yang, “A fuzzy-soft learning vector
quantization for control chart pattern recognition,”
International Journal of Production Research, Vol. 40, 2002,
PP. 2721-2731.
M. H. F. Zarandi, A. Alaeddini, I. B. Turksen, “A hybrid fuzzy
adaptive sampling – run rules for Shewhart control charts,”
Information Sciences, Vol. 178, 2008, PP. 1152-1170.
H. B. Hwarng, N. F. Hubele, “Back-propagation pattern
recognizers for X control charts: methodology and
performance,” Computers and Industrial Engineering, Vol. 24,
1993, PP. 219-235.
C. S. Cheng, “A neural network approach for the analysis of
control chart patterns,” International Journal of Production
Research, Vol. 35, No. 3, 1997, PP. 667-697.
A. M. Al-Ghanim, S. J. Kamat, “Unnatural pattern
recognition on control charts using correlation analysis
techniques,” Computers & Industrial Engineering, Vol. 29,
1995, PP. 43-47.
J. H. Yang, M. S. Yang, “A control chart pattern recognition
scheme using a statistical correlation coefficient method,”
Computers & Industrial Engineering, Vol. 48, 2005, PP.
205-221.
R. S. Guh, “Robustness of the neural network based control
chart pattern recognition system to nonnormality”,
International Journal of Quality & Reliability Management,
Vol. 19(1), 2002, PP. 97-112.
S. K. Gauri, “Control chart pattern recognition using
feature-based learning vector quantization”, International
Journal of Advanced Manufacturing Technology, 2009.