Download Type I and Type II error in SPC

The Importance of
Understanding Type I
and Type II Error in
Statistical Process
Control Charts –
Continued
Phillip R. Rosenkrantz, Ed.D., P.E.
California State Polytechnic University
Pomona
ASQ Orange Empire Section
January 10, 2017
http://www.cpp.edu/~rosenkrantz/prinfo/documents/asqdecisionrulescontd.pptx
2
Goals – Brief Review of Presentation-Part 1
in October 2016

Brief review process control and process capability

Explain Type I and Type II error and give examples

Illustrate how the improper use of decision rules creates
excessive Type I error and creates mistrust in SPC

Suggest simple approaches for reducing Type I error in SPC
3
Goals – Continued
 Give
examples of Type II error for common decision
rules and available strategies for reducing Type II error.
 Present
some additional charts available to reduce Type
II error for certain types of assignable causes
 Educate
about the common mistakes in industry related
to using the wrong control chart. This mistake seems to
more often increase Type II error (failure to detect)
rather than Type I error (false alarm).
 Discuss
the proper use of Individuals charts.
4
Assignable vs. Common Cause
Variation
 Dr.
Walter Shewhart developed Statistical Process
Control (SPC) during the 1920s. Dr. W. Edwards
Deming promoted SPC during WWII and after.
 Premise
is that there are three types of variation
 Common Cause Variation
 Assignable (or Special Cause) variation
 Tampering (or over-adjusting)
 Each
of these types of variation require a different
approach or type of action.
5
Common Cause vs. Assignable
Cause Variation
 According
to Dr. Deming’s research, more than 85% of
problems are the result of “common cause” variation.
Management is responsible for the system and it is
their responsibility to work on reducing this type of
variation. Later research puts the estimate at over
94%.
 The
work group is responsible for preventing and
reducing “assignable cause” variation.
 Management
needs to understand these concepts.
6
Tampering – The Third Type of
Variation
 Tampering
is over-adjusting the system caused by a lack
of understanding of variation.
 Sometimes large built in variation is mistaken for a
process going “out of calibration” and needing
adjustment
 Over adjusting actually increases variation by adding
more variation each time the process is changed
 Tampering is a difficult habit to break because many
machine operators consider it their “job” to constantly
adjust their machine.
 SPC
reduces or eliminates unnecessary adjustments.
7
Major Concept #1: Process
Capability
 The
ability of a process to produce within
specification limits
to produce within specifications – process is
“capable”
 Not able to produce within specifications – “not
capable”
 Able
 Often
quantified with process capability
indices
Pp – Ability to stay within specs if centered
 Cpk, Ppk – Ability based on current distribution
 Cp,
8
Major Concept #2: Process Control
Process Control refers to how stable and
consistent the process is.
 “In-control”
- stable and only experiencing systematic or
“common cause” variation.
 “Not in-control” – Process is not stable. Mean and
variation are changing due to identifiable or “special”
causes (usually controllable by those running the
operation).
 Represents
<10% of the problems
Process Capability
What it is
Process Control
Note - no reference to
specs !
In Control
(Special Causes Eliminated)
Out of Control
(Special Causes Present)
Process Capability
Lower Spec Limit
Upper Spec Limit
In Control but not Capable
(Variation from Common Causes
Excessive)
In Control and Capable
(Variation from Common
Causes Reduced)
10
Control Charts
 Walter
Shewhart developed control charts that help
management and workers identify common cause and
special cause variation
Management’s responsibility to reduce common cause
variation
 The work group is primarily responsible for controlling special
or assignable cause variation

 Small
samples are taken periodically with statistics (e.g.,
average, range) plotted on charts and reveal the amount
and type of variation. Control limits are traditionally +/- 3
standard deviations from the process average.
11
Sample Statistical Process Control
(SPC) Chart
12
Use of Control Charts
When the process remains within control limits
with only a random pattern, process variation can
be attributed to common cause variation (random
variation in the system) and is deemed “in
control.” The process is stable and continues.
When the process goes beyond control limits or
is non-random, it is assumed that an assignable
cause is present and deemed “out of control.”
The process is not stable and predictable. Find
and eliminate the assignable cause.
13
Decision or Sensitizing Rules
 Decision
Rules (a.k.a. Sensitizing rules) are used by
operators to determine if a pattern of points indicates a
process is no longer stable, that is: “out-of-control”.
 Some rules are designed to detect changes or shifts in
the process center (mean)
 Some rules are designed to detect changes in the
process variation (standard deviation)
 Some rules are designed to detect a non-normal
patterns (e.g. trends or cycles)
14
15
Types of error when you use
sampling
 Control
charts are based on sampling. Sampling is
subject to two kinds of error:
 Type I error (α): “False Alarm” – The sample
indicates the process is “out-of-control” but is not
 Type II error (β): “Failure to detect” – The sample
indicates the process is stable, but it really is “out-ofcontrol”
 In
most quality situations the larger concern is avoiding
Type II error: “Failure to detect”. However, with SPC
probably the larger concern is Type I error: “False
alarms”
16
Types of Error
Test Says
H0 True
State of
Reality
H0
True
H0
False
H0 False
Type I error: a
No error
False alarm,
producer’s risk
Type II error: b
Failure to detect,
consumer’s risk
No error
17
Examples
 Ho:
Part is good
Ha: Part is bad
 Ho:
Person did not commit the crime
Ha: Person did commit the crime
 Ho:
The appendix is good
Ha: The appendix is bad
 Ho:
The process is in control
Ha: The process in not in control
A look at two decision rules and
the probability of Type I and Type
II errors
18
Rule 1 – Any point outside the 3σ control limits
(probability shown for a sequence of 8 points)
False Alarm
Failure
To Detect
Failure
To Detect
Rule 4 – A run of 8 points on the same side of the
centerline but within the 3σ control limits
False Alarm
Failure
To Detect
Failure
To Detect
21
Overall Type I Error for both rules
22
Cumulative effect of Type I error on a sequence
of 8 points as decision rules are added
The probability of a False Alarm
Increases dramatically as decision rules
are added. It does not take too many
false alarms before operators begin
to lose faith in control charts and
start to ignore them.
23
Type I Error - A Common Problem
That Makes SPC Ineffective
 Too
much Type I error eventually renders SPC
ineffective. People get tired of chasing false alarms.
 Many
experts recommend using two decision rules
(three at the most) to minimize Type I error. Rules 1 and
4 are commonly used.
 Often,
upon set up, software installers toggle on all
decision rules thinking that is desirable.
 If
you use SPC software, ask to see which rules are in
effect.
24
Type of Error
Type I
False alarm.
Sampling error with
probability α.
SPC indicates an
assignable cause is
present but none
found.
Type II
Failure to detect.
Sampling error with
probability β.
Slow response to
presence of some
assignable causes.
Consequences
Main Causes
Solutions
Results are deadly to
SPC efforts. Operators
quickly lose faith in
SPC and cease to
search for assignable
causes.
Time wasted
collecting data
Too many decision rules. Each
additional rule adds to overall Type I
error.
Using I, MR-charts on non-normal
data
Poor measurement capability
Reduce number of
decision rules used
(recommend two decision
rules).
Test for normality.
Gage R&R studies
Not getting full
benefit of SPC. Missed
opportunity to
improve, especially
when capability is
low.
Some defects passed
on prior to detection.
Some temporary
defects never
detected.
Some loss of
confidence.
Using the wrong chart.
Wrong sampling method
Wrong Sampling frequency.
Poor choice of sample size.
Slow detection of small shifts in
processes with low Cp, Cpk.
Not using R or MR-charts.
Not recognizing non-normal
patterns.
Low Defect Level
Monitoring an attribute instead of
a variable.
Using I-chart on Non-normal data.
Use correct chart.
Use better sampling
method.
Increase sampling
frequency.
Use appropriate sample
size.
Add CUSUM, EWMA, or
Moving Average Chart.
Switch from I, MR-charts
to X-bar, R-charts.
Test for normality.
Learn to interpret control
chart patterns.
25
Using the Wrong Chart – 1
 The
underlying probability distribution of the data and
nature of the sample size (constant vs. variable) are
used to choose the proper chart.
 Continuous
random variables (things that are
measured) with n > 1 have an underlying normal
distribution because of the Central Limit Theorem.
 Use x-bar and R-charts
 Continuous
with n = 1
 Use Individuals and Moving Range charts. Very
sensitive to non-normality. Check for normality.
26
Using the Wrong Chart - 2
 Attribute
or discrete data (things that are counted)
 Number or proportion good or bad (e.g. defective
units, scrap rate)
 Underlying distribution is the binomial. Use p or npcharts depending on constant or variable sample
size.
 Number of defects.
 Underlying distribution is the Poisson. Use c or ucharts depending on constant or variable sample
size.
27
Using the Wrong Chart - 3
 Common
problems
 Using x-bar, R-charts on defect data
 Using x-bar, R-charts or Individual, MR-charts on
numbers that are averages
 Using Individuals charts for count or proportion data
 The
wrong chart would generate the wrong estimate of the
standard deviation resulting in the wrong control limits.
Both Type I and Type II error could occur.
 If
averages are treated as raw data create huge Type II
errors. Averages tend toward the mean, not toward limits.
28
Fraction Non-Conforming Data –
Use P Chart (e.g. scrap rate)
P Chart of Ex7-9Num
0.1 6
1
0.1 4
UCL=0.1289
0.1 2
Proportion
0.1 0
0.08
_
P=0.0585
0.06
0.04
0.02
0.00
LCL=0
1
3
5
7
9
11
Sample
13
15
17
19
29
Incorrect use of Individuals Chart. IChart does not show out-of-control
I-MR Chart of Ex7-9Num
UCL=17.19
Individual Value
15
10
_
X=5.85
5
0
-5
LCL=-5.49
1
3
5
7
9
11
13
15
17
19
Observation
16
1
UCL=13.93
Moving Range
12
8
4
__
MR=4.26
0
LCL=0
1
3
5
7
9
11
Observation
13
15
17
19
30
C Chart for defects shows out-ofcontrol points
C Chart of Ex7-53Num
25
1
20
1
1
Sample Count
UCL=17.38
15
10
_
C=8.59
5
0
LCL=0
1
3
5
7
9
11
Sample
13
15
17
19
21
31
Incorrect use of Individuals Chart
shows no out-of-control points
I-MR Chart of Ex7-9Num
UCL=17.19
Individual Value
15
10
_
X=5.85
5
0
-5
LCL=-5.49
1
3
5
7
9
11
13
15
17
19
Observation
16
1
UCL=13.93
Moving Range
12
8
4
__
MR=4.26
0
LCL=0
1
3
5
7
9
11
Observation
13
15
17
19
32
Not using R-charts with x-bar charts
 With
normally distributed data the mean and
standard deviation are statistically independent.
Assignable cause could affect the mean (center),
standard deviation (spread), or both.
 Using
only x-bar charts ignores assignable causes
that affect variation.
 Users
need to be educated on why and how to use
both charts together.
33
Using Individual (I) and Moving Range
(MR) Charts

The Central Limit Theorm does not apply to individuals data so
cannot assume normality. Need to check for normality.

MR method to determine standard deviation uses correlated
(not independent) data which makes interpretation of the MR
chart potentially invalid. Many experts do not trust the MR
chart.

This can be a big problem resulting in either Type I or Type II
error—depending on the underlying distribution.

Alternatives – Switch to x-bar, R-charts (n > 1) if practical,
transform data, be wary of MR-chart patterns.
34
Wrong Sampling Method (a.k.a.
Rational Subgrouping) - 1
1 – Snapshot approach. Samples taken at about
the same time. Gap of time between samples.
 Cheaper
 Can miss assignable causes that come and go
 Detects shifts in the sample mean faster
 Method
2 – Random sample approach. Every unit has an
equal chance of being in the sample.
 Can be more labor intensive and therefore cost more
 Less likely to miss causes that come and go
 Less sensitive to detecting shifts in the sample mean
 Method
Sampling Methods – Default use
Method 1
36
Sampling Method Differences
37
Wrong Sampling Method - 2
 Type
II error if:
 An assignable cause comes and goes between
samples using Method 1.
 A shift is starting to occur using Method II.
 To
decrease Type II error, the best strategies are:
 Switch to Method 1
 Increase sample frequency
 Increase sample size
 All three of the above
38
Introduction to Statistical Quality Control, 6th Edition by Douglas C. Montgomery.
Copyright (c) 2009 John Wiley & Sons, Inc.
39
Wrong Sampling Frequency
 If
frequency between sampling is not close enough,
an assignable cause can occur and make it through
the process and out the door before detected.
 Can
end up with lots of defective product throughout
the system.
– Determine how long until a defective unit
would get to a critical point would be expensive to
correct. Sample frequency should be shortened to
avoid bad product getting to that point.
 Solution
40
Sampling Frequency – Design to
avoid Type II errors
41
Poor Choice of Sample Size
 Some
companies always use the same sample size for
each application of x-bar and R-charts. (e.g., always
using n = 5)
 Inceasing
the sample size can be used to make charts
more sensitive to shifts in the mean or increases in
variation.

For processes that are very capable you can decrease n
and save money.
 For
processes that are not capable, you need to increase
n to protect against small shifts in the mean.
42
Failing to detect small shifts in low
capability processes (Cp, Cpk ≤ 1.67)
 If
Cp or Cpk are low, then small shifts can produce
defective units. One weakness of the traditional Shewart
control charts is in detecting small shifts in the mean—
under 1 or 1.5 standard deviations.
 Can increase sample size and frequency to reduce
Type II error. Added sampling can be expensive.
 Alternative is to add a third chart that is specifically
good at quickly detecting small shifts in the mean:
Either the EWMA chart, CUSUM chart, or Moving
Average chart.
43
Control Limits are based on Process
Variation to Check for Stability
44
Process Capability Affects Your
Concern for Shifts from the Mean
45
When process is not capable a shift
produces defective units
46
Increasing sample size (n)
increases sensitivity to shifts
47
Use CUSUM or EWMA charts to
detect small shifts faster
 A third
chart can be added to the traditional x-bar and Rcharts for protecting against small shifts in the process
mean (1 to 1.5 standard deviations or less). These
charts detect a small shift in half the number of points of
traditional Shewart charts. Patterns are not useful
because data points are correlated.
Sum Chart (CUSUM) – It tracks the
cumulative sum of the distance from the mean
 Cumulative
Weighted Moving Average (EWMA) –
Uses exponential weighting to predict the next point.
 Exponentially
48
Shewart Control Chart for Process
Mean. 1 unit shift after Obs. 20
49
CUSUM for Monitoring the
Process Mean
CUSUM Chart for x
Upper CUSUM
Cumulative Sum
5
5
0
-5
-5
Lower CUSUM
0
10
20
Subgroup Number
30
50
EWMA Chart for Process Mean
51
Failing to Detect Non-normal
Patterns - 1
 Some
assignable causes result in non-normal patterns
that are not easily detected with traditional decision
rules. Analogy would be EEG or EKG where trained eye
can see patterns that reflect a health problem.
trends – changing operators, changing shifts,
tool or equipment wear, or temperature or humidity
swings, can cause defects over time but go
unrecognized on the control charts.
 Cycles,
limits or centerline – Broken measurement
equipment or operators fudging data can cause this.
 Hugging
The Cyclic Pattern
52
Chapter 5
53
Failing to Detect Non-normal
Patterns - 2
 Recommendations
 Training
on how to recognize patterns
 Some sort of historical data base to help people
recognize patterns that repeat themselves.
 Use of Six Sigma Black Belts to watch how control
charts are being used. They can help recognize
patterns and train operators.
54
Dealing with Low-Defect Levels

When defect levels or count rates in a
process become very low, say under 1000
occurrences per million, then there are long
periods of time between the occurrence of a
nonconforming unit.

Zero defects occur

Control charts (u and c) with statistics
consistently plotting at zero are
uninformative.
Introduction to Statistical Quality Control,
55
Dealing with Low-Defect Levels
Alternative

Chart the time between successive
occurrences of the counts – or time
between events control charts.

If defects or counts occur according to a
Poisson distribution, then the time between
counts occur according to an exponential
distribution.
Introduction to Statistical Quality Control,
56
Dealing with Low-Defect Levels
Consideration

Exponential distribution is skewed.

Corresponding control chart very asymmetric.

One possible solution is to transform the exponential
random variable to a Weibull random variable using x
= y1/3.6 (where y is an exponential random variable) –
this Weibull distribution is well-approximated by a
normal.

Construct a control chart on x assuming that x follows
a normal distribution
Introduction to Statistical Quality Control,
57
Choice Between Attributes and
Variables Control Charts

Each has its own advantages and disadvantages

Attributes data is easy to collect and several
characteristics may be collected per unit.

Variables data can be more informative since information
about the process mean and variance is obtained
directly.

Variables control charts can indicate impending trouble
and action may be taken before defectives are produced.

Attributes control charts will not react unless the process
has already changed.
Introduction to Statistical Quality Control,
58
Managing SPC
 Black
Belt or Master Black Belt should be able to set up
the proper SPC Charts and monitor them.
 Issues
to address when designing SPC charts:
 Proper type of chart to use for the situation
 Decision rules being used
 Is the process capable or not capable
 Sample size and sample frequency
 Sampling method
 How assignable causes will be resolved
 Continous training on how to interpret/use results