Stats #18: Quality Control: A Hands-On Workshop (condensed version)
Content: This training class will show you how to use statistical tools to assess the quality of an on-going laboratory or medical process.
Objectives: In this class, you will learn how to:
- implement measures to identify common and special causes of variation and to reduce them,.
- perform Analysis of Means (ANOM) to compare results among a peer group, and
- discover root causes using a Fishbone diagram.
Teaching strategies: Didactic lectures and small group exercises.
Notes: There are no pre-requisites for this class. Please bring a pocket calculator for some simple arithmetic calculations.
Web pages included in this handout:
- Where you can find this handout
- Why I don't use PowerPoint
- Using a pocket calculator to compute a standard deviation
- Calculating an XBAR-S control chart
- Calculating a P control chart
- Calculating an Analysis of Means chart
- Table for Analysis of Means
- How to draw a Fishbone diagram
The workshop will conclude with a summary and moderated discussion. Ample time for discussion of all topics has been allocated. A more detailed version of this handout with additional supporting material is at
Where can you find this handout?
This handout and the handouts that I use for all of my seminars and training classes are a compilation of individual web pages at www.childrensmercy.org/stats. I use the "Include Page" feature of Microsoft FrontPage to combine these into a single page. You can always find the most recent version of this compilation by going to the web address listed at the bottom of this page. Links for the handouts for other seminars and classes appear at www.childrensmercy.org/stats/training.asp.
Why don't I use PowerPoint?
I stopped using PowerPoint for my presentations in the mid 1990's. This was based on Edward Tufte's advice that presenting information in a paper handout is more effective than presenting the information on a projected screen. I found this to be excellent guidance. I enjoy talking when I don't have to wrestle with a laptop computer. I look at my audience more and interact with them better. I elaborate on this in greater detail at www.childrensmercy.org/stats/weblog2004/powerpoint.asp.
Using a pocket calculator to compute a standard deviation (March 1, 2007). Category: Descriptive statistics
Most of the time, I let a computer program like SPSS compute quantities for me, but every now and then, I want to calculate a few simple statistics without the benefit of SPSS. This might involve using paper and pencil or using a pocket calculator. You should do this also, as it greatly increases your confidence level in what SPSS produces. Let me illustrate how you would calculate a standard deviation using a pocket calculator.
Your pocket calculator must have a square root key. A memory key (M+) is helpful. Here's a picture of a nice pocket calculator with both features from www.keysan.com.
Suppose you wanted to calculate the standard deviation of the following four numbers: 12, 10, 11, and 13.
First you need to clear out memory. For some calculators, the simple process of recalling the value from memory (usually the button labeled MR or MRC) will clear memory. For other calculators, you have to press the recall from memory button twice. For still others you will have to press the clear memory button (usually labeled CM or MC). Each calculator is a bit different.
Now enter each data value in succession and press the M+ button.
12 M+
10 M+
11 M+
13 M+Press the recall memory button to get 46 and clear memory. Square this number to get 2116 and divide by the sample size (in this case, 4) to get 529. Write this number down. Now enter each data value again, square it, and add it to memory. For most calculators, the sequence
12 X M+
10 X M+
11 X M+
13 X M+will suffice because the calculator will finish the squaring operation before adding to memory. For other calculators, you will have to press the equals button before adding to memory.
12 X = M+
10 X = M+
11 X = M+
13 X = M+Press the recall memory button to get 534. Subtract the number you wrote down earlier to get 5. Divide by n-1 (in this case, 3) to get 1.6666667. This quantity is the sample variance. Press the square root button to get the sample standard deviation of 1.290994.
As a quick double check of the accuracy of your calculations compute the range of the data (largest value minus the smallest value). The range is usually 4 to 6 times larger than the standard deviation . In this example the range is 3, which is a bit less than 4 times 1.3, but that's still acceptable. Don't worry about it unless the standard deviation is larger than the range (a mathematical impossibility) or smaller than the range by an order of magnitude or more (possible but only in extreme circumstances).
Repetition is the best teacher, so the first few times you do this, run through the steps twice. Getting the same number twice will make you more comfortable with the entire process.
Try to calculate the standard deviation of the following data:
2.1, 2.5, 1.1, 5.3, and 2.2.You should get 1.57734 for the standard deviation. If you didn't get this value, check some of the intermediate calculations. The sum of the five values is 13.2, and when you square this and divide by 5 (not 4!), you will get 34.848. When you square these values and add them, you will get 44.8. The difference between the two numbers is 9.952 and when you divide by n-1 you will get a variance of 2.488. The square root of this quantity, 1.57734, is the standard deviation. Note that the range (5.3 - 1.1 = 4.2) is a bit less than 4 times the standard deviation, but still acceptable.
On your own, show that the standard deviation of
442, 218, and 333is112.0134.
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Calculating an XBAR-S control chart (March 2, 2007) Category: Control charts
The following data represents a weekly evaluation of vaccine potency. The data is taken from
- An adaptation of quality control chart methods to bacterial vaccine potency testing. H. C. Batson, M. Brown, M. Oberstein. J Bacteriol 1951: 61(4); 407-19. [Medline] [PDF]
but I have taken some liberties with the data to simplify the calculations.
Week01 0.716 0.771 0.924
Week02 0.978 1.212 1.176
Week03 0.644 0.903 0.869
Week04 0.869 0.716 0.869
Week05 1.398 1.301 0.934
Week06 1.218 0.924 1.398
Week07 0.876 0.591 0.644
Week08 1.215 1.241 1.021
Week09 1.021 0.954 0.491
Week10 0.690 0.477 0.785
Week11 1.301 1.279 1.220
Week12 1.644 1.176 1.114
Week13 1.146 1.256 1.518Each week, three lots of vaccine are tested for potency. Calculate a control chart for this data.
While many experts in quality control would use an XBAR-R chart for this data, the XBAR-S chart also works well. There are three steps in calculating an XBAR-S chart.
- Compute a mean and standard deviation for each group,
- Plot the means/standard deviations in sequence (i.e., a run chart),
- Draw reference lines at the overall mean and at the three sigma limits.
The mean and standard deviation for each week are shown below
Mean Stdev
Week01 0.804 0.108
Week02 1.122 0.126
Week03 0.805 0.141
Week04 0.818 0.088
Week05 1.211 0.245
Week06 1.180 0.239
Week07 0.704 0.152
Week08 1.159 0.120
Week09 0.822 0.289
Week10 0.651 0.158
Week11 1.267 0.042
Week12 1.311 0.290
Week13 1.307 0.191Here is a plot of the means
and of the standard deviations.
I have included a single reference line at the average of all the data points. For these two charts, the data values fluctuate more or less randomly above and below the reference line. If you noticed eight or more consecutive points on the same side of the center line, you would declare the process to be out of control.
The final step is to compute control limits. These limits are placed at three sigma distance from the overall mean and variation inside these limits is considered normal variation. The formula for control limits for the XBAR chart is
where the constant A3 comes from the following table.
n A3 B3 B4
2 2.659 0 3.267
3 1.954 0 2.568
4 1.628 0 2.266
5 1.427 0 2.089
6 1.287 0.030 1.970
7 1.182 0.118 1.882
8 1.099 0.185 1.815
9 1.032 0.239 1.761
10 0.975 0.284 1.716
11 0.927 0.321 1.679
12 0.886 0.354 1.646
13 0.850 0.382 1.618
14 0.817 0.406 1.594
15 0.789 0.428 1.572
16 0.763 0.448 1.552
17 0.739 0.466 1.534
18 0.718 0.482 1.518
19 0.698 0.497 1.503
20 0.680 0.510 1.490
21 0.663 0.523 1.477
22 0.647 0.534 1.466
23 0.633 0.545 1.455
24 0.619 0.555 1.455
25 0.606 0.565 1.435This table can be found in many books and on several websites. I selected this table from
In this particular example, n=3, so A3=1.954. This produces lower and upper control limits of 0.68 and 1.34. If a single data point falls outside the control limits, your process is out of control.
It is optional, but you can also compute warning limits at two sigma units away from the mean. If you notice two out of three consecutive points outside the warning limits, then your process is out of control. Here is a control chart that includes warning limits.
Notice that the tenth week is below the lower control limit and that two consecutive weeks (11 and 12) fall above the upper warning limit. You can also compute control limits for the standard deviations using the formula
where the constants B3 and B4 come from the same table. When n is small (5 or less), the value of B3 is zero which places no effective lower control limit on the chart. What this tells you is that an individual standard deviation can be extremely small without raising any concern.
Here is a plot of the standard deviations with control limits.
There are no points outside the control or warning limits.
On your own.
Compute a control chart for the following data set (see below). This data set represents vaccine potencies, like the above data set, but it uses fixed standards to assess potency. These numbers appear in the same paper, but again some simplifying assumptions have been made. Don't peek until you've done the work, but the answers are available on a separate web page.
Week01 1.097 1.204
Week03 1.030 1.362
Week05 0.682 0.978
Week07 0.820 1.080
Week09 1.042 0.858
Week11 1.398 1.146
Week13 1.301 1.204
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Calculating a P control chart (March 7, 2007). Category: Control charts
If you are collecting data on proportions with a consistent denominator for each proportion, then you can plot this data on a control chart. This type of chart is called a P chart and it is very simple to calculate. Here is an example of some data that is appropriate for a P chart. An employee was asked to take a hearing test on 24 consecutive weeks. The hearing test consisted of listening to and trying to recognize 50 spoken words that were recorded with some background noise. The score is the percentage of words recognized correctly. This data set is loosely adapted from a larger data set at
Here is the data:
28 24 32 30 34 30 36 32 48 32 32 38
32 40 28 48 34 28 40 18 20 26 36 40The formula for the upper and lower control limits is
where pbar is the average of the individual proportions and n is the denominator for each individual proportion. If you want to compute upper and lower warning limits, the formula for these is
The average proportion is 0.3275 and n is 50. The control limits are computed as
Here is what the control chart looks like:
Notice that all data points are inside the upper and lower control limits and that we do not observe 2 out of 3 consecutive points outside the warning limits. Neither do we see eight consecutive points on the same side of the center line. Thus, this process is in statistical control. This individual's hearing may not be all that good, but there are no unusual deviations from what you would normally expect.
On your own. Two other workers also took the same series of hearing tests (see data below). Compute a P chart for each worker. Don't peek until you've done the work, but the answers are available on a separate web page.
Worker #2:
60 56 78 60 74 70 70 68 82 76 72 76
68 78 76 68 74 56 74 62 60 70 60 84
Worker #3:
34 42 30 24 42 32 30 36 36 48 40 26
46 42 48 24 36 24 48 30 24 28 32 44
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Calculation of Analysis of Means limits (March 6, 2007). Category: Analysis of means
This page shows some of the details for calculating an analysis of means (ANOM) chart.
Resistivity example. This data set comes from the National Institutes of Standards and Technology
The first three digits of the data values are constant, so you need to be careful in calculating means and standard deviations. Do not round with this data set. Resistivity measurements were recorded five times on five separate instruments. There is some concern that the instruments may have small but important differences in resistivity.
Instrument
1 2 3 4 5
196.3052 196.3042 196.1303 196.2795 196.2119
196.1240 196.3825 196.2005 196.1748 196.1051
196.1890 196.1669 196.2889 196.1494 196.1850
196.2569 196.3257 196.0343 196.1485 196.0052
196.3403 196.0422 196.1811 195.9885 196.2090Compare these five instruments using an ANOM chart.
mean var stdev
1 196.2431 0.007651577 0.08747329
2 196.2443 0.019037095 0.13797498
3 196.1670 0.008784212 0.09372413
4 196.1481 0.010863213 0.10422674
5 196.1432 0.007823043 0.08844797The average of the five means is 196.1892 and the average of the five variances is 0.01083183. The square root of this value, 0.1040761, is the pooled standard deviation.
The formulas for the decision limits in an ANOM chart are
where
In this example, I is 5 and N-I is 20. The value of h from the table of critical values for a balanced ANOM is 2.79. The ANOM decision limits are
and the ANOM chart looks like this:
All five means are inside the decision limits, so you would conclude that no individual mean differs from the overall average.
You can also use an ANOM chart for the standard deviations. I will not show the formulas or the calculations for this chart, but here is what it looks like:
All standard deviations are inside the decision limits, so you would conclude that no individual standard deviation differs from the pooled standard deviation.
Toy slingshot. I asked a group of three volunteers to collect some data on their accuracy at hitting a target with a toy slingshot. They took three shots with their dominant hand and measured the distance of each shot from the target. Then they took three shots with their non-dominant hand and took three shots at the target. Their results are recorded below
Name Shot1 Shot2 Shot3
J-D 62 20 14
J-N 21 9 37
A-D 20 24 37
A-N 43 40 26
M-D 17 75 29
M-N 27 21 59My intention was to treat these six rows as if they represented six separate individuals (I did not have enough volunteers!). This is perhaps a bit of an oversimplification, and some time in the future I want to analyze the data as a two factor study. For this weblog entry, though, I want to plot these results using an analysis of means chart with a factor having six levels.
The summary statistics are easily computed.
Name Mean Var Stdev
J-D 32.00 684.00 26.15
J-N 22.33 197.33 14.05
A-D 27.00 79.00 8.89
A-N 36.33 82.33 9.07
M-D 40.33 937.33 30.62
M-N 35.67 417.33 20.43The average of the six means is 32.28. The average variance is 399.6, and the square root of this value, 19.99, represents the pooled standard deviation.
In this example, I is 6 and N is 18. The critical value h is 3.07. The upper decision limit is 64.62, as shown below
and the lower decision limit is -0.06, as shown below
We round the lower decision limit up to zero because a negative result is impossible in this experiment. Here is a graph showing the individual means and the ANOM limits.
Even though there is some disparity in the mean values, these disparities are well within the limits of sampling error.
Although an ANOM chart for the standard deviations is theoretically possible, you should not calculate such a chart unless you have more observations contributing to each individual standard deviation.
Hypothetical change in sample size. For a process with as much randomness as this, it may make sense to ask each individual to shoot at the target five or ten times. What would the decision limits look like if these individual means and standard deviations were based on ten runs rather than three?
In this case, the value of I would remain the same (6) but the value of N would increase to 60. The critical value for h would be 2.71. The upper decision limit is 47.92
and the lower decision limit is
Improvement in toy slingshot experiment. The group then worked on the process and made some improvements. Here is the data after the process was improved
J-D 52 12 18
J-N 5 2 34
A-D 9 22 17
A-N 22 19 8
M-D 25 29 10
M-N 27 3 15The summary statistics for each group are
Name Mean Var Stdev
J-D 27.33 465.33 21.57
J-N 13.67 312.33 17.67
A-D 16.00 43.00 6.56
A-N 16.33 54.33 7.37
M-D 21.33 100.33 10.02
M-N 15.00 144.00 12.00
Avg 18.28 186.56The square root of the average variance, 13.65, represents the pooled standard deviation. The decision limits are
and
which we round to zero. The ANOM chart is
Again, all data points are inside the decision limits.
Analysis of proportions. The analysis simplifies somewhat if your data is a set of proportions rather than a set of means. You no longer need to compute a pooled standard deviation, but instead use a formula for variation that is a simple function of the average proportion. You also do not need to calculate degrees of freedom, and can treat them as an infinite number of degrees of freedom. This is effectively the same thing as replacing a t-distribution with a normal distribution. Here is the formula.
A worker is asked to compare four different hearing tests to assure that they are of comparable difficulty.
Test-1 86%
Test-2 56%
Test-3 90%
Test-4 86%The results represent the percentage of words out of fifty that are identified correctly. The average of these four proportions is 0.795. The decision limits are computed as
and the ANOM chart looks like
The second test appears to be more difficult than average.
On your own.
1. A different group of volunteers was asked to shoot a toy rocket at a target (data shown below). Six different people recorded their accuracy on two consecutive shots. Calculate an ANOM chart for this data.
A 14 39
B 60 20
C 26 9
D 9 12
E 36 21
F 53 182. The following data is fictional. Twenty separate laboratories were sent identical images of a sperm smear with exactly 100 sperm cells and were asked to estimate the proportion of normal cells on the image using WHO-3 standards (data shown below). Calculate an ANOM chart for these proportions. Don't peek but the answers are available on a separate web page.
25 23 22 18 24 30 22 28 29 15
19 35 33 35 33 17 19 19 40 26
This webpage was written by Steve Simon and was last modified on 09/25/2008.
ANOM table for alpha=0.05, part 1 (March 4, 2007). Category: Analysis of means
Here's a table of critical values for analysis of means (ANOM) at an alpha level of 0.05.
I=2 3 4 5
6 7 8 9 10 11
12 13
df=2 4.30 5.88 6.59 7.10 7.49 7.80 8.08 8.31 8.49 8.68 8.84 8.99 2
3 3.18 4.18 4.60 4.92 5.14 5.34 5.52 5.64 5.73 5.88 5.98 6.07 3
4 2.78 3.56 3.89 4.12 4.30 4.46 4.58 4.69 4.77 4.87 4.95 5.02 4
5 2.57 3.25 3.52 3.72 3.88 4.01 4.11 4.20 4.28 4.36 4.43 4.49 5
6 2.45 3.07 3.31 3.49 3.62 3.74 3.83 3.91 3.99 4.05 4.11 4.17 6
7 2.36 2.94 3.17 3.33 3.46 3.56 3.64 3.72 3.78 3.85 3.90 3.95 7
8 2.31 2.86 3.06 3.22 3.33 3.43 3.51 3.58 3.64 3.70 3.75 3.80 8
9 2.26 2.79 2.99 3.13 3.24 3.33 3.41 3.48 3.54 3.59 3.64 3.68 9
10 2.23 2.74 2.93 3.07 3.17 3.26 3.33 3.39 3.45 3.50 3.55 3.59 10
11 2.20 2.70 2.88 3.01 3.12 3.20 3.27 3.33 3.39 3.44 3.48 3.52 11
12 2.18 2.67 2.85 2.97 3.07 3.15 3.22 3.28 3.34 3.38 3.42 3.46 12
13 2.16 2.64 2.81 2.93 3.03 3.11 3.18 3.24 3.28 3.33 3.38 3.42 13
14 2.14 2.62 2.78 2.91 3.00 3.08 3.14 3.20 3.25 3.29 3.33 3.37 14
15 2.13 2.60 2.76 2.88 2.97 3.05 3.11 3.17 3.22 3.26 3.30 3.34 15
16 2.12 2.58 2.74 2.86 2.95 3.02 3.09 3.14 3.19 3.23 3.27 3.31 16
17 2.11 2.57 2.73 2.84 2.93 3.00 3.06 3.12 3.16 3.21 3.25 3.28 17
18 2.10 2.55 2.71 2.82 2.91 2.98 3.04 3.10 3.14 3.19 3.22 3.26 18
19 2.09 2.54 2.70 2.81 2.90 2.96 3.02 3.08 3.13 3.17 3.20 3.24 19
20 2.09 2.53 2.68 2.79 2.88 2.95 3.01 3.06 3.10 3.15 3.19 3.22 20
21 2.08 2.52 2.67 2.78 2.87 2.94 2.99 3.04 3.09 3.13 3.17 3.20 21
22 2.07 2.51 2.66 2.77 2.85 2.92 2.98 3.03 3.08 3.12 3.15 3.19 22
23 2.07 2.50 2.66 2.76 2.84 2.91 2.97 3.02 3.06 3.10 3.14 3.17 23
24 2.06 2.50 2.65 2.75 2.83 2.90 2.96 3.01 3.05 3.09 3.13 3.16 24
25 2.06 2.49 2.64 2.74 2.83 2.89 2.95 3.00 3.04 3.08 3.12 3.15 25
26 2.06 2.48 2.63 2.74 2.82 2.88 2.94 2.99 3.03 3.07 3.10 3.14 26
27 2.05 2.48 2.63 2.73 2.81 2.87 2.93 2.98 3.02 3.06 3.10 3.13 27
28 2.05 2.47 2.62 2.72 2.80 2.87 2.92 2.97 3.01 3.05 3.09 3.12 28
29 2.05 2.47 2.61 2.71 2.80 2.86 2.92 2.96 3.00 3.04 3.08 3.11 29
30 2.04 2.46 2.61 2.71 2.79 2.86 2.91 2.96 3.00 3.04 3.07 3.10 30
32 2.04 2.46 2.60 2.70 2.78 2.84 2.90 2.94 2.99 3.02 3.06 3.09 32
34 2.03 2.45 2.59 2.69 2.77 2.83 2.89 2.93 2.97 3.01 3.04 3.07 34
36 2.03 2.44 2.59 2.68 2.76 2.82 2.87 2.92 2.97 3.00 3.03 3.06 36
38 2.02 2.44 2.58 2.68 2.75 2.81 2.87 2.91 2.95 2.99 3.02 3.05 38
40 2.02 2.43 2.57 2.67 2.75 2.81 2.86 2.91 2.95 2.98 3.02 3.04 40
44 2.02 2.43 2.56 2.66 2.73 2.79 2.85 2.89 2.94 2.97 3.00 3.03 44
48 2.01 2.42 2.56 2.65 2.73 2.79 2.83 2.88 2.92 2.96 2.99 3.02 48
52 2.01 2.41 2.55 2.64 2.72 2.78 2.83 2.87 2.91 2.95 2.98 3.00 52
56 2.00 2.41 2.54 2.64 2.71 2.77 2.82 2.86 2.90 2.94 2.97 3.00 56
60 2.00 2.40 2.54 2.63 2.70 2.76 2.81 2.86 2.90 2.93 2.96 2.99 60
Inf 1.96 2.34 2.47 2.55 2.62 2.67 2.72 2.76 2.80 2.83 2.86 2.88 Inf
2 3 4 5
6 7 8 9 10 11
12 13
This webpage was written by Steve Simon and was last modified on 07/08/2008.
How to draw a fishbone diagram (March 8, 2007). Category: Quality control
This is a condensed version of a handout
that I have used in previous training classes.
The fishbone diagram (also called the Ishikawa diagram, or the case and effect diagram) is a tool for identifying the root causes of quality problems. It was named after Kaoru Ishikawa, the man who pioneered the use of this chart in quality improvement in the 1960's.
If you want to use a Fishbone Diagram, first list the main problem on the right hand side of the paper. Then draw a horizontal line to represent the "backbone" of the diagram. This line is not labeled. Off of the backbone, draw and label major bones: 4 to 7 major categories of causes. A commonly used list of major causes is Management, Manpower, Machines, and Materials. Another possible list is Policies, Procedures, Plant, and People. Then elicit ideas using an approach like brainstorming to place individual causes as minor bones on each major bone. Some people allow the individual causes to have subcauses, which would be attached to the minor bones. This is intended to get at the fundamental or root causes of the problem. Other people do not include this level of detail on their fishbone diagrams.
When you are done, look at the entire diagram. Does it have reasonable balance across the major bones? Are any common themes emerging? Can you identify causes that are measurable and fixable and which you believe are likely to have a large impact on the problem?
In some situations, you may find that a flow diagram of the work process may be more valuable and informative.
Further reading:
- The Memory Jogger, A Pocket Guide of Tools for Continuous Improvement. Brassard, M. (1988) Methuen, MA: GOAL/QPC.
- Basic Tools for Process Improvement: Cause-and-Effect Diagram [PDF]. US Navy Total Quality Leadership Office. Accessed on 2006-03-24. (Model, Quality control) [Excerpt] What is a Cause-and-Effect Diagram? A Cause-and-Effect Diagram is a tool that helps identify, sort, and display possible causes of a specific problem or quality characteristic (Viewgraph 1). It graphically illustrates the relationship between a given outcome and all the factors that influence the outcome. This type of diagram is sometimes called an "Ishikawa diagram" because it was invented by Kaoru Ishikawa, or a "fishbone diagram" because of the way it looks. www.hq.navy.mil/RBA/c-ediag.pdf. Also found at www.management-tools.org/files/c-ediag.pdf and www.saferpak.com/cause_effect_articles/howto_cause_effect.pdf. Other guides are available at www.hq.navy.mil/RBA/text/tools.html.
This webpage was written by Steve Simon and was last modified on 07/08/2008.~~~