Stats #18: Quality Control: A Hands-On Workshop
Content: This training class will show you how to use statistical tools to assess the quality of an on-going laboratory or medical process.
Objectives: In this class, you will learn how to:
- implement measures to identify common and special causes of variation and to reduce them,.
- perform Analysis of Means (ANOM) to compare results among a peer group, and
- discover root causes using a Fishbone diagram; and
Teaching strategies: Didactic lectures and small group exercises.
Notes: There are no pre-requisites for this class. Please bring a pocket calculator for some simple arithmetic calculations.
Web pages included in this handout:
- Information about my book
- Where you can find this handout
- Why I don't use PowerPoint
- Practice exercises
- A plea for open-mindedness
- What is a control chart?
- What is a special cause of variation?
- What is a common cause of variation?
- Statistical Koan: The very busy tailor.
- Using a pocket calculator to compute a standard deviation
- Calculating an XBAR-S control chart
- Calculating a P chart
- Calculating an Analysis of Means chart
- Table for Analysis of Means
- Examples of a Fishbone diagram
- Answers for "On Your Own" exercise with XBAR-S control chart.
- Answers for "On Your Own" exercise with P control chart.
- Answers for "On Your Own" exercise with ANOM chart.
The workshop will conclude with a summary and moderated discussion. Ample time for discussion of all topics has been allocated.
A condensed version of this handout is available at
Information about my book, Statistical Evidence in Medical Trials
 I
recently published a book, Statistical Evidence in Medical Trials, What
do the Data Really Tell Us? through Oxford University Press. A good
summary of what this book is about appears on the back cover:"Statistical Evidence in Medical Trials is a lucid, well-written and entertaining text that addresses common pitfalls in evaluating medical research. Including extensive use of publications from the medical literature and a non-technical account of how to appraise the quality of evidence presented in these publications, this book is ideal for health care professionals, students in medical or nursing schools, researchers and students in statistics, and anyone needing to assess the evidence published in medical journals." A review by Rebecca Rooney in the International Journal of Epidemiology states: "This book is a clear, concise, and interesting read and should prove to be a useful guide. The examples and case studies make it easy to understand difficult concepts and the jokes and stories make it fun. There are some salient points and hopefully the reader will be enthused about looking at the published research and be more confident about distinguishing between the good and the bad." More information about the book (supporting materials, answers to the exercises, and other updates) can be found on the web at http://www.childrensmercy.org/stats/evidence.asp. |
Where can you find this handout?
This handout and the handouts that I use for all of my seminars and training classes are a compilation of individual web pages at www.childrensmercy.org/stats. I use the "Include Page" feature of Microsoft FrontPage to combine these into a single page. You can always find the most recent version of this compilation by going to the web address listed at the bottom of this page. Links for the handouts for other seminars and classes appear at www.childrensmercy.org/stats/training.asp.
Why don't I use PowerPoint?
I stopped using PowerPoint for my presentations in the mid 1990's. This was based on Edward Tufte's advice that presenting information in a paper handout is more effective than presenting the information on a projected screen. I found this to be excellent guidance. I enjoy talking when I don't have to wrestle with a laptop computer. I look at my audience more and interact with them better. I elaborate on this in greater detail at www.childrensmercy.org/stats/weblog2004/powerpoint.asp.
Stats >> Training >> Stats #18: Practice Exercise
Form a team of four to six people. The size may be slightly bigger or smaller if the instructor agrees, but teams of size one are not teams. You will receive a packet that includes a toy and a measuring tape. Here are some examples, but your toy may be different.
Flying saucer Toy car with "pull-back". Foam disc gun. Please note that some of the toys are choking hazards, so take appropriate precautions if one of the members of your group is less than three years old.
Create a target on the floor or on a table at a reasonable distance so that there is some challenge in hitting the target with your toy. Each team member should get two practice attempts with your toy. Every member of the team is required to use the toy unless they are too young or too old. Do not measure anything at this point, but do try your best to get as close to the target as possible.
If you cannot get your toy to work, call your child on your cell phone and ask for advice. If this fails, ask the instructor for a different toy.
After every team member has had a practice turn, discuss what strategy you will use to insure consistent and high quality performance from each team member. You may wish to adjust the location of your target if you believe that this would help.
Repeat the practice runs, but now record how close each team member is to the target on two consecutive attempts. Use metric measurements if possible.
If any result is so bad that the results are not measurable (e.g., further away than the maximum length of the tape measure), you are allowed to "take a mulligan" that is, to repeat the run. If any team member requires more than two mulligans, write an outstanding performance review for this person and see if you can get him/her hired by a different team.
Write down the results for each team member and compute a mean and a standard deviation. Hint: when there are only two observations, the standard deviation is equal to the range divided by the square root of two (1.414).
Draw an Analysis of Means (ANOM) chart for your data. For four team members, the critical value for h is 3.889. For five, it is 3.724, and for six team members, it is 3.622.
Here is a sample example using real data. As a classroom exercise similar to the one described above, a group of three volunteers (labeled A, R, and V to protect their anonymity) were asked to take turns hitting a target with a hand launched foam rocket.
The equipment Shooting at the target Measuring the result Since the group was small, I asked each team member to shoot twice with their dominant hand and twice with their non-dominant hand. For each shot, the distance from the target was measured in centimeters. Here is the data (D=dominant hand, N=non-dominant hand).
A-D 14 39
A-N 60 20
R-D 26 9
R-N 9 12
V-D 36 21
V-N 53 18The means, variances, and standard deviations are:
Mean Var Stdev
A-D 26.5 312.5 17.68
A-N 40.0 800.0 28.28
R-D 17.5 144.5 12.02
R-N 10.5 4.5 2.12
V-D 28.5 112.5 10.61
V-N 35.5 612.5 24.75The overall mean is 26.42 and the pooled standard deviation is 18.20. The ANOM limits are
26.4 - 3.724 * 18.2 * sqrt(5/12) = -17.3 (round this to zero)
26.4 + 3.724 * 18.2 * sqrt(5/12) = 70.1Here is a graph of the results:
Now modify your procedure, offer extra practice trials and repeat the experiment.
A plea for open-mindedness (November 2, 2006).
Most people that I work with are quite open minded, but I do encounter, from time to time, someone who is resistant to ideas that originate from outside the sphere of medicine. In particular, some individuals are almost cynical about the application of quality control in health care. The attitude seems to be something like this:
Quality control is an approach that works on assembly lines. I am a doctor not a factory worker, and my patients are not products on an assembly line.
That's a fair statement. Patients are not widgets, and it is a mistake to treat them the same way. But it's also a mistake to think that we can't learn from how other people have approached problems that do indeed bear some semblance of similarity to the problems that you face.
Let me mention a slightly different area where healthcare professionals are indeed listening to and learning from outsiders. Patient safety is a very important issue in hospitals. Healthcare professionals recognize they make mistakes and their patients sometimes suffer from those mistakes. There are numerous well publicized examples of this, such as the following tragic report:
Boy, 6, killed in MRI accident. The Journal News. Melissa Klein and Oliver W. Prichard. (Original publication: July 31, 2001) VALHALLA — A 6-year-old boy died two days after he was smashed in the head by a metal oxygen canister that was pulled by magnetic force into the MRI machine where he was being examined, Westchester Medical Center officials said yesterday. An unidentified hospital employee brought the oxygen tank within reach of the 10-ton magnet's field, and it shot through the air to the center of the machine, the hospital said. (Source: www.mrireview.com/docs/mrideath.pdf).
Or this one:
Doctor mistakenly amputates wrong foot. Dec 28, 2004 (TUXTLA GUTIERREZ, Mexico) - A doctor at a public hospital in southern Mexico mistakenly amputated the right leg of an elderly patient who had sought treatment for an infection in his left foot, the patient's family announced Sunday. Seeking treatment for a foot wound aggravated by diabetes, Alberto Lopez, 74, was admitted to a Social Security Institute hospital in Tuxtla Gutierrez, 430 miles south of Mexico City, and underwent surgery on Friday. But the patient emerged from surgery without a right leg and still suffering from the original infection according to family members who filed a complaint Sunday with the state attorney general's office and a national medical arbitration commission. (Source: abclocal.go.com/wls/story?section=News&id=2552947).
Anecdotes like this produce a strong emotional impact among healthcare professionals and in the general public. There's also the recognition among healthcare professionals that many preventable deaths and illnesses in our hospitals go unrecognized, and simple interventions like regular handwashing are ignored. So who have the doctors and nurses and other medical professionals turned to for help with patient safety? The suprising answer is documented nicely in a recent newspaper article by Kate Murphy in the October 31, 2006 issue of the New York Times, What Pilots Can Teach Hospitals About Patient Safety. This article has a very strong lead.
Wearing scrubs and slouching in their chairs, the emergency room staff members, assembled for a patient-safety seminar, largely ignored the hospital’s chief executive while she made her opening remarks. They talked on their cellphones and got up to freshen their coffee or snag another danish. But the room became still and silent when an airline pilot who used to fly F-14 Tomcats for the Navy took the lectern. Handsome, upright and meticulously dressed, the pilot began by recounting how in 1977, a series of human errors caused two Boeing 747s to collide on a foggy runway in the Canary Islands, killing 583 people. Riveted, a surgeon gripped his pen with both hands as if he might break it, an anesthetist stopped maniacally chewing his gum, and a wide-eyed nurse bit her lip. An attention grabber, yes, but what does an airplane crash have to do with patient safety?
Apparently some pretty important people in the healthcare industry do believe that there is a link
After the Canary Islands accident, NASA convened a panel to address aviation safety and came up with a program called Cockpit or Crew Resource Management. The Federal Aviation Administration requires that all pilots for commercial airlines and the military undergo the training. They learn, among other things, to recognize human limitations and the impact of fatigue, to identify and effectively communicate problems, to support and listen to team members, resolve conflicts, develop contingency plans and use all available resources to make decisions.
Recognizing the positive impact of the program on the aviation industry’s safety record, the Institute of Medicine in 2001 recommended similar training for health care workers. The National Academies, the Agency for Healthcare Research and Quality and the Institute for Healthcare Improvement also advocate the training, as well as the use of other aviation-inspired practices like pre- and post-operative briefings, simulator training, checklists, annual competency reviews and incident reporting systems.
So is there a commonality between landing an airplane at Heathrow and excising a gall bladder?
“The trend is not surprising given the similarities between health care and aviation,” said Dr. David M. Gaba, associate dean of immersive and simulation-based learning at the Stanford University School of Medicine in Palo Alto, Calif. “Both involve hours of boredom punctuated by moments of sheer terror,” he said. In addition to sometimes having to make life-and-death decisions in seconds, pilots and physicians also tend to be highly skilled, Type A personalities, who rely heavily on technology to do their jobs.
There are differences as well and the article points these out.
The definition of an error in health care, Professor Helmreich said, is “fuzzier” than in aviation, where it is easier to identify a “foul-up” and who was responsible. Health care providers’ fear of litigation and losing their medical licenses also hinders the honest reporting of mistakes, whereas aviators are often inoculated against punishment if they promptly report incidents to the authorities. Training programs developed by pilots without knowledge of health care realities can be “appallingly bad,” he said.
I believe that a respectful attitude couched in humility is the best approach for people who are advocating new approaches and people who are listening to those advocates. We can't force fit solutions from the outside that don't respect the unique aspects of healthcare, but neither can we pretend that healthcare is so unique that only an insider can make changes and suggestions for improvement.
If healthcare professionals can learn from pilots, there may even be a sliver of hope that they can learn from Statisticians.
This webpage was written by Steve Simon on 2007-11-02, edited by Steve Simon, and was last modified on 2008-07-08. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page. Category: Quality control
What is a control chart? (November 11, 2006). Category: Definitions, Category: Control charts
A control chart is a graphical tool used in many industrial settings that monitors a work process on a continual and on-going basis. Here is an example of a control chart published in the Engineering Statistics Handbook, published by the U.S. National Institute for Standards and Technology.
Source: www.itl.nist.gov/div898/handbook/pmc/section3/pmc322.htm
There is small typographical error in this chart, but it illustrates the general structure quite well. The control chart is simply a run chart (a plot of a sequence of values) with three reference lines. The center line is typically drawn at the average of all of the data. The control chart also includes two control limits, an upper control limit (UCL) and a lower control limit (LCL). The control limits are set a certain distance away from the center line (I'm deliberately being vague here). Any data values that fall above the UCL or below the LCL are described as "out of control" and represent a "special cause" of variation. If all the data values lie inside the LCL and UCL, the work process is said to be "in control" and all of the observed variation represents "common cause" variation.
This webpage was written by Steve Simon on 2006-11-11, edited by Steve Simon, and was last modified on 2008-07-08. This page needs minor revisions. Category: Control charts, Category: Definitions.
What is a special cause of variation?
A special cause of variation is a variation from the mean that has an assignable cause. When you have a special cause in your work process, you need to investigate immediately (while the trail is still warm).
When you spot a special cause:
- The first thing to do is control any damage or problems with an immediate, short-term fix. Be careful not to view this fix as a permanent solution or the process will never be improved.
- Once a quick fix is in place, search for the cause. Ask people in the process what was different that time. What was out of the ordinary? It might not have been much – an unexpected emergency, a change in schedules, or new materials. The need for this sort of information is part of the reason for collecting very complete data the first time around, noting details and traceability factors about a sample or recorded event.
- Once you have discovered the special cause, you can develop a longer-term remedy. Most special causes have a negative impact on the output of the process and need to be removed. Occasionally, a special cause can have a positive impact depending on the nature of the process. If this is the case, finds ways to capture and integrate it into the system.
Avoid these mistakes:
- Changing the process to accommodate the special cause. This usually adds cost and bureaucracy.
- Blaming individuals. Not only does everyone makes mistakes, but also chances are that the problem would have occurred regardless of individuals involved.
- Exhorting workers to simply "do better." People can only do as well as the system allows them to do.
Source: www.skymark.com/resources/responding_to_variation.asp
This webpage was written by Steve Simon on 2006-11-11, edited by Steve Simon, and was last modified on 2008-07-08. This page needs minor revisions. Category: Control charts, Category: Definitions.
What is a common cause of variation?
A common cause of variation is a variation from the mean that is caused by the system as a whole. This variation is not due to an assignable cause, but rather represents variation inherent in the process you are studying.
When a work process has only common causes of variation and no special causes, that process is "in control." This means that it is stable, consistent, and predictable. It might be predictably good or predictably bad, or it might be a very regular mix of good and bad results.
What do you do with a common cause of variation?
Just because a process is stable, or in statistical control, does not mean that its results are satisfactory. A process may be very consistent, day in and day out making items that are nowhere near specification limits. Or, as the Japanese have done so successfully, variation can be systematically reduced, even in stable processes, enabling a gradual tightening of specification limits, and an overall increase in product quality at lower cost.
Improving a stable process is somewhat more difficult than improving an unstable process because, by definition, a stable process has no special causes of variation that jump out at you, asking to be investigated. Instead, you are faced with the task of looking at all data about the process, not simply what made one point different from the others.
Common causes of variation often lie hidden within the system, and are sometimes assumed to be unavoidable. Yet it is very possible, and often very rewarding, to improve processes and reduce common cause variation. Experience had shown that, amongst the people in and around the process, there are enough ideas for improvements to make a significant impact, even on a sound process. (Source: www.skymark.com/resources/responding_to_common_cause_variation.asp).
This webpage was written by Steve Simon on 2006-11-11, edited by Steve Simon, and was last modified on 2008-07-08. This page needs minor revisions. Category: Control charts, Category: Definitions.
The following story illustrates the problems that can occur when you fail to recognize the difference between common cause and special cause variation.
The Busy Tailor
When it was his turn to explain his recent work, Student Leaf stood up and portrayed an elegant experiment that used a central composite design with four factors. Master Stem asked, "Is this process ready for such an experiment?"
Student Leaf replied, "I do not understand."
Master Stem looked at him with an air of amusement. "If this process is not ready for an experiment, then you will make yourself very busy for no good reason."
"How can I tell, Master Stem, if a process is ready?"
"Have you computed a control chart for this process? Do you know if the process is in control?"
"I have not computed a control chart, but I do know that the process is too variable. I want to run an experiment to reduce that variation."
"I have a tailor I would like you to meet. He makes all the clothes for my family. I brought my oldest child in for a fitting and the tailor made measurements and started sewing. When I visited the next time, I had my youngest child with me. I apologized, but the tailor still insisted on doing the fitting. This required ripping out all the old seams, remeasuring and resewing. 'I am almost done with the clothes for your youngest child,' he told me, 'please come back tomorrow.' So I returned the next day, but this time I was accompanied by my middle child. 'No matter,' replied the tailor, 'I will rip out all the seams again and make the clothes fit your middle child.'"
"That is a very foolish tailor, Master Stem."
"And you, too, are foolish if you run an experiment without looking at the control chart first. If your process is out of control, that tells you that your process is not a single process, but is many instead. And you do not know which process is visiting at any time. Your experiment, carefully optimized for one process, will fit poorly for the other processes."
This webpage was written by Steve Simon and was last modified on 2008-07-08. Category: Teaching resources
Using a pocket calculator to compute a standard deviation (March 1, 2007). Category: Descriptive statistics
Most of the time, I let a computer program like SPSS compute quantities for me, but every now and then, I want to calculate a few simple statistics without the benefit of SPSS. This might involve using paper and pencil or using a pocket calculator. You should do this also, as it greatly increases your confidence level in what SPSS produces. Let me illustrate how you would calculate a standard deviation using a pocket calculator.
Your pocket calculator must have a square root key. A memory key (M+) is helpful. Here's a picture of a nice pocket calculator with both features from www.keysan.com.
Suppose you wanted to calculate the standard deviation of the following four numbers: 12, 10, 11, and 13.
First you need to clear out memory. For some calculators, the simple process of recalling the value from memory (usually the button labeled MR or MRC) will clear memory. For other calculators, you have to press the recall from memory button twice. For still others you will have to press the clear memory button (usually labeled CM or MC). Each calculator is a bit different.
Now enter each data value in succession and press the M+ button.
12 M+
10 M+
11 M+
13 M+Press the recall memory button to get 46 and clear memory. Square this number to get 2116 and divide by the sample size (in this case, 4) to get 529. Write this number down. Now enter each data value again, square it, and add it to memory. For most calculators, the sequence
12 X M+
10 X M+
11 X M+
13 X M+will suffice because the calculator will finish the squaring operation before adding to memory. For other calculators, you will have to press the equals button before adding to memory.
12 X = M+
10 X = M+
11 X = M+
13 X = M+Press the recall memory button to get 534. Subtract the number you wrote down earlier to get 5. Divide by n-1 (in this case, 3) to get 1.6666667. This quantity is the sample variance. Press the square root button to get the sample standard deviation of 1.290994.
As a quick double check of the accuracy of your calculations compute the range of the data (largest value minus the smallest value). The range is usually 4 to 6 times larger than the standard deviation . In this example the range is 3, which is a bit less than 4 times 1.3, but that's still acceptable. Don't worry about it unless the standard deviation is larger than the range (a mathematical impossibility) or smaller than the range by an order of magnitude or more (possible but only in extreme circumstances).
Repetition is the best teacher, so the first few times you do this, run through the steps twice. Getting the same number twice will make you more comfortable with the entire process.
Try to calculate the standard deviation of the following data:
2.1, 2.5, 1.1, 5.3, and 2.2.You should get 1.57734 for the standard deviation. If you didn't get this value, check some of the intermediate calculations. The sum of the five values is 13.2, and when you square this and divide by 5 (not 4!), you will get 34.848. When you square these values and add them, you will get 44.8. The difference between the two numbers is 9.952 and when you divide by n-1 you will get a variance of 2.488. The square root of this quantity, 1.57734, is the standard deviation. Note that the range (5.3 - 1.1 = 4.2) is a bit less than 4 times the standard deviation, but still acceptable.
On your own, show that the standard deviation of
442, 218, and 333is112.0134.
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Calculating an XBAR-S control chart (March 2, 2007) Category: Control charts
The following data represents a weekly evaluation of vaccine potency. The data is taken from
- An adaptation of quality control chart methods to bacterial vaccine potency testing. H. C. Batson, M. Brown, M. Oberstein. J Bacteriol 1951: 61(4); 407-19. [Medline] [PDF]
but I have taken some liberties with the data to simplify the calculations.
Week01 0.716 0.771 0.924
Week02 0.978 1.212 1.176
Week03 0.644 0.903 0.869
Week04 0.869 0.716 0.869
Week05 1.398 1.301 0.934
Week06 1.218 0.924 1.398
Week07 0.876 0.591 0.644
Week08 1.215 1.241 1.021
Week09 1.021 0.954 0.491
Week10 0.690 0.477 0.785
Week11 1.301 1.279 1.220
Week12 1.644 1.176 1.114
Week13 1.146 1.256 1.518Each week, three lots of vaccine are tested for potency. Calculate a control chart for this data.
While many experts in quality control would use an XBAR-R chart for this data, the XBAR-S chart also works well. There are three steps in calculating an XBAR-S chart.
- Compute a mean and standard deviation for each group,
- Plot the means/standard deviations in sequence (i.e., a run chart),
- Draw reference lines at the overall mean and at the three sigma limits.
The mean and standard deviation for each week are shown below
Mean Stdev
Week01 0.804 0.108
Week02 1.122 0.126
Week03 0.805 0.141
Week04 0.818 0.088
Week05 1.211 0.245
Week06 1.180 0.239
Week07 0.704 0.152
Week08 1.159 0.120
Week09 0.822 0.289
Week10 0.651 0.158
Week11 1.267 0.042
Week12 1.311 0.290
Week13 1.307 0.191Here is a plot of the means
and of the standard deviations.
I have included a single reference line at the average of all the data points. For these two charts, the data values fluctuate more or less randomly above and below the reference line. If you noticed eight or more consecutive points on the same side of the center line, you would declare the process to be out of control.
The final step is to compute control limits. These limits are placed at three sigma distance from the overall mean and variation inside these limits is considered normal variation. The formula for control limits for the XBAR chart is
where the constant A3 comes from the following table.
n A3 B3 B4
2 2.659 0 3.267
3 1.954 0 2.568
4 1.628 0 2.266
5 1.427 0 2.089
6 1.287 0.030 1.970
7 1.182 0.118 1.882
8 1.099 0.185 1.815
9 1.032 0.239 1.761
10 0.975 0.284 1.716
11 0.927 0.321 1.679
12 0.886 0.354 1.646
13 0.850 0.382 1.618
14 0.817 0.406 1.594
15 0.789 0.428 1.572
16 0.763 0.448 1.552
17 0.739 0.466 1.534
18 0.718 0.482 1.518
19 0.698 0.497 1.503
20 0.680 0.510 1.490
21 0.663 0.523 1.477
22 0.647 0.534 1.466
23 0.633 0.545 1.455
24 0.619 0.555 1.455
25 0.606 0.565 1.435This table can be found in many books and on several websites. I selected this table from
In this particular example, n=3, so A3=1.954. This produces lower and upper control limits of 0.68 and 1.34. If a single data point falls outside the control limits, your process is out of control.
It is optional, but you can also compute warning limits at two sigma units away from the mean. If you notice two out of three consecutive points outside the warning limits, then your process is out of control. Here is a control chart that includes warning limits.
Notice that the tenth week is below the lower control limit and that two consecutive weeks (11 and 12) fall above the upper warning limit. You can also compute control limits for the standard deviations using the formula
where the constants B3 and B4 come from the same table. When n is small (5 or less), the value of B3 is zero which places no effective lower control limit on the chart. What this tells you is that an individual standard deviation can be extremely small without raising any concern.
Here is a plot of the standard deviations with control limits.
There are no points outside the control or warning limits.
On your own.
Compute a control chart for the following data set (see below). This data set represents vaccine potencies, like the above data set, but it uses fixed standards to assess potency. These numbers appear in the same paper, but again some simplifying assumptions have been made. Don't peek until you've done the work, but the answers are available on a separate web page.
Week01 1.097 1.204
Week03 1.030 1.362
Week05 0.682 0.978
Week07 0.820 1.080
Week09 1.042 0.858
Week11 1.398 1.146
Week13 1.301 1.204
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Calculating a P control chart (March 7, 2007). Category: Control charts
If you are collecting data on proportions with a consistent denominator for each proportion, then you can plot this data on a control chart. This type of chart is called a P chart and it is very simple to calculate. Here is an example of some data that is appropriate for a P chart. An employee was asked to take a hearing test on 24 consecutive weeks. The hearing test consisted of listening to and trying to recognize 50 spoken words that were recorded with some background noise. The score is the percentage of words recognized correctly. This data set is loosely adapted from a larger data set at
Here is the data:
28 24 32 30 34 30 36 32 48 32 32 38
32 40 28 48 34 28 40 18 20 26 36 40The formula for the upper and lower control limits is
where pbar is the average of the individual proportions and n is the denominator for each individual proportion. If you want to compute upper and lower warning limits, the formula for these is
The average proportion is 0.3275 and n is 50. The control limits are computed as
Here is what the control chart looks like:
Notice that all data points are inside the upper and lower control limits and that we do not observe 2 out of 3 consecutive points outside the warning limits. Neither do we see eight consecutive points on the same side of the center line. Thus, this process is in statistical control. This individual's hearing may not be all that good, but there are no unusual deviations from what you would normally expect.
On your own. Two other workers also took the same series of hearing tests (see data below). Compute a P chart for each worker. Don't peek until you've done the work, but the answers are available on a separate web page.
Worker #2:
60 56 78 60 74 70 70 68 82 76 72 76
68 78 76 68 74 56 74 62 60 70 60 84
Worker #3:
34 42 30 24 42 32 30 36 36 48 40 26
46 42 48 24 36 24 48 30 24 28 32 44
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Calculation of Analysis of Means limits (March 6, 2007). Category: Analysis of means
This page shows some of the details for calculating an analysis of means (ANOM) chart.
Resistivity example. This data set comes from the National Institutes of Standards and Technology
The first three digits of the data values are constant, so you need to be careful in calculating means and standard deviations. Do not round with this data set. Resistivity measurements were recorded five times on five separate instruments. There is some concern that the instruments may have small but important differences in resistivity.
Instrument
1 2 3 4 5
196.3052 196.3042 196.1303 196.2795 196.2119
196.1240 196.3825 196.2005 196.1748 196.1051
196.1890 196.1669 196.2889 196.1494 196.1850
196.2569 196.3257 196.0343 196.1485 196.0052
196.3403 196.0422 196.1811 195.9885 196.2090Compare these five instruments using an ANOM chart.
mean var stdev
1 196.2431 0.007651577 0.08747329
2 196.2443 0.019037095 0.13797498
3 196.1670 0.008784212 0.09372413
4 196.1481 0.010863213 0.10422674
5 196.1432 0.007823043 0.08844797The average of the five means is 196.1892 and the average of the five variances is 0.01083183. The square root of this value, 0.1040761, is the pooled standard deviation.
The formulas for the decision limits in an ANOM chart are
where
In this example, I is 5 and N-I is 20. The value of h from the table of critical values for a balanced ANOM is 2.79. The ANOM decision limits are
and the ANOM chart looks like this:
All five means are inside the decision limits, so you would conclude that no individual mean differs from the overall average.
You can also use an ANOM chart for the standard deviations. I will not show the formulas or the calculations for this chart, but here is what it looks like:
All standard deviations are inside the decision limits, so you would conclude that no individual standard deviation differs from the pooled standard deviation.
Toy slingshot. I asked a group of three volunteers to collect some data on their accuracy at hitting a target with a toy slingshot. They took three shots with their dominant hand and measured the distance of each shot from the target. Then they took three shots with their non-dominant hand and took three shots at the target. Their results are recorded below
Name Shot1 Shot2 Shot3
J-D 62 20 14
J-N 21 9 37
A-D 20 24 37
A-N 43 40 26
M-D 17 75 29
M-N 27 21 59My intention was to treat these six rows as if they represented six separate individuals (I did not have enough volunteers!). This is perhaps a bit of an oversimplification, and some time in the future I want to analyze the data as a two factor study. For this weblog entry, though, I want to plot these results using an analysis of means chart with a factor having six levels.
The summary statistics are easily computed.
Name Mean Var Stdev
J-D 32.00 684.00 26.15
J-N 22.33 197.33 14.05
A-D 27.00 79.00 8.89
A-N 36.33 82.33 9.07
M-D 40.33 937.33 30.62
M-N 35.67 417.33 20.43The average of the six means is 32.28. The average variance is 399.6, and the square root of this value, 19.99, represents the pooled standard deviation.
In this example, I is 6 and N is 18. The critical value h is 3.07. The upper decision limit is 64.62, as shown below
and the lower decision limit is -0.06, as shown below
We round the lower decision limit up to zero because a negative result is impossible in this experiment. Here is a graph showing the individual means and the ANOM limits.
Even though there is some disparity in the mean values, these disparities are well within the limits of sampling error.
Although an ANOM chart for the standard deviations is theoretically possible, you should not calculate such a chart unless you have more observations contributing to each individual standard deviation.
Hypothetical change in sample size. For a process with as much randomness as this, it may make sense to ask each individual to shoot at the target five or ten times. What would the decision limits look like if these individual means and standard deviations were based on ten runs rather than three?
In this case, the value of I would remain the same (6) but the value of N would increase to 60. The critical value for h would be 2.71. The upper decision limit is 47.92
and the lower decision limit is
Improvement in toy slingshot experiment. The group then worked on the process and made some improvements. Here is the data after the process was improved
J-D 52 12 18
J-N 5 2 34
A-D 9 22 17
A-N 22 19 8
M-D 25 29 10
M-N 27 3 15The summary statistics for each group are
Name Mean Var Stdev
J-D 27.33 465.33 21.57
J-N 13.67 312.33 17.67
A-D 16.00 43.00 6.56
A-N 16.33 54.33 7.37
M-D 21.33 100.33 10.02
M-N 15.00 144.00 12.00
Avg 18.28 186.56The square root of the average variance, 13.65, represents the pooled standard deviation. The decision limits are
and
which we round to zero. The ANOM chart is
Again, all data points are inside the decision limits.
Analysis of proportions. The analysis simplifies somewhat if your data is a set of proportions rather than a set of means. You no longer need to compute a pooled standard deviation, but instead use a formula for variation that is a simple function of the average proportion. You also do not need to calculate degrees of freedom, and can treat them as an infinite number of degrees of freedom. This is effectively the same thing as replacing a t-distribution with a normal distribution. Here is the formula.
A worker is asked to compare four different hearing tests to assure that they are of comparable difficulty.
Test-1 86%
Test-2 56%
Test-3 90%
Test-4 86%The results represent the percentage of words out of fifty that are identified correctly. The average of these four proportions is 0.795. The decision limits are computed as
and the ANOM chart looks like
The second test appears to be more difficult than average.
On your own.
1. A different group of volunteers was asked to shoot a toy rocket at a target (data shown below). Six different people recorded their accuracy on two consecutive shots. Calculate an ANOM chart for this data.
A 14 39
B 60 20
C 26 9
D 9 12
E 36 21
F 53 182. The following data is fictional. Twenty separate laboratories were sent identical images of a sperm smear with exactly 100 sperm cells and were asked to estimate the proportion of normal cells on the image using WHO-3 standards (data shown below). Calculate an ANOM chart for these proportions. Don't peek but the answers are available on a separate web page.
25 23 22 18 24 30 22 28 29 15
19 35 33 35 33 17 19 19 40 26
This webpage was written by Steve Simon and was last modified on 09/25/2008.
ANOM table for alpha=0.05, part 1 (March 4, 2007). Category: Analysis of means
Here's a table of critical values for analysis of means (ANOM) at an alpha level of 0.05.
I=2 3 4 5
6 7 8 9 10 11
12 13
df=2 4.30 5.88 6.59 7.10 7.49 7.80 8.08 8.31 8.49 8.68 8.84 8.99 2
3 3.18 4.18 4.60 4.92 5.14 5.34 5.52 5.64 5.73 5.88 5.98 6.07 3
4 2.78 3.56 3.89 4.12 4.30 4.46 4.58 4.69 4.77 4.87 4.95 5.02 4
5 2.57 3.25 3.52 3.72 3.88 4.01 4.11 4.20 4.28 4.36 4.43 4.49 5
6 2.45 3.07 3.31 3.49 3.62 3.74 3.83 3.91 3.99 4.05 4.11 4.17 6
7 2.36 2.94 3.17 3.33 3.46 3.56 3.64 3.72 3.78 3.85 3.90 3.95 7
8 2.31 2.86 3.06 3.22 3.33 3.43 3.51 3.58 3.64 3.70 3.75 3.80 8
9 2.26 2.79 2.99 3.13 3.24 3.33 3.41 3.48 3.54 3.59 3.64 3.68 9
10 2.23 2.74 2.93 3.07 3.17 3.26 3.33 3.39 3.45 3.50 3.55 3.59 10
11 2.20 2.70 2.88 3.01 3.12 3.20 3.27 3.33 3.39 3.44 3.48 3.52 11
12 2.18 2.67 2.85 2.97 3.07 3.15 3.22 3.28 3.34 3.38 3.42 3.46 12
13 2.16 2.64 2.81 2.93 3.03 3.11 3.18 3.24 3.28 3.33 3.38 3.42 13
14 2.14 2.62 2.78 2.91 3.00 3.08 3.14 3.20 3.25 3.29 3.33 3.37 14
15 2.13 2.60 2.76 2.88 2.97 3.05 3.11 3.17 3.22 3.26 3.30 3.34 15
16 2.12 2.58 2.74 2.86 2.95 3.02 3.09 3.14 3.19 3.23 3.27 3.31 16
17 2.11 2.57 2.73 2.84 2.93 3.00 3.06 3.12 3.16 3.21 3.25 3.28 17
18 2.10 2.55 2.71 2.82 2.91 2.98 3.04 3.10 3.14 3.19 3.22 3.26 18
19 2.09 2.54 2.70 2.81 2.90 2.96 3.02 3.08 3.13 3.17 3.20 3.24 19
20 2.09 2.53 2.68 2.79 2.88 2.95 3.01 3.06 3.10 3.15 3.19 3.22 20
21 2.08 2.52 2.67 2.78 2.87 2.94 2.99 3.04 3.09 3.13 3.17 3.20 21
22 2.07 2.51 2.66 2.77 2.85 2.92 2.98 3.03 3.08 3.12 3.15 3.19 22
23 2.07 2.50 2.66 2.76 2.84 2.91 2.97 3.02 3.06 3.10 3.14 3.17 23
24 2.06 2.50 2.65 2.75 2.83 2.90 2.96 3.01 3.05 3.09 3.13 3.16 24
25 2.06 2.49 2.64 2.74 2.83 2.89 2.95 3.00 3.04 3.08 3.12 3.15 25
26 2.06 2.48 2.63 2.74 2.82 2.88 2.94 2.99 3.03 3.07 3.10 3.14 26
27 2.05 2.48 2.63 2.73 2.81 2.87 2.93 2.98 3.02 3.06 3.10 3.13 27
28 2.05 2.47 2.62 2.72 2.80 2.87 2.92 2.97 3.01 3.05 3.09 3.12 28
29 2.05 2.47 2.61 2.71 2.80 2.86 2.92 2.96 3.00 3.04 3.08 3.11 29
30 2.04 2.46 2.61 2.71 2.79 2.86 2.91 2.96 3.00 3.04 3.07 3.10 30
32 2.04 2.46 2.60 2.70 2.78 2.84 2.90 2.94 2.99 3.02 3.06 3.09 32
34 2.03 2.45 2.59 2.69 2.77 2.83 2.89 2.93 2.97 3.01 3.04 3.07 34
36 2.03 2.44 2.59 2.68 2.76 2.82 2.87 2.92 2.97 3.00 3.03 3.06 36
38 2.02 2.44 2.58 2.68 2.75 2.81 2.87 2.91 2.95 2.99 3.02 3.05 38
40 2.02 2.43 2.57 2.67 2.75 2.81 2.86 2.91 2.95 2.98 3.02 3.04 40
44 2.02 2.43 2.56 2.66 2.73 2.79 2.85 2.89 2.94 2.97 3.00 3.03 44
48 2.01 2.42 2.56 2.65 2.73 2.79 2.83 2.88 2.92 2.96 2.99 3.02 48
52 2.01 2.41 2.55 2.64 2.72 2.78 2.83 2.87 2.91 2.95 2.98 3.00 52
56 2.00 2.41 2.54 2.64 2.71 2.77 2.82 2.86 2.90 2.94 2.97 3.00 56
60 2.00 2.40 2.54 2.63 2.70 2.76 2.81 2.86 2.90 2.93 2.96 2.99 60
Inf 1.96 2.34 2.47 2.55 2.62 2.67 2.72 2.76 2.80 2.83 2.86 2.88 Inf
2 3 4 5
6 7 8 9 10 11
12 13
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Examples of a fishbone diagram (March 24, 2006)
The fishbone diagram (also called the Ishikawa diagram, or the case and effect diagram) is a tool for identifying the root causes of quality problems. It was named after Kaoru Ishikawa, the man who pioneered the use of this chart in quality improvement in the 1960's. Surprisingly, I have had to hunt very hard to find any good examples of a fishbone diagram.
Here's one example.
This diagram identifies problems with a speech recognition and interaction system called The Carnegie Mellon Communicator System which is used to automate travel-planning. The major bones are
- System Failure,
- Understanding,
- Task,
- System Output,
- Dialog, and
- Recognition.
The first two minor bones are
- System Crash, and
- Airline Information Access Error.
This image appears in
Here's a second example:
This diagram outlines causes of defects in a computer user interface. The major bones are
- Guidelines Not Followed,
- Lack of Feedback,
- Lack of Guidelines,
- Different Perspectives, and
- Oops! (Forgotten).
The first minor bone is
- Don't Read Them
with
- No Time and
- No Central Location
attached as root causes. This diagram appears at
and a similar fishbone diagram on specification defects also appears in this article.
I could not find a good medical example that appears on the web. There were some examples in journal articles that do not appear on the web, such as this one:
This diagram does not follow the form but does capture the spirit of the fishbone diagram. The major bones are
- People,
- Environmental & Other,
- Patient Factors,
- Drugs & Devices,
- Technology, and Measures,
- Process Tools & Communication.
The first two minor bones are
- Family and
- RN.
This diagram appears in the following journal publication.
- Management of the agitated intensive care unit patient. Ian L. Cohen, T. James Gallagher, Anne S. Pohlman, Joseph F. Dasta, Edward Abraham, Peter J. Papadokos. Crit Care Med 2002: 30(1 (Suppl.)); S97-S123. [Medline]
The American Statistician has a hypothetical example of a manufacturing environment involving fluid mechanics. Students examine a system that involves dropping a bead into a glycerin/water mix with a few other chemicals like baking soda thrown in. The goal is to produce a drop time of 7.5 plus or minus 01. seconds. As part of preparing a statistical experiment that will identify appropriate manufacturing conditions, students are encouraged to produce a fishbone diagram. Here is the example shown in the article itself.
The major bones are
- Operator,
- Methods,
- Measurement, and
- Materials.
The first minor bone is
- Bead Orientation,
with
- Edge and
- Radius
attached as root causes. This diagram appears in
- Process Improvement Exercises for the Chemical Industry. Dale A. Kopas, Paul R. McAllister. The American Statistician 1992: 46(1); 34-41.
This article also discusses and explores other important Quality Control techniques like the Plan-Do-Check-Act cycle, and Evolutionary Operation (EVOP).
If you want to use a Fishbone Diagram, first list the main problem on the right hand side of the paper. Then draw a horizontal line to represent the "backbone" of the diagram. This line is not labeled. Off of the backbone, draw and label major bones: 4 to 7 major categories of causes. A commonly used list of major causes is Management, Manpower, Machines, and Materials. Another possible list is Policies, Procedures, Plant, and People. Then elicit ideas using an approach like brainstorming to place individual causes as minor bones on each major bone. Some people allow the individual causes to have subcauses, which would be attached to the minor bones. This is intended to get at the fundamental or root causes of the problem. Other people do not include this level of detail on their fishbone diagrams.
When you are done, look at the entire diagram. Does it have reasonable balance across the major bones? Are any common themes emerging? Can you identify causes that are measurable and fixable and which you believe are likely to have a large impact on the problem?
In some situations, you may find that a flow diagram of the work process may be more valuable and informative.
Further reading
- The Memory Jogger, A Pocket Guide of Tools for Continuous Improvement. Brassard, M. (1988) Methuen, MA: GOAL/QPC.
- Basic Tools for Process Improvement: Cause-and-Effect Diagram [PDF] Description: This website offers simple explanations of the cause and effect diagram, a classic tool used in quality improvement. This same guide is also found at www.management-tools.org/files/c-ediag.pdf and www.saferpak.com/cause_effect_articles/howto_cause_effect.pdf. Other guides are available at www.hq.navy.mil/RBA/text/tools.html.
This webpage was written by Steve Simon and was last modified on 07/14/2008. Category: Quality control
XBAR-S control chart, answers to on your own exercise (March 2, 2007). Category: Control charts
On the web page
you were asked to calculate an XBAR-S control chart. Here is the data for that problem.
Week01 1.097 1.204
Week03 1.030 1.362
Week05 0.682 0.978
Week07 0.820 1.080
Week09 1.042 0.858
Week11 1.398 1.146
Week13 1.301 1.204Here are the summary statistics.
Mean Stdev
Week01 1.151 0.076
Week03 1.196 0.235
Week05 0.830 0.209
Week07 0.950 0.184
Week09 0.950 0.130
Week11 1.272 0.178
Week13 1.252 0.069The mean of the seven means is 1.09 and the mean of the seven standard deviations is 0.15. For subgroups of size 2, the constants are
A3=2.659,
B3=0,
B4=3.267.The upper and lower control limits for the XBAR chart are
1.09+2.659*0.15=1.49 and
1.09-2.659*0.15=0.69.The warning limits are
1.09+2.659*0.15*2/3=1.36 and
1.09-2.659*0.15*2/3=0.82.Here is the control chart for the means (note that the numbers are slightly different because of rounding).
The upper control limit for the S chart is
3.267*0.15=0.49.The distance from the center line to the upper control limit is
0.49-0.15=0.34.The warning limit is placed at 2/3 of the distance from the center line or
0.15+0.34*2/3=0.38.Here is the S chart (note again some slight discrepancies due to rounding).
This webpage was written by Steve Simon and was last modified on 07/08/2008.
P control chart, answers to on your own exercises (March 7, 2007) Category: Control charts
On the web page
you were asked to calculate P charts for the following two data sets:
Worker #2:
60 56 78 60 74 70 70 68 82 76 72 76
68 78 76 68 74 56 74 62 60 70 60 84
Worker #3:
24 32 20 14 32 22 20 26 26 38 30 16
36 32 38 14 26 14 38 20 14 18 22 34For worker #2, the average of all 24 proportions is 0.6967. The control limits are calculated as
and the control chart is
Notice that the computer rounds these numbers slightly differently, reporting the lower control limit as 0.50 rather than 0.51. Do not worry about these small differences, but the computer's value is slightly more accurate.
There are no points out of control on this chart.
For worker #3, the average proportion is 0.2525. The control limits are calculated as
and the control chart is
Again, the answers diverge slightly because of rounding. There are no points out of control on this chart.
This webpage was written by Steve Simon and was last modified on 07/08/2008.
Analysis of Means answers to on your own exercises (March 6, 2007) Category: Control charts
On the web page
you were asked to calculate ANOM charts for two different data sets.
The first data set is reproduced below.
A 14 39
B 60 20
C 26 9
D 9 12
E 36 21
F 53 18The summary statistics are
Mean Var Stdev
A 26.5 312.5 17.68
B 40.0 800.0 28.28
C 17.5 144.5 12.02
D 10.5 4.5 2.12
E 28.5 112.5 10.61
F 35.5 612.5 24.75The average of the individual means is 26.4 and the average of the individual variances is 331.1. The square root of this value is 18.2 which represents the pooled standard deviation.
In this example, I is 6 and N is 12. The critical value h is 3.62. The upper decision limit is
26.4+3.62*18.2*√5/18=61.4The lower decision limit is less than zero. Since negative values are impossible in this setting, you should not plot this value. Here is a graphical display.
The second data set represents proportions and is reproduced below.
25 23 22 18 24 30 22 28 29 15
19 35 33 35 33 17 19 19 40 26The average of these 20 proportions is 0.26. In this example, I=20 and N=2000. The critical value h is 3.02. The upper and lower decision limits are
0.26-3.02*√(0.26*0.74)*√(19/2000)=0.13
0.26+3.02*√(0.26*0.74)*√(19/2000)=0.39Here is a graphical display of the individual proportions and the decision limits.
This webpage was written by Steve Simon and was last modified on 07/08/2008.