Seminar #68: The use of control charts to track adverse events in clinical trials

This seminar is represents my attempt to outline some of the issues facing Institutional Review Boards (IRBs) as they provide continual monitoring of clinical trials. There are several important questions that they face:

  1. Is the accrual rate lagging to the point at which you might want to question the ability of the research to meet its planned goals?
  2. Is there an unusual rate of dropouts in the trial which might compromise the quality of the research?
  3. Is there an unusual and unexpected number of adverse events in this trial?
  4. Are there disparities in accrual, dropouts, and adverse events between the two arms of the trials.

 Most IRBs are uncomfortable with their role in ongoing monitoring, and they often provide only a qualitative assessment at their continual review.

I'm thinking about writing a grant that would use control charts for the ongoing monitoring of clinical trials. The control charts would supplement the qualitative assessment at continuing review with some hard numbers. I want to develop some simple and useful charts that would track trends in accrual, dropouts, and (most importantly) the accumulation of adverse events in clinical trials. entry. This seminar will outline some of my initial efforts, focused primarily at the assessment of accrual rates.

I would greatly appreciate any feedback or assistance as I develop these ideas further.

Where can you find this handout?

This handout and the handouts that I use for all of my seminars and training classes are a compilation of individual web pages at www.childrensmercy.org/stats. I use the "Include Page" feature of Microsoft FrontPage to combine these into a single page. You can always find the most recent version of this compilation by going to the web address listed at the bottom of this page. Links for the handouts for other seminars and classes appear at www.childrensmercy.org/stats/training.asp.

Why don't I use PowerPoint?

I stopped using PowerPoint for my presentations in the mid 1990's. This was based on Edward Tufte's advice that presenting information in a paper handout is more effective than presenting the information on a projected screen. I found this to be excellent guidance. I enjoy talking when I don't have to wrestle with a laptop computer. I look at my audience more and interact with them better. I elaborate on this in greater detail at www.childrensmercy.org/stats/weblog2004/powerpoint.asp.

Stats >> Model >> Steps for establishing a quality control program (March 12, 2004)

So you've decided to implement a quality control program in your laboratory. What will it take to make that program successful? There are three steps, and you must follow these steps in order.

Every work situation is different, of course, but these steps are a useful guideline. Start first with management support.

Step 1. Establish management support.

If you are already the boss, congratulations. A quality control program is easiest to implement from the top down. Make sure you understand the commitment of time and money that is involved. Every workplace is different, but think about allocating 10% of your time and 10% of the time of all your employees to quality control. Don't pretend that you can implement a serious quality control program in the margins of free time. You have to re-allocate time that normally would be used "getting the real work done."

If you are not the boss, you have to convince someone else that your company has to make a substantial re-investment of resources. That's not an easy thing to do, but you do need support from your boss. Otherwise, you'll be asked to put the quality control project on hold once the first new crisis appears at work. What, you have no crises at your job? Maybe I should come and work for you.

To make your case with management, you need to convince them that

  1. variation exists in all processes,
  2. variation comes in two flavors, and
  3. there are established ways to reduce variation.

Variation exists in all processes. This is something that should be obvious, but we often forget it. Here's a simple exercise to illustrate this. Perform the exercise in the box before you read any further.

There are several reasons why two different people could come up with different answers to this exercise. First, there is human error, it's easy to miss things. Second there are ambiguous situations-did you include the sentence of instructions in your count? Third, people may apply different standards-do you include capital letters in your count?

If variation exists in a simple exercise like this, how much more likely is it to occur in a complex work process?

Exercise: Select a work process in your area. List all of the possible causes for variation in that work process.

Variation comes in two flavors. Not all types of variation are the same, and treating them so creates problems. Common causes of variation are factors that are present at all times that contribute to variation in all of the output from a work process. Special causes of variation are factors which appears sporadically and contribute to variation for only some of the output from a process. Special causes can usually be assigned to a particular source.

Deming had a "red bead" exercise that he used in all his course. Five volunteer workers would dip their paddles into a bag of red and white beads in order to "produce" white beads. Inspectors count the number of defective (red) beads and then Deming praises or chastises the workers according to their output. A second round of production ensues and the workers who did well the first time slip back and those who did poorly end up doing better. The cycle continues with Deming trying to exhort the workers to do better, when it was actually the process itself that was causing the variation.

The red bead experiment is a classic example of taking a common cause of variation and labeling it a special cause. Managers love to do this. Any small change in the process has to have a "variance report" that finds a specific person of thing to blame.

Exercise: examine the sources of variation in the previous exercise. Classify them as common cause or special cause variation.

There are established ways to reduce variation. There are good and bad ways to reduce variation, but unfortunately there are more bad ways than good ways. Here are some examples of bad ways to reduce variation:

The best way to reduce variation is to identify special causes of variation and then remove them. Once the special causes are gone, you have a stable process that is amenable to experimentation. Run experiments to remove common causes of variation.

Identify and remove special causes. This step has to come first. Monitor your process using a control chart when you find a point out of control, investigate and find the assignable cause to that out of control point. Fix that problem and continue to monitor the work process.

Identify and remove common causes. This step has to come second. Once the process is in control, you then have to identify common factors that influence all of the output. Manipulate these common factors in experiments to reduce variation.

It's very important to remove all special causes first. You can't run an experiment effectively if there are special causes which appear at unexpected times. They'll contaminate the experiment and make the results much harder to interpret.

Step 2. Measure your process.

It was Socrates who first said  that an unexamined life is not worth living. As a professional statistician, I believe that an unanalyzed data set is not worth collecting. There are several ways to measure a work process.

  1. Flow diagrams
  2. Pareto charts
  3. Cause and effect diagrams
  4. Control charts

There are plenty of resources for how to use each of these tools, but the key question should be, what should I measure? My advice is to start small and start well downstream.

Start small. There is a large start-up cost for a quality control program and you don't want to try to do everything at once. Look for the low hanging fruit--those areas where it is easy to measure things and where there is a strong prospect for time/money savings.

Start well downstream. By "downstream," I mean near the end of the entire work process. Measure something that represents the end-product--what the customer sees. As you start to understand the end-product better then you will need to look at some of the intermediate processes that contribute to the end-product. This is moving upstream.

Step 3. Experiment.

Once you have established a stable process, you want to improve that process, either by lowering the mean or shrinking the width of the control limits. Experimentation is the key to either approach.

When you run an experiment, you should follow the PDSA cycle (Plan-Do-Study-Act).

Plan. What sort of change to the work process are you considering? What is the current work process? How well/poorly does the current work process do?

Do. Implement the planned change. Measure the impact of the change.

Study. Analyze the measurements. Did the change make things better or worse?

Act. If the change is better, implement that change in all future work. Make sure that the change persists.

The whole experiment will probably raise new questions to study, which starts the whole cycle over again.

Exercise: All four steps in the PDSA cycle are important. What happens if skip the Study step? What happens if you skip the Act step?

Summary

Setting up a quality program is not easy. First get management support. Without that support all your efforts will be refocussed at the first work crisis. Second, measure your process. Start small and start well downstream. Third, experiment. Follow the plan, do, study, act cycle.

Web Resources

A Few Definitions. Clark TJ. Accessed on 2004-03-11. www.successthroughquality.com/glossary.htm

Turning Top Management Reluctance into Six Sigma Support. Devane T, iSixSigma. Accessed on 2004-03-11. www.isixsigma.com/library/content/c040308a.asp

PDCA Cycle. HCi Services. Accessed on 2004-03-11. www.hci.com.au/hcisite2/toolkit/pdcacycl.htm

Flow Charts. Kimbler DL. Accessed on 2004-03-10. deming.eng.clemson.edu/pub/tutorials/qctools/flowm.htm

Deming's Red Bead Experiment. Martin JR. Accessed on 2004-03-11. www.maaw.info/DemingsRedbeads.htm

Understanding Variation [pdf]. Nolan TW, Provost LP. Accessed on 2004-03-11. www.apiweb.org/UnderstandingVariation.pdf

Pareto Chart. Simon K, iSixSigma. Accessed on 2004-03-11. www.isixsigma.com/library/content/c010527a.asp

Cause & Effect Diagram. Skymark Corporation. Accessed on 2004-03-11. www.skymark.com/resources/tools/cause.asp

Control Charts as a tool in SQC (Statistical Quality Control). Sweat S, Terala K, Troha K, Williamson K. Accessed on 2004-03-11. deming.eng.clemson.edu/pub/tutorials/qctools/ccmain1.htm

Common Control Chart Cookbook. Sytsma S, Manley K, Ferris State University. Accessed on 2004-03-11. www.sytsma.com/tqmtools/charts.html

Deming cycle PDSA model framework: Plan Do Study Act continuous improvement. Value Based Management. Accessed on 2004-03-11. www.valuebasedmanagement.net/methods_demingcycle.html

Books

Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. Carey RG, Lloyd RC (1995) New York: Quality Resources.

Understanding Statistical Process Control, Second Edition. Wheeler DJ, Chambers DS (1992) ISBN: 0-945320-13-2.

Understanding Variation: The Key to Managing Chaos. Wheeler DJ (1993) Knoxville, TN: SPC Press Inc. ISBN: 0-945320-35-3.

Appendix: Examples of control chart calculations

Here is some data on a work process that involves drilling. I'm not quite sure where I originally found this data.


Notice that the drill bit is replaced after nine days. The right had side shows some simple calculations for a control chart. You compute an average and a range for each subgroup. The averages are then averaged to get an overall mean (Xbarbar) and an average range (Rbar).

The formulas for the control limits are

where the constant, A2, can be found in any quality control text book. The chart for the range looks like

and the formulas for the control limits are

Again, the constants, D3 and D4, are in any quality control text book.

Exercise: What change to the work process does this control chart suggest?

Rules for identifying special causes.

There are many rules that have been proposed for deciding when a special cause has occurred. A good summary of these rules appears on pages 135-140 of Wheeler (1995).

The simplest rule, and the one that appears to be universally adopted is when a single point lies outside the control limits. This indicates a sudden large shift in the process.

A second rule is to declare a special cause variation when eight consecutive points lie on the same side of the center line. This rule is attractive because it is simple to apply and it is sensitive to gradual shifts or trends that the first rule might miss.

Other rules involve dividing the control chart into zones (e.g., more than one standard deviation from the center line, more than two standard deviations from the center line) and noting whether two out of three or three out of four points fall into certain zones. There are also rules about a sequence of consecutively increasing or consecutively decreasing points or a sequence that hops too frequently from one side to the other of the center line or a sequence that hugs too closely to the center line.

On page 139, Wheeler warns that too many rules increase the chances for a false alarm. The first two rules (one point outside control limits; eight consecutive points on the same side of the center line) work very well in practice and there is little justification for adopting more complex rules. This is especially true if you are just starting out with a quality control program.

Control charts for individual values

Sometimes, there is no easy way to grab a batch of two or more measurements. The measurements that you get are distinct and cannot be put into batches without some degree of "ad hoc"ery. For these situation, an XmR chart works well. Here is an example.


This example appears in section 6.4 of Wheeler and Poling (1998). The data represents accounts receivable as a percentage of sales. This is an important accounting ratio that describes how quickly bills are collected. A large percentage is bad and indicates that accounts receivable tends to be slow or overdue.

Rather than compute a range within a batch, a moving range is computed. The moving range is the range between each successive pair of observations. The moving range provides an alternative method for measuring short term variation. The average of the moving ranges is used to compute control limits using formulas similar to those shown above.

Here is what the control limits look like

There is also a moving range chart for this data, which I do not display.

Control charts for counts

There are several control chart choices when your data represents counts. The simplest choice is the c-chart. Here's an example that appears on page 272 of

The data appears below:

The average number of defects is

The formula for the control limits is

In this example, the lower control limit is negative, which is impossible. When this happens, you should set the lower control limit to zero. The control chart looks like the following:

Note that any number of defects from 0 to 5 would be considered common cause variation. There are two special causes to investigate: hoods #6 and #13. Although the author of this example does not give a specific explanation for these two points, he does offer the following possibilities: differences in the diligence or training among inspectors; a counting/recording error; or some change in the manufacturing process itself.

The C chart has potential application to the surveillance of employee injuries and illnesses. An example of such a chart even appears in the book Wheeler and Poling (page 186). Nevertheless, there are several difficulties with using a C chart for injuries and illnesses.

First, the assumptions of a Poisson distribution are questionable. If injuries and illnesses events in one month are correlated with one another, then the assumptions are a Poisson distribution are violated. A simple example of correlation would be if one of the illness categories represented an infectious disease like influenza. The assumptions of the Poisson distribution are also violated if there is heterogeneity in the distribution of injury and illness events. This might occur if certain times of the months, certain employees, or certain areas of the workplace, are more prone (or less prone) to have specific injuries and illnesses.

Which chart should I use?

Eventually, you should learn how to run a wide range of control charts. But to start with, I would encourage you to learn the XmR chart first. This is the simplest and most versatile chart and can handle most commonly encountered situations. It is not always as efficient as an Xbar/Rbar chart, but simplicity rather than efficiency is more important when you are just starting.

Stats >> Model >> Steps for establishing a quality control program

Page last modified on 09/26/2007. Send feedback to ssimon at cmh dot edu or click on the email link at the top of the page.

Monitoring accrual rates (May 30, 2006). Category: Accrual problems in clinical trials

This scenario is based on real data, but has been adapted slightly to serve as an illustration of the use of control charts in monitoring a clinical trial.

Suppose a clinical trial was set up in 1997 and the goal was to recruit one patient per month over a ten year period, for a total sample size of 120 patients. Here are the dates of recruitment for the first 42 patients.

 [1] "2/26/1997" "4/4/1997"   "7/7/1997"   "7/25/1997"  "2/5/1998"
 [6] "2/15/1998" "3/6/1998"   "7/3/1998"   "8/3/1998"   "2/8/1999"
[11] "3/19/1999" "4/20/1999"  "5/29/1999"  "6/21/1999"  "7/27/1999"
[16] "9/6/1999"  "1/10/2000"  "1/11/2000"  "2/28/2000"  "3/3/2000"
[21] "4/13/2000" "5/30/2000"  "11/21/2000" "12/18/2000" "2/6/2001"
[26] "4/30/2001" "8/3/2001"   "11/20/2001" "12/3/2001"  "12/7/2001"
[31] "9/27/2002" "10/1/2002"  "2/2/2003"   "3/3/2003"   "10/31/2003"
[36] "11/4/2003" "11/11/2003" "1/5/2004"   "2/2/2004"   "4/15/2004"
[41] "5/23/2004" "6/2/2004"

If you set 1/1/1997 as Day 0, then the days on which patients were recruited to the study are

 [1]   56   93  187  205  400  410  429  548  579  768  807  839  878
[14]  901  937  978 1104 1105 1153 1157 1198 1245 1420 1447 1497 1580
[27] 1675 1784 1797 1801 2095 2099 2223 2252 2494 2498 2505 2560
[40] 2588 2661 2699 2709

Distinguishing between monthly, quarterly, and yearly rates

You might ask some questions about this data, like

A simple approach to help answer these questions is to compute the number of patients recruited each month.

 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
 0  1  0  1  0  0  2  0  0  0  0  0  0  2  1  0  0  0  1  1  0  0  0  0
 
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
 0  1  1  1  0  1  1  1  1  0  0  0  2  0  2  1  0  1  0  0  0  0  0  1
 
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
 1  1  0  0  1  0  0  1  0  0  0  2  1  0  0  0  0  0  0  0  0  2  0  0

73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91
 0  0  1  1  0  0  0  0  0  0  0  3  0  1  1  0  1  1  1

Here is a plot of the data:

The accrual could also be summarized as the number of patients recruited per quarter.

 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
 1  1  2  0  3  0  2  0  2  2  3  0  4  2  0  1  2  1  1  2  1  0  0  2

which looks like this when you plot the data

or as a yearly rate

1 2 3 4 5 6 7 8
4 5 7 7 6 3 5 5

which looks like this when you plot it.

It is not clear what scale is most logical for this data. Should you analyze the data monthly? That allows you to respond quickly if a trend appears, but such a short time span leads to a lot of imprecision in any individual monthly value. A yearly trend provides a more stable estimate, but does not allow you to respond quickly to sudden shifts. Quarterly data offers the best (worst?) of both worlds.

Applying control charts to rates

The trouble with rates becomes more apparent when you try to apply control chart limits to the data. Here is a simple control chart for the monthly rates.

Note that the minus signs got clipped off the values of -0.06 and =0.58. Sorry!)

The center line for the plot is 0.46, which shows that the average rate is well below the target. But the control limits are -1.10 and 2.03, which are unrealistic on both ends of the scale.

There are many different algorithms for deciding whether a signal has occurred in a control chart. A commonly used standard involves dividing the control chart into zones. Zone A is 2-3 sigma away from the mean and is represented in the chart above as the region between the thin solid line and the dotted line. There are actually two A Zones, one above the center line and one below. Zone B is 1-2 sigma away from the mean and is represented by the region between the two dotted lines. Again there are two zones, above and below the centerline. Zone C is 0-1 sigma away and is represented by the region between the thick solid line and the dotted line.

A commonly used set of rules, called the Western Electric Company (WECO) rules declare that a signal occurs (also known as an out of control condition, or a special cause) if

  1. a single point falls outside of all the Zones
  2. two out of three points fall in Zone A or beyond and one the same side of the centerline
  3. four out of five points fall in Zone B or beyond and on the same side of the centerline
  4. eight consecutive points fall in Zone C or beyond and on the same side of the centerline.

These rules are documented at the NIST Statistics Engineering handbook in section 6.3.2

These rules appear to work well in a wide range of applications and have a good balance between false positives (declaring the process out of control when everything was actually running consistently) and false negatives (failing to declare the process out of control when a special cause of variation has indeed occurred). There are variations of these rules (some people, for example, will ask for nine consecutive points in Zone C) and additional rules that others will add (such as seven consecutive increasing or decreasing points).

 A negative value for the lower control limit means that it is impossible for a single month to produce a rate that is so low that you would identify it as a special cause. Since Zone A is entirely in the negative range, you can't use the two out of three rules either. For this chart, even Zone B is entirely in the negative range. So the only rule that works when the rate slows down is eight consecutive points below the centerline. So for this chart that means eight consecutive months without recruiting any patients at all! In a study which only seems to accrue a half a patient per month, perhaps this is not such an outrageous time frame.

In a typical research study, no one is usually too interested in discovering whether the accrual rate has taken a sharp upturn, so perhaps the upper control limit is not all that important. Still, it is worth pointing out that the upper control limit of 2.03 is also a bit unrealistic. I suspect that recruiting 3 patients in a month hardly seems like an indication that the process of accrual has suddenly accelerated. I want to formally investigate this to verify my hunch. My suspicion is that when the data are seriously skewed, the control limits also need to be skwed.

The control chart for the quarterly data exhibits much the same issues as the monthly data.

The control limits are -2.37 and 5.08. The value of zero falling into Zone B (between the one and two sigma limits) is fortunate, because otherwise you have to wait eight consecutive quarters before you could declare a slowing in the accrual rate. Applying the third of the four WECO rules, you could declare a change in the accrual process if four out of five consecutive quarters show no new patients. This means that a signal could occur in as little as four quarters (one year).

The control chart for yearly data is a bit better, perhaps.

The control limits are 1.83 and 8.67. The problem with the yearly chart is not the control limits but rather the delay in discovering a trend. If you apply one classic rule for declaring a point out of control (4 out of 5 data points outside the 1 SIGMA limits), you would have to wait 4-5 years before you would accumulate evidence that the accrual process has changed. You could get a quick reaction if you observe a single year with one or fewer patients being recruited.

Which of these control charts is best? You may have to apply the Goldilocks rule to the time interval (not too short and not too long). Donald Wheeler has some rules for control charts that are too "granular" (meaning the chart only hops around a very small number of values). The monthly rate chart is too granular, by Wheeler's rules, I suspect, but I have to double check this.

I'd like to develop some simple rules, such as if you expect to accrue X patients per month, use a quarterly chart, but if you expect to recruit Y patients per month instead a monthly chart is preferable.

Avenues of research exploration

There are two avenues of possible exploration for this question. First, you could model the number of patients using a Poisson distribution and compute some simple probabilities. For example, I mentioned how 2.03 seemed like a bad upper control limit for the monthly control chart. If the number of patients is actually 0.46, then a Poisson random variable with a mean of 0.46 would exceed the value of 2.0 with probability 0.011. This seems like a small value, but keep in mind that the control chart looks at a whole sequence of values from the process, so the standards need to be set a bit higher. Traditionally, the three sigma limits were designed so the probability of declaring a point out of control when the process was stable (and with a normal distribution) is only 0.0013, which is an order of magnitude smaller.

Second, you could compute the average run length (ARL) for the chart under the assumption that the process is stable, and make sure that the ARL is reasonably large. Then compute the ARL for a process that suddenly shifts and is now out of control. You want the ARL to be small here, otherwise the chart is insensitive to changes in the accrual process. James Westgard has a nice summary of simple ARL calculations at

You can improve these rate control charts somewhat by transforming them on a log scale, but the zeros that occur in the monthly and quarterly rates offer a problem.

Date gap calculations

What I propose as a better method for evaluating accrual is to look at the date gap, the number of days between recruiting consecutive patients and using this as the basic unit of measure. The  date gap in this example is

 [1]  56  37  94  18 195  10  19 119  31 189  39  32  39  23  36  41 126
[18]   1  48   4  41  47 175  27  50  83  95 109  13   4 294   4 124  29
[35] 242   4   7  55  28  73  38  10

So, for example, you had to wait 56 days for the first patient, 37 between the first and second patients, 94 days between the second and third patient, etc. Here is what a plot of the date gaps look like:

Notice that the number of data points is equal to the number of patients recruited. If you converted the data into the number of weeks between events or the number of months between events, you would still have exactly the same amount of data and exactly the same pattern..

This is the first advantage of recording the data as date gaps. It liberates you from having to worry about the time units--the chart for day gaps looks the same as the chart for week gaps, month gaps, etc.

Another advantage of using date gaps for monitoring accrual rates is that you don't have to wait for an artificial calendar boundary to be crossed before you evaluate the data. Every time a new patient joins the research study, you have a new point on your graph.

Finally, the average date gap has a simple interpretation. When you sum all the date gaps, the terms telescope.

When you invert the average date gap, you get a familiar formula

The numerator on the right hand side is the total number of patients recruited and the denominator is the total amount of time it took to recruit these patients. So the inverse of the average date gap is the overall recruitment rate. This is actually quite intuitive. If you have to wait an average of 15 days (0.5 months) between patients, you are recruiting 2 patients per month.

The control limits for a date gap chart have the same issues that control limits for a rate have.

The control limits are -5 months (-150 days) and 9.3 months (280 days). The center line is at 2.15 months (64 days). Since the date gap is inversely related to the accrual rate, large date gaps represent slower accrual rates. If you have to wait more than 9.3 months for the next patient in this clinical trial, that tells you that the accrual process has slowed substantially from the previous norm.

A log transformation for the date gaps

One intriguing possibility is to draw a control chart after a log transformation. The log transformation removes much of the skewness from the data.

The control limits for the log transformed data are interesting.

The control limits are at 0.5 days and 6.9 years (approximately 2500 days). Notice that the log scale places Zones B and C in the range of the data. The value at the low end of Zone C, 0.5 is intriguing. You could theoretically say that if two patients were recruited in the same day, that there was a gap of 0.5 days between them, and that if three patients were recruited in the same day, that there was a gap of 0.33 days between them. In order to fit the lower zones into the range of the data, though, you have to extend the upper zones. The upper control limit is now a disappointingly high value (6.9 years). Is this realistic and the 9.3 month limit on the original scale too liberal? Or is the 6.9 year value too conservative? I need to explore this further.

Other avenues for research

Once I develop a good model for accrual rates, I want to apply and adapt these methods to dropout rates. Is there a problem with excessive dropouts? Then the next natural extension is to adverse event reports. Is there a sudden surge in adverse events? I believe that a control chart represents a good way of tracking accrual, dropouts, and adverse events. I'm a bit leery of examining efficacy in a control chart, and when I have time, I want to document why efficacy might be different.

Also, I want to examine two group differences. Are we seeing a disproportionate number of dropouts in one arm of the study. I think that coding the events as +1 for the first arm of the study and -1 for the second arm of the study, and then cumulating this information over time will provide a continuous monitoring of whether a disproportionate impact is occuring. This becomes even more valuable when looking at safety data and adverse event reports.

There is a relationship between the control chart monitoring of adverse events and the traditional Number Needed to Harm (NNH) calculation used in Evidence Base Medicine. I need to establish and explore this link, because the NNH is the statistic that has a simple clinical interpretation. I would like to promote the control chart as a continual monitoring of NNH in a clinical trial.

Finally, I want to explore the use of CUSUM charts in monitoring accrual, dropouts, and adverse events in a clinical trial. There is not a lot written about CUSUM charts and there are a lot of variations to consider. The CUSUM chart is very good at showing gradual shifts in a work process. Also, the CUSUM chart appears to avoid some of the issues with interval size. If you select every third data point from a cumulative sum of monthly events that subset will be the same as a cumulative sum of quarterly events. A cumulative sum of date gaps is also intriguing, because of the telescoping that occurs when you add date gaps together. Finally, a deviation of the cumulative sum from a target value will rapidly show if you are consistently falling short of your goal. As such it might be a more sensible indicator of whether the original research plan needs to be reviewed or revised.

A good reference for CUSUM charts appears in section 6.3.2.3 of the NIST Engineering Statistics Handbook

Related pages on my web site

This web page was written by Steve Simon, edited by Steve Simon, and was last modified on 09/24/2007.