Selection bias (August 24, 2004).

The CarTalk radio show has an interesting puzzle every week and often these puzzles involve mathematics. These puzzles can sometimes help you understand complex mathematical concepts that are important in Statistics. In the summer, they re-use puzzlers from earlier years, and just last week, they re-used one of my favorites. A "nameless mathematician" during World War II was asked to help with a military problem. A lot of bombers were not returning from their missions, so the Royal Air Force wanted to put armor on the bombers. But where to put it? They couldn't put it everywhere because it would be too heavy. So this mathematician looked at the planes that returned and noted where they had holes from enemy fire. These holes were distributed more or less randomly throughout the plane except for two regions where there were nothing. His recommendation was to place the armor only in those two areas where no enemy fire was found. This seems counterintuitive, which is why it makes such a good puzzle.

This mathematician hypothesized that any plane hit in those regions did not survive to return. The other areas could be hit and the plane could still limp back to safety. This is an example of selection bias. The bombers in the study were not a random sample of all bombers, they were a sample of bombers that returned safely.

This case was actually discussed in a delightful article in Chance magazine (see below). That article gives the name of the mathematician (Abraham Wald) and a few more details on the logic behind this choice. It has several other amusing examples of selection bias, such as the average age of death among various professions. The profession that had the worst result, with an average age at death of only 20.7 years was "student".

An excellent book by Freedman, Pisani, and Purves also has a delightful story about selection bias.

The original Car Talk puzzler is on the web at cartalk.com/content/puzzler/transcripts/199838/index.html.

You can read the original answer on the web at cartalk.com/content/puzzler/transcripts/199839/answer.html.

A Selection of Selection Anomalies. Wainer H, Pamer S, Bradlow ET. Chance 1998: 11(2); 3-7.

Statistics Third Edition. Freedman D, Pisani R, Purves R (1998) New York: W.W. Norton & Company. [BookFinder4U Link]

This webpage was written and was last modified on 2008-07-14. dot edu or click on the email link at the top of the page. Category: Exclusions in research