Faulty Sampling

The government believes that too much money is being spent on healthcare in the US, and it is trying to figure out how to reduce the amount of money spent. If this is to be accomplished without reducing the quality of care, then wasteful spending must be reduced. The only way to know where the waste is occurring is to measure all the waste in the system. But obviously, this is not feasible.

Standard practice for determining some quantity in the population without measuring every occurrence entails:

(1) Selecting a sample, that is, taking a small, but sufficient, sample from the population that’s representative of the population. In the case of medical system waste, the Dartmouth study cited in the article chose UCLA hospital as the sample used to represent all medical care in the US.

(2) Measuring the quantity at issue in the sample, that is, measuring every occurrence of the quantity itself or of some proxy for the quantity in the sample. The Dartmouth study cited in the article used a proxy for medical waste. The proxy was medical care provided to people who died despite receiving care.

(3) Generalizing from the sample to the population. The Dartmouth study generalized from the sample, UCLA hospital, to the population, the US healthcare system, by concluding that too much care was being provided for people who ended up dying shortly after the care was given.

There are two common mistakes made when attempting to gather information about a population by using a sample: choosing a sample that is not representative of the population and using a proxy in lieu of the true quantity that’s not actually representative of the true quantity at issue. In both cases the conclusions drawn from the study about the population will generally be inaccurate at best, and completely invalid at worst.