Proxy Measurements

How do you measure a nation’s well-being? Gross national product (GNP), a measure of the total amount of goods and services produced within the nation, has long been used as a proxy for the more abstract measure of national well-being. Recently, however, it has been noted that GNP is an inadequate measure of a nation’s well-being, since it does not take into account the amount of environmental resources used to produce the goods and services (see my earlier post Problems with Aggregate Measures). As such, other measures that account for both economic and environmental conditions will probably appear in the near future in lieu of GNP as proxies for national well-being.

It often difficult for analysts to measure the quantity they are interested in measuring (e.g., national well-being), either because the quantity is too abstract or it is too difficult to measure directly. In such cases, analysts will choose some proxy (e.g., GNP) to measure or use in lieu of the quantity of interest (e.g., national well-being). If analysts do use proxies, then they must be sure that the proxies are valid substitutes for the quantities of interest; that is, they must be sure that by measuring the proxies, they will get a good estimate of the quantity of interest; otherwise, any conclusions drawn about the quantities of interest that are based on the proxies will be problematic, as seen in the case of national well-being.

In lieu of measuring all the waste in the hospitals, the analysts for the Dartmouth study measured the care provided to patients who ended up dying soon after they received the care. In other words in the Dartmouth study, medical care provided to patients who died soon after was used as a proxy for healthcare system waste. Unfortunately, this proxy suffers from information set problems described above in the beach vs. skiing example: For many (most?) of the patients who end up dying soon after expensive medical tests or procedures were performed, the doctors did not know the patients would die until after-the-fact. In other words, the only way doctors could refuse to provide care to patients who will die anyway is if at the time that they must decide whether or not to provide the care (before-the-fact or ex ante), they have after-the-fact (ex post) information as to whether or not the patient will die anyway, which, obviously, they don’t have.

When analyses are performed in an attempt to measure some quantity, you can get a better idea of whether or not the analyst is actually measuring what he purports to measure by asking yourself the following three questions:

(1) What is it that the analysis is trying to measure (in theory)?

(2) What is the analyst actually measuring?

(3) Will estimates of what the analyst is actually measuring provide good estimates of the quantity that the analyst is trying to measure?