The Problems with Aggregate Measures

Normal.dotm 0 0 1 239 1364 QuantAA 11 2 1675 12.0 0 false 18 pt 18 pt 0 0 false false false

GNP, Net Income, SAT, GPA. Everybody loves aggregate metrics because they’re so easy to use. They provide simple snapshots of complex situations. And because everyone wants things to be easy, they convince themselves (and/or others convince them) that the single number proffered is all they really need to know to have a good understanding of what’s going on. Unfortunately, this is generally not the case.

Some of the bigger problems associated with aggregate metrics include the following.

1. Aggregate metrics often do not capture all issues of importance.

As the article points out, aggregate metrics often fail to capture important issues associated with the picture the metric is trying to paint:

One of the most glaring problems with using economic growth as a proxy for well-being was the fact that it excluded the damage to society and ultimately to the economy of environmentally non-sustainable activities.

In particular, aggregate metrics often fail to include important issues when those factors are difficult to measure. GNP does not include the costs associated with depletion of beneficial resources or the increase in harmful emissions. Net income does not capture the depletion or generation of intangible assets. SAT does not capture perseverance. GPA does not capture class difficulty.

Because aggregate measures are generally incomplete, comparisons of aggregate measures across actors (companies, countries, individuals, etc.) often provide apples-to-oranges comparisons. This means any conclusions drawn from the cross-actor comparisons of the aggregate statistics are bound to be flawed.