Book Review: The Flaw of Averages

In the summer of 2003, Red Lobster launched the infamous “Endless Crab” campaign, touted as “a celebration of all the hot, steaming snow crab legs you can eat.” It turned into one of the most highly publicized statistical miscalculations in recent memory. Red Lobster executives grossly underestimated the average American’s appetite for crab legs. “It wasn’t the second helping, it was the third one that hurt,” recounted chief executive Joe Lee. Company executives also failed to account for a rise in wholesale crab prices, which eroded the company’s profit margin as the promotion drove up demand for crab nationwide. This incident and many others are richly described in Sam Savage’s aptly titled volume The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty.

Savage’s objectives are to reveal the pitfalls of using single-number averages in the face of uncertainty and to lift “the algebraic curtain separating the real-life manager from management science.” Savage, a professor of management science at Stanford University, is a gifted writer whose accessible writing style is a refreshing departure from the droning prose of many academics. As a pioneer in the emerging field of probability management (the practice of managing risk in a corporate environment), he has produced what, in essence, is a general management book disguised as a dissertation on statistical analyses.

The opening chapters provide a concise overview of statistical concepts for those uninitiated to “steam era” statistical jargon such as standard deviation and covariance. Readers with deeper quantitative backgrounds may feel compelled to skip these chapters, but Savage’s passion for the subject and keen sense of imagery make for quick reading. One of his memorable depictions of the flaw of averages is of a drunken man stumbling down a busy highway. While the man’s position is an average position in the middle of the road, he is still, on average, a dead man.

Students of history will also enjoy Savage’s rendition of the evolution of statistical analysis over the last 50 years. Particularly fascinating is the retelling of how the SRG (Statistical Research Group) was able to accurately estimate the number of German tanks during World War II. The SRG also advised the military on the placement of protective armor on bombers by examining where returning aircraft were not hit by bullets.

Savage establishes core principles to mitigate the flaw of averages. His five principles, or “mindels,” dictate that, for example, uncertain numbers should be thought of as probability distributions and that interrelationship between uncertain numbers can have a profound effect on risk. Though such assertions will hardly be provocative and disagreeable to most investment professionals, he cites numerous case studies, including evidence from the recent banking crisis and collapse in the housing market, as discrete examples of the blatant disregard for these principles.

In working toward a cure for the flaw of averages, Savage offers several remedies, some more progressive than others. Like many statisticians, he is a strong advocate of employing Monte Carlo simulations to better characterize and visualize risk. Recognizing the weakness of Monte Carlo–style risk modeling, he sees the need for a standardized library of probability distributions for everything from the return expectations of stocks and bonds to the successful development of a blockbuster pharmaceutical drug and the discovery of a new natural gas field. In this new world, in which unknown numbers are represented by a defined probability distribution, companies would rely on “chief probability officers” to more effectively manage systematic risk across an organization.

Readers indoctrinated into Nassim Nicholas Taleb’s world of “fat tails”—in which the variables with the greatest impact are those that are unknowable—will likely be Savage’s most vociferous critics. Taleb’s adherents argue that it was the overreliance and overconfidence in the financial system’s complex Monte Carlo–derived risk management models that created the most recent financial crisis in the first place. While Savage is likely the first to admit that Monte Carlo simulation is not the holy grail of risk modeling, he has successfully raised the bar of statistical understanding that will, at the very least, help prevent many avoidable risk-modeling missteps. The Flaw of Averages is a rare mix of entertainment and education. It will appeal to both general business managers and experienced quantitative professionals.

--Jeffrey S. Chang is an investment associate at Performance Equity Management, a private equity investment company based in Greenwich, Connecticut.