Gustav Heinrich Ralph von Koenigswald was a German-Dutch palaeontologist. His discoveries of hominid fossils in Java and his other studies in human fossil research made him the leading figures of 20th Century paleo-anthropology.
During one of his excavation trips to the Solo River at Ngandong, he thought of engaging the locals to assist him in finding the hominid bones. This move turned out to be one of the biggest tactical errors.
He decided to offer the locals 10 cents for every piece of hominid bone they could come up with. The scheme seemed to work until Koenigswald discovered that the locals had been enthusiastically smashing large pieces into small ones to maximize their income.
He failed to set the incentives thoughtfully and people optimized for maximum return on their efforts.
[show_to accesslevel=’almanack-all’]
You’d find scores of stories with similar patterns where someone used a proxy for value and the proxy soon becomes the target of optimization which inevitably results in the original idea backfiring.
This phenomenon — a measure ceasing to be a good measure because it becomes the target — is known as Goodhart’s law. It is named after English economist Charles Goodhart. He was the chief economic advisor to the Bank of England.
Goodhart stated —
Any observed statistical regularity will tend to collapse once the pressure is placed upon it for control purposes.
Robert Heilbroner, in his 1990 article, explained the idea with the example of the Soviet Union where economic planning backfired during the later part of the twentieth century. He wrote –
For many years, targets were given in physical terms—so many yards of cloth or tons of nails—but that led to obvious difficulties. If cloth was rewarded by the yard, it was woven loosely to make the yarn yield more yards. If the output of nails was determined by their number, factories produced huge numbers of pin-like nails; if by weight, smaller numbers of very heavy nails. The satiric magazine Krokodil once ran a cartoon of a factory manager proudly displaying his record output, a single gigantic nail suspended from a crane.
You may want to laugh at such ridiculous stories because we like to think that our modern capitalistic societies are generally much better run than Soviet Russia. But Goodhart’s law is everywhere because in one way it explains the tendency of the people (and the institutions built by people) to achieve their explicitly defined targets in the easiest way possible, often disobeying the spirit of the law.
Gabriel Weinberg, in his excellent book Superthinking, writes —
When you try to incentivize behavior by setting a measurable target, people focus primarily on achieving that measure, often in ways you didn’t intend. Most importantly, their focus on the measure may not correlate to the behavior you hoped to promote.”
Cobra effect is another well-known example of this about which I had written earlier in the context of unintended consequences in a complex adaptive system.
It’s like a wish-granting genie who finds loopholes in your wishes, adds Weinberg, “meeting the letter of the wish but not its spirit, and rendering you worse off than when you started.”
Charlie Munger never gets tired talking about the power of incentives. On one instance he explained with this example —
Early in the history of Xerox, Joe Wilson, who was there in the government, had similar experience. He had to go back to Xerox because he couldn’t understand why this new machine was selling so poorly in relation to its older and inferior machine. When he got back to Xerox, he found out that the commission arrangement with the salesmen gave a large and perverse incentive to push the inferior machine on customers, who deserved a better result.
There’s no limit to human ingenuity when it comes to finding loopholes to game the system. In his book, Tyranny of Metrics, Jerry Muller fleshes out a few patterns to spot Goodhart’s law in action —
- Gaming through creaming. This takes place when practitioners find simpler targets or prefer clients with less challenging circumstances, making it easier to reach the metric goal, but excluding cases where success is more difficult to achieve.
- Improving numbers by lowering standards. One way of improving metric scores is by lowering the criteria for scoring. Thus, for example, graduation rates of high schools and colleges can be increased by lowering the standards for passing. Or airlines improve their on-time performance by increasing the scheduled flying time of their flights.
- Improving numbers through omission or distortion of data. This strategy involves leaving out inconvenient instances, or classifying cases in a way that makes them disappear from the metrics. Police forces can “reduce” crime rates by booking felonies as misdemeanors, or by deciding not to book reported crimes at all.
- Cheating. One step beyond gaming the metrics is cheating—a phenomenon whose frequency tends to increase directly with the stakes of the metric in question.
One place where Goodhart’s law is conspicuous is our education system. The thing called examination imposed by the authorities in school and colleges is supposed to be a proxy for something else. It’s thought of as a measure not just how well you did on that particular exam, but how well you learned in the class.
Paul Graham argues –
The most damaging thing you learned in school wasn’t something you learned in any specific class. It was learning to get good grades. In theory, tests are merely what their name implies: tests of what you’ve learned in the class. In theory you shouldn’t have to prepare for a test in a class any more than you have to prepare for a blood test. In theory you learn from taking the class, from going to the lectures and doing the reading and/or assignments, and the test that comes afterward merely measures how well you learned. In practice, the phrase “studying for a test” was almost redundant, because that was when one really studied.
The test-taking system in educational institutions is completely hackable. And any system that’s hackable will be hacked.
GDP is one such proxy to the economic strength of a country. In his article title GDP fetishism, David Henderson reasons –
…many economists go on to give GDP far more credit than it deserves. They tend to consider fiscal and monetary policy positive if these policies increase GDP, but they often fail to ask, let alone answer, whether those same policies increase or reduce welfare. I have a term for giving GDP such a sacred place in economists’ reasoning: GDP fetishism.
The concept of GDP, handled carefully, can be useful. But for many people, and even many good economists, GDP has been used to judge well being even when using it that way leads to highly misleading conclusions.
Think about it, if everyone increased their consumption by 2x, the GDP of our country would double. But no one is asking what kind of things people are consuming. And what about the impact of all that consumerism on the planet earth? After all, every bit of increase in GDP can be traced back to the exploitation of some natural resource, isn’t it?
Ditto with money. It was invented as a means to store value and exchange goods and services. In other words, humans brought in this new tool to reduce the friction in their affairs. But over the centuries, this imagined reality called money became the main thing and everyone started optimizing their lives to earn and hoard it.
I hope Goodhart’s law will give you new lenses to relook at your life’s goals and arrive at the real metric rather than the proxy.
[/show_to] [hide_from accesslevel=’almanack-all’]- Spotlight: Big ideas from Value Investing and why applying them in your investment decision making will be a great deal
- InvestorInsights: Interviews with experienced value investors, learners, and deep thinkers
- StockTalk: Thorough analysis of business models of companies (without any recommendations)
- Behaviouronomics: Deep analysis of human behaviour and how it impacts investment decision making
- BookWorm: Reviews of the best books on Value Investing and related subjects
- Free Course – Financial Statement Analysis for Smart People (otherwise priced at Rs 5,900)
- Archives: Instant access to our huge archive from the past three years