Most people learn about probability concepts through examples of coins and dice. But that rarely helps in developing an intuitive mind for probabilities. The cure for it is to leverage the idea of natural frequencies.
The day remains deeply etched in my memory. I was in the fifth standard and the maths teacher had introduced the concept of percentages that day. It sounded freakishly alien and I just couldn’t fathom it. I literally had tears in my eyes because it seemed that everybody in the class understood it except me.
The teacher had repeated the definition – percentage is when you normalize any ratio to base of hundred – at least a dozen times in the class that day. It didn’t help.
Then my father explained, “Look. When you divide something in two equal parts, each part is called half, right? Instead of calling it half, let’s call it 50 percent. Now, when you divide something in four equal parts each portion is one fourth, correct? Let’s call that 25 percent.”
[show_to accesslevel=’almanack-all’]
That made some sense. It took a while, but finally, I was able to wrap my head around this strange human invention called percentages.
Unfortunately, mathematicians didn’t stop at percentages. They got busy and went on to invent probability. If percentage is normalization, then probability is normalization on steroids.
In his remarkable book, Innumeracy, mathematician John Allen Paulos recounts watching a newsreader forecasting the weather forecast on the TV. He announced that there was a 50 percent chance of rain for Saturday and a 50 percent chance for Sunday. He then concluded that there was therefore 100 percent chance of rain that weekend. Paulos was horrified.
For most people, the first formal encounter with probability happens in high school. The route to the knowledge of probability often goes through the stories of flipping coins, rolling dice, spinning roulette wheels and shuffled cards. Unlike percentages, developing an intuition for probability never happens for most people.
Being innumerate in probability can get you in all sorts of troubles like the man who travelled a lot and was always worried about the possibility of a bomb on board his plane. He determined the probability of it to be low but not low enough for him, so he decided to carry a bomb in his suitcase. He reasoned that the probability of two bombs being on board would be infinitesimal.
I am sure you’d find the man’s reasoning pretty absurd. But most people can’t pinpoint the exact flaw in the above logic.
Losing Money in Goa
Imagine you’re walking on a beach in Goa. Every few seconds, a sea wave splashes on your legs and then on its way back tries to steal away all the sand below your feet. Just observing that mischief of waves for few minutes is a transcendental experience.
Suddenly, your meditative state is disrupted by a commotion on the beach. A screaming crowd surrounds a man holding few colorful cards. You reckon that some kind of betting game is going on.
The man is furtively playing a card game with rupee notes stacked tall on a small makeshift table. The game is simple. There are three cards. One is red on both sides, the second white on both sides, and the third is red on one side and white on the other. The guy lets you draw one card blindly and places it on the table face up. Say it’s red. Then he offers you a Rs. 100 bet. If the hidden side is also red, he wins; if it’s white, you do.
Seems a fair bet, isn’t it? After all, the card you drew could be either the red-red or the red-white card, so the probability that the other side is red or white is equal, i.e., 0.5 or 50% (the 5th-standard kid in me still prefers percentages over probabilities)
Well, that’s the wrong answer. The Goan trickster’s chance of winning is 67 percent. Don’t feel bad if you don’t understand it because our brains are just not wired to do probability problems very well.
One way to solve these problems is by using conditional probabilities and plugging the Bayes’ equation. You’ll get the right answer, but unless you’ve spent a good part of your life dreaming about mathematical equations, Bayes’ equation is not easy to visualize. The trick is to forget the equations and make use of natural frequencies.
Natural Frequencies
Frequencies that correspond to the way humans encountered information before the invention of books and probability theory. Gerd Gigerenzer, in his book Risk Savvy, writes –
Unlike probabilities and relative frequencies, they are “raw” observations that have not been normalized with respect to the base rates of the event in question. For instance, a physician has observed one hundred persons, ten of whom show a new disease. Of these ten, eight show a symptom, whereas four of the ninety without disease also show the symptom. Breaking these one hundred cases down into four numbers (disease and symptom: 8; disease and no symptom: 2; no disease and symptom: 4; no disease and no symptom: 86) results in four natural frequencies: 8, 2, 4, and 86. Natural frequencies facilitate Bayesian inferences. For instance, a physician who observes a new person with the symptom can easily see that the chance that this patient also has the disease is 8/(8 + 4), that is, two thirds. This probability is called the posterior probability…“Natural frequencies help people to “see” the posterior probabilities,whereas conditional probabilities tend to cloud minds.
Now using natural frequencies let’s try to understand how you were conned on the beaches of Goa.
Let’s say you played the card game three hundred times. Since the draw is random, we can expect that each card will come up in 100 draws, i.e., the middle level in the visual representation of the natural frequency tree. The bottom level shows the range of all the outcomes.
For each of the two red-red cards, a red side will show faceup; for the two red-white cards, we expect only one red. (The white-white card will of course never show red.) In the three cases where red is faceup, two are with the red-red card, and one with the red-white card. In other words, if you draw a red face, the other side will be red as well two out of three times. That’s why the card guy bets on red and makes money. Your chance of winning is not 50:50, but only one in three.
Although these kinds of riddles are much more interesting than the textbook problems of coin tosses and dice rolls, they are still far removed from the real-world situations that we face every day (unless you frequent the Goa beaches a lot).
In many countries worldwide, doctors prescribe that all pregnant women who are above thirty-five (in India it’s thirty) to attend the screening for Down syndrome. For a pregnant woman, prenatal screening is an extremely stressful situation. The joy of expecting a child is hijacked by the anxiety of Down syndrome or other birth defects.
The Down syndrome screening involves two tests. First, they do a blood test during the first trimester. If the result of the blood test is positive, the woman needs to decide whether to undergo for the second test which is an invasive test and has the probability of miscarriage which is about 0.5 percent. Ironically, for a woman in her thirties, the likelihood of giving birth to a baby with Down syndrome is same, i.e., 0.5 percent. So it’s an excruciating dilemma if the first test shows positive. Whether to go for the second test and risk a miscarriage or to avoid the invasive test and face the risk of having a baby with Down syndrome?
Here are some stats –
1. The prevalence of Down syndrome is only one percent, i.e., for every 100 babies that take birth, only one is born with Down syndrome.
2. If the baby has Down syndrome, there is a 90 percent chance that the test result will be positive.
3. If the baby is unaffected, there is still a 5 percent chance that the test result will be positive, i.e., the false positive rate is 5 percent.
What happens when a thirty-five-year-old woman is tested positive in the first-trimester blood test and decides to forego the invasive test? The remaining months of the pregnancy would be highly stressful for her. But should she really be so worried?
Let’s apply the natural frequency method that we learnt in Goa. We would first need to convert the percentages into numbers –
1. About 10 out of every 1,000 babies have Down syndrome.
2. Of these 10 babies with Down syndrome, 9 will get a positive test result.
3. Of the remaining 990 unaffected babies, about 50 will still have a positive test result.
So out of every one thousand women who go through screening, 59 are tested positive. But only 9 have Down syndrome. Which means, only one out of every six or seven women with a positive screening result actually has a baby with Down syndrome.
Put simply, even if the test is positive, the baby most likely doesn’t have Down syndrome. However, the stress caused by worry may actually harm the baby.
Gerd again –
The younger a pregnant woman is, the less likely the chances. If she is thirty-five rather than forty years old, her baby’s chance of having Down syndrome after a positive test is only about one in twenty. That means the baby most likely does not have it. The tree also helps to understand that a negative test result is not certain either; about one out of every ten cases of Down syndrome is overlooked.
By making the numbers clearer, natural frequencies help to make an informed decision. There are good reasons for not taking the test: when a woman is young, when she does not want to risk a miscarriage, or when she would not abort a baby with Down syndrome. If she decides for prenatal screening, she needs to know what a test result means.
Conclusion
Monty Hall problem first appeared in a Sunday Newspaper in the US in 1990. Ninety-two percent of people gave an incorrect answer, and that included 1000 maths PhDs. The Monty Hall riddle goes like this –
Suppose the contestants on a game show are given the choice of three doors: Behind one door is a car; behind the others, goats. After a contestant picks a door, the host (Monty Hall), who knows what’s behind all the doors, opens one of the unchosen doors, which always reveals a goat. He then says to the contestant, “Do you want to switch to the other unopened door?” Is it to the contestant’s advantage to make the switch?
Like the card problem, most people estimate that the probability of winning a car in Monty Hall problem remains 0.5 (switch or no switch). And like the card problem, the reality is that the likelihood of winning a car isn’t 50-50.
If you make the switch, it’s 67 percent and only 33 percent if you don’t switch. Now that you understand how natural frequencies work; go figure why you should switch.
Charlie Munger rightly said, and I am paraphrasing him, someone who doesn’t understand elementary probability is like a one-legged man in an ass kicking contest.
When you develop intuitive understanding of conditional probability, you become the man who brings a flamethrower to a stick fight.
[/show_to]
[hide_from accesslevel=’almanack-all’]
- Spotlight: Big ideas from Value Investing and why applying them in your investment decision making will be a great deal
- InvestorInsights: Interviews with experienced value investors, learners, and deep thinkers
- StockTalk: Thorough analysis of business models of companies (without any recommendations)
- Behaviouronomics: Deep analysis of human behaviour and how it impacts investment decision making
- BookWorm: Reviews of the best books on Value Investing and related subjects
- Free Course – Financial Statement Analysis for Smart People (otherwise priced at Rs 5,900)
- Archives: Instant access to our huge archive from the past three years
