• Skip to primary navigation
  • Skip to main content

The One Percent Almanack

Wit and Wisdom on Investing, Business, and Life

  • Home
  • Members
  • Log In|Log Out
  • Show Search
Hide Search

The Grave Consequences of Misunderstanding Probability

April 17, 2020

In 1996, Sally Clark, an English Solicitor, gave birth to her first child. Unfortunately, her infant son died at the age of 11 weeks. Doctors reported the cause as sudden infant death syndrome (SIDS) — a diagnosis when an infant dies unexpectedly and postmortem doesn’t reveal anything.

SIDS usually occurs during sleep and affects infants under the age of one year, with the peak incidence occurring when the infant is at 2 to 4 months of age. As an aside, Elon Musk’s first child had also succumbed to SIDS in 2002.

Clark’s life took a nasty turn when in January 1998, Clark gave birth to her second son, and the baby died at 8 weeks, again reportedly of SIDS. It’s hard to imagine her situation but her plight worsened when a month later, she was arrested and accused of smothering both her children.
[show_to accesslevel=’almanack-all’]

The prosecution argued that SIDS is a rare cause. Their reasoning was based on the statistical evidence presented by paediatrician Professor Sir Roy Meadow, who testified that the chance of two children from an affluent family suffering SIDS was 1 in 73 million.

There was no substantive evidence against Sally Clark but based on just this argument, she was sent to prison.

But Meadow’s logic was flawed. His calculation relied on the assumption that two SIDS deaths in the same family are independent. He claimed that, for an affluent non-smoking family like the Clarks, the probability of a SIDS death was 1 in 8,543, so the likelihood of two in the same family was around “1 in 73 million” (8543 × 8543).

It is not the probability that two children will die of SIDS that we seek but the probability that the two children who died, died of SIDS, wrote Leonard Mlodinow in his book The Drunkard’s Walk where I read about Clark’s story for the first time.

In other words, the probability that two children will die of SIDS is not the same as the probability that two children who have died, have died of SIDS. It’s a subtle distinction but a crucial one when it comes to dealing with conditional probabilities.

To wrap your head around this, you need to understand the idea of Bayes theorem which helps us calculate the conditional probability which measures the probability of an event, given that another event has occurred.

In Clark’s case, the right way to size up the probability was to ask — given the condition that the first child has died of SIDS, what’s the probability that the cause of the second death is also SIDS?

The two drivers in Bayes rule are — the base rate (or prior odds) and the likelihood ratio (strength of the new evidence). So the equation becomes —

Posterior odds (Conditional Probability) = Prior odds (Base Rate) × Likelihood ratio

People who aren’t familiar with Bayesian thinking make two kinds of mistakes.

The first is when they completely ignore the base rates and tend to get influenced by the story of single evidence. Base rate neglect is a serious thinking error where people forget about the historical statistical evidence and tend to believe the anecdotal evidence.

A simple example is when someone argues – “My grandfather was a lifelong chain smoker and still lived up to a ripe age of 90 years. So don’t tell me that smoking kills.” He’s ignoring the fact that statistically a chain smoker isn’t likely to live a long life.

The second mistake is when we underestimate the strength of the new evidence (or a compelling story) and fail to update the base rates.

After one case of SIDS death in a family, it becomes much more likely to have another SIDS in the same family owing to unknown genetic or environmental factors.

Two years after Clark was imprisoned, the Royal Statistical Society (RSS) weighed in on this subject with a press release, declaring that the jury’s decision was based on a serious error of logic. There were two mistakes in the prosecution’s reasoning.

First, the assumption of base rates of SIDS death was not used in accordance with Bayes theorem. RSS argued that there may well be unknown genetic or environmental factors that predispose families to SIDS, so that a second case within the family becomes much more likely than would be a case in another, apparently similar, family.

Put simply, once a family has a SIDS death, the base rate assumption of 1 in 8543 is no more valid.

In 2004, Ray Hill, a Mathematics Professor, did extensive research on SIDS statistics for England and concluded that after a first SIDS death, the chances of a second go up significantly — as much as by 5 to 10 times.

The second error that the jury made is known as the Prosecutor’s Fallacy.

Meadow argued that the probability of double SIDS was 1 in 73 million. Hence it wasn’t likely. So the only other possibility was murder.

The inherent assumption in the above statement is that there are only two competing explanations — double SIDS or double murder. However, the base rate of the double murder is also very small.

In order to calculate the probability of Clark’s innocence, the jury needed to weigh up the relative likelihood of the two competing explanations for the children’s deaths. In other words, murder was not the only alternative explanation for the deaths in the way that one might have inferred from asserted probability of double SIDS. Although double SIDS is very rare, double infant murder is likely to be rarer still, so the probability of Clark’s innocence was quite high. Hill calculated the odds ratio for double SIDS to double homicide at between 4.5:1 and 9:1. (Source: Wikipedia)

Also, even the initial assumption about SIDS base rate of 1 in 8543 was questionable because SIDS is not that uncommon. SIDS was the third leading cause of death in children less than one year old in the United States in 2011. In fact, it is the most common cause of death between one month and one year of age.

Excerpt from Mlodinow’s book —

Mrs. Clark appealed the case and hired their own statisticians as expert witnesses. Unfortunately, they lost the appeal. Fortunately, they continued to seek medical explanations for the deaths and in the process uncovered the fact that the pathologist working for the prosecution had withheld the fact that the second child had been suffering from a bacterial infection at the time of death, an infection that might have caused the infant’s death. Based on that discovery, a judge quashed the conviction and Sally Clark was released after having served three years in prison.

Clark’s release in January 2003 prompted reopening and review of hundreds of other similar cases. At least three other women — Donna Anthony, Angela Cannings, and Trupti Patel — who were convicted of murdering their children, were released from prison.

In each case, Meadow had testified about the unlikelihood of multiple SIDS deaths. In 2005, Meadow’s medical practice license was revoked on charges of serious professional misconduct.

For most of my life, I had imagined that decisions in the court of law would be based on hard facts and reliable witnesses. Case studies like this opened my eyes to how innocuous flaws in logical reasoning and misunderstanding the probability can make or break lives.

In the case of Sally Clark, the damage had already been done. After being acquitted and released from prison, she was could never recover from the effects of her conviction and imprisonment.

In March 2007, she died after a prolonged mental illness attributed to her traumatic experiences caused by accusations of murdering her children.

[/show_to] [hide_from accesslevel=’almanack-all’]

Want to Read More? This content is exclusive for members of Value Investing Almanack. Login to read if you are a member. Else, click here to subscribe.

Benefits to VIA Members
 
  • Spotlight: Big ideas from Value Investing and why applying them in your investment decision making will be a great deal
  • InvestorInsights: Interviews with experienced value investors, learners, and deep thinkers
  • StockTalk: Thorough analysis of business models of companies (without any recommendations)
  • Behaviouronomics: Deep analysis of human behaviour and how it impacts investment decision making
  • BookWorm: Reviews of the best books on Value Investing and related subjects
  • Free Course – Financial Statement Analysis for Smart People (otherwise priced at Rs 5,900)
  • Archives: Instant access to our huge archive from the past three years
Become a VIA Member. Click to Subscribe
[/hide_from]

Filed Under: Special Report

Handcrafted with in India