chanceIt is now well established that bad lawyering, bad expert evidence and bad police work can contribute to wrongful convictions but do innocent people spend time in prison owing to bad math?  Photo: Diacritica (Wikipedia)

Author: Claire Horsnell, Osgoode Hall Law Student

The truism, originally attributed to nineteenth-century British prime minister Benjamin Disraeli and popularized by Mark Twain, about there being three kinds of lies—“lies, damned lies, and statistics”—nods ironically toward the ways in which statistics can be manipulated to support a speaker’s point. However, for non-statisticians, the use of impressive-sounding numbers to support a contention can be very convincing; numbers, of course, are a fundamentally reliable way to quantify and understand the world around us. However, an error in calculation or misunderstanding—or worse, a deliberate misrepresentation of statistical evidence—in a courtroom can have dire consequences.

Probably the most notorious instance in which statistics played a part in a wrongful conviction is the British case of Sally Clark. Clark, a solicitor based in Manchester in the north of England, was not only a victim of an egregious miscarriage of justice, but also of tragic circumstances. Her first son, Christopher, was born healthy but passed away at the age of two-and-a-half months, after falling unconscious in the family home. Clark’s second son, Harry, was born two years later—and died at the age of eight weeks, after he was found unconscious, and attempts to resuscitate him failed.

Clark and her husband were arrested shortly afterward; the charges against Steve Clark were dropped, but Sally Clark faced two counts of murder. She denied the charges.

The most controversial element of Clark’s trial was the testimony by then eminent paediatrician Professor Sir Roy Meadow. Meadow testified on the stand that the chances of two cot deaths occurring in the same family were around one in 73 million. He followed up by asserting that, since there were over 700,000 live births every year in England, Scotland, and Wales, such an instance of double cot death would only take place once every hundred years. This was pretty damning evidence— Clark was convicted and received a mandatory sentence of life imprisonment.

The problem was that the statistic was—predictably—incorrect. Meadow had arrived at “one in 73 million” by squaring the figure of 1 in 8,543—the chance of one cot death in a household similar to that of the Clarks, according to a government-sponsored study which ran from 1993–1995, and for which Meadows wrote the preface.[1] The problem was that to square the original statistic effectively meant treating the two deaths as independent events, without considering the possible effects that genetics and even environment could have had on the two children.[2]

But the calculation wasn’t the only significant problem. Meadow had unwittingly been blinded by the infamous Prosecutor’s Fallacy.[3] Gerd Gigerenzer has summarized the fallacy as follows:

[T]he prosecutor’s fallacy is to reason that the probability of a random match is the same as the probability that the defendant is not guilty, or equivalently, that the guilt probability is 1 minus the probability of a random match. For instance, assume that the p(match) is 1 in 1,000. The person who commits the fallacy reasons that, therefore, the chances that the defendant is not guilty are 1 in 1,000 or, equivalently, that the chances that the defendant is guilty are 999 in 1,000. In fact, these probabilities are not the same.[4]

In other words, the fallacy states that the chances of a particular set of circumstances occurring at random are the same as the possibility that the defendant is innocent—which is not actually the case. The errors in the Clark case drew the ire of the Royal Statistical Society, which issued a press statement in October 2001 raising its concerns. The Society did not mince words: “The case of R v Sally Clark,” it said, “is one example of a medical witness making a serious statistical error, one which may have had a profound effect on the outcome of the case. The Society urges the Courts to ensure that statistical evidence is presented only by appropriately qualified statistical experts, as would be the case for any other form of expert evidence.”[5]

Steven J Watkins, writing in the British Medical Journal, also drew attention to the problem saying “Guidelines for using probability theory in criminal cases are urgently needed. The basic principles are not difficult to understand, and judges could be trained to recognise and rule out the kind of misunderstanding that arose in this case.”[6] He went on to point out that “It is possible to be an extremely good doctor without being numerate, and not every eminent clinician is best placed to give epidemiological evidence. Doctors should not use techniques before they have acquainted themselves with the principles underlying them.”[7] This is, obviously, good advice for lawyers as well.

That said, lawyers cannot and should not be expected to turn themselves into expert statisticians. However, an awareness of the need to question how statistics are formulated and how they can be abused can lead us to question seemingly damning figures such as “a chance of one in seventy-three million”—and may help to prevent further wrongful convictions and tragic consequences. (Clark was eventually exonerated and released after serving three years of her sentence, but never recovered from the experience, and died in 2007 as a result of alcohol poisoning.) Given the prevalence of probabilities used in, for example, DNA test results, an improvement in the ability of lawyers to understand statistical evidence, to be able to present it clearly to juries, and to challenge accurately the formulation of suspicious statistics can only be of benefit to the justice system as a whole.


 

[1] Department of Health (UK), Confidential Enquiry into Sudden Deaths in Infants (CESDI), 5th Annual Report.

[2] The effect of genetics was a key element in the trial of Trupti Patel, at which Meadow was also an expert witness.

[3] The Prosecutor’s Fallacy was first named by William C Thompson and Edward Schumann in their 1987 paper “Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor’s Fallacy and the Defense Attorney’s Fallacy,” published in 11 Law & Human Behaviour, 167–187.

[4] Gerd Gigerenzer, Reckoning with Risk: Learning to Live with Uncertainty (London: Penguin, 2002).

[5] Quoted in John Batt, Stolen Innocence (London: Ebury Press, 2004), at 401.

[6] Steven J Watkins, “Conviction by Mathematical Error?” (2000) 320 Brit Med J 2 at 3 (ProQuest).

[7] Ibid.