More Perils of Probability

British statistician, geneticist and probability guru, Peter Donnelly, recently gave a TED lecture about some of the common mistakes that people -- all people -- make when considering some fairly common probability scenarios. Peter is a very good speaker and present a self-deprecating wit that I find appealing. He is also very good at explaining what he does.

I was surprised, and delighted, to discover that he used as his primary example a scenario virtually identical to the one that I regularly employ to introduce lawyers to the perils of probability in tort litigation. The example involves a hypothetical person who receives a positive result to an HIV test. The underlying question is: "What are the chances the person is actually HIV positive?" My version of the example was derived from actual statistics concerning the incidence of HIV in a particular population (white women with no symptoms or risk factors). While Peter's example involves numbers that run in roughly the same ballpark, it is unclear whether he based them on real data or fabricated them for ease-of-use. Peter focuses his talk on the fact that such probabilities are hard to work with and that people are known to make particular kinds of mistakes when interpreting them. While I recognize such difficulties in my own presentation, I go on to focus on presentation strategies litigators can use to help juries correctly interpret probabilities. Anyway, it must be a really great example!

I am including here both Peter Donnelly's TED lecture and my own presentation slides on the topic. Peter gets to the common example about 40% of the way through his talk and you can find my treatment about 2/3 of the way through my slides (for those disinclined to watch both all the way through).

First Peter's talk.



Now, mine (No voice-over, I'm afraid. Feel free to invite me to present to your firm or lawyers group).

[Download presentation or view online]

The basic lesson to be learned here is that jurors are just not very good at math. Few have any experience with probability theory, especially anything related to very low-probability events. As I have discussed before, in relation to hindsight bias, tort cases typically stem from such extremely unlikely events. When jurors are faced with information processing tasks that are beyond their abilities, they typically resort to cognitive short-cuts. The most common of these is to use "intuition." Unfortunately, as Peter Donnelly so astutely illustrates, our intuitions about probabilities can often be completely off-the-mark. But the evaluation of reasonable care requires an accurate evaluation of the risks facing the care-taker. Litigators, then, have a Herculean task in getting jurors to understand the true underlying probabilities of a case.

I believe that two basic strategies are critical to getting jurors to appreciate the true risks faced by parties in a tort dispute. The first is reasoning by analogy. It is important to connect the choice problem faced by a decision-maker to something with which jurors are themselves familiar. Second, visual learning is key. Represent probabilities in a way that allows an average person to just "get it" by visual inspection. Ask yourself whether your exhibit passes the ol' "interocular impact test."

Finally, I want to point readers who, might not have watched Peter Donnelly's presentation all the way through, to a very interesting and disturbing example he presents at the end of his talk. In England, a woman was convicted of having murdered her two children, both of whom died of "crib death." The jury was largely convinced to convict on the testimony of an expert who testified that the chance she was innocent was simply the probability of any baby dying of crib death squared. That is, the expert chose to ignore the obvious dependence of the two events. If environmental or genetic factors made it more likely that one child would die in this way, those same factors would, of course, increase the likelihood that another child in the same family would also die in this way. That is, the fact that the defendant's two children died of crib death actually made it less likely that either resulted from foul play -- exactly the opposite of the expert's testimony. And the scariest thing is that not a single person in the courtroom called him on it. No one appreciated the enormous error in reasoning that was being committed, resulting in an innocent woman being sent to prison. Fortunately, her conviction was later reversed on appeal and the expert in question was discredited.

So, in your next case, make sure that (1) you have the probability theory right, (2) your expert does, too, (3) your expert is prepared to teach the jury how to evaluate such probabilities, and (4) you support such efforts with well-designed visual aids, both to reinforce your expert's testimony and also to guide jurors who might remain "confused" by the math.