Recent Jury Box Blog Entries

Subscribe to The Jury Box Blog

Showing posts with label probabilities. Show all posts
Showing posts with label probabilities. Show all posts

Thursday, January 07, 2010

Murder Trial to test public attitudes towards mental illness

Preschooler diagnosed with ADHD dies from overmedication: Parents on Trial for Murder

Rebecca Riley was diagnosed with ADHD and a bi-polar disorder before her third birthday. Her treatment was supervised by a respected pediatric psychologist, who prescribed two powerful medications, normally reserved for adults. While not approved for such use by the FDA, such "off label" uses are at the professional discretion of the treating physician, and are not uncommon.

Just over a year later, Rebecca Riley died.

The Massachusetts Medical Examiner concluded that her death was caused by an overdose of her medicine. Her parents, Michael and Carolyn Riley, on trial for her murder, claim that she died from pneumonia contracted shortly before her death. A grand jury refused to indict the girl's physician, concluding that the prescribed dosage was not to blame. Rather, the grand jury indicted the parents for murder on the theory that they intentionally overmedicated their daughter to make her easier to manage.

Read about the trial here.

There is some incriminating evidence that the Rileys will have to counter during their trial. School officials, including the school nurse, reported that the girl was often listless and non-responsive in school, sometimes requiring assistance just to get on or off the school bus. In addition, according to the girl's doctor, Mrs. Riley often requested additional refills of her daughter's medicine, claiming that she had lost or damaged bottles of pills. Finally, the Rileys' other two children have also been diagnosed with similar conditions and are being treated in a similarly aggressive way.


Are all these children really mentally ill?

Without a "smoking gun," per se, the outcome of this case will depend, to a great deal, on public perceptions of pediatric mental illness in the modern era. There are clearly countervailing trends and it remains to be seen how they will balance out.

On the one hand, psychiatry (with a big assist from neuroscience) has come a long way in identifying the physiological and neurological causes (or at least traces) of many previously undiagnosed (or misdiagnosed) conditions. We no longer throw women into insane asylums for "bad humors" or "hysteria." Most members of the public believe that depression actually exists and has a neurological foundation. People now appreciate the hereditary influences on alcoholism and other forms of "abhorrent" behavior. Americans generally recognize that soldiers and witnesses to war really do express post traumatic stress disorders. Mental illness is no longer a taboo subject the way it was even 50 years ago.

On the other hand, the public is clearly skeptical of the increasing propensity of mental health professionals to assign a syndrome name to virtually every form of unusual or undesirable behavior. Most current adults were raised during a time when behavioral problems among students were handled behaviorally, rather than psychologically. Children were expected -- required -- to behave themselves in school and at home. There were certainly allowances made for kids who were fidgety or had difficulty paying attention, but children were generally held accountable for their own actions. The primary objection to the current climate of diagnosing every other kid with some kind of disorder or syndrome is that we are raising a generation of people who won't know how to be responsible for their own decisions and actions. By making excuses for our kids, we are raising a nation of excuse-makers.

Modern pediatric psychiatry and parenting on trial

With these countervailing trends in play, how will jurors in this case perceive of the actions of Rebecca Riley's parents? Some may be inclined to defer to the girl's psychiatrist, excusing the parents for doing the same. They gave her these drugs because their doctor said it was OK. The doctor must have seen Rebecca on a regular basis. If she didn't seem concerned, then the parents had no way of knowing that Rebecca was in any kind of danger.

Other jurors will be skeptical of the parents' motives. A parent can get some doctor to diagnose a kid with some kind of disorder if they push hard enough. These folks obviously wanted to medicate their daughter to make their own lives easier. Why didn't they focus on trying to be better parents?

The wild card in this debate might just be the Rileys' handling of their other children. Jurors are notoriously bad at understanding and processing arguments based on probabilities. Unfortunately for the lawyers in this case, conveying the implications of the diagnoses of Rebecca's siblings requires the jurors to wrestle with such issues.

Consider the numbers: It is estimated that roughly 1 out of every 9 American children has some level of diagnosable ADHD. That is about 11%. One might then conclude that the chance of all three Riley children having ADHD is 0.11 cubed, or 0.0013 -- roughly one in a thousand. (Apparently only about 1/3 of children with ADHD receives the treatment she requires. As such, public perception would be that only about 4% of children have ADHD.) By such logic, it is extremely unlikely that the Riley kids got similarly diagnosed purely by chance. With such astronomical odds against it, the parents must really have been pushing doctors to diagnose ADHD.

Such logic is spurious, however, because it assumes that the incidence of ADHD among siblings are independent events. In truth, due to the shared genetics and physiology of siblings, having a brother or sister with ADHD substantially increases the chance of a child having the disorder.
When one child in a family has ADHD, a sibling will also have the disorder 20% to 25% of the time, says geneticist Susan Smalley, PhD, co-director of the Center for Neurobehavioral Genetics at the David Geffen School of Medicine at UCLA (www.adhd.ucla.edu). About 15% to 40% of children with ADHD will have at least one parent with the same condition.
By this logic, the fact that both of Rebecca's siblings have been diagnosed with ADHD makes it substantially more likely that Rebecca was correctly diagnosed with ADHD. If having one sibling with ADHD would increase the likelihood of Rebecca having it by almost three times, having two afflicted siblings would likely increase the chances five-fold.

This leaves the jurors in a tricky bind. Everything turns on the reliability of the doctor's diagnostic techniques. If her diagnoses of Rebecca's siblings were correct, her diagnosis of Rebecca almost certainly was, too. On the other hand, if her methods were suspect, the likelihood that she properly diagnosed Rebecca's condition, in particular, drops dramatically.

How to explain this to jurors

I have written in other blog entries about the "perils of probability." I also regularly discuss this issue with attorneys to whom I give presentations. It is very difficult to teach enough probability theory to jurors that they actually "get it." Don't expect them to be able to replicate the math. On the other hand, it is possible to provide them with enough insight to appreciate the nature of the results. My advice: Use visual aids to convey comparisons between probabilities. Analogize to decisions-under-risk with which jurors might be familiar. Make sure that your expert witnesses are excellent teachers. Jurors perceive as more reliable those witnesses who are seen as "helpful."

Regardless of the exact presentation strategies a litigator employs, she would always be wise to test its efficacy in advance. You might think your expert's explanation is crystal clear, but you can't be sure until you present it to a group of ordinary people and find out what they think. If your case turns on the "perils of probability," as the Riley case seems to, some sort of pretrial jury study (focus group research) is absolutely essential.

Friday, October 09, 2009

More Perils of Probability

British statistician, geneticist and probability guru, Peter Donnelly, recently gave a TED lecture about some of the common mistakes that people -- all people -- make when considering some fairly common probability scenarios. Peter is a very good speaker and present a self-deprecating wit that I find appealing. He is also very good at explaining what he does.

I was surprised, and delighted, to discover that he used as his primary example a scenario virtually identical to the one that I regularly employ to introduce lawyers to the perils of probability in tort litigation. The example involves a hypothetical person who receives a positive result to an HIV test. The underlying question is: "What are the chances the person is actually HIV positive?" My version of the example was derived from actual statistics concerning the incidence of HIV in a particular population (white women with no symptoms or risk factors). While Peter's example involves numbers that run in roughly the same ballpark, it is unclear whether he based them on real data or fabricated them for ease-of-use. Peter focuses his talk on the fact that such probabilities are hard to work with and that people are known to make particular kinds of mistakes when interpreting them. While I recognize such difficulties in my own presentation, I go on to focus on presentation strategies litigators can use to help juries correctly interpret probabilities. Anyway, it must be a really great example!

I am including here both Peter Donnelly's TED lecture and my own presentation slides on the topic. Peter gets to the common example about 40% of the way through his talk and you can find my treatment about 2/3 of the way through my slides (for those disinclined to watch both all the way through).

First Peter's talk.



Now, mine (No voice-over, I'm afraid. Feel free to invite me to present to your firm or lawyers group).

[Download presentation or view online]

The basic lesson to be learned here is that jurors are just not very good at math. Few have any experience with probability theory, especially anything related to very low-probability events. As I have discussed before, in relation to hindsight bias, tort cases typically stem from such extremely unlikely events. When jurors are faced with information processing tasks that are beyond their abilities, they typically resort to cognitive short-cuts. The most common of these is to use "intuition." Unfortunately, as Peter Donnelly so astutely illustrates, our intuitions about probabilities can often be completely off-the-mark. But the evaluation of reasonable care requires an accurate evaluation of the risks facing the care-taker. Litigators, then, have a Herculean task in getting jurors to understand the true underlying probabilities of a case.

I believe that two basic strategies are critical to getting jurors to appreciate the true risks faced by parties in a tort dispute. The first is reasoning by analogy. It is important to connect the choice problem faced by a decision-maker to something with which jurors are themselves familiar. Second, visual learning is key. Represent probabilities in a way that allows an average person to just "get it" by visual inspection. Ask yourself whether your exhibit passes the ol' "interocular impact test."

Finally, I want to point readers who, might not have watched Peter Donnelly's presentation all the way through, to a very interesting and disturbing example he presents at the end of his talk. In England, a woman was convicted of having murdered her two children, both of whom died of "crib death." The jury was largely convinced to convict on the testimony of an expert who testified that the chance she was innocent was simply the probability of any baby dying of crib death squared. That is, the expert chose to ignore the obvious dependence of the two events. If environmental or genetic factors made it more likely that one child would die in this way, those same factors would, of course, increase the likelihood that another child in the same family would also die in this way. That is, the fact that the defendant's two children died of crib death actually made it less likely that either resulted from foul play -- exactly the opposite of the expert's testimony. And the scariest thing is that not a single person in the courtroom called him on it. No one appreciated the enormous error in reasoning that was being committed, resulting in an innocent woman being sent to prison. Fortunately, her conviction was later reversed on appeal and the expert in question was discredited.

So, in your next case, make sure that (1) you have the probability theory right, (2) your expert does, too, (3) your expert is prepared to teach the jury how to evaluate such probabilities, and (4) you support such efforts with well-designed visual aids, both to reinforce your expert's testimony and also to guide jurors who might remain "confused" by the math.

Friday, October 02, 2009

Avoiding the Black Swan Effect at trial

Nicholas Nassim Taleb's book, The Black Swan: The Impact of the Highly Improbable, introduced a wide lay audience to the the "black swan effect," whereby people tend to overgeneralize based upon extremely rare, but impactful events. More generally, in psychology, this is referred to hindsight bias. The basic idea is that highly improbable events tend to get noticed. They stick in the brain and are easily accessed mentally. As such, people tend to underestimate just how improbable the event was. In addition, most people are really crummy at comprehending either really large or really small numbers. So, for example, an average person can't really distinguish the difference between a one-in-a-million chance and a one-in-ten-million-chance. Both are pretty much "never gonna happen." By contrast, people are pretty good at appreciating the difference between a coin flip and a one-in-twenty chance. Mathematically, the two comparisons are identical. The first event in each pairing is ten times more likely to happen than the second one.

Recently, Taleb has teamed up with Daniel G. Goldstein, and Mark W. Spitznagel to apply the black swan effect to corporate risk management in an article in the Harvard Business Review, enititled, "The Six Mistakes Risk Managers make in Risk Management." The focus of the article is to help managers recognize hindsight bias in their own decision-making. They caution these risk managers not to think they could have predicted these events from the past. Not should they use them to predict the future.

Given how much difficulty even seasoned risk managers have overcoming hindsight bias, it should not be surprising that ordinary people find the problem almost insurmountable. The problem is that many torts are the results of Black Swan events. A very improbable event takes place, adversely impacting someone in a profound way. That person sues and some company finds itself defending a failure to predict and prevent the unforeseeable. So, a risk manager also needs to understand this issue because it profoundly affects her company's performance in court.

I wrote an article on how (poorly) jurors handle issues of statistics and probabilities for The Jury Box, back when it was a newsletter. In addition, I have given several presentations on this topic, some slides for which can be found here. Given the timeliness of the topic (Really, it's timeless), I am reprinting that original article below. I hope you find it instructive.

A jury trial can be Risky Business (Don’t worry – not all issues will make reference to Tom Cruise). In addition to the inherent uncertainty associated with putting your case in the hands of a group of laypersons, a jury trial is complicated by the fact that jurors themselves are notoriously unreliable at evaluating risk. In this issue I focus on how juries handle probabilities, risks and cost-benefit analyses.

If it weren’t for bad luck, I’d have no luck at all
Most litigation involves a dispute over an unlikely event. A patient develops an allergic reaction, brakes fail, a grape rolls down an aisle to precisely the place where a little old lady is about to take her next step. These low-probability events turn into legal questions of foresee-ability, reasonable care and adequate notice, all of which turn, to one degree or another, on just how unlikely the jury believes the event to have been.

One common problem that jurors have evaluating probabilities is known as hindsight bias. When someone learns of a low-probability event having actually occurred, there is a tendency to treat it as if were much more likely than it was. If a juror places greater likelihood on the event, she will believe it to have been more easily anticipated and will assign greater urgency to guarding against it. The result is that defendants are often blamed for not anticipating and preventing truly freak accidents.

An alarming second order effect is that the more bizarre the circumstances, the greater the hindsight bias. This may be because really weird circumstances are more easily remembered and recalled by jurors.

One strategy for overcoming hindsight bias is to argue by analogy to something with which jurors are likely to be familiar. The problem usually confronts defense counsel, so it is also wise to choose an analogy for which jurors might feel some responsibility. For instance, if one argues that an allergic reaction to a medication is as common as an automobile accident caused by a sneezing driver, jurors may conclude that the situation was not very dangerous, given that they never pull over their cars just because they need to sneeze.

All costs with no benefits
The economic theory of reasonable care in tort goes back at least as far as Learned Hand’s opinion in Carroll Towing. A cost-benefit analysis showing that all economically efficient precautions were taken is supposed to be a defense to a charge of negligence. Many industry regulations make such calculations mandatory. One might imagine, then, that jurors would look favorably upon companies who perform cost-benefit analyses. One would be dead wrong. Typically, whatever appreciation that jurors might have for a company thinking hard about safety is overwhelmed by their discomfort in reducing human pain and suffering to a mathematical calculation, especially one involving money.

Several empirical studies have shown that defendants are almost always punished for performing cost-benefit analyses, regardless of how clearly the calculations support the measures taken. Plaintiffs’ attorneys are wise to play up the cold, callous, calculating nature of the defendant’s methods. By contrast, defense counsel has the difficult task of convincing the jury that her client cares about safety without the testimony being reduced to probabilities, statistics and dollar signs. Again, reasoning by analogy is often the best policy, alerting jurors to the many cost-benefit calculations they perform in their every day lives, with a focus on those costly precautions most people choose not to take. For instance, it is clearly safer for children to wear helmets on playgrounds but almost no parent makes her children wear them.

It is also worth noting that companies are actually punished for placing a higher value on human life in their cost-benefit analyses. While this high value might help marginally in avoiding liability, it creates a costly anchor when jurors are calculating damages.

The zero risk fallacy
Many jurors mistakenly believe that it is possible to make products, services and treatments absolutely safe. They conclude that any risk of loss or injury is unacceptable. They have essentially adopted a strict liability standard despite the law to the contrary. Others have simply decided that manufacturers or service providers, rather than consumers, should be responsible for all safety precautions because of perceived wealth or knowledge advantages.

Many jurors are troubled by the idea of bad things happening to innocent people. Some conclude that the world is unfair and that the poor victim is entitled to be compensated for her loss. The only source from which the jury can take money is the defendant, so liability is attached despite conclusions that the defendant did nothing wrong. The inadmissibility of evidence of insurance can exacerbate this problem since jurors often assume that the absence of any mention of insurance means that the plaintiff had none.

Plaintiffs face their own risks
Juror difficulties with risk and probabilities do not always benefit plaintiffs. Plaintiffs who are engaged in risky activities are sometimes entitled to compensation because defendant’s conduct unacceptably increased the risk. Jurors sometimes conclude that risk-takers implicitly assume all responsibility for their well-being. For instance, a juror might think, “Hey, skiing is a dangerous sport. If you get hurt, you have no-one to blame but yourself.” A plaintiff will have trouble convincing such a juror that the ski-binding manufacturer is liable for her injuries.

In most cases, both parties have behaved imperfectly. Some jurors implicitly adopt a contributory negligence rule, whereby any fault by the plaintiff bars compensation. I recommend that counsel make sure that the jury is given a very clear instruction on negligence. Ideally, the jury should be given the instruction in advance of opening arguments (an increasingly common practice, endorsed by the ABA). The idea is to get the jury to focus as quickly as possible on the defendant’s conduct.
I also recommend that plaintiff’s counsel consider a “de-fanging” strategy, whereby the plaintiff owns up to any personal mistakes. This will prevent defense counsel from raising the plaintiff’s failings in a manner that suggests to the jury that they somehow excuse the defendant’s conduct. If the plaintiff conveys confidence in the legitimacy of her claim despite a full appreciation of her own shortcomings, the jury is more likely to do so, as well.