[xoxoxoBruce]

A very interesting article about eyewitness reliability to crimes.

In an excellent new paper, Too Good to Be True, Lachlan J. Gunn et al. show that more evidence can reduce confidence. The basic idea is simple. We expect that in most processes there will normally be some noise so absence of noise suggests a kind of systemic failure. The police are familiar with one type of example. When the eyewitnesses to a crime all report exactly the same story that reduces confidence that the story is true. Eyewitness stories that match too closely suggests not truth but a kind a systemic failure, namely the witnesses have collaborated on telling a lie.

What Gunn et al. show is that the accumulation of consistent (non-noisy) evidence can reverse one’s confidence surprisingly quickly. Consider a police lineup but now consider a more likely cause of systemic failure than witness conspiracy. Suppose that there is a small probability, say 1%, that the police arrange the lineup, either on purpose or by accident, so that the “suspect” is the only one who is close to matching the description of the criminal. Now consider what happens to our rational (Bayesian) probability that the suspect is guilty as the number of eyewitnesses saying “that’s the guy” increases.

The first eyewitness to identify the suspect increases our confidence that the suspect is guilty and our confidence increases when the second and third eyewitness corroborate but when a fourth eyewitness points to the same man our rational confidence should actually decrease. Even though the systemic failure rate is only 1%, that small probability starts to weigh more heavily the more consistent (less noisy) the evidence becomes. The red line in the graph at right shows–using a 1% systemic failure rate and realistic probabilities of eyewitness identification–that after 3 witnesses more evidence decreases our confidence and when more than 10 witnesses identify the same suspect we should be less certain of guilt than when one witness identifies the suspect!

Bold mine.