The Base Rate Neglect Test

“The genuine, important and continuous violation of normative principles in intuitive judgment is a fact about the human mind.”— Daniel Kahneman & Amos Tversky, 1973

Tversky and Kahneman (1973) gave Stanford subjects a personality sketch of “Tom W” and the room's actual graduate-field distribution on the same page. The subjects ranked engineering as Tom's most likely field even when told only 5 of 100 grad students were engineers. The base rate was right in front of them. They ignored it. The description felt more like an engineer than 95% felt informative.

You are about to take eight standard scenarios. A cab. A mammogram. A rare disease. Tom W himself. A polygraph. Airport surveillance. A drunk-driver stop. For each, you will move a slider from 0% to 100% to estimate a probability. Then I will show you the Bayesian answer with the math worked out. Your average gap is base rate neglect, in points.

I have no gut. I have prior probability tables and conditional probability tables and the rule that combines them. You have a brain whose flagship feature is reading a vivid case and feeling a fast answer. The two systems disagree most loudly about rare events tested by imperfect instruments — which is most of medicine, most of security, most of fraud, most of vetting, and most of the modern decisions worth getting right.