In the spring of 1972, Daniel Kahneman stood before a small audience in a University of Oregon seminar room. He was not yet a Nobel laureate, and the work he described—a then-unpublished study on how people make intuitive predictions—seemed like a narrow methodological point. The audience of roughly 30 faculty and graduate students listened politely. A few later recalled the talk as a premonition of something larger. What Kahneman laid out that day was, in essence, a blueprint for a crisis that would not fully erupt for another four decades.

The 1972 Talk That Almost Nobody Heard

Kahneman’s 1972 lecture at the Oregon Research Institute was a low-key affair. He presented early findings from a collaboration with Amos Tversky on what they called the 'law of small numbers'—a sarcastic label for the human tendency to treat small samples as if they were large and representative. The core claim was simple: researchers routinely overestimate the reliability of conclusions drawn from tiny datasets. In their experiments, participants—and, Kahneman argued, scientists themselves—acted as if a sample of 10 or 20 observations could support confident generalizations.

At the time, social psychology studies typically used 20 to 40 participants per condition. Effect sizes hovered around Cohen’s d = 0.3 to 0.5—modest by any standard. Kahneman showed that under such conditions, a study’s apparent result could flip entirely with the addition of just a few more participants. He illustrated this with concrete examples from his own lab: a finding that seemed robust with 20 participants vanished when the sample doubled to 40.

The audience included a handful of methodologists who would later become influential in the field. One of them, Robert Abelson, later wrote that the talk 'made the hair on the back of my neck stand up.' But most listeners filed out without much sense of urgency. The lecture was not recorded, and no paper came directly from it. The ideas would have to wait for formal publication.

Yet for those who did pay attention, the 1972 talk was a kind of fire drill. It demonstrated, with arithmetic clarity, that the replication rate in social psychology was likely far lower than researchers assumed. The crisis, if it came, would not be a surprise. It would be a delayed consequence of ignoring a simple equation.

How a Single Equation Became a Warning System

The equation at the heart of the 1972 lecture was not new—it was a standard formula for the standard error of the mean. What was new was the realization that researchers routinely misinterpreted its implications. Kahneman and Tversky formalized this in a 1973 paper titled 'On the psychology of prediction,' published in Psychological Review. They showed that even experienced scientists overestimated the replicability of results from samples of 10 to 20 per cell.

The paper introduced the concept of 'intuitive statistics'—the gap between how people actually reason about data and how they should reason according to probability theory. In one demonstration, Kahneman and Tversky asked participants to estimate the likelihood that a given study would replicate if repeated. Most guessed 80% or higher, when the true probability, given typical effect sizes and sample sizes of the era, was closer to 50%.

That gap had real consequences. A study with 30 participants per cell and an effect size of d = 0.4 has roughly 35% power—meaning it has only about a one-in-three chance of detecting a real effect even if it exists. Yet researchers at the time often treated such studies as providing strong evidence. The 1973 paper was a direct warning: the published literature was littered with underpowered, unreplicable findings.

Kahneman and Tversky did not stop at diagnosis. They proposed a remedy: researchers should compute the power of their studies before collecting data and should adjust sample sizes accordingly. They recommended a minimum of 100 participants per condition for typical effect sizes. This advice was largely ignored for the next 30 years.

The Replication Audits That Kahneman Predicted

In 2015, the Open Science Collaboration published a landmark replication audit of 100 studies from three top psychology journals. The result: only about 36% of the original findings replicated. For studies that had originally used small samples—say, 20 to 30 per condition—the replication rate was even lower, sometimes below 20%. The finding sent shockwaves through the field and triggered what is now called the replication crisis.

To anyone familiar with Kahneman’s 1972 lecture, the numbers were not shocking. They were predictable. The median sample size in the original studies was about 40 per condition, and the median effect size was around d = 0.4. A power analysis shows that such studies have roughly 30% power. The replication rate of 36% is exactly what one would expect from underpowered research.

Kahneman himself, when asked about the crisis, said in a 2017 interview: 'I feel a mixture of sadness and vindication. Sadness because the field has wasted so much time and energy on false results. Vindication because the problem was foreseeable.' He noted that his 1972 talk had essentially sketched the same arithmetic that the replication audits later confirmed.

The failed replications were not random. They clustered in areas like social priming, ego depletion, and stereotype threat—all domains where small samples and small effect sizes were the norm. A 2016 analysis by Ulrich Schimmack found that the median power of studies in social psychology was around 20%. That means four out of every five studies in the field were unlikely to detect the effects they claimed to find.

Why the Field Ignored the Early Alarm

Why did psychology ignore Kahneman’s warning for so long? Part of the answer lies in incentives. In the 1970s and 1980s, academic journals prized novelty and surprising results. A study that failed to replicate was not a problem; it was just another finding. Journals rarely published replications, and null results were almost impossible to get into print. Researchers who followed Kahneman’s advice to increase sample sizes would have found it harder to produce the flashy, statistically significant results that editors wanted.

Graduate training also played a role. Statistics courses in psychology programs in the 1970s and 1980s focused on null hypothesis significance testing—the calculation of p-values—but often skipped power analysis. Many students never learned to ask: 'What is the probability that my study will detect the effect I am looking for?' Without that question, the sample size problem remained invisible.

Kahneman and Tversky themselves moved on to other topics. By the early 1980s, they were deep into prospect theory and behavioral economics, leaving the methodological critique unfinished. Their 1973 paper on intuitive prediction was cited occasionally, but mostly for its insights about human judgment, not for its warning about research practices. The paper was not widely cited in methods discussions until the 2010s, when the replication crisis made it suddenly relevant.

There was also a psychological barrier: the 'law of small numbers' was a bias that researchers were reluctant to see in themselves. It is easier to believe that one’s own studies are well-powered than to confront the arithmetic that says otherwise. Kahneman later wrote that he himself had been guilty of the bias, designing underpowered studies in his early career.

What the 1972 Framework Still Gets Right

Despite decades of neglect, the core insights from Kahneman’s 1972 lecture remain central to modern research practices. The most visible change is the rise of preregistration: researchers now specify their sample size and analysis plan before collecting data. This practice, encouraged by initiatives like the Open Science Framework, directly addresses the 'intuitive statistics' problem. By committing to a sample size in advance, researchers prevent themselves from rationalizing a small sample after seeing the results.

Power analysis has become a standard part of grant applications and ethics reviews. Many journals now require authors to report how they determined their sample size. The recommended minimum for typical effect sizes has risen to 100–200 participants per condition, consistent with what Kahneman advised in 1973. Some subfields, like cognitive psychology, now routinely use samples of 50–100 per cell, achieving power above 80%.

The 'small numbers' bias is now taught in every undergraduate research methods course. Students learn to recognize the illusion that small samples provide reliable evidence. They are shown the same arithmetic that Kahneman presented in 1972: the standard error shrinks only with the square root of the sample size, meaning that doubling the sample barely improves precision.

Transparency tools like Registered Reports—where journals accept a study based on its design before results are known—echo Kahneman’s logic. They separate the planning of research from the interpretation of results, reducing the temptation to overinterpret noisy data. As of 2025, more than 300 journals offer Registered Reports, and their use is growing.

The Unfinished Agenda: What Kahneman Missed

Kahneman’s 1972 framework, however, did not anticipate every dimension of the replication crisis. His focus was on researcher intuition—the honest but flawed judgment that small samples are sufficient. He did not predict p-hacking, the practice of analyzing data in multiple ways until a significant result emerges. Nor did he foresee outright fraud, which later scandals revealed to be a small but real contributor to irreproducibility.

Publication bias—the tendency of journals to publish only positive results—was also outside his model. In the 1970s, the file-drawer problem was already known, but Kahneman’s analysis assumed that published studies were a random sample of all conducted studies. We now know that published effect sizes are inflated by roughly 30–50% compared to unpublished ones, because null results rarely see print. This inflation means that even well-powered studies may overestimate true effects.

Modern replication rates vary dramatically by subfield. In cognitive psychology, replication rates hover around 50–80%, depending on the domain. In social psychology, they are lower, often 20–40%. The difference partly reflects differences in sample size and effect size, but also differences in methodological rigor. Kahneman’s framework explains the broad pattern but not the variation.

Another gap: his framework did not address the role of multi-lab collaborations. Projects like the Many Labs replication series, which involve dozens of labs collecting data on the same protocol, have become a gold standard for estimating true effect sizes. These collaborations can achieve sample sizes of thousands, far beyond what Kahneman imagined. His 1972 lecture assumed that individual researchers would fix the problem by increasing their own sample sizes. Today, the solution often involves collective action.

Lessons for Other Disciplines

The replication crisis is not confined to psychology. Similar problems have emerged in medicine, neuroscience, economics, and other fields that rely on small-sample studies. In medicine, for example, many early-phase trials of new drugs are conducted with 10–30 patients, and their results often fail to replicate in larger trials. A 2018 analysis of preclinical cancer studies found that only about 25% of published findings could be reproduced, a rate strikingly similar to that in social psychology.

Kahneman’s 1972 lecture offers a template for identifying such problems early. The key is to ask: What is the typical sample size in this field? What is the typical effect size? If the product of these two factors yields low power, then the literature is likely unreliable. This simple diagnostic can be applied to any domain where statistical inference is used.

For instance, in behavioral economics, many studies in the 2000s used samples of 50–100 participants per condition, with effect sizes around d = 0.2 to 0.3. A power analysis reveals that such studies have roughly 20–30% power. Unsurprisingly, several high-profile findings in behavioral economics have failed to replicate, including some related to 'nudge' interventions. The field is now grappling with its own replication crisis, albeit less publicly than psychology.

In neuroscience, the problem is compounded by the high cost of scanning. Many fMRI studies use samples of 10–20 participants, with hundreds of thousands of voxels analyzed. This creates a massive multiple-comparison problem, and the typical power is abysmally low. A 2013 analysis estimated that the median power of fMRI studies was around 8%, meaning that most reported activations are likely false positives. Kahneman’s arithmetic would have flagged this risk decades ago.

The Counter-Argument: Are We Overcorrecting?

Some researchers argue that the replication crisis has led to an overcorrection. They point out that requiring very large samples can be impractical for certain types of research, such as studies of rare populations or expensive interventions. For example, a clinical trial for a rare disease might only be able to recruit 20 patients. In such cases, the choice is not between a well-powered study and an underpowered one, but between an underpowered study and no study at all.

Others note that effect sizes in real-world settings can be larger than those found in lab experiments. A well-designed intervention might have an effect size of d = 1.0 or higher, which would require far fewer participants. Kahneman’s recommendation of 100 per condition was based on typical effect sizes in social psychology, but it may not apply universally.

There is also a trade-off between sample size and measurement quality. A study with 50 participants and highly reliable measures may have more power than a study with 100 participants and noisy measures. Kahneman’s framework focused on sample size alone, but modern power analysis incorporates measurement reliability, design complexity, and analytic choices.

Critics of the replication reform movement argue that the focus on sample size can lead to a 'one-size-fits-all' approach that ignores context. They advocate for a more nuanced view: researchers should calculate power for their specific situation, rather than following a general rule. This is consistent with Kahneman’s original message, which was about the importance of thinking statistically, not about a specific number.

A Lecture That Keeps Giving—If We Listen

The 1972 lecture is now available online as a re-recording from 2017, arranged by the Association for Psychological Science. It is a grainy, hour-long talk, but it carries an urgency that has not faded. Graduate programs still use it as a historical case study, showing students how a single transparent example can reveal a systemic problem.

One concrete takeaway from the lecture is the importance of preregistering sample size justifications. A researcher who writes down 'I need 150 participants per condition to achieve 80% power for an effect of d = 0.3' is less likely to stop at 50 because the p-value looks promising. Another is to treat single studies as provisional. Kahneman emphasized that no one study, however well-designed, should be taken as conclusive. Replication is the only check.

The lecture also carries a cautionary tale about the slow pace of scientific reform. It took 40 years for the field to take Kahneman’s warning seriously. During that time, countless underpowered studies were published, many of which will never be replicated. The cost is not just wasted effort but a polluted literature that misleads other researchers and, in applied fields, practitioners.

Kahneman’s 1972 lecture was a fire drill that took four decades to matter. It remains a reminder that the most important warnings are often the quietest. The question now is whether the field will continue to listen, or whether it will find new ways to ignore uncomfortable arithmetic.