In the 1970s, UC Berkeley faced a discrimination lawsuit. The numbers seemed airtight: 44% of male applicants were admitted versus only 35% of female applicants. By any measure, that's a 9-point gap. The university appeared caught red-handed, perpetuating systemic gender bias in one of America's most prestigious institutions.
The intuitive explanation was straightforward. Admissions committees were biased against women. Maybe unconsciously, maybe not. The data was the data. This is how discrimination works—it hides in plain sight, in aggregate statistics that reveal institutional preference. The lawsuit had legs. The evidence seemed damning.
Except it wasn't. When statisticians disaggregated the data by department, something remarkable happened: the trend inverted. According to analysis covered in Scientific American's exploration of statistical paradoxes, women had equal or higher acceptance rates in nearly every individual department. English, chemistry, geology, history—department after department showed no bias against women, or even favored them. The 9-point gap that seemed to prove systemic discrimination simply vanished when you looked at where it actually mattered: where the admissions decisions were being made.
This wasn't a small difference buried in noise. The reversal was stark enough to become a canonical case of what statisticians call Simpson's Paradox—a phenomenon where a trend visible in aggregate data reverses or disappears entirely when you break the data into subgroups. As explored on Statology's analysis of statistical paradoxes, this reveals how a single hidden variable can completely invert our understanding of reality. In Berkeley's case, that variable was departmental selectivity. Women were disproportionately applying to the university's most competitive departments—engineering, physics, chemistry—programs with lower acceptance rates overall. Men were more likely to apply to less selective departments with higher acceptance rates. The aggregate gap wasn't evidence of bias; it was evidence of different strategic choices about where to apply.
The mechanism here is simple but subtle. Imagine Department A accepts 30% of applicants (very competitive), and Department B accepts 80% (less competitive). If women cluster in Department A and men in Department B, the overall female acceptance rate will be lower even if each department treats them identically. The aggregate statistic conflates two separate phenomena: departmental selectivity and applicant distribution. Strip away the departmental context, and the "bias" evaporates.
What's striking about Berkeley's case is that it demolished its own accusers not through defensive spin but through methodological honesty. The university's own statisticians, examining their data more carefully, revealed that the aggregate accusation was misleading. This wasn't a coverup; it was the opposite. More sophisticated analysis actually exonerated them—because the disaggregated data told a different story than the headline numbers.
The lesson here transcends Berkeley's admissions office. Whenever you see a striking aggregate statistic—whether about hiring, healthcare, or hiring practices—the first question should be: what happens when you break this down by relevant subgroups? Hidden variables lurk everywhere in data. They invert correlations, manufacture false trends, and make discrimination invisible when it exists or visible when it doesn't. The Berkeley case teaches us that the most damning statistical evidence might be the least trustworthy, and that the instinct to dig deeper often reveals truths that superficial numbers conceal.