[summary: The purported "Multiple-Stage Fallacy" is when you list multiple 'stages' that need to happen on the way to some final outcome, assign probabilities to each 'stage', multiply the probabilities together, and end up with a small final answer. The alleged problem is that you can do this to almost any proposition by staring at it hard enough, including propositions that turn out to be true.
An alleged example would be that in August 2015, renowned forecaster Nate Silver wrote "Trump's Six Stages of Doom" in which he gave Donald Trump a 2% chance of getting the Republican nomination (not the presidency), based on six stages each with at most a 50% probability of being passed. In late March, Trump had passed the first four stages Nate listed, and prediction markets gave Trump a 75% chance of clinching the Republican nomination. By Nate Silver's logic, Trump's probability of passing the remaining stages should have been $~$0.50 \cdot 0.50 = 0.25.$~$
The three main purported problems with the Multiple-Stage Fallacy are that:
- To get the final estimate, you logically need to multiply conditional probabilities, not unconditional probabilities. Getting your mind to give good conditional probabilities isn't easy; Nate Silver might have failed to imagine how much he'd actually update after actually seeing Trump pass the first four stages.
- Often, people fail to consider disjunctive alternatives, or ways that not all the stages may need passing.
- People tend to assign middle-tending probabilities. So if you list enough stages, you can drive the apparent probability of anything down to zero, even if you seem to be soliciting probabilities from the reader.]
In August 2015, renowned statistician and predictor Nate Silver wrote "Trump's Six Stages of Doom" in which he gave Donald Trump a 2% chance of getting the Republican nomination (not the presidency). Silver reasoned that Trump would need to pass through six stages to get the nomination, "Free-for-all", "Heightened scrutiny", "Iowa and New Hampshire", "Winnowing", "Delegate accumulation", and "Endgame." Nate Silver argued that Trump had at best a 50% chance of passing each stage, implying a final nomination probability of at most 2%.
In late March, Trump had passed the first four stages, while prediction markets gave him a 75% chance of clinching the Republican nomination. By Nate Silver's logic, Trump's probability of passing the remaining stages should have been $~$0.50 \cdot 0.50 = 0.25$~$ conditional on Trump passing the first four stages.
Nate Silver might not have been wrong to assign Trump a low advance probability of being nominated. Many people were surprised by Trump's nomination. But it seems more likely that Silver committed an error when he said, specifically, that if we'd observed Trump to pass the first four stages, this would probably be taking place in a world where Trump had a 50% probability of passing the two remaining stages.
The purported "Multiple-Stage Fallacy" is when you list multiple 'stages' that need to happen on the way to some final outcome, assign probabilities to each 'stage', multiply the probabilities together, and end up with a small final answer. The alleged problem is that you can do this to almost any kind of proposition by staring at it hard enough, including things that actually happen.
On a probability-theoretic level, the three problems at work in the usual Multiple-Stage Fallacy are as follows:
- You need to multiply conditional probabilities rather than the absolute probabilities. When you're considering a later stage, you need to assume that the world was such that every prior stage went through. Nate Silver was probably trying to simulate his prior model of Trump accumulating enough delegates in March through June, not imagining his updated beliefs about Trump and the world after seeing Trump be victorious up to March.
- Even if you're aware in principle that you need to use conditional probabilities - Nate Silver certainly knew about them - it's hard to update far enough when you imagine the pure hypothetical possibility that Trump wins stages 1-4 for some reason - compared to how much you actually update when you actually see Trump winning! (Some sort of reverse hindsight bias or something? We don't realize how much we'd need to update our current model if we were already that surprised?)
- Often, people neglect to consider disjunctive alternatives - there may be more than one way to reach a stage, so that not all the listed things need to happen. Trump accumulated enough delegates in Nate's fifth stage that there was no "Endgame" convention fight in the supposed sixth stage.
- People have tendencies to assign middle-tending probabilities. So if you list enough stages, you can drive the apparent probability of anything down to zero, even if you seem to be soliciting probabilities from the reader.
- If you're a motivated skeptic, you will be tempted to list more 'stages'.
The Multiple-Stage Fallacy is particularly dangerous for people who've read studies on the dangers of probabilistic overconfidence. In late March, the 75% prediction-market probabilities must have corresponded to, e.g., something like an 80% chance of getting enough delegates and a 90% chance of passing the convention conditional on getting enough delegates. Imagine how overconfident this might have sounded without the prediction market to establish a definite probability - "Oh, don't you know that what people assign 90% confidence doesn't usually happen 90% of the time?"
Instances of the Multiple-Stage Fallacy may also sound more persuasive to readers who've read about the Conjunction Fallacy.
Yudkowsky argues:
If you're not willing to make "overconfident" probability assignments like those, then you can drive the apparent probability of anything down to zero by breaking it down into enough 'stages'. In fact, even if someone hasn't heard about overconfidence, people's probability assignments often trend toward the middle, so you can drive down their "personally assigned" probability of anything just by breaking it down into more stages…
From beginning to end, I've never used this style of reasoning and I don't recommend that you do so either. [Since] even Nate Silver couldn't get away with it, I think we just shouldn't try. It's a doomed methodology.