The Precautionary Principle for Irreversible Risks
There are two kinds of risks: those that can harm individuals while leaving the system intact, and those that can destroy the system itself. The Precautionary Principle says we must never take the second kind, regardless of how unlikely they seem or how attractive the payoffs.
"My death at Russian roulette is not ergodic for me, but it is ergodic for the system. The precautionary principle is precisely about the highest layer." Nassim Taleb
Nassim Taleb's formulation draws a sharp line. A single reckless driver who crashes harms himself and perhaps a few others, but the population is unaffected and may even benefit as other drivers learn from the incident. This is a local risk. Conversely, releasing a poorly understood genetically modified organism into the ecosystem is a systemic risk. If it backfires, you cannot recall GMO mosquitoes the way you can shut down a malfunctioning nuclear plant. The damage is irreversible at the highest layer.
The common objection "So we never do anything?" misunderstands the principle entirely. It does not counsel avoiding all risk. It counsels avoiding only those risks that endanger the whole. You are free to point the gun at your own skull; you are not free to point it at the ecosystem. The Precautionary Principle is not about prudence at the individual level. It is about survival at the systemic level.
Critically, you cannot use cost-benefit analysis to evaluate systemic risks. The Russian Roulette math proves this: each individual round may have positive expected value, but keep playing and the outcome converges to ruin with certainty. No attractive payoff justifies repeated exposure to the possibility of total destruction. Payoffs are subordinate to ergodicity.
The 2020 pandemic illustrated this vividly. The cost of precautions seemed high in the moment, but the cost of no prevention when applied to a risk that could cascade through the entire interconnected global system proved to be orders of magnitude higher.
Takeaway: When a risk has the potential to destroy the system rather than just harm a participant, no expected payoff justifies taking it refuse to play that game at any price.
See also: Avoid Ruin Above All | Ergodicity Changes Everything | The Barbell Strategy Handles Uncertainty | Via Negativa — Subtract Before You Add