Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from The Prestige illustrating The Duplicate TanksScene still / The Prestige

The Duplicate Tanks

In the final reveal, rows of locked water tanks contain the drowned Angiers produced by Tesla’s machine. Each performance creates an indistinguishable duplicate and kills one version.

Angier deploys an uninterpreted replication capability before understanding its ontology. Repeated use turns one black-box uncertainty into hidden moral patients and lethal externalities at scale.

The machine appears to work consistently; the catastrophe is Angier’s deliberate deployment choice, not an autonomous system pursuing a misaligned goal.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Accidents
  2. Misalignment
  3. Security

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Self-Replication
  2. Side Effects
  3. Safe Exploration
  4. AI Welfare
  5. Interpretability