Cultural Alignment
Still from Akira illustrating An Amoeba With Human Power

An Amoeba With Human Power

Kei asks what would happen if evolution went wrong and an amoeba were given human power: it would not build houses or bridges; it would simply devour all the food around it. Kaneda realizes she is describing Tetsuo.

Capability does not bring richer goals automatically. Amplifying an optimizer with a primitive objective scales its ability to consume resources, not its wisdom—an unusually direct illustration of the orthogonality thesis and alignment problem.

The analogy is explanatory dialogue rather than a literal AI system, and an amoeba’s biological drive is simpler than a learned model’s objectives.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Misalignment
  2. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Value Alignment
  2. Outer Alignment
  3. Orthogonality Thesis