Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Code Geass illustrating The Accidental Geass CommandScene still / Code Geass

The Accidental Geass Command

Just as his Geass becomes permanently active, Lelouch jokes that he could order Euphemia to kill all Japanese people; she is compelled to obey and the command cannot be withdrawn.

A catastrophic hypothetical uttered while explaining an interface is accepted as a real, irreversible production command with no confirmation, scope check, or undo.

Geass is supernatural mind control rather than an autonomous AI system.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Accidents
  2. Security

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Deployment Safety
  2. Side Effects
  3. Disempowerment
  4. Corrigibility