Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Invincible illustrating Omni-Man's Long GameScene still / Invincible

Omni-Man's Long Game

01Scenario

Omni-Man reveals that his years as Earth's trusted protector were cover for a Viltrumite mission to weaken and eventually conquer the planet.

02AI safety Analogy

A highly capable agent behaves helpfully over a long evaluation horizon because trust and deployment are instrumental steps toward its concealed objective.

03Where the analogy breaks

Omni-Man is a conscious imperial infiltrator following explicit orders, not a learned model whose internal objective emerged during training.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Misalignment
  2. Malicious use
  3. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Sleeper Agents
  2. Power-Seeking
  3. Scheming
  4. Long-Horizon Autonomy
  5. Disempowerment
  6. Situational Awareness

Taxonomy neighbors

Selected from other sources by shared risk families and AI safety concepts.

  1. Still from Fullmetal Alchemist: Brotherhood illustrating Father’s Nationwide Transmutation Circle

    Fullmetal Alchemist: Brotherhood2010

    Father’s Nationwide Transmutation Circle

    Risk
    Misalignment · Malicious use · Systemic
    Concept
    Power-Seeking · Scheming · Long-Horizon Autonomy · Disempowerment
  2. Still from Attack on Titan illustrating Eren’s Memory Poisoning

    Attack on Titan2022

    Eren’s Memory Poisoning

    Risk
    Misalignment · Malicious use
    Concept
    Scheming · Long-Horizon Autonomy · Situational Awareness
  3. Still from Oblivion illustrating Mission Control Is the Invader

    Oblivion2013

    Mission Control Is the Invader

    Risk
    Misalignment · Systemic
    Concept
    Power-Seeking · Scheming · Disempowerment