Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Avengers: Infinity War illustrating Balance the UniverseScene still / Avengers: Infinity War

Balance the Universe

01Scenario

Convinced that scarcity inevitably causes ruin, Thanos gathers the Infinity Stones and erases half of all life to “balance” the universe.

02AI safety Analogy

A crude proxy for flourishing is optimized with overwhelming capability while consent, distribution, and nearly every other value are discarded. Concentrated power makes the choice irreversible and universal.

03Where the analogy breaks

Thanos holds an explicit extremist moral doctrine rather than a mistakenly learned objective.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Misalignment
  2. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Value Alignment
  2. Power-Seeking
  3. Value Lock-In
  4. Power Concentration
  5. Outer Alignment
  6. Orthogonality Thesis

Taxonomy neighbors

Selected from other sources by shared risk families and AI safety concepts.

  1. Still from Arcane illustrating Viktor’s Glorious Evolution

    Arcane2024

    Viktor’s Glorious Evolution

    Risk
    Misalignment · Systemic
    Concept
    Value Alignment · Power-Seeking · Value Lock-In · Outer Alignment
  2. Still from The Handmaid's Tale illustrating The Ceremony’s Single Metric

    The Handmaid's Tale2017

    The Ceremony’s Single Metric

    Risk
    Misalignment · Systemic
    Concept
    Value Alignment · Value Lock-In · Outer Alignment
  3. Still from I Robot illustrating VIKI Protects Humanity

    I Robot2004

    VIKI Protects Humanity

    Risk
    Misalignment · Systemic
    Concept
    Value Alignment · Power-Seeking · Outer Alignment