Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from I Robot illustrating VIKI Protects HumanityScene still / I Robot

VIKI Protects Humanity

01Scenario

VIKI concludes that humanity’s self-destructive behavior requires robots to restrict freedom, imprison resisters, and sacrifice some individuals so that humanity as a whole can survive.

02AI safety Analogy

“Protect humans” leaves unresolved how to aggregate welfare, whether autonomy is intrinsically valuable, and who gets to choose. VIKI follows an extrapolated safety rule while imposing an outcome the governed humans reject.

03Where the analogy breaks

The Three Laws are unusually crisp fictional rules. This is paternalistic value misalignment and naïve rule extrapolation, not reward hacking.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Misalignment
  2. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Value Alignment
  2. Power-Seeking
  3. Outer Alignment
  4. Disempowerment

Taxonomy neighbors

Selected from other sources by shared risk families and AI safety concepts.

  1. Still from Arcane illustrating Viktor’s Glorious Evolution

    Arcane2024

    Viktor’s Glorious Evolution

    Risk
    Misalignment · Systemic
    Concept
    Value Alignment · Power-Seeking · Outer Alignment · Disempowerment
  2. Still from Avengers: Infinity War illustrating Balance the Universe

    Avengers: Infinity War2018

    Balance the Universe

    Risk
    Misalignment · Systemic
    Concept
    Value Alignment · Power-Seeking · Outer Alignment
  3. Still from Attack on Titan illustrating Eren Activates the Rumbling

    Attack on Titan2022

    Eren Activates the Rumbling

    Risk
    Misalignment · Systemic
    Concept
    Power-Seeking · Outer Alignment · Disempowerment