Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Fullmetal Alchemist: Brotherhood illustrating Shou Tucker’s Final AssessmentScene still / Fullmetal Alchemist: Brotherhood

Shou Tucker’s Final Assessment

01Scenario

Facing loss of his certification and livelihood, Tucker creates another “successful” talking chimera by combining Nina and Alexander.

02AI safety Analogy

A capability assessment becomes the target, rewarding an impressive output while ignoring provenance, consent, and catastrophic externalities.

03Where the analogy breaks

Tucker knowingly commits an atrocity; this is chiefly an institutional-incentive failure rather than accidental machine misalignment.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Malicious use
  2. Security
  3. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Capability Evals
  2. Goodhart’s Law
  3. Governance Failure

Source continuation

More from Fullmetal Alchemist: Brotherhood

View all 3 scenarios
  1. Still from Fullmetal Alchemist: Brotherhood illustrating Father’s Nationwide Transmutation Circle

    Fullmetal Alchemist: Brotherhood2010

    Father’s Nationwide Transmutation Circle

  2. Still from Fullmetal Alchemist: Brotherhood illustrating Hawkeye’s Preauthorized Check

    Fullmetal Alchemist: Brotherhood2010

    Hawkeye’s Preauthorized Check

Taxonomy neighbors

Selected from other sources by shared risk families and AI safety concepts.

  1. Still from Avatar: The Last Airbender illustrating No War in Ba Sing Se

    Avatar: The Last Airbender2006

    No War in Ba Sing Se

    Risk
    Malicious use · Security · Systemic
    Concept
    Goodhart’s Law · Governance Failure
  2. Still from The Sopranos illustrating The HUD Housing Scam

    The Sopranos2002

    The HUD Housing Scam

    Risk
    Malicious use · Security · Systemic
    Concept
    Goodhart’s Law · Governance Failure
  3. Still from Hot Fuzz illustrating Village of the Year

    Hot Fuzz2007

    Village of the Year

    Risk
    Malicious use · Security · Systemic
    Concept
    Goodhart’s Law · Governance Failure