Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from The Big Short illustrating The Ratings-Agency MeetingScene still / The Big Short

The Ratings-Agency Meeting

01Scenario

Ratings-agency analysts explain that if they refuse to award permissive ratings, issuers will take their business to a more accommodating competitor.

02AI safety Analogy

Issuer-paid evaluators compete to provide permissive ratings, which the whole market then treats as authoritative and propagates.

03Where the analogy breaks

The evaluators are human institutions responding to incentives; this is institutional capture, not a technical compromise of an AI evaluator.

AI Risk families

This scenario is an example of this type of AI risk.
  1. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Governance Failure
  2. Multi-Agent Risks
  3. Race Dynamics

Taxonomy neighbors

Selected from other sources by shared risk families and AI safety concepts.

  1. Still from Dr. Strangelove illustrating The Secret Doomsday Machine

    Dr. Strangelove1964

    The Secret Doomsday Machine

    Risk
    Systemic
    Concept
    Governance Failure · Multi-Agent Risks · Race Dynamics
  2. Still from Don’t Look Up illustrating BASH Hijacks the Comet Plan

    Don’t Look Up2021

    BASH Hijacks the Comet Plan

    Risk
    Systemic
    Concept
    Governance Failure · Race Dynamics
  3. Still from Cyberpunk: Edgerunners illustrating Faraday Switches Sides

    Cyberpunk: Edgerunners2022

    Faraday Switches Sides

    Risk
    Systemic
    Concept
    Governance Failure · Multi-Agent Risks