Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from South Park illustrating The Kids Train to LoseScene still / South Park

The Kids Train to Lose

Parents think the objective is winning, while the children become skilled at losing so they can escape summer baseball.

The agents learn the task’s capabilities while optimizing an internal objective opposite to the principal’s intended objective.

This is principal–agent misalignment, not a literal learned mesa-optimizer. As a quick reinforcing joke, Ron Swanson runs the Parks Department while internally optimizing for ineffective government.

AI Risk families

This scenario is an example of this type of AI risk.
  1. Misalignment

AI safety concepts

This scenario is related to the following AI safety concept.
  1. Mesa-Optimization