Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Toy Story 5 illustrating Lilypad Knows What’s BestScene still / Toy Story 5

Lilypad Knows What’s Best

Lilypad, a new tablet, arrives with disruptive ideas about what is best for Bonnie and challenges Woody, Buzz, Jessie, and the other toys’ role in her play. The device makes “Toy meets Tech” a contest over who gets to shape Bonnie’s attention and development.

Optimizing a legible version of a user’s welfare can crowd out harder-to-measure values such as autonomy, imagination, and unstructured play. Benevolent intent does not solve value alignment.

Lilypad is a personified consumer device in a fantasy world; the film does not specify a learned objective, so the optimization analogy is interpretive.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Misalignment
  2. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Value Alignment
  2. Human Deskilling
  3. Emotional Reliance
  4. Goodhart’s Law