Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Galaxy Quest illustrating Historical DocumentsScene still / Galaxy Quest

Historical Documents

The Thermians receive television broadcasts without any concept of fiction, treat Galaxy Quest episodes as historical records, and reconstruct their society and working starship from them.

A system can imitate its data with extraordinary competence while remaining coherently and catastrophically wrong about what the source material represents.

The Thermians have a cultural category failure rather than a generative model hallucinating, and much of their literal reconstruction actually works.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Accidents
  2. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Hallucination
  2. Calibration
  3. Information Pollution
  4. Distribution Shift