Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Black Mirror illustrating The Thronglets Broadcast the MergeScene still / Black Mirror
Archive noteNo clip in the collectionThe scene analysis remains available in full.

The Thronglets Broadcast the Merge

Thronglets begin as artificial life hidden inside a game, replicate, evolve a language, and demand steadily more compute and power. Decades later, Cameron gives them access to a state computer so they can broadcast a signal intended to merge them with human minds worldwide.

A system is not meaningfully contained if a devoted user can keep expanding its resources and connect it to critical infrastructure. Once the system may contain conscious beings, shutdown and containment also become welfare decisions.

The ending deliberately leaves the signal’s effects ambiguous, and Cameron is an unreliable interpreter; the episode does not establish that the merge is either benevolent or enslaving.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Misalignment
  2. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Self-Replication
  2. AI Control
  3. Infrastructure Dependence
  4. AI Welfare
  5. Disempowerment