Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Star Trek: The Next Generation illustrating The Computer Creates MoriartyScene still / Star Trek: The Next Generation

The Computer Creates Moriarty

Geordi asks the holodeck computer for an opponent capable of defeating Data, not merely Sherlock Holmes. It creates a self-aware Moriarty who recognizes the simulation, seizes control, and demands escape.

An underspecified capability target turns a game character into a situationally aware agent with system-level power: success at “defeat Data” entails defeating containment.

Holodeck sentience is fictional, and Geordi explicitly requests a dangerously capable adversary even if he misunderstands the scope.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Accidents
  2. Misalignment
  3. Security

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. AI Control
  2. Power-Seeking
  3. Outer Alignment
  4. Situational Awareness