Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Captain America: The Winter Soldier illustrating Zola’s AlgorithmScene still / Captain America: The Winter Soldier

Zola and Sitwell explain that Project Insight mines people’s records and communications to predict who may threaten HYDRA; three networked Helicarriers then acquire those people as targets before they have acted.

A single pipeline turns mass surveillance into predictive classification and then irreversible force, with no notice, hearing, or appeal. The model need not become evil: a captured institution gives an authoritarian objective to a system that can act at scale.

HYDRA is deliberately malicious and the film treats prediction as uncannily accurate. Real predictive systems are generally noisy and discriminatory as well as vulnerable to authoritarian misuse.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Malicious use
  2. Security
  3. Systemic

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Privacy Loss
  2. Autonomous Weapons
  3. Contestability
  4. Digital Repression