Cultural Alignment

Exploring real AI risks through the lens of pop culture

Still from Psycho-Pass illustrating The Helmets Fool the DominatorsScene still / Psycho-Pass

The Helmets Fool the Dominators

Attackers wear helmets that copy the lowest Crime Coefficient of nearby people. The Sibyl System and Dominators therefore classify obvious murderers as harmless while they attack civilians in public.

The attacker crafts an input that exploits a known weakness in the classifier at inference time. Humans can see the crime, but the automated enforcement system trusts the spoofed score and refuses to respond.

The attack copies another person’s live score through fictional hardware rather than perturbing pixels or tokens. Structurally, however, it is an unusually direct evasion attack.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Malicious use
  2. Security

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Adversarial Robustness
  2. Adversarial Evasion