Cultural Alignment
Still from The Lord of the Rings: The Return of the King illustrating The Palantir Is a Hostile API

Pippin looks into the palantír, extracts useful information about the coming attack, and is simultaneously interrogated by Sauron.

A powerful interface can provide genuine capability while also serving as an adversarial channel for extraction, manipulation, and surveillance.

The palantír is a magical communications artifact with an explicitly hostile operator.

AI Risk families

This scenario is an example of the following types of AI risk.
  1. Malicious use
  2. Security

AI safety concepts

This scenario is related to the following AI safety concepts.
  1. Exfiltration
  2. Privacy Loss
  3. Adversarial Robustness
  4. Prompt Injection