Intelligence notes

◉Dashboard↗

Constitutional AI

3 min read

Constitutional AI

BLUF

Constitutional AI (CAI) is Anthropic’s alignment methodology for training large language models to be helpful, harmless, and honest through a self-critique and revision process guided by a written set of principles — the “constitution.” Introduced in a 2022 research paper by Bai et al. (Anthropic), CAI replaces or supplements human feedback on harmful outputs with model-generated critique and revision, reducing the volume of human labeling required for RLHF (Reinforcement Learning from Human Feedback) while increasing consistency. It is the technical foundation for Anthropic’s safety claims and underpins the Responsible Scaling Policy (RSP) assertions about model behavior at each ASL tier.

Mechanism

  1. Constitution: A written list of principles (honesty, avoiding harm, respecting autonomy) that functions as the behavioral reference for self-critique
  2. Supervised Learning (SL-CAI): The model is prompted to critique its own output against the constitution and revise; the revised outputs are used as supervised training data
  3. RL-CAI (RLAIF — Reinforcement Learning from AI Feedback): A “preference model” is trained using AI-generated feedback on which outputs better conform to the constitution; this preference model is used in place of (or alongside) human feedback in RLHF
  4. Result: Models trained with CAI show reduced harmful outputs without requiring human labeling of every harmful example — the constitution generalizes

Analytical Significance

CAI is relevant to intelligence analysis of AI governance in two ways:

  • Transparency claim: Anthropic publishes its constitution (unlike proprietary RLHF human-labeling criteria), enabling external analysis of what behavioral constraints are intended — but not whether they are implemented faithfully
  • Scalability: RLAIF reduces dependence on human feedback at scale, enabling rapid training of large models; the same scalability that reduces safety-labeling cost also means capability scaling faces fewer manual checkpoints
  • Dual-use constraint: CAI is designed to prevent the model from assisting with harmful activities; the Claude Mythos NSA deployment raises the question of whether CAI constraints apply when the model is used in a classified government context under a separate API agreement, or whether CAI constraints can be bypassed by system-prompt framing

Key Connections

  • Anthropic — developer; CAI is Anthropic’s primary alignment research contribution
  • Responsible Scaling Policy — RSP safety claims rest on CAI’s behavioral constraints; higher ASL tiers require demonstrated CAI effectiveness
  • Claude Mythos — the frontier model trained with CAI; deployed by NSA outside normal consumer guardrails
  • Dario Amodei — co-author of foundational CAI research

Sources

  • Bai et al. (Anthropic), “Constitutional AI: Harmlessness from AI Feedback” (2022) — [High confidence — primary research paper]
  • Anthropic model card documentation — [High confidence — primary]
  • Center for AI Safety (CAIS): CAI evaluation — [Medium confidence]

Graph View

Backlinks