constitutional-ai

AI safety alignment via constitutional AI trains models to be harmless through self-critique and AI feedback. It applies a constitution of formal principles guiding outputs toward helpfulness and honesty without requiring human labels.

You train AI to be harmless without human labels by using Reinforcement Learning from AI Feedback (RLAIF). This workflow generates responses, critiques them against a constitution, and fine-tunes the model on revised outputs.

Yes, you can use optional chain-of-thought reasoning prompts for transparent safety alignment. These prompts expose the model's safety reasoning process during the self-critique and revision loops.

Reinforcement learning from AI feedback scales safety improvements without human labels by using AI to evaluate and critique model outputs. It follows supervised learning with a constitution to iteratively reduce harmful outputs.

The limitation of using self-critique for safety alignment is that the model's revision quality depends entirely on the defined constitution. If principles are incomplete, harmful outputs may not be fully corrected during the RLAIF phase.