Training AI models in morality is important. But who decides what what is right, and how?
“In January 2026, Anthropic announced a new approach to AI safety. Rather than hard-coding a list of rules and decision procedures, the company would train its model, Claude, on an 84-page “constitution” meant to instill “good judgment” and “ethical character.”
“Rules often fail to anticipate every situation and can lead to poor outcomes when followed rigidly in circumstances where they don’t actually serve their goal,” the constitution explained. “Good judgment, by contrast, can adapt to novel situations and weigh competing considerations in ways that static rules cannot.”
With the exception of a few bright lines—”Claude should never” create bioweapons, for example—the model was given remarkable leeway to decide whether, and how, to comply with requests. Anthropic acknowledged there would be “edge cases” in which Claude was “uncertain” about what to do. In such cases, the constitution said, Claude should “imagine” how a “senior Anthropic employee”—”someone who cares deeply about doing the right thing”—would approach the situation.
Left unsaid was an awkward fact about the company’s senior employees: Several of them do not believe in objective morality.”
A longish but very interesting article about training AI in morality, with a thoughtful and nuanced example of how that reasoning works in practice to provide responsible answers to moral questions.
Leave a Reply