AI Safety
Capability without understanding is a liability. Our safety research develops the tools to look inside models, evaluate them rigorously, and characterize how they behave when conditions change — so that progress can be matched by justified confidence.
- Interpretability
- Alignment
- Robustness
- Model evaluation
- Distribution shift
- Uncertainty
- Responsible deployment
Why it matters
A model that performs well on a benchmark may still fail in deployment, misrepresent its own confidence, or pursue an objective that diverges subtly from what was intended. These problems become harder to detect as systems become more capable, because their outputs become harder for people to check directly.
We regard safety and capability as connected rather than opposed. The methods that let us understand a model — interpretability, careful evaluation, calibrated uncertainty — are the same methods that let us improve it with confidence. Safety research is therefore integral to our work across every other domain.
Key research questions
Can we identify, from a model's internals, the features and circuits responsible for a specific behavior?
How can evaluations remain informative as models approach or exceed the abilities of their evaluators?
When should a model abstain, and how can we make its confidence a trustworthy signal?
How do models behave under distribution shift, and can we detect shift before it causes failure?
What training methods produce behavior that remains aligned with intent in situations not covered by training?
Primary subfields
- Interpretability
- Explaining model behavior in terms of internal representations and computation.
- Evaluation
- Designing measurements that remain valid, hard to game, and predictive of real behavior.
- Robustness
- Behavior under adversarial inputs and distribution shift.
- Uncertainty quantification
- Calibration, abstention, and honest reporting of confidence.
- Alignment
- Methods for specifying intent and checking that systems pursue it.
Representative methodologies
Mechanistic interpretability
Reverse-engineering circuits and features in trained networks.
Red-teaming and adversarial testing
Searching systematically for inputs that cause failure.
Calibration analysis
Comparing stated confidence with observed accuracy across conditions.
Held-out and dynamic evaluation
Test sets constructed to resist contamination and overfitting.
Relationships to other domains
- Machine Learning
Interpretability depends on a science of representations.
- Generative AI
Generative systems are where evaluation is currently hardest.
- Artificial Intelligence
Autonomous agents amplify the consequences of misalignment.
Working on questions in AI Safety?
We welcome conversations with researchers and organizations working on related questions — whether to compare notes, discuss a collaboration, or share work in progress.