Staff+ Software Engineer, Safeguards Evals

Anthropic · San Francisco, CA · $320,000-$485,000 USD · hybrid Anthropic's mission is to create reliable, interpretable, and steerable AI systems that are safe and beneficial for users and society. The Staff+ Software Engineer, Safeguards Evals role focuses on building methods and infrastructure for evaluating model behavior and agentic systems. Responsibilities Evaluation research and methodology: Design experiments to improve evaluation quality, develop methods to generate representative test data, and validate grading accuracy. Agentic investigation evals: Build and own the evaluation harness for an agentic investigation system, defining metrics, test cases, and grading approaches. Datasets grounded in real harm: Construct high-quality eval datasets representing real-world misuse across harm areas. Productionization and tooling: Ship successful research into evaluation, regression, and release pipelines, and build tooling for policy experts to author, run, and iterate on evaluations. Minimum qualifications Experience building and maintaining data pipelines Experience working with LLMs and a working understanding of their capabilities and failure modes Strong data analysis skills Ability to move fluidly between research prototyping and production-quality code Ability to translate ambiguous problems into concrete, testable experiments Care deeply about AI safety and want your work to have real impact Preferred qualifications 8+ years of industry software engineering or ML engineering experience Expertise in building or contributing to LLM or agent evaluation frameworks, benchmarks, or automated grading systems Extensive experience in trust and safety, content moderation, or abuse detection systems Experience in red teaming, adversarial testing, or jailbreak research on AI systems Logistics Annual Salary: $320,000-$485,000 USD