The AI Behavior Engineer will measure and shape AI model behaviors, collaborating with stakeholders to conduct evaluations and adapt methods for various contexts.
Salary range: $310,000 - $500,000/year + benefits
Description: Transluce is a fast-moving nonprofit research lab building the public tech stack for scalable AI evaluation and oversight. We specialize in behavioral evaluations of frontier AI systems, assessing how models actually behave in deployment, not just how they perform on benchmarks. We are an independent non-profit with a mission to steer the development of AI for the public good.
About the role: We're looking for an engineer to work on measuring and shaping AI model behaviors, someone who thrives on turning hard questions into evidence fast. Think of this as a forward-deployed engineering role working directly with policymakers, civil society partners, and frontier labs to rapidly answer key questions about why AI systems act the way they do, and when and why they fail.
You'll build relationships with external domain experts, adapt our methods to new contexts, and help ensure our work is both technically credible and immediately useful to the people making consequential AI governance decisions.
This is a high-autonomy role with direct exposure to senior stakeholders and a clear line of sight from your work to real-world impact.
Core responsibility: Build and extend Transluce’s AI evaluation methods for measuring important evolving AI model behaviors. This includes:
- Scope, prototype, and run behavioral evaluations in response to emerging policy and oversight needs, including rapid-turnaround work for government and civil society partners.
- Execute on Transluce's contracts with government evaluators, including building evaluations for harmful manipulation with the EU AI Office.
- Design and run privileged-access evaluations and external oversight exercises with frontier labs.
- Work with civil society organizations and domain experts to adapt our behavioral evaluation pipelines to their contexts (e.g., mental health, persuasion, evaluation awareness).
Qualities of a strong candidate:
- Hands-on experience designing and running AI evaluations, particularly behavioral or interactive evaluations (multi-turn, agentic, or red-teaming contexts)
- Strong engineering instincts and good judgment about when "good enough to ship" is actually good enough.
- Experience in customer-facing, consulting, or forward-deployed roles translating ambiguous stakeholder needs into concrete deliverables.
- Experience running evaluations at scale or in a production context.
- Ability to understand and balance between the needs of AI researchers and domain experts, as well as between researchers and senior decision makers.
- Strong communication skills, low ego, openness to giving and receiving feedback.
We are located in San Francisco and enthusiastic to work together in-person. We are open to sponsoring international visas.
Transluce San Francisco, California, USA Office
1301 Sansome St, San Francisco, California, United States, 94111 1122
Similar Jobs
.png)
Consumer Web • eCommerce • Marketing Tech • Retail • Software • Analytics • Generative AI
Own the positioning, messaging, launches, packaging, pricing, adoption, and revenue growth strategy for Klaviyo Personalization. Partner with Product, Sales, Customer Success, Enablement, and Marketing to translate AI and data science capabilities into compelling customer value. Shape the roadmap through market insight, develop enablement and thought-leadership materials, influence cross-functional teams, and measure launch and program performance.
Top Skills:
Ai/MlAnalyticsCustomer Data Platforms (Cdp)Marketing AutomationPredictive Models
Blockchain • Fintech • Payments • Financial Services • Cryptocurrency • Web3
Lead the technical vision, architecture, and roadmap for Circle’s identity and access platform. Manage and develop backend and frontend engineers, guide architectural changes, review designs and code, troubleshoot production issues, participate in incident response, and partner with Security, Product, Risk, and engineering teams. The role focuses on authentication, authorization, customer and agent identity, secure delegation, platform reliability, migrations, and operational readiness.
Top Skills:
Auth0CedarJavaKotlinMcpOauth 2.XOidcOktaOpen Policy Agent (Opa)SAMLSpiffeSpireTypescript
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Supports RF network performance and RAN optimization across 4G, 5G, 6G, and small-cell deployments. Responsibilities include developing generative AI automation tools, launching carriers and sites, conducting verification and drive testing, monitoring KPIs, troubleshooting radio and baseband issues, and delivering engineering projects through customer acceptance. The role requires telecommunications or engineering knowledge, automation familiarity, structured problem-solving, and collaboration with customers and cross-functional teams.
Top Skills:
4G5G6GGenerative AiPower BIPythonRanRf OptimizationSmall CellsTableau
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
