Lead embedded assessments of frontier AI labs’ safety practices. Scope evaluations, audit existing experiments, identify coverage gaps, design safety experiments, lead teams of 3–5 evaluators, and own the scientific content of public reports. Between engagements, conduct practice evaluations, train scientific staff, and improve evaluation tooling with engineers. The role requires in-person work at host labs and includes collaboration with AI developers, governments, and civil society organizations.
Salary range: $400,000 - $650,000/year + benefits
Description: Transluce is a fast-moving nonprofit research lab building the public tech stack for AI evaluation and oversight. We partner with frontier AI developers, governments, and civil society organizations to evaluate how AI systems behave, from misaligned agent swarms to child safety and mental health. Our evaluations directly underpin policy, including through collaborations with the EU AI Office and Common Sense Media. We are now building a program of embedded evaluations, in which independent teams assess frontier labs’ safety practices with employee-level access.
About the role: As Scientific Lead for Embedded Assessments, you will scope, design, and lead the assessments carried out by a team of 3-5 evaluators embedded at a frontier AI lab. We expect each engagement to last 1-3 months, during which you’ll work on-site at the host lab to assess the coverage of its internal safety stack. This involves identifying the most important open questions about model safety, auditing the lab’s existing evaluations to find gaps in coverage, and designing experiments to investigate risks and failure modes that haven’t been examined. You’ll own the scientific content of each engagement’s public report. Between engagements, you’ll lead practice evaluations on public models, train other scientific staff, and work with our engineering team to improve our tooling.
Core responsibilities:
- Refine the scientific scope of the assessment based on information gained while embedded.
- Review the output of already-conducted experiments or evals within the host organization with the aim of auditing coverage.
- Design and help execute experiments with the aim of evaluating safety practices of the host organization.
- Lead the scientific input into the final report that will be the culmination of the embedded assessment.
It is crucial for this work to happen in person, since much of the benefit of embedded evals comes from working physically within the host company’s facilities.
Preferred qualifications:
We are looking for candidates with an excellent track record of scientific research, ideally in an area adjacent to AI alignment. Other qualities we value for this role:
- Experience building AI evaluations, such as benchmarks, eval suites, or red-teaming work.
- Experience leading or managing researchers, ideally small, talented teams in a scrappy environment.
- Demonstrated ability to produce high-quality results while prioritizing under uncertainty and tight deadlines.
- Writing ability: both technical writing and the ability to communicate results to less technical team members as well as the general public.
- Ability to quickly incorporate a lot of technical context, in the form of code or other technical artifacts, and begin making progress towards a goal.
We are hiring at all levels of experience and would encourage those enthusiastic about the role who do not meet all of the qualifications to apply. We are located in San Francisco and excited to work together in-person. We are open to sponsoring international visas.
Transluce San Francisco, California, USA Office
1301 Sansome St, San Francisco, California, United States, 94111 1122
Similar Jobs
.png)
Consumer Web • eCommerce • Machine Learning • Software • Sports • Analytics
Manage a high-volume book of smaller and newer B2B VaaS partners. Lead standardized onboarding, launches, renewals, revenue retention, account health monitoring, escalations, forecasting, and partner communications. Improve onboarding documentation and playbooks, identify expansion opportunities, and transition growing accounts to Senior Account Managers. Collaborate with Business Development, Product, Engineering, Operations, Support, and Finance while maintaining partner terms, SLAs, and invoice accuracy.
Top Skills:
APIsChatgptClaudeCRMSlack
Events • Food • Travel • Hospitality • Hotels & Resorts
Processes hotel linens by collecting, sorting, washing, drying, ironing, folding, and delivering them. Maintains laundry equipment and work areas, monitors linen and supply inventory, and responds courteously to guest laundry requests. The role requires flexibility for shifts, weekends, and holidays, plus the ability to perform physical tasks including lifting, pushing, and pulling up to 100 pounds. Prior laundry, housekeeping, or hospitality experience is preferred but not required.
Fintech • Payments • Financial Services
Leads end-to-end creative production across marketing and brand initiatives, including campaigns, digital content, events, experiential activations, and customer-facing deliverables. Defines scopes, budgets, timelines, resources, and production plans; coordinates internal teams, vendors, agencies, and freelancers; manages reviews, risks, approvals, costs, and change orders. The role also improves creative intake and production processes while maintaining visibility across multiple concurrent projects.
Top Skills:
AIProject Management Tools
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
