Lead embedded assessments of frontier AI labs’ safety practices. Scope evaluations, audit existing experiments, identify coverage gaps, design safety experiments, lead teams of 3–5 evaluators, and own the scientific content of public reports. Between engagements, conduct practice evaluations, train scientific staff, and improve evaluation tooling with engineers. The role requires in-person work at host labs and includes collaboration with AI developers, governments, and civil society organizations.
Salary range: $400,000 - $650,000/year + benefits
Description: Transluce is a fast-moving nonprofit research lab building the public tech stack for AI evaluation and oversight. We partner with frontier AI developers, governments, and civil society organizations to evaluate how AI systems behave, from misaligned agent swarms to child safety and mental health. Our evaluations directly underpin policy, including through collaborations with the EU AI Office and Common Sense Media. We are now building a program of embedded evaluations, in which independent teams assess frontier labs’ safety practices with employee-level access.
About the role: As Scientific Lead for Embedded Assessments, you will scope, design, and lead the assessments carried out by a team of 3-5 evaluators embedded at a frontier AI lab. We expect each engagement to last 1-3 months, during which you’ll work on-site at the host lab to assess the coverage of its internal safety stack. This involves identifying the most important open questions about model safety, auditing the lab’s existing evaluations to find gaps in coverage, and designing experiments to investigate risks and failure modes that haven’t been examined. You’ll own the scientific content of each engagement’s public report. Between engagements, you’ll lead practice evaluations on public models, train other scientific staff, and work with our engineering team to improve our tooling.
Core responsibilities:
- Refine the scientific scope of the assessment based on information gained while embedded.
- Review the output of already-conducted experiments or evals within the host organization with the aim of auditing coverage.
- Design and help execute experiments with the aim of evaluating safety practices of the host organization.
- Lead the scientific input into the final report that will be the culmination of the embedded assessment.
It is crucial for this work to happen in person, since much of the benefit of embedded evals comes from working physically within the host company’s facilities.
Preferred qualifications:
We are looking for candidates with an excellent track record of scientific research, ideally in an area adjacent to AI alignment. Other qualities we value for this role:
- Experience building AI evaluations, such as benchmarks, eval suites, or red-teaming work.
- Experience leading or managing researchers, ideally small, talented teams in a scrappy environment.
- Demonstrated ability to produce high-quality results while prioritizing under uncertainty and tight deadlines.
- Writing ability: both technical writing and the ability to communicate results to less technical team members as well as the general public.
- Ability to quickly incorporate a lot of technical context, in the form of code or other technical artifacts, and begin making progress towards a goal.
We are hiring at all levels of experience and would encourage those enthusiastic about the role who do not meet all of the qualifications to apply. We are located in San Francisco and excited to work together in-person. We are open to sponsoring international visas.
Transluce San Francisco, California, USA Office
1301 Sansome St, San Francisco, California, United States, 94111 1122
Similar Jobs
Fintech • Mobile • Software • Financial Services
Own the technical vision, architecture, roadmap, and operations of SoFi’s enterprise Temporal workflow orchestration platform. Design multi-region resiliency, self-service provisioning, access controls, SDKs, and platform features; manage the Temporal Cloud partnership and infrastructure costs; resolve high-severity incidents; establish reliability and developer-experience standards; and mentor engineers across teams. The role also champions AI-assisted engineering and partners with business and engineering leaders on distributed workflow use cases.
Top Skills:
Apache KafkaAWSAzureGCPGoJavaKafka ConnectKotlinKubernetesPythonRabbitMQTemporal CloudWarpstream
HR Tech • Information Technology • Professional Services • Sales • Software
The SMB Account Executive prospectively generates pipeline, manages full sales cycles, conducts product demonstrations, engages decision-makers, forecasts revenue, and closes SaaS deals with companies of 10–199 employees. The role requires generating 40–50% of pipeline, maintaining Salesforce data, attending industry events, and building relationships with market influencers, consultants, and partners. This is a hybrid role requiring New York Metro candidates to work in the NYC office three days weekly.
Top Skills:
HrisSaaSSalesforce
eCommerce • Fashion • Retail • Sales • Wearables • Design
Provides personalized styling advice, product recommendations, and customer service in a retail environment. Builds customer relationships, drives sales through product knowledge and storytelling, supports checkout, maintains stockroom organization, and contributes to operational excellence. Requires flexible scheduling, physical activity, teamwork, communication skills, and willingness to learn omnichannel and virtual selling techniques.
Top Skills:
Omni-Channel Selling ToolsPoint-Of-Sale (Pos) SystemsSocial Media
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
