About the company
Our client is a software company.
The role
Raydar is recruiting for this role on behalf of our client. Lead the measurement and improvement of AI agent quality, owning evaluation design and error analysis while building the backend systems that run and monitor agents. The role also mentors engineers and helps set the reliability direction for the AI platform.
What you'll do
- Set the strategy for measuring agent quality, including evaluation design and error-analysis practice.
- Design and build the knowledge and evaluation infrastructure that AI agents depend on.
- Lead investigations into agent failures and turn the findings into platform-level reliability fixes.
- Build and harden backend services that run, evaluate and monitor agents at scale.
- Help set the reliability and quality roadmap for the AI platform, working closely with the wider AI team.
- Mentor engineers on evaluation rigor and error analysis.
Requirements
What we're looking for
- A seasoned engineering background, with founders who have a strong track record also considered.
- 1 to 2 years of experience building end-to-end AI agents or production AI platforms, including agent loops, evaluation and error analysis.
- Backend, data platform or full-stack engineering experience.
- Hands-on history shipping backend code for AI features that end users rely on in live environments.
- Degree from a top 50 computer science program, in the US or internationally.
- Comfortable being on-site in the city roughly three days each week.
Bonus points
- Experience at a recently IPO'd technology company or an AI-native startup.
- Distributed systems experience.
Benefits
Compensation and benefits
- Base salary: $220K to $315K
- Equity
Location and work model
- San Francisco, CA, United States
- Hybrid, about 3 days in-office per week on average with no set days
- Full-time
Similar Jobs
What you need to know about the San Francisco Tech Scene
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
