xAI Logo

xAI

Software Engineer - Evals

Reposted One Month Ago
In-Office
Palo Alto, CA, USA
175K-275K Annually
Mid level
In-Office
Palo Alto, CA, USA
175K-275K Annually
Mid level
Build, own, and improve the evals platform that measures model capabilities and behaviors. Design datasets and grading schemes, manage execution infrastructure, diagnose failures, and deliver robust tools engineers use to clean data, provide experiment feedback, and track model growth.
The summary above was generated by AI

SpaceXAI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company’s mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All employees are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates.

You will own and elevate our mission-critical research platform that directly accelerates SpaceXAI's frontier work.

The evals team creates and executes instruments for measuring model capabilities and behaviors. These instruments are used to clean datasets, provide feedback for experiments, and track growth throughout training. The evals team also diagnoses why agents fail and how to make improvements both short and long term. We manage the execution infrastructure, design data sets and grading schemes.

You will design, build, and ship delightful and bulletproof systems that engineers love to use every day. This role values a strong attention to details, first principles reasoning, and great taste.  

We are open to all levels from new grads to senior engineers.

Requirements

  • Software engineering proficiency and 3+ years of experience actively writing code.
  • A track record of shipping high quality projects and a strong sense of urgency.
  • Exceptional product taste -  you have a sharp sense for what “good” looks like in outputs, and you can translate fuzzy notions of quality, helpfulness, truthfulness, and safety into concrete, measurable criteria.
  • Strong attention to detail.

Bonus Points

  • Previously founded a startup or joined a fast-growing early-stage startup.
  • Familiarity with AI/ML workflows, annotation tools, or evals.
  • Experience creating and maintaining component libraries, design systems, or internal tools.

COMPENSATION AND BENEFITS:

$175,000 - $275,000 USD

Base salary is just one part of our total rewards package at SpaceXAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

SpaceXAI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.

HQ

xAI Palo Alto, California, USA Office

1450 Page Mill Road, Palo Alto, CA, United States

xAI San Francisco, California, USA Office

3180 18th St., San Francisco, CA, United States

Similar Jobs

One Month Ago
Hybrid
San Francisco, CA, USA
155K-400K Annually
Senior level
155K-400K Annually
Senior level
Cloud • Information Technology • Software • Analytics • Business Intelligence
The Senior Software Engineer will build evaluation infrastructure for AI systems, ensuring reliability and accuracy. Responsibilities include designing datasets, benchmarks, and test harnesses for AI behavior assessment.
Top Skills: AIMlPythonTypescript
One Month Ago
In-Office
San Francisco, CA, USA
266K-445K Annually
Mid level
266K-445K Annually
Mid level
Artificial Intelligence • Machine Learning • Generative AI
The Backend Software Engineer will design and build infrastructure for evaluating and enhancing OpenAI's support automation, collaborating with teams to develop pipelines and APIs.
Top Skills: FastapiPostgresPython
One Month Ago
In-Office
San Francisco, CA, USA
155K-195K Annually
Mid level
155K-195K Annually
Mid level
Information Technology • Software • Database
As a Backend Engineer, you will design and build backend systems and APIs for LangChain's observability and evals platform, optimizing performance and reliability while collaborating with cross-functional teams.
Top Skills: AWSAzureClickhouseGCPGoPostgresPythonRedis

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account