xAI Logo

xAI

Software Engineer, Evals

Reposted 2 Days Ago
In-Office
Palo Alto, CA, USA
175K-275K Annually
Mid level
In-Office
Palo Alto, CA, USA
175K-275K Annually
Mid level
Build, own, and improve the evals platform that measures model capabilities and behaviors. Design datasets and grading schemes, manage execution infrastructure, diagnose failures, and deliver robust tools engineers use to clean data, provide experiment feedback, and track model growth.
The summary above was generated by AI

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceX is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars.
SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack: from our Colossus supercomputers, to Grok models, to our suite of applications, including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next.

SOFTWARE ENGINEER, EVALS

The evals team creates and executes instruments for measuring model capabilities and behaviors. These instruments are used to clean datasets, provide feedback for experiments, and track growth throughout training. The evals team also diagnoses why agents fail and how to make improvements both short and long term. We manage the execution infrastructure, design data sets and grading schemes. You will own and elevate our mission-critical research platform that directly accelerates SpaceXSI's frontier work.

You will design, build, and ship delightful and bulletproof systems that engineers love to use every day. This role values a strong attention to details, first principles reasoning, and great taste.

BASIC QUALIFICATIONS:

  • 3+ years of experience actively writing code

PREFERRED SKILLS AND EXPERIENCE:

  • A track record of shipping high quality projects and a strong sense of urgency
  • Exceptional product taste -  you have a sharp sense for what “good” looks like in outputs, and you can translate fuzzy notions of quality, helpfulness, truthfulness, and safety into concrete, measurable criteria
  • Strong attention to detail
  • Previously founded a startup or joined a fast-growing early-stage startup
  • Familiarity with AI/ML workflows, annotation tools, or evals
  • Experience creating and maintaining component libraries, design systems, or internal tools

COMPENSATION AND BENEFITS:

$175,000 - $275,000 USD

Base salary is just one part of our total rewards package at SpaceXSI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

SpaceXSI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.

HQ

xAI Palo Alto, California, USA Office

1450 Page Mill Road, Palo Alto, CA, United States

xAI San Francisco, California, USA Office

3180 18th St., San Francisco, CA, United States

Similar Jobs

One Month Ago
In-Office
San Francisco, CA, USA
180K-240K Annually
Senior level
180K-240K Annually
Senior level
Information Technology • Software • Database
Lead architecture and implementation of LangSmith platform features across Go, Python, and TypeScript. Collaborate with product and design teams, mentor engineers, drive project timelines, and deliver clean, tested software. Build and scale APIs and data-intensive applications using backend and frontend technologies, PostgreSQL, and Redis while engaging with enterprise customers and internal stakeholders.
Top Skills: APIsGoPostgresPythonReactRedisTypescript
2 Months Ago
In-Office
San Francisco, CA, USA
155K-195K Annually
Mid level
155K-195K Annually
Mid level
Information Technology • Software • Database
As a Backend Engineer, you will design and build backend systems and APIs for LangChain's observability and evals platform, optimizing performance and reliability while collaborating with cross-functional teams.
Top Skills: AWSAzureClickhouseGCPGoPostgresPythonRedis
2 Hours Ago
Remote or Hybrid
United States
107K-160K Annually
Mid level
107K-160K Annually
Mid level
Cloud • Fintech • Software • Business Intelligence • Consulting • Financial Services
The role involves managing outsourced accounting for nonprofits, leading teams, preparing financial reports, and improving operational efficiencies.
Top Skills: Bill.ComIntaactMicrosoft Office SuiteNetSuiteQuickbooks Online

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account