Judgment Labs Logo

Judgment Labs

Forward Deploy AI Engineer

Reposted 19 Days Ago
In-Office
San Francisco, CA, USA
Entry level
In-Office
San Francisco, CA, USA
Entry level
Embed Judgment Labs' agent behavior monitoring into customer production systems: integrate monitoring and evaluation into agent workflows, diagnose failures in live environments, guide customers on monitoring and evaluation strategy, and own multiple customer engagements end-to-end to ensure sustained adoption.
The summary above was generated by AI

We're building the infrastructure for continual learning. Agents are

We’ve raised $32M across two rounds led by Lightspeed.

The Role:

You'll own the end to end customer deployment process from understanding their pain points to building solutions end to end.

This role centers on deep technical execution and customer ownership. You will work directly with customer teams to reason about agent behavior, translate high-level goals into concrete ABM deployments, and own outcomes end-to-end across real production environments. The scope, judgment, and autonomy required in this role mirrors a training ground for what it takes to found or lead a technical company.

What You'll Do:
  • Deploy and embed Judgment Labs’ ABM platform and AI components directly into customer codebases and production AI systems

  • Work inside customer systems to integrate monitoring, evaluation, and agent-facing components into real workflows

  • Guide customers through technical decisions around agent monitoring, evaluation strategy, and integrating these capabilities into existing production systems.

  • Own multiple customer engagements end-to-end, ensuring successful integration and sustained adoption of monitoring and evaluation systems within production agent workflows.

What We're Looking For:

You identify with at least one of the following:

  • Experience deploying AI or LLM-based systems into real production environments

  • Ability to quickly learn new tools and systems, and integrate AI infrastructure into existing customer workflows and codebases

  • Ability to translate ambiguous customer goals into concrete technical solutions and evaluation strategies

  • Strong customer-facing skills, including explaining complex technical concepts clearly and building trust with both technical and non-technical stakeholders

  • Comfort owning deployments end-to-end, from initial integration through successful production adoption

  • You want to be a technical founder in the future.

Why Judgment?
  • Agents can’t work without this. Today’s agents hallucinate, drift, and break in production. We’re building the infrastructure that fixes this: the monitoring layer that makes agents self-improving.

  • We’re wired to win. We're a team of 20 but we ship like 50+ on the daily. You'll be working with olympiad medalists, debate champions, and competitive athletes who bring that same intensity to company building.

  • Fast track to founding. Our engineers interface directly with customers, ship code into their environments, and use their feedback to dictate what’s next on the roadmap. Everyone on the team is either an ex-founder or a founder-to-be.

  • We make sure our people do their best work. If you deserve a spot on the team, money will never get in the way of it. Full benefits, Equinox, and a private chef to take care of you. We sprint hard but we play hard, ask us about our Smash/Mario Kart tournaments.

    We work in person in San Francisco.

HQ

Judgment Labs San Francisco, California, USA Office

425 Bush St, San Francisco, California, United States, 94108 3708

Similar Jobs

An Hour Ago
Hybrid
15-22 Hourly
Junior
15-22 Hourly
Junior
eCommerce • Fashion • Retail • Sales • Wearables • Design
Provide friendly, efficient customer service at the cash wrap and sales floor; operate POS and handle cash accurately; receive and organize shipments; manage stock levels and visual merchandising; maintain stockroom cleanliness and loss prevention standards; assist customers and support team activities.
Top Skills: Cash Register SystemsInternetIpadLaptopMobile PosPosWalkie Talkie
2 Hours Ago
Remote or Hybrid
United States
80K-160K Annually
Senior level
80K-160K Annually
Senior level
Cloud • Insurance • Payments • Software • Business Intelligence • App development • Big Data Analytics
Build and deploy production micro-apps and LLM-powered agents on a governed GCP platform. Integrate AI with systems (Salesforce, Slack, NetSuite, SQL), harden apps with auth, secrets, CI/CD, logging and monitoring, automate maintenance, and implement evaluation and observability for model quality and drift.
Top Skills: AgentforceAuthentication/IdentityBraintrustCi/CdClaudeGCPLangfuseLangsmithLlmsLoggingMonitoringNetSuitePgvectorPineconePythonSalesforceSecrets ManagementSlackSQLVertex Ai Vector Search
2 Hours Ago
Remote or Hybrid
US
90K-140K Annually
Senior level
90K-140K Annually
Senior level
Cloud • Insurance • Payments • Software • Business Intelligence • App development • Big Data Analytics
Design, deploy, and operate enterprise network infrastructure across data centers and cloud (GCP, AWS, Azure). Manage VPCs, VPNs, BGP, F5 BIG-IP, hybrid connectivity, and observability. Build IaC and automation with Terraform, GitLab CI/CD, and Ansible. Maintain monitoring, documentation, and participate in on-call support for critical incidents.
Top Skills: AnsibleAWSAzureBgpCloud RouterCloudflareDnsF5 Big-IpFirewallsGCPGitlab Ci/CdHa VpnIp Address ManagementIpsec VpnLoad BalancingTerraformVpc

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account