Own the strategy and product systems for AI agents that automate complex operational workflows. Define agent reasoning, tool use, autonomy, human escalation, quality evaluation, monitoring, and continuous improvement. Work closely with customers, operators, engineers, and AI practitioners to transform messy processes into reliable products. Establish outcome-based metrics including resolution rate, accuracy, intervention rates, and time to resolution while building trust in production AI.
About BackOps
The Role
What You'll Own
Define how BackOps agents work
Turn messy operations into great products
Own the quality bar for production AI
Design human + AI collaboration
Make intelligence compound
Measure outcomes, not demos
What We're Looking For
Particularly Compelling
This Probably Isn't the Role for You If
Why BackOps
BackOps is building the AI resolution layer for companies that make or move physical goods.
The physical economy still runs on a fragmented web of ERPs, WMSs, TMSs, carrier and vendor portals, email, spreadsheets, legacy systems, and the people who know how to hold all of it together. When something breaks, whether a shipment is delayed, an order changes, a claim needs to be filed, a document is missing, someone has to gather context across systems, determine what should happen next, take action, and make sure the problem is actually resolved.
BackOps automates that work.
Our platform, Relay, deploys AI operators that understand operational workflows, work across the systems our customers already use, take action, navigate exceptions, and carry work through to resolution. We aren't building another dashboard or copilot. We're building the execution layer that sits between systems and gets the work done.
We recently raised a $42M Series B led by Insight Partners, just six months after our Series A, and are scaling across some of the largest companies in logistics, manufacturing, retail, and other industries that power the physical world.
The opportunity ahead is much larger than any single workflow or vertical. We're building the infrastructure for AI to operate reliably inside the real world.
The Role
As Product Manager, AI Agents, you'll define what it means for an AI operator to be truly excellent at operational work.
This isn't a role focused on adding AI features to an existing SaaS product.
AI is the product.
Relay has to understand an incoming problem, reconstruct context scattered across multiple systems, interpret company-specific rules, determine the right next action, use tools to execute it, recognize when something has gone wrong, know when human judgment is necessary, and carry the work through to a measurable outcome.
And it has to do this in environments where the end user may be a logistics coordinator, warehouse operator, customer service representative, dispatcher, buyer, or operations manager, not an AI expert.
You'll own the product systems and experiences that make those agents intelligent, trustworthy, controllable, and increasingly autonomous.
This is a senior individual contributor role for someone who wants to help define a new category of software from first principles.
Define how BackOps agents work
Own product strategy across the agent lifecycle: how agents understand work, reason about it, take action, handle ambiguity, recover from failure, learn from outcomes, and collaborate with humans.
You'll help answer fundamental questions such as:
- What context does an agent need to make the right decision?
- Which parts of a workflow should be deterministic and which should be agentic?
- When should an agent act autonomously versus request approval?
- How should agents interpret SOPs, business rules, historical outcomes, and unstructured information?
- How should an agent behave when reality doesn't match the documented process?
- How do we make agent behavior configurable without forcing customers to become prompt engineers?
Turn messy operations into great products
The workflows BackOps automates were rarely designed cleanly in the first place.
You'll spend time with customers and operators understanding how work really happens, including the workarounds, exceptions, judgment calls, and tribal knowledge that never appear in an SOP.
Then you'll determine what the best AI-native version of that workflow should be.
You'll work across areas like order management, shipment exceptions, claims, billing, document processing, scheduling, vendor coordination, and entirely new categories we haven't built yet.
The objective isn't to reproduce the manual process step for step.
It's to build a better one.
Own the quality bar for production AI
For traditional software, you can usually define what the product will do.
For AI products, you have to define what good looks like and build systems for continuously determining whether you're achieving it.
You'll own the product strategy for agent quality, partnering with engineering to develop:
- Evaluation frameworks and success criteria
- Golden datasets and representative test cases
- Simulation and regression testing
- Human evaluation and annotation
- LLM-based evaluation where appropriate
- Production quality monitoring
- Failure taxonomies and root-cause loops
- Feedback systems that turn real-world outcomes into better agent behavior
You'll define the domain-specific quality bar; our platform team will help build the infrastructure that makes evaluating and operating agents at scale possible.
Design human + AI collaboration
The goal isn't autonomy for autonomy's sake.
Some tasks should happen automatically. Some require approval. Some need a human because judgment or accountability matters.
You'll define those boundaries and build experiences that allow operators to understand what Relay is doing, trust it, correct it when necessary, and progressively hand over more work as confidence grows.
Great AI UX at BackOps might mean making something disappear entirely. It might also mean showing exactly the right context at exactly the right moment for a five-second human decision.
Knowing the difference is core to this job.
Make intelligence compound
Every workflow creates data about how operations actually function: which exceptions occur, where people intervene, which decisions work, where processes break, and what ultimately resolves the issue.
You'll help determine how BackOps turns that data into a product advantage, improving agent quality, workflow design, and customer outcomes over time.
Measure outcomes, not demos
You'll define and own metrics such as:
- End-to-end resolution rate
- No-touch / autonomous resolution rate
- Task accuracy and quality
- Human intervention and escalation rate
- Time to resolution
- Failure and recovery patterns
- Customer adoption and trust
- Operational impact generated for customers
A compelling demo is useful.
A production system that completes mission-critical work every day is the standard.
What We're Looking For
You may be a strong fit if you:
- Have 6+ years of relevant product-building experience, with meaningful ownership at the Senior or Staff level.
- Have shipped AI/ML, LLM, agentic, automation, or other technically sophisticated products into production and owned the outcome, not simply added an AI feature to an existing roadmap.
- Understand the unique challenges of non-deterministic products: evaluation, quality measurement, grounding, tool use, context, failure modes, guardrails, human escalation, and continuous improvement.
- Have exceptional product instincts and can take a complicated workflow and make the resulting user experience feel obvious.
- Are technically fluent enough to work deeply with engineers and AI practitioners without outsourcing the hard product decisions to them.
- Have built zero-to-one products or substantially rethought how an existing category should work.
- Stay extraordinarily close to users and are comfortable observing messy real-world processes rather than relying exclusively on feature requests.
- Can distinguish between an impressive AI capability and a product customers will trust with real work.
- Move quickly, think rigorously, and are energized by a product surface where many of the right answers haven't been discovered yet.
Particularly Compelling
We'd be especially interested if you've:
- Built core products at a breakout AI company or an exceptional, high-growth B2B software company.
- Directly owned agentic systems, LLM applications, AI workflow products, AI-native interfaces, or evaluation systems.
- Built products where reliability, accuracy, explainability, or trust had meaningful real-world consequences.
- Worked in logistics, manufacturing, commerce, construction, healthcare, field services, or another operationally complex industry.
- Built for users who aren't highly technical and have little patience for software that requires them to change how they work.
- Developed products where software interacts with both digital systems and real-world operational processes.
You don't need to arrive knowing supply chain.
You do need to be the kind of product builder who can become dangerous in a complicated domain very quickly.
This Probably Isn't the Role for You If
- Most of your AI experience is adding a chatbot or generative feature to an otherwise traditional SaaS product.
- You think the objective is to maximize autonomy rather than maximize customer outcomes.
- You prefer product problems where behavior can be completely specified in advance.
- You stay primarily at the roadmap/PRD layer and expect engineering or research to make the difficult AI product decisions.
- Your first instinct is to replicate a customer's existing workflow exactly as it exists today.
Why BackOps
AI has become remarkably good at generating information.
The next frontier is much harder: AI that can be trusted to do the work.
In the physical economy, that means more than producing the right answer. An agent has to understand what's happening, make a decision, interact with real systems, navigate exceptions, communicate with real people, and remain accountable until the problem is actually resolved.
That creates an unusually rich product problem.
You'll help determine how agents reason about real-world operations, where humans remain in the loop, how quality is measured, how trust is earned, and how AI progresses from assisting an operator to owning an outcome.
Few product teams get the opportunity to define those interaction models while the category itself is still being invented.
That's the opportunity at BackOps.
BackOps AI San Francisco, California, USA Office
San Francisco, CA, United States
BackOps AI San Ramon, California, USA Office
San Ramon, United States
Similar Jobs
.png){kind=logo}
Artificial Intelligence • Big Data • Consumer Web • eCommerce
Own the AI agent-facing product, including the API and MCP surfaces, interface contracts, versioning, schemas, reliability, and per-query costs. Define eval-gated specifications and machine-checkable rubrics that coding agents and engineers can build against. Model system architecture, developer needs, latency, trust, provenance, confidence, and schema stability. Work directly with the founder and direct product development inside Cortex.
Top Skills:
Ai AgentsAPIsCortexKnowledge GraphsMcp
.png)
Software
Lead strategy, vision, and execution for Amplitude’s AI agents and MCP products. Define 12–24 month priorities, improve agent quality through evaluations and feedback systems, expand MCP integrations across coding agents, CLIs, and IDEs, engage customers to identify use cases, and enable field teams to position and support agentic products. The role partners closely with design and engineering while serving as an internal and customer-facing thought leader on agentic product development.
Top Skills:
Ai/MlLarge Language Models (Llms)Mcp
.png)
Artificial Intelligence • Big Data • Machine Learning
Own the Finance vertical for Agents Data & RL Environments: set product and data strategy, design and scale finance-specific RL simulations and datasets, partner with frontier research teams, and drive cross-functional execution and go-to-market efforts to train and evaluate AI agents on real financial workflows.
Top Skills:
Agentic AiMachine LearningModel TrainingReinforcement LearningRl Environments
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
