Tolmo, Inc. Logo

Tolmo, Inc.

Research Engineer, AI Security

Posted 4 Days Ago
Hybrid
San Francisco, CA, USA
Entry level
Hybrid
San Francisco, CA, USA
Entry level
Research Engineer improving AI security agents through agent reasoning, evaluation, benchmarking, model selection, security hardening, and reliability engineering. The role develops production-quality systems that detect vulnerabilities, resist prompt injection and data exfiltration, validate findings, and learn from customer engagements. Responsibilities span research and engineering, including experiments, graders, benchmarks, multi-step planning, model tuning, and security research for LLM-based applications.
The summary above was generated by AI
About Tolmo

Tolmo is cybersecurity for the age of AI. Coding agents now ship software faster than any human team can review or secure it; 100+ pull requests per engineer per week is no longer an outlier, and attackers are using the same agents to find and exploit weaknesses in days, not months. The old playbook of quarterly pentests and endless alert queues breaks at this speed.

We're building a new security playbook. Tolmo runs a fleet of AI security agents that reason over a live graph of your entire production system (code, CI, cloud, and telemetry) and continuously attack a faithful copy of it like a real adversary. Every finding arrives validated, with its blast radius and a remediation plan that lands in your workflow, ready for a coding agent to fix. Hours to integrate, days to your first validated finding.

We've done this before: Tolmo's founders built Sqreen (acquired by Datadog) and led security products that thousands of organizations trusted. We're backed by Accel and Y Combinator and just getting started.

The role

Our product is only as good as our agents' judgment. As a Research Engineer on AI Security, you make that judgment better and measurably so: how our agents plan, reason over a graph of a customer's production system, decide what is worth testing, and know when a finding is real.

This sits between research and engineering. In one week you might design an eval that catches a new class of false positive, change how an agent plans a multi-step investigation, compare models on a hard benchmark, and harden the agents themselves against prompt injection from the systems they inspect. You'll work directly with the founders, our engineers, and the security leads who run pentests with our agents, and your work ships to production.

What you'll do
  • Agent reasoning: improve how our agents plan, use tools, and reason over a live graph of code, cloud, and telemetry, so they reach real findings faster and with fewer dead ends.

  • Evaluation: build the evals, benchmarks, and graders that tell us whether an agent actually got better, and make them part of how every change ships.

  • Model selection and tuning: compare models, prompts, and agent designs on cost, latency, and quality, and turn the results into production decisions.

  • Model and agent security: defend our agents against prompt injection, data exfiltration, and manipulation by the very systems they test, and research the security of LLM-based applications our customers build.

  • Reliability at scale: make long-running agents dependable across thousands of runs: verification, self-checks, and sensible stopping.

  • Learning from the field: study agent runs from customer engagements, find where reasoning broke, and turn each failure into an eval and a fix.

What we're looking for
  • Hands-on experience building, evaluating, or improving LLM-based agents or systems in production.

  • A rigorous, empirical approach: you design experiments, distrust single anecdotes, and let the evals decide.

  • Strong engineering skills: you write production-quality code (Python, TypeScript, or Go) and build tools other people rely on.

  • A working understanding of security, or a strong drive to learn it fast: how real systems are attacked and how findings are validated.

  • Familiarity with LLM security risks such as prompt injection, jailbreaks, and tool misuse.

  • You build with AI coding agents daily (e.g., Claude Code, Cursor) and have opinions about what makes them reliable.

Nice to have
  • Research background in machine learning, AI safety, or security: publications, open-source work, or talks.

  • Offensive security experience: penetration testing, red teaming, bug bounty, or CTFs.

  • Experience with reinforcement learning, fine-tuning, or synthetic data generation for agents.

  • Experience with graph data models or reasoning over structured data.

  • Prior early-stage startup experience.

Compensation & benefits
  • Competitive salary, depending on level and experience.

  • Early-stage equity: you're joining early; you'll own a real piece of what we build.

  • Comprehensive medical, dental, and vision coverage, and the equipment you need to do your best work.

  • Based in San Francisco; we work in person.

Tolmo is building a team with wide-ranging backgrounds, perspectives, and paths into security; we think it makes our product and judgment better. If you're excited about this role but don't match every point above, we'd still like to hear from you. We're an equal opportunity employer and welcome applicants of every background.

Similar Jobs

6 Days Ago
In-Office or Remote
5 Locations
224K-431K Annually
Expert/Leader
224K-431K Annually
Expert/Leader
Artificial Intelligence • Computer Vision • Hardware • Robotics • Metaverse
Develop and formalize methods to validate reported vulnerabilities and generate/verify safe patches for internal NVIDIA targets. Define correctness, regression, and revalidation standards; produce repeatable, auditable processes and evidence. Apply AI-assisted workflows cautiously, perform bounded applied research on vulnerability classes, and document findings for independent security review.
Top Skills: Ai-Assisted ToolsCC++FuzzingLlm-Based AgentsProgram AnalysisPythonStatic AnalysisSymbolic Execution
15 Minutes Ago
In-Office
Entry level
Entry level
Events • Food • Travel • Hospitality • Hotels & Resorts
Maintains cleanliness and appearance of hotel public areas, including restrooms, hallways, elevators, stairwells, offices, and event spaces. Performs dusting, vacuuming, mopping, carpet shampooing, window washing, trash removal, and periodic deep cleaning. Greets guests, answers questions, resolves service requests, and assists with guest room cleaning and housekeeping operations. Requires availability for days, weekends, and holidays.
18 Minutes Ago
In-Office or Remote
Senior level
Senior level
Events • Food • Travel • Hospitality • Hotels & Resorts
Execute property-specific hotel marketing campaigns across digital, social media, and branded channels to increase visibility, revenue, and guest engagement. Monitor hospitality marketing trends, tools, and consumer behavior; prepare monthly campaign performance reports and analyses; and collaborate with Sales, Revenue Management, and other department leaders on cross-functional initiatives.
Top Skills: Digital Marketing PlatformsSocial Media Platforms

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account