Model Evaluation and Threat Research, Inc. Logo

Model Evaluation and Threat Research, Inc.

Platform Engineer, Application Security

Posted Yesterday
Be an Early Applicant
Hybrid
Berkeley, CA, USA
328K-579K Annually
Senior level
Hybrid
Berkeley, CA, USA
328K-579K Annually
Senior level
Design and build security controls for METR’s AI evaluation platform, including sandboxing, isolation, networking, access boundaries, IAM, and agent permissions. Identify, triage, and remediate vulnerabilities across cloud infrastructure, codebases, and access systems. Conduct secure code reviews, harden production systems, automate provisioning and access reviews, and develop monitoring and guardrails for agentic systems.
The summary above was generated by AI
About METR

We are a nonprofit research organization that develops scientific methods to assess AI capabilities, risks, and mitigations, with a specific focus on threats related to AI R&D automation and misalignment.

We believe it is robustly good for policymakers and civil society to have a clear understanding of risks from AI systems, and we are extremely excited to build a team of ambitious, excellent people to tackle one of the most important challenges of our time.

About the role

    METR’s mission of enabling transparency and coordination about the risks of frontier AI requires a high degree of trust from frontier AI labs, governments, and the public. As misalignment incidents become more extreme and confidential information about models and frontier AI labs becomes more valuable, we expect to be under increasingly intense pressure from external actors and internal agents. 

    METR is looking to expand the security expertise on our platform team. This would span application security in our evaluation platform, sandboxing agents and evaluations, cloud platform security, networking, access control for both people and agents, and securing our development environments and workflows. This role will have a large engineering component: expect to write and review code, fix vulnerabilities, and design and develop secure systems.

What this role looks like

  • Securing a unique attack surface. METR's evaluation infrastructure runs frontier AI agents, including early checkpoints of unreleased models, executing untrusted, model-generated code at scale on multi-day tasks. You will design and build the isolation, networking, and permission boundaries that contain evaluated agents. 

  • Remediation and hardening. You will find, triage, and fix vulnerabilities across our cloud infrastructure and access control systems.

  • Fixing known vulnerabilities in our codebase. This includes code reviews and resolving known vulnerabilities in our backlog.

  • Identity and access as a system. You will design and implement IAM policies, least-privilege access, and automated provisioning and access review for both people and agents.

  • Securing agentic systems. You will design and implement systems to monitor and control agents, making sure they can operate securely and with appropriate permissions and guardrails.

Required skills

  • 7+ years of experience working in security engineering, software development, or an adjacent field.

  • Production software engineering. You have experience building and operating backend or infrastructure in practice.

  • Cloud and container security. You have deep familiarity with AWS (especially non-trivial IAM), Kubernetes, and infrastructure-as-code environments.

  • Vulnerability remediation. You have found and fixed security flaws in large production systems and can prioritize a remediation backlog.

  • Security fundamentals. Strong security knowledge across systems, networks, cloud, and identity, and a track record of applying it to real systems. Experienced in designing secure software and cloud architectures.

  • Code review. Reviewing and giving constructive security feedback on PRs, including from the FOSS community.

  •  

Nice to haves

  • Detection engineering at scale: Experience with detection pipelines (DataDog SIEM, AWS SecurityHub), writing and tuning detections, and threat hunting. 

  • AI/LLM engineering: You build with AI: agent pipelines, LLM-powered tooling, automated workflows, and understand current limitations of those tools.

  • AI security research: Familiarity with agent control, hardware security, or red teaming AI systems themselves.


  • Ideally you have experience with a portion of these technologies:

  • AWS: cloud-native software platforms

  • EKS

  • Lambda

  • ECS

  • IAM (in-depth)

  • CloudWatch

  • SecurityHub & GuardDuty

  • PostgreSQL: RLS, serverless Aurora

  • Pulumi: IaC

  • DataDog: SIEM

  • Okta: IdP

  • Google Workspace: IdP

  • Tailscale: networking

  • CrowdStrike Falcon: endpoint security


Our Culture
 
METR is a mission-driven organization. We believe our work can meaningfully shape humanity's future for the better, and we want to be the best people in the world doing this work. We have a tight-knit, collaborative research culture rooted in truth-seeking and integrity. We're fiercely committed to producing high-quality, trustworthy science. We're honest and transparent about our results, especially when they may go against the grain. We've earned trust as reliable partners who handle confidential information with care. We maintain a low-ego, drama-free environment focused on what matters.
 
Hybrid Preferred: Our technical team members are in our office in Berkeley 3-5 days/week. We would ideally like for you to be in person too, but we are happy to be flexible here. If you lack US work authorization and would like to work in-person, we can likely sponsor a cap-exempt H-1B visa for this role.
 
We encourage you to apply even if your background may not seem like the perfect fit! We would rather review a larger pool of applications than risk missing out on a promising candidate for the position.
 
We are committed to diversity and equal opportunity in all aspects of our hiring process. We do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We welcome and encourage all qualified candidates to apply for our open positions.

Similar Jobs

57 Minutes Ago
In-Office
126K-165K Annually
Expert/Leader
126K-165K Annually
Expert/Leader
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Leads safety, strategy, financial performance, governance, operations, customer relationships, and service delivery across a defined geographic or business area. Oversees project teams, budgets, cost-efficiency initiatives, compliance, execution quality, performance reviews, coaching, customer escalations, and adjacent sales opportunities. Requires extensive project management and leadership experience with financial responsibility, strong stakeholder management skills, and regional travel as needed.
An Hour Ago
Hybrid
San Jose, CA, USA
249K-399K Annually
Expert/Leader
249K-399K Annually
Expert/Leader
AdTech • eCommerce • Information Technology • Software • Travel • Generative AI
Lead AI-powered reporting and insights strategy, design reporting data architecture and pipelines, and implement LLM-powered workflows (RAG, prompt engineering, agentic pipelines). Integrate BI tools and enterprise AI platforms, establish governance for AI outputs, and provide technical leadership to deliver trusted executive insights and scalable self-service reporting.
Top Skills: AnthropicAws BedrockAzure OpenaiDatabricksDelta LakeFlinkGoogle Vertex AiIcebergLangchainLlamaindexLlmLookerPythonRagSnowflakeSparkSQLTableau
An Hour Ago
Remote or Hybrid
USA
Senior level
Senior level
Machine Learning • Payments • Security • Software • Financial Services
Owns the vision, customer focus, and product backlog for a near-real-time data product. Prioritizes work based on business value, leads backlog grooming, communicates product direction, and partners with Scrum Masters and development teams to ensure delivery aligns with client requirements and business objectives.
Top Skills: Agile DevelopmentData VisualizationScrumUx Design

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account