About the company
Our client is a technology company.
The role
Raydar is recruiting for this role on behalf of our client. Run and scale highly available container clusters across several public clouds and specialist GPU providers, owning the production reliability of a model-serving platform. The work spans accelerator troubleshooting, deployment tooling, performance monitoring and automated failover. You will also partner with optimization engineers so performance gains hold up under live traffic.
What you'll do
- Build and operate highly available Kubernetes clusters across multiple public clouds and GPU providers.
- Own the production serving layer, troubleshooting GPU stacks from drivers through to the network fabric.
- Work with optimization engineers so performance improvements hold under real traffic and hardware.
- Extend deployment tooling so models roll out consistently across different accelerator types.
- Build monitoring that tracks latency, throughput and availability by model, hardware type and region.
- Design alerting, automated failover, self-healing systems and traffic routing for reliability.
Requirements
What we're looking for
- At least 5 years of infrastructure engineering experience.
- Currently at staff level or above in an individual contributor infrastructure role.
- Direct, hands-on experience running inference infrastructure day to day.
- Deep experience running production Kubernetes at scale across AWS and GCP.
- Hands-on skill debugging GPU systems, including drivers, CUDA and collective communication libraries.
- Hands-on experience with LLM serving frameworks.
- Experience with high-performance cluster networking.
- Track record of observability and reliability for latency-sensitive services, including metrics, alerting and automated failover.
Bonus points
- Experience deploying across mixed accelerator types.
- Contributions to open-source inference frameworks.
- Background on a production-scale inference platform.
Benefits
Compensation and benefits
- Base salary: USD 350,000 to 450,000
- Equity
Location and work model
- San Francisco, CA, United States
- On-site, 5 days per week in office
- Full-time
Similar Jobs
What you need to know about the San Francisco Tech Scene
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
