About the company
Our client is a software company.
The role
Raydar is recruiting for this role on behalf of our client. Own the systems that run and train language models, from early experimentation through live production use. The focus is on speed, dependability and efficient hardware spend, working side by side with senior technical leaders and researchers in a small team.
What you'll do
- Take wide ownership of the compute platform used to train and serve models, covering both research and production needs.
- Measure and reduce response times, processing capacity and spend on accelerators under realistic load.
- Refine how requests are grouped, how results are stored for reuse and how memory is managed, and test compression and faster decoding approaches where they pay off.
- Support model inference for evaluation runs, synthetic data creation and reinforcement learning workloads.
- Build dependable release and observability workflows for models.
- Work with founders and researchers to split GPU capacity between training and serving, and to roll out models in customer environments.
Requirements
What we're looking for
- Professional experience at a company building or tuning GPU-based infrastructure for large language models in production; coursework or personal projects alone are not enough.
- Deep expertise in diagnosing GPU performance limits, tuning model serving and setting up monitoring.
- Solid debugging and low-level systems skills, with a good grasp of accelerator memory and compute limits.
- Track record of owning an infrastructure system from start to finish, ideally launching one from scratch at a startup or in a small team; strong infrastructure engineers from large technology companies are also welcome.
- Hands-on work with an LLM serving framework or a comparable in-house engine, using PyTorch or JAX.
- Sound judgment in balancing model accuracy against speed, scale and cost to pick pragmatic solutions.
Bonus points
- Experience at an AI-focused startup that trains its own models.
- Low-level optimization experience, such as custom GPU kernels, quantization, speculative decoding, or distributed and multi-model serving.
- Technical degree from a leading program.
- Publications at top machine learning conferences or public writing about systems work.
Benefits
Compensation and benefits
- Base salary: USD 200,000 to 350,000 per year
- Equity
Location and work model
- California, United States
- On-site, 5 days a week in the office
- Full-time
Similar Jobs
What you need to know about the San Francisco Tech Scene
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
