Responsible for defining product roadmaps for AI inference capabilities, collaborating with engineers to optimize resources and improve latency and throughput. Engage with technical users and validate features with customers.
About Alluxio:
Alluxio powers the data layer for modern AI and analytics. Proven in production at eight of the top ten internet companies and seven of the ten highest-valued enterprises globally, Alluxio’s open-source data-orchestration platform unifies data across storage systems, regions, and clouds — enabling memory-speed access for large-scale AI and analytics workloads.
Spun out of UC Berkeley’s AMPLab and backed by Andreessen Horowitz, Hillhouse Capital, and Seven Seas Partners, Alluxio sits at the intersection of data, distributed systems, and AI infrastructure.
Our technology is deployed at scale by Meta, Uber, TikTok, Alibaba, Microsoft, and Salesforce, orchestrating data for billions of operations per day.
Learn more at alluxio.io or on Wikipedia.
The Role
We’re hiring a Technical Product Manager to define and execute Alluxio’s AI systems strategy — spanning inference, training, and emerging agentic workloads.
This role bridges the worlds of AI infrastructure and distributed data systems, guiding how Alluxio evolves to serve next-generation model architectures and large-scale data flows.
You’ll partner with engineering, research, and enterprise AI teams to build features that improve latency, throughput, and GPU efficiency for model inference and training — and design the data layer powering the shift toward autonomous, agentic AI systems.
What You'll Own
- AI Product Strategy — define the long-term vision and roadmap for Alluxio’s AI data platform, covering inference, training, and agentic workloads.
- Systems Optimization for AI — collaborate with engineering to design features that deliver high-throughput, low-latency data access (e.g., GPU-aware caching, streaming reads, tiered prefetching).
- AI Ecosystem Integration — ensure seamless integration with frameworks like PyTorch, TensorFlow, Ray, and Triton; evolve Alluxio’s APIs for AI-native workloads.
- Customer & Partner Collaboration — engage directly with enterprise AI teams to understand workload patterns, validate impact, and prioritize roadmap direction.
- Market Awareness & Foresight — stay ahead of trends in multi-model serving, retrieval-augmented generation (RAG), and agentic orchestration; translate them into actionable product plans.
What We’re Looking For
- 5–9 years of experience in product management or technical leadership within AI infrastructure, ML platforms, or distributed systems.
- Strong understanding of AI/ML workflows — from model training and deployment to inference and data-access pipelines.
- Proven track record of delivering infrastructure features that improve latency, GPU utilization, or total cost of ownership.
- Technical fluency with distributed systems, caching, and cloud orchestration (Kubernetes, AWS/GCP/Azure).
- Familiarity with AI frameworks such as PyTorch, TensorFlow, Triton, Ray, or LangChain.
- Exceptional communication and strategic thinking — ability to translate complex systems work into clear, prioritized roadmaps.
Why Join Alluxio?
- Shape how the world’s most advanced AI systems access and process data.
- Work at the intersection of distributed systems, AI acceleration, and open source.
- Collaborate with world-class engineers, researchers, and customers driving the AI frontier.
- Competitive compensation and equity package with comprehensive benefits.
- A culture built on curiosity, empathy, and deep technical rigor.
Alluxio is an equal opportunity employer and does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, retaliation, parental status, military service, or other non-merit factors.
The salary range for this full-time position in the United States is $190,000 - $240,000 depending on experience and level, subject to standard withholding and applicable taxes. All candidates will also receive equity (stock options) and access to a comprehensive benefits package. The base salary range reflects the minimum and maximum target for candidates across all US locations. The specific compensation awarded will be based on factors such as work location, skills, experience, and relevant education or training. Our Recruiting Team or Hiring Manager will provide more details about the specific salary range during the recruitment process.
Top Skills
AWS
Azure
GCP
Kubernetes
PyTorch
TensorFlow
Triton Inference Server
Alluxio San Mateo, California, USA Office
1825 S Grant St, Suite 800, San Mateo, CA, United States, 94402
Similar Jobs
Angel or VC Firm
Lead a team managing production support for Workday Financials, ensuring stability and reliability while driving improvements and innovations.
Top Skills:
AzureCore ConnectorsData FactoryEibsFunctionsLogic AppsService BusStudioWorkday FinancialsWorkday Integrations
Angel or VC Firm
Lead a team to enhance the Workday Financials platform, collaborating with stakeholders for requirements, managing enhancements, and promoting innovation.
Top Skills:
AIAzure Integration TechnologiesReporting ToolsWorkday FinancialsWorkday Integrations
Healthtech • Information Technology • Biotech
The Technical Product Manager leads product strategy and lifecycle management for EHR interconnect services, enhancing interoperability and user experience in healthcare. Collaborates with cross-functional teams to drive adoption of standards and ensure compliance, while mentoring junior staff and managing Agile processes.
Top Skills:
APIsAWSEhr SystemsFhirGCP
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
