Distributed ML Systems Engineer- Inference

Together AI is seeking a Distributed ML Systems Engineer to design and build scalable machine learning systems that power our accelerated AI initiatives. This role involves developing large-scale, fault-tolerant distributed systems that handle high-load and high-performance requirements. If you are passionate about designing ML systems that operate at scale and eager to create impactful solutions, we want to hear from you. This position offers the chance to work closely with our AI researchers and infrastructure teams to ensure our systems are robust and efficient. Join us in shaping the future at Together AI!

• Design and build large-scale, distributed machine learning systems that are fault-tolerant and high-performance.
• Develop and optimize distributed processing frameworks and storage systems.
• Collaborate with researchers, engineers, and product managers to integrate ML systems into our infrastructure.
• Conduct architecture and design reviews to ensure best practices in system design.
• Implement robust monitoring and logging systems to ensure the health and performance of our ML systems.

• 3+ years of experience in building large-scale, fault-tolerant, high-performance distributed systems.
• Strong programming skills in one or more of Python, Go, Rust, or C/C++.
• Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, and storage, performance, and scale.
• Experience with cloud computing platforms (AWS, GCP, Azure etc.) and large-scale infrastructure.
• Strong problem-solving skills and ability to work in a fast-paced environment.
• Preferred: Experience with Kubernetes
• Preferred: Experience with Pytorch

Together AI is a research-drven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society. Together, we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI. Our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey to build the next-generation AI infrastructure.

We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $160,000 - $230,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy