Rust Systems Engineer - Inference

• Demonstrated proficiency in Rust programming language and ecosystem
• Strong experience with Rust frameworks namely axum, tokio, jemalloc, serde, rayon, and/or crossbeam
• Deep understanding of concurrent programming and multiprocessing patterns
• Experience developing and optimizing orchestration and scheduling algorithms
• Knowledge of distributed systems principles and design
• Experience interfacing Rust services with RPC components
• Ability to optimize systems for performance, reliability, and resource efficiency
• Solid understanding of memory management and performance profiling.

• Experience with one or more ML inference frameworks (HuggingFace TGI, NVIDIA Dynamo, Candle, Pixi, uv, llguidance)
• Background in designing and implementing high-throughput, low-latency systems
• Knowledge of disaggregated serving architectures and continuous batching algorithms for ML workloads
• Familiarity with ML/AI inference systems and performance optimization techniques
• Experience with system benchmarking and performance analysis

Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society. Together, we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI. Our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey to build the next-generation AI infrastructure.

We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $160,000 - $240,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.