Turing’s mission is to accelerate superintelligence to drive real economic progress. Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality datasets, reinforcement learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work, and advanced STEM reasoning. In software engineering, Turing is the largest and longest-running data provider in the category. Turing also works with Fortune 500 enterprises across financial services, life sciences, healthcare, retail, automotive, and CPG to build and deploy end-to-end agentic AI systems inside mission-critical workflows. By operating on both sides, Turing closes the loop between frontier research and enterprise deployment, turning real-world deployment signals into better data, evaluations, and more capable models. Learn more at www.turing.com.
Overview
We are seeking a seasoned techno-functional leader to drive the development and execution of large-scale LLM training programs. This leader would partner with our clients (leading LLM labs) research teams to:
- Identify opportunities for building training datasets to improve model capabilities and performance
- Generate these datasets with high quality and speed
- Build automation tools and processes for scalability
- Deliver the datasets so that they are easily usable by our clients
Key Responsibilities
Operational Leadership & Performance Management
- Lead and scale global delivery teams of 100+, distributed across functions, regions, and levels (ICs, leads, and managers)
- Implement performance management systems that go beyond managerial reporting using data-driven metrics, tools, and products to assess productivity, quality, and output consistency
- Build strong operational structures that allow for transparency, accountability, and early detection of underperformance
- Partner with cross-functional leads to optimize workflows and improve internal tool adoption for delivery efficiency
Data Quality & Scripting-Driven Automation
- Own the quality, accuracy, and scalability of data generated for LLM training
- Move beyond manual QA layers by leveraging Python scripting, APIs, and automation frameworks to measure, validate, and improve dataset integrity
- Design and oversee tools or scripts for data validation, annotation accuracy checks, and pipeline consistency
- Ensure datasets adhere to compliance standards (PII, GDPR, HIPAA) and can be programmatically tested for usability and quality
LLM Training & Evaluation
- Lead generation and delivery of high-quality, scalable datasets focused on SFT, RLHF, reasoning, and agentic workflows
- Oversee the entire data lifecycle from client intake and annotation workflow design to delivery
- Partner with product, research, and engineering teams to implement evaluation metrics (e.g., win rate, inter-annotator agreement, and pairwise preference scoring)
Client Partnership & Communication
- Serve as the primary point of contact for enterprise AI clients; manage expectations, delivery timelines, and escalations
- Build relationships with engineering and research stakeholders by delivering consistently high-quality data
- Communicate effectively across technical and non-technical audiences; provide transparency through structured updates and quality reporting
Team Development & Tooling
- Recruit, mentor, and coach cross-functional leaders (Eng, Data, Ops, and Program Management)
- Drive adoption and improvement of internal tools (e.g., task management systems, quality dashboards)
- Champion continuous improvement across data quality, tools, and delivery processes
Required Qualifications
- 10+ years of experience leading large-scale technical delivery organizations, ideally across AI, ML, or data operations
- Bachelor's degree in Engineering, Computer Science, or equivalent technical discipline
- Demonstrated ability to act as a strategic business partner with our clients, researchers, and engineers at leading LLM labs
- Proven success in building and scaling multi-level high performance teams, with distributed global operations
- Experience managing managers
- Skip-level performance management
- Hands-on technical fluency: ability to write and review data validation scripts
- Demonstrated experience managing dataset generation or annotation for machine learning model evaluation and/or training
- Familiarity with ML tools and data workflows (e.g., HuggingFace, LangChain, Weights & Biases, Databricks)
Preferred Qualifications
- Experience evaluating large language model performance and/or improving model performance via fine-tuning
- Strong understanding of data quality frameworks, including automation, toolings and manual processes
- Experience in AI data annotation, model evaluation, and fine-tuning platforms
- Strong communication and storytelling skills with executive stakeholders
Location SF Bay Area (Hybrid)
Compensation: $255,000 to $325,000 OTE + Equity
- We are client first: We put our clients at the center of everything we do, because their success is the ultimate measure of our value.
- We work at Start-Up Speed: We move fast, stay agile and favor action because momentum is the foundation of perfection
- We are AI forward: We help our clients build the future of Al and implement it in our own roles and workflow to amplify productivity.
- Work at the frontier of AI, helping the world’s leading AI labs improve their most advanced models by building expert datasets, RL environments, and first-of-a-kind benchmarks.
- Contribute to leading-edge AI research and showcase your work at top conferences such as ICLR, ICML, and NeurIPS.
- Bring frontier AI innovation to the enterprise, applying lessons learned from leading AI labs to solve real-world business challenges.
- Collaborate with and learn from exceptional colleagues with deep AI experience from Google, Meta, Amazon, and other leading technology companies.
- Move at the pace of AI innovation, with the speed, ownership, and impact of a startup.
Turing is proud to be an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, gender identity, sexual orientation, age, marital status, disability, protected veteran status, or any other legally protected characteristics. At Turing we are dedicated to building a diverse, inclusive and authentic workplace and celebrate authenticity, so if you’re excited about this role but your past experience doesn’t align perfectly with every qualification in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.
For applicants from the European Union, please review Turing's GDPR notice here.
Turing Palo Alto, California, USA Office
1900 Embarcadero Rd, Palo Alto, CA, United States
Similar Jobs
What you need to know about the San Francisco Tech Scene
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
