ReflectionAI Logo

ReflectionAI

Human Data Operations Manager

Posted 15 Days Ago
Be an Early Applicant
In-Office
San Francisco, CA, USA
Mid level
In-Office
San Francisco, CA, USA
Mid level
Own human data acquisition programs for AI model development. Translate research needs into sourcing strategies, coordinate vendors, define task and quality requirements, monitor campaign metrics, investigate bottlenecks, and optimize data workflows. The role requires SQL and Python for analytics, dashboards, API integrations, and automation, plus familiarity with LLM training, evaluation, and prompt design.
The summary above was generated by AI
Our Mission

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.

Role Overview

The Data Acquisition Team is responsible for ensuring that Reflection has all the data it needs to train the world’s best models. We're looking for a Human Data Operations Manager to own the pipelines that turn human expertise into better AI models. You will partner closely with researchers, deeply understand their needs, and source from vendors the data that improves our models’ usefulness and performance.

This role combines program ownership with hands-on work on task design, contributor workflows, data quality frameworks, and success metrics. The ideal candidate brings organizational rigor, technical curiosity, and sound judgment about when to move quickly and when to investigate the details.

What You'll Do
  • Collaborate with research: Translate critical model capabilities, and researchers’ plans for developing them, into executable data acquisition strategies.

  • Orchestrate providers: Assess vendors’ capabilities, forecast capacity, define task instructions, acceptance criteria and then hold vendors accountable for quality, delivery, and cost.

  • Track and optimize campaigns: Build dashboards tracking throughput, turnaround time, acceptance rates, rework, and cost per accepted task. Leverage data analytics to investigate bottlenecks and guide improvements.

  • Ensure alignment: Experiments and priorities evolve. You will surface vulnerabilities early and turn feedback into concrete changes to subsequent sourcing. Work with research to close the loop; tracking how our data acquisitions improve model capabilities

What We're Looking For
  • 4+ years of experience delivering a portfolio of complex programs and prioritizing across changing research needs.

  • An understanding of LLM training, evaluation, and prompt design, with the ability to learn unfamiliar research concepts quickly.

  • Proficiency in SQL for querying datasets and building dashboards, and Python for data analysis, API integrations, and workflow automation.

  • Clear written communication, strong stakeholder judgment, and a willingness to inspect individual tasks and datasets when the headline numbers do not tell the full story.

  • High agency, bias towards action, and the ability to drive impact in fast-paced, dynamic, and ambiguous environments

  • Experience with annotation, expert data collection, human feedback, or AI evaluation is a plus.

What We Offer:

We believe that to make intelligence open and accessible to all, you need to start at the foundation. Joining Reflection means building from the ground up as part of a talent-dense team. You will help define our future as a company, and help define the future of open foundational models.

We want you to do the most impactful work of your career with the confidence that you and the people you care about most are supported.

  • Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.

  • Stock options: Everyone who joins and contributes to Reflection's success gets to share in the upside through stock options.

  • Health & wellness: Comprehensive medical, dental, vision, and life, with an annual wellness allowance.

  • Meals: Lunch and dinner are provided in the office daily.

  • Life & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.

  • Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.

  • Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.

  • Team building: We have regular off-sites, happy hours, and team celebrations.

Export Control Notice: This position may require access to technology or source code subject to the U.S. Export Administration Regulations. Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export control laws, which may require the Company to seek government authorization.

Equal Employment Opportunity Statement
We are an Equal Opportunity Employer. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We are committed to creating a diverse and inclusive environment and encourage applicants from all backgrounds to apply.

Candidate and Employee Privacy Notice: We collect and process personal data about applicants for the purposes described in our Candidate and Employee Privacy Notice

Similar Jobs

5 Days Ago
In-Office
San Francisco, CA, USA
270K-365K Annually
Mid level
270K-365K Annually
Mid level
Artificial Intelligence • Natural Language Processing • Generative AI
Build and scale human-data operations supporting frontier AI research. Own data strategy, vendor partnerships, collection pipelines, evaluation frameworks, quality standards, and process improvements across RLHF, safety, tool use, and agentic workflows. Partner with researchers and senior leaders to align data operations with model roadmaps, identify risks and bottlenecks, and improve research velocity while maintaining rigorous data quality.
Top Skills: Constitutional AiHuman-In-The-Loop WorkflowsLarge Language Models (Llms)PythonReinforcement Learning From Human Feedback (Rlhf)SpreadsheetsSQLTableau
An Hour Ago
Hybrid
125K-140K Annually
Senior level
125K-140K Annually
Senior level
eCommerce • Fashion • Retail • Sales • Wearables • Design
Lead strategy and execution of ecommerce discovery experiences across brands. Own roadmap, research, experimentation (A/B tests), analytics, and cross-functional delivery with UX, design, engineering, and brand teams to drive measurable product impact and incorporate AI and emerging discovery technologies.
Top Skills: A/B TestingAIFigmaGa4JIRATableau
An Hour Ago
Remote or Hybrid
United States
54K-68K Annually
Junior
54K-68K Annually
Junior
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Engage high-potential and prospective VIP players across communication channels, build relationships, evaluate VIP program eligibility, and optimize player engagement and onboarding strategies. The role supports acquisition efforts for new state launches, gathers customer feedback, and requires adaptability, travel, nights, weekends, and state gaming licensure.

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account