Top Lead Data Engineer Jobs in San Francisco, CA

The Lead Data Engineer will design, develop, and implement technical solutions while collaborating with Agile teams and leveraging data technologies to solve business problems.

Lead Data Engineer builds and maintains foundational, cloud-based data platforms for transaction analytics. Collaborates with Agile teams to design, develop, test, and deploy data solutions using big data, streaming, and warehousing technologies, mentors engineers, and ensures performant, production-ready code.

As the first in-house Data Engineer, you will own the data infrastructure, build and maintain data pipelines, and collaborate with teams to ensure quality data access and insights.

Lead design and implement scalable data infrastructure and transformations. Build data repositories, convert data science algorithms into robust pipelines, mentor a small data engineering team, ensure delivery quality, and collaborate with Product, Data Analysts, and Data Scientists.

Lead the development of healthcare applications and data systems, focusing on audience intelligence and compliance. Design scalable data pipelines, APIs, and reporting systems while ensuring high data quality and security in regulated environments.

Lead diverse technology projects and a team to create solutions for regulatory needs, leveraging distributed microservices and collaboration with product managers.

The Lead Data Engineer will oversee the architecture and operationalization of the Finance reporting data platform, focusing on Databricks, SQL, and Power BI. Responsibilities include designing scalable data solutions, optimizing data workflows, developing reporting datasets, implementing data quality frameworks, and leading data engineering projects.

Lead architecture and implementation of cloud-native data platforms on Google Cloud (BigQuery). Build unified batch/streaming pipelines, enforce IaC with Terraform, implement open table formats and governance (Dataplex/Data Catalog), automate data quality/lineage, partner with analytics and product teams, and mentor data engineers.

The Lead Data Engineer will architect and develop a healthcare data platform on GCP, guiding teams in pipeline development, data modeling, and ensuring compliance with HIPAA. They will collaborate across functions, set technical directions, and mentor junior engineers.

As a Lead Data Engineer, you'll optimize data workflows, manage data engineering projects, mentor junior engineers, and ensure robust data solutions meet business needs.

Lead the analytics data layer: design and operate a ClickHouse-backed warehouse, define and validate trusted metrics, build customer-facing React dashboards and internal Grafana/self-serve tooling, and ensure data quality, lineage, and documentation for telemetry-derived insights.

As a Lead Data Engineer, you will design and maintain data platforms, oversee data ingestion pipelines, enforce standards, collaborate on solutions, and ensure best practices in healthcare data operations.

Lead the design and delivery of scalable data platform services, guide engineers, and improve engineering practices while collaborating with cross-functional teams.

Lead Data Engineer responsible for defining architecture, engineering standards, and best practices for the enterprise data platform, collaborating with distributed teams for modernization and scalability.

The Data Architect/Data Governance Lead supports the Medicaid Enterprise Data Warehouse by leading data architecture, metadata management, and governance processes, ensuring compliance and scalability.

The Lead Data Engineer drives enterprise-scale data solutions, oversees project scoping, and leads a team, ensuring best practices in data engineering.

Lead the architecture and implementation of data integrations with healthcare payers, build data pipelines for AI/ML systems, and establish engineering best practices.

The Technical Lead will build software systems for AI-native drug discovery, design data pipelines, develop APIs, and collaborate with scientists to create efficient workflows and solutions.

The Senior Data Engineer will design and manage petabyte-scale data pipelines, ensuring data quality and architecture for operational decision-making and ML training.

Lead and own the data science function: design and build production ML systems (recommendation, forecasting, segmentation), develop configurable multi-tenant models, run rigorous experimentation and causal analyses, connect model outputs to business impact, set scientific standards, and hire and mentor a high-velocity data science team.

Lead data engineering for a healthcare scheduling platform rebuild: map legacy multi-TB schema, design target schema, build translation layer (views/materialized views and write mappers), ensure data consistency during parallel systems, plan per-tenant cutover with verification and rollback, tune DB performance, instrument data quality/lineage/audit, and mentor engineers.

The Data Engineer will design and manage data pipelines, develop datasets for product metrics, and collaborate with teams to ensure robust data integration and processing.

The Scientific Data Engineer will design data architecture, build ETL/ELT pipelines, implement data quality checks, and develop semantic layers for AI-consumable data products in healthcare research.

The Lead AI Engineer will develop AI and ML systems with a focus on intelligent decision-making and data pipelines to improve agent performance and continuous learning.

The Lead Data Platform Engineer will design and build Virtuous's data platform, ensuring it is secure, scalable, and AI-ready. Responsibilities include architecting access controls, optimizing performance, and enabling self-service analytics while maintaining governance and compliance standards.