Top Data Engineer Jobs in San Francisco Bay Area, CA

Owns the architecture and implementation of Absentia Labs’ biomedical data platform. Designs schema-driven models, scalable batch and streaming pipelines, and cloud-native infrastructure supporting machine learning workflows. Establishes data quality, validation, provenance, lineage, and versioning practices while partnering with scientists, ML engineers, and product leads. Provides architectural leadership, mentorship, and guidance on performance, cost, scalability, and correctness.

Own data pipelines into and out of the warehouse, dbt models, data integrity, governance, identity resolution, reverse ETL, and quality monitoring across growth, lifecycle, and brokerage data. Partner with product, marketing, and operations teams to establish data contracts, ensure accurate and timely business metrics, and support regulated financial data requirements.

Lead product data engineering across architecture, data modeling, pipelines, monitoring, lineage, and data quality. Build reliable analytical datasets, support experimentation and machine learning use cases, and resolve business-critical pipeline issues. Partner with product and engineering stakeholders to translate business needs into automated data solutions. The role requires ownership of end-to-end data engineering projects, on-call participation, and development of AI-assisted monitoring and automation.

Lead product data engineering initiatives by designing data models, owning critical pipelines, building data quality and lineage monitoring, and supporting experimentation and ML-ready datasets. Translate business needs into automated data solutions, maintain pipeline SLAs through on-call support, and develop AI-assisted tools for issue detection and resolution. Collaborate with Product, Engineering, Data Science, Machine Learning, and Applied AI teams across Block’s products.

Architects, deploys, and operates scalable data platform infrastructure supporting analytics. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, enforcing security and compliance, developing DBT data models, cataloging assets in OpenMetadata, writing Python automation, and mentoring engineers. The role requires extensive infrastructure and data engineering experience across cloud platforms, containers, data security, and workflow orchestration.

Design and build scalable data infrastructure, including lakehouse architecture, ClickHouse deployments, ETL/ELT pipelines, and data models connecting Postgres with analytics systems. Establish data quality, observability, performance, cost, and reliability standards. Collaborate with product and engineering teams, define data contracts and specifications, support real-time analytics, and use AI-assisted coding and LLMs to accelerate pipeline development and transformation.

Lead design and build of ETL pipelines and Gold Layer tables in Databricks; develop semantic layers, data dictionaries, and AI-ready documentation; create dashboards and ad-hoc analyses to surface marketing and sales KPIs; partner with stakeholders to translate requirements into analytics-ready datasets and enable self-serve BI and agentic AI workflows.

Build and maintain dbt data models and pipelines for billing, payments, usage, and revenue data. Ensure data accuracy, reconciliation, integrity, and timely delivery while defining metrics and monitoring systems. Develop dashboards and self-service data products, support finance data operations, and collaborate with Billing Engineering, Finance, Accounting, and Product teams. Influence billing data architecture and engineering roadmaps.

Build, optimize, and maintain batch and streaming ETL/ELT pipelines, data models, and automated workflows using AWS, Snowflake, Python, SQL, dbt, and shell scripting. Containerize and test data workloads, integrate systems, support business intelligence and API access, improve data architecture, and implement documentation and security governance. Collaborate cross-functionally on time-bound projects while developing expertise in education data and advancing CCGI’s educational equity mission.

Leads the design, development, governance, and productization of enterprise data models, pipelines, data products, and ontologies. Builds machine-learning-ready datasets, implements security and lineage controls, optimizes Databricks and Delta Lake environments, establishes data-product standards, collaborates with architects and data scientists, and mentors engineering talent. The role also defines product roadmaps, service levels, usage metrics, and continuous-improvement processes for enterprise data assets supporting AI agents, analytics, and applications.

Build and maintain data pipelines to ingest, normalize, and deliver messy healthcare data (FHIR, HL7, CCDAs, PDFs) into structured models for AI and clinical use. Implement ETL, real-time webhooks, document extraction, PII sanitization, storage on AWS, and monitor data quality while ensuring compliance with healthcare interoperability standards.

Own Mux’s end-to-end data platform, including ingestion, Snowflake administration, dbt transformations, orchestration, data quality, observability, and reverse ETL. Build semantic models supporting AI agents, establish governance and reliability standards, and partner with Engineering, Finance, Product, and GTM. Use AI coding tools to develop and maintain pipelines while validating output quality. The role requires strong SQL, Python, data modeling, cloud warehouse administration, and production data engineering experience.

The AVP, Data Engineer will design and build data ingestion, transformation, and replication pipelines using SQL, Python, dbt, and Dagster. Responsibilities include translating requirements into technical designs, developing data products, debugging production issues, supporting testing, and improving data engineering practices. The role also involves data modeling, mentoring junior engineers, conducting code reviews, and providing technical leadership while remaining hands-on. Experience with modern data platforms, workflow orchestration, cloud ecosystems, and insurance or financial services is preferred.

Design and build petabyte-scale data pipelines for genomic and imaging data, transforming public and proprietary datasets into AI-ready training data. Develop agent-based curation and quality-control systems, data cataloging tools, and reliable observable infrastructure. Collaborate with AI researchers and scientists to define data requirements and integrate diverse biological datasets while addressing performance, bandwidth, scalability, and scientific correctness.

Develop enterprise data warehouse solutions and ETL pipelines using Azure Data Factory, Databricks, Snowflake, and related technologies. Process structured and unstructured healthcare data, including HL7 and FHIR files. Build stored procedures, perform data warehouse modeling, prepare ETL specifications, and use Python, source control, and scripting tools to support data integration and engineering initiatives.

Design, develop, and maintain data pipelines, ETL processes, integrations, data infrastructure, and complex data models. Build and support data warehouses and analytics solutions using SQL, PL/SQL, Oracle, Snowflake, and cloud technologies. Collaborate with analysts, architects, developers, data scientists, and business stakeholders throughout the delivery lifecycle. Provide production support, troubleshoot issues, tune database performance, and contribute to data-driven reporting and AI tools.

Design, build, and operate large-scale ETL pipelines, data lakes, and data platforms processing billions of daily events. Own data quality, freshness, reliability, monitoring, and observability for foundational datasets. Develop self-service ETL frameworks and tooling, contribute to architecture and technical strategy, and partner with product, engineering, machine learning, data science, and leadership teams to deliver scalable data solutions.

Build and manage dbt data pipelines, canonical datasets, dashboards, reporting, and foundational data products. Partner with Product, Engineering, and cross-functional teams to define reliable metrics, data models, technical requirements, and analytics systems for consumer products. Establish data integrity standards and SLAs, enable self-service analytics, and influence Product roadmaps through data architecture expertise.

Designs, builds, and operates scalable batch and streaming data pipelines, ETL/ELT workflows, cloud solutions, and secure Python REST APIs. Develops data foundations for machine learning, generative AI, embeddings, and retrieval-based applications. Ensures data quality, security, observability, lineage, and reliability while optimizing performance and cost. Leads architecture and code reviews, establishes engineering standards, mentors engineers, and collaborates with product, analytics, data science, and machine learning teams.

Develops, architects, and maintains production ETL/ELT and C#/.NET data pipelines across cloud data platforms. Responsibilities include supporting Iceberg, DynamoDB, AWS Lambda, and Trino/Starburst systems; onboarding data sources; tuning performance; monitoring data quality; resolving incidents; coordinating schemas; documenting processes; mentoring engineers; and supporting releases, deployments, architecture, and roadmap decisions.

Build and operate data-intensive web products and APIs, extend high-throughput ETL pipelines and scrapers, ensure reliability/observability, design self-scaling services, participate in code review, and collaborate across product, data, and infrastructure teams to meet production SLAs.

Build and maintain SQL-based integrations connecting behavioral healthcare EHR/EMR systems to the Eleos platform. Investigate pipeline failures, data discrepancies, and quality issues to identify root causes. Develop reusable extraction, automation, and validation tooling; document data models, mappings, and business logic; and create standardized data models. Collaborate with platform engineers, product teams, and customer technical teams to establish secure access and deliver scalable, reliable integrations.

Design, build, and maintain scalable data pipelines, data lakes, and databases; ingest and map customer financial datasets; monitor pipeline reliability; translate business requirements into data flows and analytical insights to ensure high-quality, usable data.

Design and operate Alpaca’s scalable data platform on GCP, including lakehouse infrastructure, Kubernetes-based deployments, streaming and CDC pipelines, batch ingestion, BI access, cataloging, monitoring, and governance. Build infrastructure as code with Terraform and Ansible, operate distributed query engines and Apache Iceberg, maintain reliability practices, and collaborate with DevOps and analytics teams to support rapidly evolving data requirements.

Own and improve Life360’s Ads Data Engineering platform on Databricks. Build, operate, and optimize batch and streaming pipelines, Delta Lake tables, dbt models, semantic layers, and core metrics. Integrate advertising and business data, implement quality and governance controls, manage AWS infrastructure with Terraform, reduce compute costs, and mentor engineers. Partner with analysts, ad strategists, and sales teams to deliver trusted data products while developing AI-assisted engineering tools and workflows.