Top Senior Data Engineer Jobs in San Francisco Bay Area, CA

As a Data Engineer, you will design, implement, and maintain data systems for Notifications at Whatnot, ensuring data quality and performance while collaborating with cross-functional teams on data architecture and workflows.

Lead design and implementation of enterprise data architecture and cloud-based data solutions. Manage large data engineering projects, develop data models and pipelines, ensure governance and security compliance, advise clients strategically, and build high-performing inclusive teams.

Lead design and implementation of data architecture, ETL/ELT pipelines, data models, warehouses and lakes. Ensure data governance, security, cloud deployments, performance and cost optimization. Coach and manage team members, partner with stakeholders, and drive data-driven solutions for clients.

Design and implement data architecture and scalable ETL/ELT pipelines on cloud platforms (AWS/Azure/GCP). Analyze complex problems, mentor junior staff, ensure data governance/security compliance, liaise with stakeholders, and deliver high-quality client solutions while developing technical and client-facing skills.

Design and build data infrastructure and pipelines using cloud and big-data tools. Develop data lakes/warehouses, ensure data quality and security, collaborate with clients, and apply data architecture, transformation, and analytics to deliver insights.

The Data Engineer will architect and develop robust data frameworks for data platforms, focusing on data transformations and automated workflows. Responsibilities include collaborating with stakeholders, ensuring data quality, and maintaining high engineering standards.

Own and evolve the data lake and analytics stack powering fleet observability. Build and maintain scalable ETL/ELT pipelines, operate data lake infrastructure (Iceberg, Trino, Airflow, Spark, Superset), produce analytics and executive reporting, create dashboards and runbooks, and implement data governance and observability.

Design and maintain data architecture and pipelines to support compliance and risk teams. Build and optimize data models, standardize metrics, and create data dictionaries. Implement data quality, lineage monitoring, AI-driven agents for false-positive reduction and automation, and participate in on-call rotations to ensure SLAs are met.

Own and scale end-to-end data infrastructure: build production ETL pipelines, design scalable schemas, enforce data quality/governance/security, create self-serve data models for analytics and ML, instrument observability, and partner cross-functionally while participating in on-call incident response.

Lead design and implementation of enterprise data platforms, real-time pipelines, feature stores, and agentic data infrastructure. Ensure data governance, quality, lineage, and reliability while partnering with scientists, engineers, and product leaders. Own technical roadmap, mentor engineers, and drive platform scalability and engineering best practices.

Own and scale the data platform powering ReadyOn's real-time labor OS. Design and operate batch and real-time pipelines, transformations, and data services (Python, PySpark, Airflow, Glue, Snowflake, Postgres). Build connectors, observability, data quality, and feature-store infrastructure while partnering with AI, backend, and product teams and providing technical leadership and mentorship.

Lead design and optimization of data models and pipelines for compliance and risk; standardize metrics and documentation; build data quality, lineage, and monitoring (including AI agents for automation); manage ETL scheduling, on-call pipeline support, and collaborate with product and non-technical partners to translate business needs into automated, production-ready data solutions.

Own and architect large-scale data pipelines, orchestration, and integrations. Drive IaC strategy (Terraform), third-party API and AI/LLM tool integrations, build export pipelines and internal tooling, mentor engineers, and align cross-team technical strategy for reliable, scalable data infrastructure.

Build and operate scalable data ingestion, transformation, and connector frameworks; design and maintain a knowledge-graph-based data fabric; normalize and vectorize enterprise data for LLM/AI workflows; implement governance, lineage, access controls, and secure APIs to serve ML/agent pipelines.

As a Data Engineer, you'll build data infrastructure for analytics, drive business growth strategies, and enhance personalized user experiences at Eight Sleep.

Design, build, and operate scalable PostgreSQL databases and cloud-native data infrastructure. Build and maintain ETL/ELT pipelines, optimize query and server performance, ensure high availability and disaster recovery, improve observability, troubleshoot production issues, guide schema and application design, evaluate database technologies, and mentor junior engineers to support reliable, secure data-driven applications.

Lead design and scaling of the data platform and analytics pipelines. Build and maintain ETL/ELT workflows, integrations, serverless data services, and reliable data models. Mentor teammates, improve automation, observability, and data governance to support analytics and operational insights.

Design, build, and maintain ingestion and transformation pipelines for a knowledge-graph analytics platform. Profile and prepare structured and semi-structured source data, ensure data lineage and traceability, implement transformation logic for graph loading, collaborate with data scientists on graph modeling and entity resolution, improve pipeline reliability, and document ingestion patterns using version control and issue tracking.

Design, build, and maintain Keboola-based data pipelines integrating multiple business systems to deliver accurate, scalable data for supply chain, logistics, and transportation analytics; collaborate with analysts and engineers; support documentation, governance, and pipeline automation.

Own and evolve the data infrastructure powering identity and fraud products. Build and operate scalable ETL/ELT pipelines, ensure data quality and observability, optimize storage and access, support product squads, respond to production incidents, mentor engineers, and drive data platform innovation.

Lead the Data Platform Engineering team owning ETL/reverse-ETL pipelines, replication, and production datastores. Define architecture, roadmap, standards (IaC, observability, schema contracts), mentor engineers, drive incident response and durable fixes, and partner on cross-team technical decisions and operational reliability for data correctness, completeness, and timeliness.

Own and optimize data infrastructure and ETL/ELT pipelines using SSIS and Microsoft Fabric. Build and maintain SQL Server data warehouse staging, transformations, and lakehouse ingestion. Implement CI/CD, version control with GitHub, monitor pipeline health, troubleshoot data issues, document data flows, and translate stakeholder requirements into data engineering solutions that support BI reporting.

The role involves architecting intelligence infrastructure, building data pipelines, collaborating with other teams, and ensuring data accessibility and security for threat detection systems.

Design and implement scalable data models and ETL pipelines (batch and streaming), ensure data quality/SSOT, collaborate with product and platform teams to resolve issues, contribute to data governance, and explore practical integrations between data engineering and AI.

Lead design and delivery of scalable, secure cloud data pipelines and products. Translate requirements into architecture, apply advanced data modeling (Data Vault 2.0/dimensional), embed data quality, observability, security, and DataOps/CI-CD practices, optimize performance and cost, mentor engineers, and enable AI/ML-ready datasets within an agile DevSecOps model.