Top Senior Data Engineer Jobs in San Francisco Bay Area, CA

Build and maintain scalable ETL/ELT pipelines and data foundations using Databricks, dbt, and AWS. Transform raw structured and unstructured data into reliable datasets for product analytics and data science, including telemetry, streaming data, and feature engineering. Develop dashboards in Tableau and Databricks, optimize workflows, and collaborate with Product, Engineering, analytics, and data science stakeholders to support product health, adoption, and performance insights.

Design and optimize Snowflake data warehouses, dbt transformation models, and scalable ETL/ELT pipelines. Develop ingestion utilities, improve SQL and warehouse performance, establish data quality and governance controls, and maintain CI/CD workflows using Git. Collaborate with analysts, data scientists, and business stakeholders across manufacturing, supply chain, and operations. The role also supports modern DataOps practices and may involve orchestration tools and cloud platforms.

Own Orum’s data platform architecture, roadmap, reliability, and scalability. Build batch and real-time ETL/ELT pipelines using PostgreSQL, event streams, ClickHouse, BigQuery, and dbt. Administer Looker and develop LookML models, dashboards, and a trusted semantic layer. Establish data quality, governance, lineage, observability, and metric standards while partnering with Product, Finance, GTM, and Operations on analytics, AI, and business intelligence initiatives.

Designs, builds, and maintains cloud data pipelines and ETL/ELT processes supporting VA healthcare analytics. Responsibilities include patient matching, record linkage, data harmonization, quality management, documentation, performance optimization, and pipeline support. Collaborates with analysts and stakeholders to develop scalable solutions and Power BI reports using Databricks SparkSQL. The role requires federal healthcare data experience, strong SQL or Spark SQL skills, and the ability to obtain a VA Public Trust clearance.

Build and configure a government Databricks workspace, develop ingestion pipelines from enterprise systems, and prepare data for reporting. Implement Unity Catalog governance, metadata, auditing, and lineage; configure MLflow workflows; assess security controls for sensitive data; and create architecture documentation, data dictionaries, and operating procedures. The role also requires integrating legacy systems, optimizing Delta Lake pipelines, and supporting long-term platform sustainment.

Designs and maintains ELT pipelines, warehouse integrations, data models, and data quality processes. Collaborates with data scientists, analysts, engineers, and stakeholders to deliver scalable analytical solutions. Responsibilities include data integration, governance, security, performance optimization, exploratory analysis, visualization, and cloud-based API ingestion. Requires strong SQL, DBT, Snowflake, Azure Data Factory, cloud, and scripting experience, plus leadership and communication skills.

Build and maintain production data pipelines, Airflow DAGs, Snowflake infrastructure, and Data Vault 2.0 models. Ensure data quality, observability, security, governance, reliability, and cost efficiency through automated testing and CI/CD. Lead technical projects end-to-end, collaborate with stakeholders, and support SOX-relevant reporting while developing expertise in commercial insurance data and processes.

Build and operate reliable ETL and ELT pipelines across lakehouse and transactional systems. Develop data models, enforce governance, integrate diverse data sources, and support analytics, APIs, and predictive tools. Monitor performance, data quality, lineage, cost, and uptime while collaborating with data science, AI, product, and innovation teams.

As a Data Engineer, you will design, implement, and maintain data systems for Notifications at Whatnot, ensuring data quality and performance while collaborating with cross-functional teams on data architecture and workflows.

Own Serval’s complete data function as its first dedicated data hire. Build a unified warehouse, ETL/ELT pipelines, data models, quality and governance systems, documentation, and self-serve analytics. Partner with technical and business teams to define trusted metrics, consolidate product and SaaS data, and create AI-powered internal data tools. Select the data stack, establish engineering standards, and support scalable, cost-conscious infrastructure.

Designs and develops complex cloud-native data solutions, scalable batch and real-time pipelines, APIs, data models, and enterprise data platforms. Implements data governance, quality, lineage, security, observability, CI/CD, automated testing, and operational reporting capabilities. Collaborates with architects, product owners, analysts, governance, security, and platform teams on technology strategy and cloud modernization. Mentors engineers, leads technical design discussions, presents recommendations, and supports AI-ready data products and emerging development tools.

Lead design and optimization of data models and pipelines for compliance and risk; standardize metrics and documentation; build data quality, lineage, and monitoring (including AI agents for automation); manage ETL scheduling, on-call pipeline support, and collaborate with product and non-technical partners to translate business needs into automated, production-ready data solutions.

Design, develop, and optimize enterprise-scale data platforms, pipelines, data lakes, warehouses, and analytical datasets on Azure and Snowflake. Build ETL/ELT processes using ADF, Databricks, PySpark, SQL, Python, and Snowflake. Manage data models, governance, security, quality, CI/CD, cloud migrations, and performance optimization. Collaborate with stakeholders, troubleshoot production issues, document solutions, and support Agile delivery.

Designs, builds, and manages cloud-based data systems, including pipelines, data lakes, warehouses, lakehouses, databases, and ETL processes. Develops optimized Python and SQL queries, improves workflow performance, and supports data movement between on-premises and cloud environments. Collaborates with leadership, analysts, research teams, and other departments to define requirements and reporting needs. Applies data governance, access controls, security, compliance, troubleshooting, and system support practices.

Own the analytics lifecycle from data ingestion, ETL, modeling, orchestration, validation, and warehousing through BI dashboards and actionable insights. Build SQL/Python analyses and automation, define trusted metrics, quantify customer ROI, improve data quality, and support customer-facing presentations. Prototype machine learning models for product and business problems while designing scalable analytics infrastructure and partnering with Product, Engineering, and CX teams.

Build and maintain ELT pipelines, data models, semantic layers, and personalization signals for client retention and marketing tests. Partner with machine learning, product, marketing, design, and architecture teams to prepare predictive data, connect signals to CRM channels, support real-time event processing, and document a scalable data foundation. Use AI coding assistants daily to accelerate development, testing, and reviews in a fast-moving discovery environment.

Design and maintain scalable ELT/ETL pipelines, data models, ingestion frameworks, and automation systems supporting cloud cost visibility and optimization. Use Python, SQL, dbt, Airflow, AWS services, and Snowflake to process telemetry and usage data. Optimize performance, reliability, and costs while ensuring data quality, validation, auditability, and production troubleshooting. Partner with Engineering and Finance stakeholders on usage-based insights and cost optimization initiatives.

Designs and operates batch, streaming, and real-time data platforms and pipelines using Spark, Kafka, Iceberg, Airflow, and cloud infrastructure. Owns data lake evolution, data quality, observability, reliability, governance, event instrumentation, schema management, and privacy controls for PII, retention, deletion, and residency. Leads complex initiatives end-to-end, troubleshoots production issues, mentors engineers, and develops AI agent harnesses that enforce data engineering standards and quality gates.

Builds and maintains scalable data pipelines, ETL infrastructure, data warehouses, and analytics tools. The role assembles complex datasets, automates processes, optimizes data delivery architecture, and supports cross-functional data initiatives. Collaborates with executive, product, data, and design stakeholders to resolve technical issues and improve analytics systems. Requires proficiency in SAS, Python, Java, MATLAB, SQL, NoSQL, Hadoop, MapReduce, Hive, Presto, UNIX, and Linux, with JavaScript, R, and basic machine learning preferred.

Own and scale the company’s analytics infrastructure, including BigQuery data warehouses, dbt data marts, external data exchange, DataOps, data quality, governance, security, and cloud cost optimization. Partner with application database teams, analytics engineers, and data scientists to enable reliable metrics and scalable client data delivery.

Design and operate scalable data pipelines, transformation workflows, layered data models, and production data infrastructure supporting analytics, experimentation, and AI/ML. Implement monitoring, data quality, observability, and incident response processes; modernize legacy systems; collaborate with product, analytics, and engineering teams; and mentor engineers through reviews and architectural discussions.

Designs and maintains scalable cloud data architectures, pipelines, ETL processes, data warehouses, databases, and data models. Ensures data quality, accuracy, security, availability, and consistency across sources. Develops SQL queries for analytics and reporting, and documents pipelines, schemas, architectures, and models. The role requires expertise with Python, Java, Scala, R, SQL, relational and NoSQL databases, cloud platforms, and Snowflake.

Designs and delivers large-scale, governed data pipelines and infrastructure across the full data lifecycle. Responsibilities include coding in Python and SQL, optimizing Airflow, dbt, AWS, Kafka, and Iceberg/Parquet workflows, implementing data quality and observability, contributing to architecture and tool evaluations, mentoring engineers, collaborating with stakeholders, and participating in on-call support.

Design, build, and own core data models, pipelines, and infrastructure for the Eclipse Platform and Data Marketplace. Implement data quality monitoring, collaborate on governance, build dashboards, support leadership reporting, and deliver analytics and ML/NLP solutions.

Designs and manages scalable data pipelines, contracts, transformation layers, and analytics models. The role focuses on data governance, observability, lineage, configuration validation, and collaboration with product engineering teams. It requires processing large datasets, integrating cloud and on-premises industrial technologies, and implementing streaming and message-queuing solutions, ideally in cybersecurity or industrial environments.