Top Senior Data Engineer Jobs in San Francisco Bay Area, CA

Design, build, and operate scalable cloud data pipelines and platforms (Azure preferred). Implement DevSecOps, CI/CD, automated testing, monitoring, performance tuning, and production readiness. Mentor engineers, drive operational excellence, and translate requirements into robust technical designs. Apply healthcare domain knowledge and collaborate in agile teams to deliver secure, reliable data products.

As the founding engineer of the Data Intelligence team, you will design semantic layers, develop AI tooling, and automate data processes to drive business insights.

The Senior Data Engineer will design and implement scalable data models in BigQuery using dbt for analytics and reporting, ensuring data governance and optimal performance.

Design, build, and automate scalable data platforms (lakehouses/warehouses). Implement and optimize ETL/ELT pipelines using Databricks and dbt, apply Medallion Architecture, manage IaC/CI-CD, ensure data security and governance, and integrate pipelines with BI tools while engaging with stakeholders.

Build and maintain production-grade data pipelines and ELT processes using Python, SQL, Airflow, and dbt within AWS. Design optimized schemas in PostgreSQL and Redshift, leverage AI/LLMs to improve development, automation, testing, and data quality, and scale infrastructure for large healthcare datasets.

As a Senior Data Engineer, you will automate processes, optimize data handling, build data products, and govern data while mentoring junior engineers.

Lead design and operation of large-scale data pipelines and data APIs. Build Spark/PySpark workflows on Databricks, optimize job performance, manage data quality and observability, and develop MCP servers and AI-agent integrations. Mentor engineers, define standards, support production incidents and on-call rotations, and collaborate with stakeholders to deliver scalable data platform products.

Lead architecture and buildout of a new graph-backed enterprise data platform: design ingestion, graph and relational storage, entity resolution pipelines, temporal models, ETL/ELT pipelines, governance, APIs, and production connectors. Ship scalable graph data models, traversal queries, and platform roadmap while enabling observability, security, and containerized deployments.

Lead architecture and reliability of the data platform, build and harden ingestion (batch and CDC), own BigQuery+dbt transformations, establish testing/observability/governance, mentor engineers, develop Python tooling and APIs, run POCs for lakehouse/open table formats, and define AI-assisted development practices.

As a Data Engineer, you will develop solutions for customer-centric technology, ensuring platform integration and managing data architecture. Responsibilities include building data processes, maintaining data integrity, and collaborating cross-functionally to meet business challenges.

Design, build, and optimize enterprise-scale data pipelines on Databricks and Azure. Implement batch, streaming, CDC, and incremental ETL/ELT solutions, Delta Lake optimizations, data quality, CI/CD, and secure, production-ready data products for analytics and AI/ML initiatives.

The Scientific Data Engineer will design data architecture, build ETL/ELT pipelines, implement data quality checks, and develop semantic layers for AI-consumable data products in healthcare research.

Design, build, and operate large-scale data platforms and Spark/PySpark pipelines. Enable data integration, modeling, quality, and observability. Build MCP servers and AI-augmented tooling, mentor engineers, and lead cross-functional projects to deliver reliable data products.

Design, build, and maintain scalable Spark-based ETL pipelines and computed tables in a central data lake. Integrate structured and unstructured IoT, sensor, and external data for analytics, model training, and dashboards. Collaborate with Data Science, Analytics, and ML teams to ensure reliable, high-quality customer-facing datasets.

The role involves architecting and improving data systems, defining governance standards, optimizing performance, and mentoring engineers in data engineering.

Design, develop, and maintain enterprise data integration (ETL) systems and data platforms. Lead technical designs, optimize performance and scalability, support production issues and off-hours rotations, build reporting and BI solutions, document data processes, and collaborate with stakeholders and vendors to align business requirements.

Design, develop, and optimize ETL/data integration, data platforms, and BI solutions. Lead technical designs, troubleshoot production issues, participate in on-call support, and document processes to ensure scalable, secure, high-performance enterprise data solutions.

Design, build, and optimize scalable ETL/ELT data pipelines using Databricks and cloud platforms. Implement data quality checks, monitor and troubleshoot jobs, audit pipelines for compliance, and collaborate with analysts and business stakeholders to define data models and support reporting.

Senior Exasol data engineer to design, build, optimize, and validate SQL-based data pipelines for healthcare analytics. Responsibilities include complex query tuning, extracting/staging from SQL Server, chart validation, applying clinical terminologies, quality measure development, and maintaining disciplined version-controlled engineering practices.

Design, implement, and maintain scalable data infrastructure and real-time processing for analytics platforms. Migrate systems from AWS to on-prem Kafka/Spark, implement data transformations, support production, optimize pipelines, query and analyze data, and build scripts and APIs to operationalize data science work.

Own and build the end-to-end data layer: extraction pipelines from messy supplier emails and PDFs, the unified data model, reliable observable pipelines, and ML-driven extraction evaluation. Drive extraction accuracy, data quality, lineage, monitoring, and automation to scale coverage and enable engineering and product to depend on the data layer.

Own and scale ReadyOn's data platform: design/build real-time and batch pipelines, implement ingestion and transformation workflows, model schemas in Snowflake and PostgreSQL, ensure observability and data quality, build connectors and feature-store roadmap, and provide technical leadership and mentorship across data, AI, and backend teams.

Develop and maintain data infrastructure for safety data analysis, build scalable data pipelines, conduct exploratory analyses, and support regulatory reporting in AV safety.

As an Analytics Engineer, you will create scalable solutions for data reporting, build data pipelines, and establish high data integrity standards while collaborating with multiple teams.

Lead design and development of web UI and REST middleware within an Agile team. Define requirements and acceptance criteria, implement Java/Spring and Angular solutions, build CI/CD pipelines, work with Product Owner and testers, and produce/update design documentation.