Top Data Engineer Jobs in San Francisco Bay Area, CA

Lead the reverse-engineering and migration of legacy ETL pipelines from SAP BusinessObjects Data Services and JD Edwards into dbt and Databricks architectures. Build automated data validation, maintain SQL Server write-back mechanisms, collaborate with client stakeholders, and lead and mentor a small engineering team. The role requires hands-on expertise in SQL, Python, Spark, orchestration, cloud platforms, and legacy ERP and ETL migrations.

Designs, builds, tests, and maintains scalable data infrastructure and ETL/ELT pipelines for mission partners. Uses AWS Glue, Glue Catalog, and Step Functions to integrate, migrate, transform, and validate data. Develops data models, schemas, queries, and production-ready datasets while collaborating with analytics and business teams. The role may also support AI/ML pipelines, data governance, DevOps workflows, Databricks, and real-time streaming architectures.

Architects, builds, tests, and maintains scalable data infrastructure and ELT/ETL pipelines using AWS Glue, Glue Catalog, and Step Functions. Performs data modeling, SQL querying, data migration, schema optimization, validation, and data quality improvements. Partners with analytics and business teams to transform raw data into production-ready datasets. The role may also support AI/ML pipelines, governance, access controls, DevOps workflows, and real-time data architectures.

Build and own scalable finance and revenue data pipelines, models, and datasets supporting ARR, NRR, bookings, billing, and revenue attribution. Ensure data quality, reliability, monitoring, reconciliation, and self-service access while partnering with Finance, Revenue Operations, Analytics, Product, and Engineering.

Own Eve’s data platform end to end, including ingestion, orchestration, incremental dbt models, Snowflake administration, Terraform infrastructure, observability, access controls, and pipeline reliability. Build schema-change detection, freshness SLAs, CI environments, and incident processes while managing performance and compute costs. Partner with analytics engineers and stakeholders to maintain trusted data for reporting and AI systems, using AI-assisted development and documenting platform standards.

Supports an enterprise data warehouse and analytics platform by developing data flows, models, dashboards, visualizations, automation, and actionable business insights. The role gathers stakeholder requirements, performs data cleaning and transformation, develops solutions using SQL, Python, and dbt, and builds Tableau or other BI dashboards. It also promotes analytics best practices, monitors adoption of initiatives, and collaborates with cross-functional teams and junior engineers.

Designs, builds, operates, and modernizes secure ETL/ELT data pipelines for public health interoperability. Responsibilities include using NiFi, Kafka, Python, and SQL; deploying pipelines on Azure or AWS; implementing CI/CD with GitHub Actions; monitoring with Grafana; troubleshooting production systems; supporting legacy migrations; and ensuring data security, compliance, scalability, and reliability. The role collaborates with architects, integration engineers, data scientists, DevOps/SRE, and product teams.

Leads and develops scalable Databricks data platforms and lakehouse solutions. Designs batch, micro-batch, and streaming pipelines; implements orchestration, data quality, observability, and CI/CD practices; collaborates with architects and cross-functional teams; contributes to Databricks Center of Excellence initiatives and solution accelerators.

Leads data architecture and engineering modernization across business domains. Designs and implements complex pipelines, integrations, data models, and orchestration using Snowflake, AWS, and Airflow. Establishes data quality, governance, security, testing, observability, CI/CD, and reliability practices. Provides architectural guidance, evaluates technical solutions, leads cross-team initiatives, mentors engineers, and improves platform performance and cost efficiency.

Build and maintain production ETL pipelines for structured and unstructured data, integrate machine learning models for enrichment, develop data transformations and quality monitoring, design warehouses and dimensional models, establish governance frameworks, and contribute to data architecture and platform improvements. Collaborate with data scientists, analysts, and engineering teams to deliver reliable, actionable intelligence.

Build and maintain scalable Snowflake-based ELT/ETL pipelines, production data models, and observability frameworks. Translate business requirements into reliable solutions supporting analytics, BI, and ML. Establish data quality, governance, lineage, security, and compliance practices while optimizing warehouse performance, cost, and data freshness. Collaborate with Analytics, Product, and Engineering teams, contribute to architecture decisions, evaluate Snowflake tooling, and use AI-assisted development tools to accelerate delivery.

Designs, builds, and optimizes scalable data collection, storage, access, and analytics systems. Creates and automates reliable data pipelines, supports data quality and governance, ensures SLA and performance compliance, and monitors pipeline operations. The role requires collaboration with analytics, business, and customer teams, with potential work involving AI/ML pipelines, Databricks, DevOps practices, streaming data, and event-driven architectures.

Build and maintain scalable batch and incremental data pipelines using Python, SQL, and Snowflake. Develop data warehouse models, transformations, automation, quality checks, lineage, alerting, and performance optimizations. Collaborate with analysts, data scientists, and application teams to deliver trusted datasets, while documenting pipelines, schemas, and data contracts.

Build and maintain scalable batch and incremental data pipelines using Python, SQL, and Snowflake. Develop warehouse models, transformations, automation, data quality checks, lineage, alerting, and performance optimizations. Collaborate with analysts, data scientists, and application teams to deliver trusted datasets, while documenting pipelines, schemas, and data contracts.

Designs and maintains a scalable data platform supporting analytics, machine learning, and AI workloads. Builds tooling for reliable data pipelines, semantic layers, data agents, LLM applications, and evaluation systems. Ensures data quality, availability, observability, and reliability while establishing modeling, testing, and platform best practices. Collaborates with Data Science and Product Engineering, leads solutions from design through implementation, and contributes individually in a fast-paced startup environment.

Own and scale automated patient-data ingestion and transformation pipelines from EMR systems into clinical-trial matching workflows. Design reliable asynchronous, queue-based, containerized architectures; manage schemas, validation, data quality, monitoring, and production issues. Partner with AI and product engineering teams on healthcare data foundations, APIs, scalability, and trustworthy clinician-facing workflows. Operate autonomously in a startup environment while applying strong systems thinking and pragmatic technical judgment.

Build, test, and maintain batch and event-driven data pipelines and ETL/ELT workflows. Transform and validate large datasets, troubleshoot data-quality issues, improve workflow reliability, and support platform modernization. Collaborate with engineering, product, and data stakeholders while following strong practices for version control, testing, monitoring, documentation, and code review. The role may occasionally require work outside standard business hours.

Own EULER’s data lifecycle, including instrumentation, validation, reconciliation, privacy, anonymization, canonical data modeling, analytics datasets, documentation, and production pipelines. The role requires strong SQL, software engineering practices, independent ownership, skepticism toward data quality, and clear communication. Preferred experience includes GDPR/LGPD, multi-tenant B2B SaaS, dbt, statistics, Cloudflare Workers, Postgres, Parquet, DuckDB, and Bubble.

Build and maintain scalable batch and streaming data pipelines, analytics warehouses, transformation layers, and dimensional data models. Perform SQL-based analysis, support dashboards and reporting, monitor data quality and delivery SLAs, troubleshoot incidents, and improve operational reliability. Collaborate with engineers, analysts, data scientists, and business stakeholders while contributing to documentation, governance, observability, and automation.

Build scalable data ingestion, transformation, and loading infrastructure using Spark, SQL, AWS, and Databricks. Develop entity resolution systems, CI/CD pipelines, anomaly detection, and data quality processes. Design scalable processing systems, orchestration workflows, storage patterns, and data warehouse solutions. Collaborate remotely, document technical designs, scope projects, and solve complex data engineering and data science problems.

Build and scale distributed data pipelines and architectures supporting AI products and research. Responsibilities include processing large datasets with Spark, designing AWS data systems, managing SQL and NoSQL storage, writing Python and SQL, ensuring data quality and reliability, and implementing automation, orchestration, monitoring, and scalable data operations.

Design, build, and operate scalable batch and streaming data pipelines, lakehouse and warehouse models, and data services. Own datasets end to end, improve Snowflake, Iceberg, Spark, and Flink performance and reliability, implement governance and observability, and support graph-serving data models. Partner with product and engineering teams, participate in on-call, review code and designs, and mentor junior engineers.

Build and maintain reliable data pipelines supporting money movement, fraud response, and client reporting. Integrate internal and external APIs, orchestrate workflows with Airflow on Kubernetes, and implement testing, monitoring, alerting, and observability. Debug production issues, document root causes, improve pipeline efficiency, collaborate with stakeholders, and contribute to technical designs. The role requires Python, SQL, Airflow, Snowflake, dbt, and GCP experience, with opportunities for increased end-to-end ownership.

Build, deploy, and operate production Spark pipelines on Databricks powering analytics products. Partner with product teams on pipeline requirements, data quality, freshness, monitoring, and observability. Diagnose performance and cost issues, establish scalable pipeline standards, and support streaming, logical replication, RAG pipelines, AWS infrastructure, and platform administration as needed.

Build and maintain large-scale data processing pipelines using Apache Spark (PySpark/Scala) and Python. Design ETL workflows, implement distributed data processing, use version control (Git), and work with cloud platforms (GCP/AWS/Azure).