Top Staff Data Engineer Jobs in San Francisco, CA

Lead design and operation of a cloud-native data platform: set technical direction, build scalable ingestion/processing/warehousing pipelines (batch & streaming), support ML data needs, mentor engineers, drive projects end-to-end, run planning/standups/on-call, and eventually lead a data engineering team.

Own production-grade analytics pipelines from design through maintenance, build dbt-based analytical models using Kimball dimensional modeling, and deliver reliable data features. Partner with Product, Engineering, and Data Science teams to translate retail signals into business insights. Establish standards for monitoring, documentation, reproducibility, and code quality. Use AI tools to accelerate development and prototype conversational data experiences for natural-language retail data exploration.

Own and scale the company’s analytics infrastructure, including BigQuery data warehouses, dbt data marts, external data exchange, DataOps, data quality, governance, security, and cloud cost optimization. Partner with application database teams, analytics engineers, and data scientists to enable reliable metrics and scalable client data delivery.

Leads the design and delivery of scalable data pipelines, dbt models, orchestration workflows, and analytical data models. Establishes data architecture, engineering standards, quality frameworks, governance, observability, and self-service analytics practices. Partners with stakeholders, leads technical reviews, mentors engineers, supports hiring, oversees analyses, and participates in incident response and continuous improvement.

Lead architecture and implementation of fleet telemetry, time-series storage, streaming pipelines, and CRM/warehouse integrations on AWS. Build tools for proactive fleet health, partner cross-functionally (firmware, manufacturing, ops), support on-call as systems mature, and design LLM-driven automations to increase team leverage.

Lead design and evolution of SentiLink's data platform for fraud detection. Architect scalable batch and streaming pipelines, improve reliability, performance, and observability, set engineering standards, mentor engineers, drive cross-team technical initiatives, participate in production support/on-call, and evaluate build-vs-buy and emerging technologies.

Own and scale automated patient-data ingestion and transformation pipelines from EMR systems into clinical-trial matching workflows. Design reliable asynchronous, queue-based, containerized architectures; manage schemas, validation, data quality, monitoring, and production issues. Partner with AI and product engineering teams on healthcare data foundations, APIs, scalability, and trustworthy clinician-facing workflows. Operate autonomously in a startup environment while applying strong systems thinking and pragmatic technical judgment.

Own the architecture and technical direction of Northbeam’s data ingestion and integration platform. Design scalable API-based ETL, event-driven and batch processing systems; establish standards for observability, freshness, and failure handling; and guide build-versus-buy decisions. Collaborate across engineering, infrastructure, product, and business teams while mentoring engineers, leading design reviews, and driving complex platform initiatives to completion.

As a Staff Data Engineer, you'll architect a scalable data platform, optimize data pipelines, establish data standards, and mentor teams to drive technical excellence.

Lead the technical strategy for Coinbase's Data Platform, architecting data systems for AI/ML workloads and ensuring data reliability, performance, and cross-functional alignment.

Design and build scalable backend services, data-processing workflows, APIs, databases, cloud infrastructure, and AI-powered systems. Lead architecture and technical strategy, own reliability and production operations, mentor engineers, and guide cross-team initiatives from design through deployment. The role focuses on transforming unstructured customer data into reliable structured outputs and developing LLM-powered and agentic workflows.

Design, build, optimize, and scale Ray Data’s distributed data processing infrastructure for large-scale AI training and inference. Responsibilities include distributed execution, scheduling, resource management, data partitioning, fault tolerance, performance optimization, and system architecture. The role also involves collaborating with customers and AI-native companies to solve complex workload-scaling challenges.

Build and own the data platform backbone: design ingestion patterns, Snowflake warehouses, governance, and freshness strategy. Implement reliable batch and streaming pipelines, data quality, observability, and onboarding processes. Lead legacy migrations, create data contracts and SLAs, and apply AI-native tooling to automate debugging, documentation, and operational workflows while partnering with Analytics, Engineering, and vendors.

Lead design and ownership of a modern data platform: ingestion, Snowflake governance, pipeline reliability, migrations, data contracts, observability, and AI-enabled tooling. Partner with Analytics, Engineering, and vendors to deliver trustworthy data products and operate production pipelines.

The role involves architecting and improving data systems, defining governance standards, optimizing performance, and mentoring engineers in data engineering.

Own Orum’s data platform architecture, roadmap, reliability, and scalability. Build batch and real-time ETL/ELT pipelines using PostgreSQL, event streams, ClickHouse, BigQuery, and dbt. Administer Looker and develop LookML models, dashboards, and a trusted semantic layer. Establish data quality, governance, lineage, observability, and metric standards while partnering with Product, Finance, GTM, and Operations on analytics, AI, and business intelligence initiatives.

Own and drive the architecture and roadmap of ClickUp's data platform. Build scalable, reliable data pipelines and AI/ML infrastructure using AWS serverless, Snowflake, dbt, and Terraform. Lead cross-team technical initiatives, optimize cloud costs, establish engineering standards (observability, testing, CI/CD), mentor senior engineers, and influence org-wide architecture decisions.

Founding data engineer to design and build Payabli's data platform: architect lakehouse/warehouse, build batch and streaming pipelines, model canonical datasets, ensure data quality/observability, enforce access/masking/lineage for regulated financial data, and enable analytics/ML feature pipelines while establishing team standards and CI/CD.

Lead operational reliability and platform enablement for Databricks: build monitoring, CI/CD, deployment standards, compute and job policies, observability, runbooks, and governance to support secure, cost-aware, production data workloads across regulated environments. Mentor engineers and align platform with cloud/infrastructure and compliance requirements.

Lead architecture and evolution of Headway's data platform (warehouse, ingestion, orchestration, CI/CD, monitoring, cloud infra). Serve as technical anchor across analytics, product, and ML teams, drive platform roadmaps, set standards, mentor engineers, and own end-to-end infrastructure decisions for scale and performance.

As a Senior Staff Data Engineer, you'll set the technical vision for data architecture, lead initiatives across teams, mentor engineers, and ensure platform reliability, while integrating data systems with ML pipelines effectively.

Own the 2-3 year architecture and reliability of high-volume catalog, pricing, and inventory ingestion. Lead migrations from legacy batch to modern systems, define data SLAs, build observability and orchestration, mentor senior engineers, and resolve cross-team, high-severity data problems.

Own the architecture and implementation of production data ingestion systems for manufacturing data. Build scalable backend services and cloud infrastructure using Python, PostgreSQL, and Kubernetes; solve integrations across heterogeneous enterprise systems; establish validation, lineage, monitoring, and alerting; and partner with Engineering, Product, and customer-facing teams. Provide technical mentorship, influence system design, and balance rapid delivery with production-grade reliability.

Build and operate full-stack interfaces, backend services, APIs, and data pipelines that collect human feedback for reinforcement learning. Partner with researchers to launch data collection campaigns, improve reliability and latency, create monitoring and inspection tools, and move high-quality feedback into training systems. Own projects end to end, make architectural decisions, and improve the usability and throughput of researcher- and annotator-facing tooling.

Build and operate full-stack interfaces, backend services, APIs, data pipelines, dashboards, and monitoring for human feedback and RL data collection. Partner with RL researchers to translate data needs into reliable collection campaigns and training-ready datasets. Own projects end to end, improve system latency and usability, and identify bottlenecks between data collection and model training.