Top Staff Data Engineer Jobs in San Francisco, CA

The role involves deep troubleshooting, root cause analysis, and architectural optimization in the Data and AI ecosystem to enhance platform reliability and supportability.

Lead design, build, and support of cloud-based ETL, data warehousing, and reporting solutions. Develop/optimize ETL pipelines, data models, and queries; deploy containerized services on Kubernetes; automate workflows; mentor teammates; collaborate with architects, data scientists, and stakeholders to deliver scalable analytics and reporting products.

Build and automate large-scale ML data pipelines and tooling for autonomous trucking. Improve dataset curation, deduplication, labeling, evaluation, and metrics infrastructure. Collaborate with robotics, autonomy, and infra teams to support continuous learning, deployment, and scalable AI infrastructure for Kodiak's fleet.

Architect and develop robust, petabyte-scale data pipelines, provide technical leadership, and collaborate with ML Engineers at Watney Robotics.

The Staff Data Engineer will design and maintain data infrastructure using Azure and Databricks, optimize data workflows, ensure data quality, and collaborate with teams for data-driven decisions.

Design and implement data-intensive platform components, mentor engineers, and optimize streaming infrastructure to enable scalable data services at Okta.

You will create large datasets and training recipes, develop methods for data mining, create scalable infrastructure solutions, and collaborate with ML infrastructure teams.

As a Staff Software Engineer, you will build distributed data systems, focusing on reliability and performance for large-scale data processing, using technologies like Apache Spark™ and Delta Lake.

Design and deploy scalable data infrastructure and models, maintain CDC pipelines, build reporting tools and dashboards, write high-performance SQL, partner with product and engineering to enable enterprise-wide analytics and data-driven decision making.

As a Staff Software Engineer, you'll design and deliver a core data platform, focusing on architecture, pipeline components, and AI integration. This hands-on role emphasizes creating production code and establishing technical standards within a small team.

Own and develop concurrent C++ backend services for Webviz to stream time-series and sensor data, integrate offboard storage and WebRTC, optimize latency and throughput, build APIs for automated triage/evaluation, plan technical roadmaps, and mentor engineers.

Build scalable ML data pipelines for Waabi's autonomous driving platform. Design, optimize, and manage datasets and training processes while collaborating with scientists and engineers.

Design and build large-scale data pipelines, lakes, and warehouses; develop ETL, data models, monitoring, and governance; collaborate with product and cross-functional teams; lead and mentor engineers while driving technical strategy for advertising data platforms.

Design, build, and operate petabyte-scale data infrastructure: real-time ingestion, storage (open table formats), and serving. Architect scalable Iceberg-based data lakes, optimize Spark pipelines and streaming (Kafka/Flink), drive reliability, cost efficiency, and data contracts; collaborate with product and platform teams and establish best practices and tooling.

Architect and build core data governance systems (fine-grained permissions, auditability, policy enforcement, metadata, labeling, compliance) for a multi-tenant platform. Drive projects end-to-end with Product, balance security and usability, own large components, mentor engineers, and ensure scalable, secure handling of billions of data points.

Design and build scalable, semantically consistent 360 entity data models and transformation pipelines. Implement cleansing, enrichment, KPIs, scoring, and business rules in SQL and Python/Scala, ensure data quality, lineage, and performance at scale. Use AI coding assistants and SDD to accelerate development, create tests and documentation, and collaborate with domain experts, data scientists, and product teams to deliver trusted, reusable data assets.

As a Senior Staff Machine Learning Engineer, you will drive AI product development, collaborate with cross-functional teams, and enhance ML models at scale.

Design and implement scalable backend systems for analytics, optimizing performance and resource efficiency while mentoring engineers and managing petabytes of data.

Build and scale AI data and ML infrastructure: evaluation systems, fine-tuning pipelines, agent-first product surfaces, high-throughput data workflows, and integrations of new models into production.

The Staff Software Engineer will design, build, and maintain scalable data infrastructure, ensure reliability, mentor engineers, and enhance data workflows in a collaborative environment.

Lead architecture and technical direction for Pinterest's conversion data privacy platform. Own design and operation of de‑identification pipelines, access controls, policy enforcement, deletion workflows, monitoring, and tooling. Partner cross‑functionally with product, data science, legal, and infra to translate privacy requirements, balance privacy and utility, and drive rollouts. Mentor engineers, lead reviews, and define privacy‑by‑design best practices for large‑scale data systems used in ads reporting and monetization.

Lead architecture and strategic direction for Pinterest's data warehouse, analytics tools, and data governance. Drive cross-functional initiatives to build scalable data platforms, AI-assisted pipeline tooling, and analytics capabilities. Mentor engineers, define policies and tooling, and deliver measurable adoption and business impact across the company.

Lead design and operation of identity resolution and data governance systems. Build batch and streaming pipelines, APIs, and tooling for identity ingestion, matching, lineage, quality, access controls, and privacy compliance (GDPR/CCPA). Collaborate with product, analytics, legal, and security to ensure privacy-by-design, monitoring, and documented runbooks.

Lead design and implementation of identity resolution and data governance systems. Build batch and streaming pipelines, APIs, and services to ingest, normalize, link, and version identity data. Ensure auditable matching, data lineage, quality checks, schema enforcement, access controls, and privacy-by-design. Collaborate cross-functionally with product, analytics, legal, privacy, and security teams and create documentation, monitoring, and runbooks.

Design and maintain a dbt-based business data layer for GTM metrics, centralize and reconcile multi-source data, implement automated data quality and reconciliation checks, align and document metrics with stakeholders, mentor analytics team, and lead root-cause investigations into complex data anomalies.