Build and maintain scalable ETL/ELT pipelines, integrate data from APIs, databases, and third-party systems, and develop reliable data models for analytics and reporting. Ensure data quality, governance, security, and accessibility while supporting batch and real-time processing. Monitor pipeline performance, troubleshoot failures, and collaborate with Product, Analytics, Engineering, and Cloud teams to improve data architecture and enable data-driven decisions.
This is a remote position.
The Data Engineer will be responsible for building reliable data pipelines, integrating multiple data sources, and ensuring high-quality, accessible, and well-structured data for analytics and product teams. The role works closely with Product, Analytics, Engineering, and Cloud teams to enable data-driven decision-making.
Key Responsibilities
- Design, build, and maintain scalable data pipelines and ETL/ELT processes.
- Integrate data from multiple sources, including APIs, databases, and third-party systems.
- Develop and optimize data models for analytics and reporting use cases.
- Ensure data quality, integrity, and consistency across systems.
- Support real-time and batch data processing workflows.
- Collaborate with Product and Analytics teams to define data requirements.
- Monitor and troubleshoot data pipeline performance and failures.
- Implement data governance, security, and access controls.
- Support BI and analytics teams with clean and structured datasets.
- Continuously improve data architecture, performance, and scalability.
Required Qualifications
- 6-8 years of experience in data engineering or related roles.
- Strong understanding of data structures, databases, and data modeling.
- Experience building ETL/ELT pipelines in production environments.
- Good problem-solving and analytical skills.
- Experience working in Agile and cross-functional teams.
- Strong communication and documentation abilities.
Technical Skills
- Programming: Python, SQL (advanced level)
- Data Processing: Spark, Kafka, or similar distributed systems (preferred)
- Databases: PostgreSQL, MySQL, MongoDB, or equivalent
- Data Warehousing: Snowflake, BigQuery, Redshift (preferred)
- Cloud Platforms: Amazon Web Services / Azure / GCP
- ETL/ELT Tools: Airflow, dbt, or similar
- Version Control: Git / GitHub / GitLab
- BI Tools: Tableau, Power BI, or similar (supporting role) Tableau
Similar Jobs
Blockchain • Fintech • Payments • Financial Services • Cryptocurrency • Web3 • Infrastructure as a Service (IaaS)
Build and maintain scalable batch and streaming data pipelines for payments, ledger, transaction, and blockchain data. Own ingestion from databases, files, APIs, and onchain sources; develop incremental dbt models; orchestrate workflows in Dagster; and implement data quality, freshness, reconciliation, monitoring, deduplication, and late-data handling. Partner with engineering, analytics, data science, and risk teams to support reliable reporting and machine-learning features.
Top Skills:
AirbyteAirflowApache BeamBigQueryCdcClickhouseDagsterDataflowDatastreamDbtDebeziumEvmFivetranFlinkGCPGcsKafkaMeltanoPub/SubPythonRustSftpSnowflakeSolanaSQL
Fintech • Machine Learning • Payments • Software • Financial Services
Design, build, and support scalable cloud data platforms, pipelines, and applications. Collaborate with Agile teams, product managers, software engineers, analysts, and data scientists. Develop distributed data solutions using Python, Spark, SQL, Databricks, Snowflake, and relational or NoSQL databases. Establish reusable engineering patterns, data security controls, and operational practices while mentoring colleagues and communicating technical concepts to stakeholders.
Top Skills:
AirflowAWSCassandraDagsterDatabricksDynamoDBEmrGlueGCPJavaAzureMongoDBMonte CarloNosql DatabasesPythonRedshiftRelational DatabasesScalaSnowflakeSparkSplunkSQL
Blockchain • Fintech • Payments • Financial Services • Cryptocurrency • Web3
Build and operate reliable data pipelines and models supporting risk monitoring, compliance screening, regulatory reporting, and audit evidence. Work with large-scale on-chain and internal data using SQL, Python, cloud data warehousing, Airflow, dbt, and AI. Responsibilities include data quality checks, alerting, runbooks, on-call ownership, blockchain data modeling, backfills, access controls, peer reviews, and collaboration with engineering, compliance, analytics, and risk teams.
Top Skills:
Apache AirflowBlockchainCloud Data WarehouseDbtPythonSQL
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
