Build and maintain scalable ETL/ELT pipelines using Azure Data Factory, Snowflake, dbt, and SQL. Develop data models, transformations, tests, and documentation; ingest data from APIs, databases, files, and enterprise applications; optimize performance; ensure data quality; troubleshoot failures; and support production deployments and monitoring.
This is a remote position.
We are looking for an experienced Data Engineer with strong hands-on experience in Azure Data Factory (ADF), Snowflake, dbt, and SQL. The ideal candidate will be responsible for building and maintaining scalable data pipelines, transforming data, and developing reliable data solutions for analytics and reporting.
- Design, develop, and maintain ETL/ELT data pipelines using Azure Data Factory (ADF).
- Build and optimize data solutions using Snowflake as the cloud data warehouse.
- Develop data transformation models using dbt (Data Build Tool).
- Write complex and optimized SQL queries, stored procedures, views, and transformations.
- Develop data pipelines to ingest data from various sources including APIs, databases, files, and enterprise applications.
- Perform data cleansing, transformation, validation, and reconciliation to ensure data quality and accuracy.
- Implement incremental data loads, scheduling, dependency management, and error handling in ADF.
- Develop scalable and optimized Snowflake data models, tables, views, and SQL transformations.
- Build and maintain dbt models, tests, macros, and documentation.
- Troubleshoot pipeline failures, data issues, and performance problems across ADF, Snowflake, and dbt.
- Work with business analysts, data architects, and other engineering teams to understand data requirements and deliver solutions.
- Support production deployments, monitoring, and ongoing enhancement of data pipelines.
- 4+ years of experience in Data Engineering.
- Strong hands-on experience with:
- Azure Data Factory (ADF)
- Snowflake
- dbt
- Advanced SQL
- Azure Data Factory (ADF)
- Strong understanding of ETL/ELT concepts and data warehousing.
- Experience developing complex data pipelines and data transformations.
- Good understanding of dimensional data modeling, including fact and dimension tables.
- Experience with Snowflake performance optimization and query tuning.
- Hands-on experience with dbt models, tests, incremental models, macros, and Jinja.
- Strong troubleshooting and production support skills.
- Experience with Azure cloud services, particularly Azure Storage, Azure Functions, Databricks, or Key Vault.
- Experience integrating data through REST APIs, JSON, CSV, and relational databases.
- Experience with Python or PySpark is a plus.
- Experience with Git and CI/CD using Azure DevOps or similar tools.
- Understanding of data governance, data quality, and security best practices.
- Experience working with large-scale enterprise data platforms.
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
Drives Rapid7 solution adoption and joint revenue growth through CDW by delivering technical demos, proof-of-concept evaluations, training, enablement plans, and solution guidance. Partners with account managers, sales engineers, sellers, and customers to support opportunities, explain technical value, resolve complex questions, and strengthen channel relationships. Requires consistent travel to CDW headquarters in Chicago and partner offices.
Top Skills:
Application SecurityAWSCloud SecurityDevOpsGoogle Cloud Platform (Gcp)Incident ResponseAzureRapid7Security AutomationVulnerability Management
Fintech • Payments • Software • Financial Services
Lead commercialization of emerging AI products by securing enterprise customers and strategic partners, structuring pilots, and converting them into scaled relationships. Own engagements from prospecting and negotiation through launch and optimization, while partnering with Product and Engineering to shape product strategy, pricing, implementation, and go-to-market models.
Top Skills:
AICloud InfrastructureData InfrastructureDeveloper Platforms
Fintech • Payments • Software • Financial Services
Own commercialization of Block’s emerging AI products from early pilots through scaled enterprise adoption. Develop market strategy, positioning, pricing, partnerships, and routes to market; personally secure lighthouse enterprise customers and strategic partners; design pilots that convert to production relationships; establish repeatable GTM, sales, implementation, and expansion processes; and build the commercial organization as product-market fit develops. Partner closely with Product and Engineering while shaping an emerging AI business.
Top Skills:
AICloud InfrastructureData InfrastructureDeveloper Platforms
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
