Key Responsibilities:
Design, develop, and manage complex Airflow DAGs for data workflows, ETL/ELT pipelines, and task automation.
Integrate Airflow with various data sources (e.g., databases, APIs, cloud storage) and targets.
Optimize and monitor pipeline performance and troubleshoot failures in a timely manner.
Collaborate with data engineers, analysts, and platform teams to support end-to-end data solutions.
Implement best practices for code quality, testing, version control, and deployment.
Maintain documentation and provide technical mentorship to junior engineers.
Required Skills:
3+ years of hands-on experience with Apache Airflow in production environments.
Strong programming skills in Python.
Solid understanding of data engineering concepts, including ETL/ELT processes, workflow orchestration, and data validation.
Experience with SQL and relational databases (e.g., PostgreSQL, MySQL).
Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and their storage and compute services.
Knowledge of containerization tools like Docker and orchestration with Kubernetes (a plus).
Compensation, Benefits and Duration
Minimum Compensation: USD 36,000
Maximum Compensation: USD 126,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Photon San Francisco, California, USA Office
San Francisco, United States
Similar Jobs
What you need to know about the San Francisco Tech Scene
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine



.png)