The Sr. Data Engineer will build and manage data pipelines for large datasets, collaborate with teams, and leverage cloud technologies for data processing.
Windfall is seeking a Sr. Data Engineer to join our data team. As a Sr. Data Engineer on our data team, you will be building out the core data asset that everything else at Windfall is built on top of. Communication and collaboration are at the heart of Windfall, and you will work closely with our product, data science, and other engineering teams. You will personally design and build the pipelines for massive datasets, taking them all the way from inception to exploration to production and customer use.
We’re on a mission to change how organizations perceive and use people data. And we hold true to our core values of: (1) Be an excellent communicator; (2) Operate with transparency; (3) Provide leverage, not optimization; (4) Win When Our Customers Win; and (5) Act with integrity and trust.
Responsibilities:
- Construct data pipelines to ingest and merge billions of individual entities into Windfall’s core data asset
- Work closely with our data science team to run ML models on top of billions of data points
- Build supporting data services and applications to orchestrate and monitor our data systems
Some technology you will use:
- Cloud platform - GCP
- Programming languages - Java, Python, and Kotlin
- Data warehouse & databases - BigQuery, Postgres, Scylla/Cassandra
- Distributed processing frameworks - Dataflow (Apache Beam) and Apache Spark
- Orchestration - Airflow
Requirements:
- 4-8 years of professional data engineering experience
- Significant experience working with Apache Beam/Spark/Flink or MapReduce
- Strong Object-oriented programming ability in a JVM language
- Expert knowledge of distributed data processing
- Familiarity with different datastores, their differences, and appropriate usages
- Experience at a sub-200 person company
- You communicate as well as you code
- You can simplify complex problems into simple solutions
- You balance a strong sense of ownership and responsibility in your work with collaboration and team alignment
- You are comfortable making trade-offs between quality, complexity, and speed-of-delivery
Preferred Qualifications:
- Proven experience taking a large project from ideation to production
- Experience leading greenfield projects
- Working knowledge of cloud-native data engineering infrastructure
Additional Information:
The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across California and Colorado. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. We also offer a comprehensive benefits package, which you can explore on our careers site
Compensation range:About Windfall
Windfall is a people intelligence and AI company that gives go-to-market teams actionable insights. By democratizing access to people data, organizations can intelligently prioritize go-to-market resources to drive greater business outcomes. Powered by best-in-class machine learning and artificial intelligence, Windfall activates insights into workflows that engage the right people for each respective organization. More than 1,500 data-driven organizations use Windfall to power their business. For more information, please visit www.windfall.com.
We comply with CCPA. For more information on how we comply, review our privacy notice.
Windfall is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
We may use AI tools to assist parts of the hiring process, such as reviewing applications or analyzing resumes. These tools assist our team and do not replace human judgment. Final hiring decisions are made by people.
Windfall San Francisco, California, USA Office
430 Pacific Ave, , San Francisco, CA, United States, 94133
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Design, build, and maintain scalable Spark-based ETL pipelines and computed tables in a central data lake. Integrate structured and unstructured IoT, sensor, and external data for analytics, model training, and dashboards. Collaborate with Data Science, Analytics, and ML teams to ensure reliable, high-quality customer-facing datasets.
Top Skills:
AirflowAWSAzureDagsterData LakeDatabricksDelta LakeETLGCPGitGitPrefectPysparkPythonRest ApisSparksqlSQL
Fintech • HR Tech
Build and deliver end-to-end, production-grade data solutions: design and maintain scalable ETL pipelines, ingest from diverse sources, implement dbt transformations, ensure data quality and observability, optimize performance and cost, and partner with analytics, product, and engineering teams to drive business impact.
Top Skills:
AIAlertingAPIsAutomated TestingAutomationBigQueryCi/CdData ObservabilityDatabricksDbtETLEvent StreamsJavaMonitoringPythonRedshiftScalaSnowflakeSQL
Big Data • Cloud • Productivity • Software • Database • Analytics • Automation
As a Senior Data Engineer at Jellyfish, you'll build and maintain data pipelines, optimize orchestration, automate CI/CD processes, and enhance data integration while ensuring high performance and reliability.
Top Skills:
AirflowBigQueryDagsterDatabricksDbtPrefectPysparkPythonRedisSnowflakeSQLTerraform
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
