The Punch Agency Logo

The Punch Agency

Data Engineer

Reposted One Month Ago
In-Office
San Francisco, CA, USA
150K-150K Annually
Expert/Leader
In-Office
San Francisco, CA, USA
150K-150K Annually
Expert/Leader
Lead design and implementation of a scalable Big Data Audience Management platform (500+ TB). Own data pipelines (ingest, validation, transformation, storage) using Spark and Scala, mentor engineers, perform R&D on new technologies, and partner with product managers to meet roadmap and business needs.
The summary above was generated by AI
Company Description

Punch is a full service digital agency based out of New York, Silicon Valley, LA, Boston, Stockholm and Phuket, Thailand. Here at Punch, we take pride in tackling our clients’ most challenging tasks. Founded by a group of former Google Engineers and now with over 75 staff, Punch is now leading the way in creating new cutting-edge technology.

We're looking for innovative thinkers with a hunger to expand their knowledge!
We are building a more technical world.Go ahead and check out some of our work here: www.punch-agency.com

Job Description

We are seeking an extremely talented senior/lead data engineer to design & implement highly reliable & scalable Audience Management Big Data Platform (500+ Terabytes).

Responsibilities:

- Build, enhance and scale our existing Big Data Platform to process 500+ Terabytes of data from various 3rd party providers.

- Technically own the complete data pipeline module including data ingestion, validation, transformation & storage in the data platform.

- Design and write clean, robust, server-side code using Spark & Scala.

- Perform R & D to evaluate new technologies, generate different ideas & patentable IP to communicate value for company

- Work closely with team members & mentor them technically

- Partner with product managers to support product roadmap & business needs

Required Skills:

- 10 - 15 years experience in software development primarily in Big Data domain using Spark & Hadoop eco system.

- 5+ Years of experience in Spark & Scala (Skill Level: 8 or more out of 10)

- Bachelors degree in Computer Science or equivalent.

- 5+ years experience designing, building and launching extremely efficient & reliable data pipelines to move data (both large and small amounts) using modern data architecture/tools.

- Strong background (5+ years) in ETL processing for large amount of data (250+ Terabytes)

- Multiple years of experience working with 250+ Terabytes of data for ETL using Spark & Scala

- 3+ years experience working in Agile environment (SCRUM)

Nice To Have Skills:

- Experience in digital marketing (Good Domain knowledge on digital marketing Industry)

- Experience working with AWS (S3, EMR, EC2, RDS)

Job Type: Full-time

Required education:

  • Bachelor's

Required experience:

  • designing, building: 5 years

Job Type: Full-time

Required education:

  • Bachelor's

Required experience:

  • designing, building: 5 years

Qualifications

Required Skills:

- 10 - 15 years experience in software development primarily in Big Data domain using Spark & Hadoop eco system.

- 5+ Years of experience in Spark & Scala (Skill Level: 8 or more out of 10)

- Bachelors degree in Computer Science or equivalent.

- 5+ years experience designing, building and launching extremely efficient & reliable data pipelines to move data (both large and small amounts) using modern data architecture/tools.

- Strong background (5+ years) in ETL processing for large amount of data (250+ Terabytes)

- Multiple years of experience working with 250+ Terabytes of data for ETL using Spark & Scala

- 3+ years experience working in Agile environment (SCRUM)

Nice To Have Skills:

- Experience in digital marketing (Good Domain knowledge on digital marketing Industry)

- Experience working with AWS (S3, EMR, EC2, RDS)

Job Type: Full-time

Required education:

  • Bachelor's

Required experience:

  • designing, building: 5 years

Job Type: Full-time

Required education:

  • Bachelor's

Required experience:

  • designing, building: 5 years

Additional Information

We are a team of former Google Engineers and Designers based in Silicon Valley, New York and Phuket, Thailand.

All of our core engineers are paid in the $150,000 range or greater and we also provide our engineers the opportunity to work, all expenses paid, from our Phuket, Thailand office one month a year as a creative outlet to work and enjoy life to the fullest.

In this role you will work with some of the most exciting projects in the industry today with a team of fast moving engineers and designers who have a passion for building disruptive products. 

Similar Jobs

10 Days Ago
Remote or Hybrid
USA
125K-159K Annually
Mid level
125K-159K Annually
Mid level
AdTech • Automotive • Big Data • Consumer Web
Administer and enhance Edmunds’ Databricks data platform and AWS infrastructure. Build and maintain ETL pipelines, infrastructure-as-code tooling using Terraform or CDK, and operational dashboards, alerts, and reports. Collaborate with business, engineering, analytics, security, and infrastructure teams to support data platform users and AI solutions. Evaluate new technologies, troubleshoot platform issues, and improve operational, cost, and security visibility.
Top Skills: SparkAWSAws CdkDatabricksInfrastructure As Code (Iac)PythonScalaSQLTerraform
11 Days Ago
Easy Apply
Hybrid
San Francisco, CA, USA
Easy Apply
50-50 Annually
Internship
50-50 Annually
Internship
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Build and maintain data pipelines and scalable cloud infrastructure supporting analytics and machine learning. Integrate internal and external data sources, ensure data integrity, contribute to analytical investigations, document findings, and apply data governance, security, and performance optimization practices while collaborating with senior engineers and cross-functional teams.
Top Skills: APIsAWSAzureCloud-Native DesignDistributed SystemsGCPGenerative AiLarge Language Models (Llms)MicroservicesPythonRelational DatabasesSQL
11 Days Ago
Remote or Hybrid
USA
120K-180K Annually
Senior level
120K-180K Annually
Senior level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Architects, deploys, and operates CrowdStrike’s data platform infrastructure and services. Responsibilities include administering Airflow and Superset, managing Terraform infrastructure, building CI/CD pipelines, implementing security and access controls, developing DBT data models, cataloging assets in OpenMetadata, writing Python automation, and mentoring engineers. The role requires cloud, container orchestration, SQL, data modeling, compliance, and AI technology expertise, along with eligibility for CJIS clearance.
Top Skills: AIApache AirflowApache SupersetAWSAzureCi/CdDbtDockerGCPIamKubernetesMachine Learning PipelinesOciOpenmetadataPythonSQLTerraform

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account