Comspark International Inc. Jobs

Senior GCP Data Engineer

Comspark International Inc.

Senior GCP Data Engineer

Reposted 22 Days Ago

Be an Early Applicant

In-Office

Sunnyvale, CA, USA

Senior level

In-Office

Sunnyvale, CA, USA

Senior level

Design, build, and maintain large-scale GCP data pipelines and data warehouse solutions using Dataproc, GCS, BigQuery, Hadoop/Spark/Hive and Airflow. Mentor engineers, run agile ceremonies, prioritize backlog, provide on-call support, and deliver data models and products at multi-TB/PB scale.

The summary above was generated by AI

Senior GCP Data Engineer
Sunnyvale, CA (5 Days onsite)
Long Term

GCP Experience
• 4+ years of recent GCP experience
• Experience building data pipelines in GCP
• GCP Dataproc, GCS & BIGQuery experience
• 12+ years of hands-on experience with developing data warehouse solutions and data products.
• 6+ years of hands-on experience developing a distributed data processing platform with Hadoop, Hive or Spark, Airflow or a workflow orchestration solution are required
• 5+ years of hands-on experience in modeling and designing schema for data lakes or for RDBMS platforms.
• Experience with programming languages: Python, Java, Scala, etc.
• Experience with scripting languages: Perl, Shell, etc.
• Practice working with, processing, and managing large data sets (multi TB/PB scale).
• Exposure to test driven development and automated testing frameworks.
• Background in Scrum/Agile development methodologies.
• Capable of delivering on multiple competing priorities with little supervision.
• Excellent verbal and written communication skills.
• Bachelor's Degree in computer science or equivalent experience.
The most successful candidates will also have experience in the following:
• Gitflow
• Atlassian products – BitBucket, JIRA, Confluence etc.
• Continuous Integration tools such as Bamboo, Jenkins, or TFS
As a Senior Data Engineer, you will
• Design and develop big data applications using the latest open source technologies.
• Desired working in offshore model and Managed outcome
• Develop logical and physical data models for big data platforms.
• Automate workflows using Apache Airflow.
• Create data pipelines using Apache Hive, Apache Spark, Apache Kafka.
• Provide ongoing maintenance and enhancements to existing systems and participate in rotational on-call support.
• Learn our business domain and technology infrastructure quickly and share your knowledge freely and actively with others in the team.
• Mentor junior engineers on the team
• Lead daily standups and design reviews
• Groom and prioritize backlog using JIRA
• Act as the point of contact for your assigned business domain

Similar Jobs

EXL

Data Engineer

19 Days Ago

Remote or Hybrid

United States

Senior level

Information Technology • Database • Consulting

Lead end-to-end data engineering on GCP: design scalable data platforms and architectures (lake/lakehouse/warehouse), build and optimize BigQuery/Spark pipelines, ensure data quality/security, enable ML workflows with Data Science, mentor engineers, and own delivery and performance.

Top Skills: SparkBatch Processing FrameworksBigQueryCloud Composer (Airflow)Cloud StorageData LakeData LakehouseData WarehouseDataprocEtl FrameworksGCPPythonReal-Time Processing FrameworksScalaSQL

Shield AI

Executive Assistant

An Hour Ago

In-Office

San Francisco, CA, USA

110K-170K Annually

Expert/Leader

110K-170K Annually

Expert/Leader

Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software

Provide high-level executive support to the CMO and Marketing leadership: manage complex calendars and travel, coordinate meetings and events, prepare executive presentations, track action items and budgets, handle confidential information, support cross-functional initiatives, and serve as a senior EA partner and mentor across the Executive Assistant community.

Top Skills: Ai-Powered Productivity ToolsCalendaring SystemsConcurExpensifyGoogle WorkspaceMicrosoft Office SuiteMicrosoft TeamsPowerPointSlackVideo Conferencing Platforms

PagerDuty

Consultant

An Hour Ago

Easy Apply

Hybrid

San Francisco, CA, USA

Easy Apply

144K-197K Annually

Senior level

144K-197K Annually

Senior level

Artificial Intelligence • Cloud • Information Technology • Machine Learning • Software • Big Data Analytics • Automation

Run technical discovery, design integrations, and provide pre-sales technical guidance to support sales cycles. Answer RFP/RFI/security questions, demonstrate PagerDuty value to stakeholders, manage multiple accounts, and collaborate with sales, product, and cross-functional teams to ensure successful implementations.

Top Skills: Ai/MlAutomationCloud MigrationDevOpsJavaLinuxNagiosNew RelicPagerdutyPerlPHPRubySaaSServicenowSplunkZabbixZenoss

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
Major Tech Employers: Google, Apple, Salesforce, Meta
Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine