The Staff Product Support Engineer will design, optimize, and manage Hadoop and Spark applications, solve data challenges, and ensure operational stability in a 24/7 environment.
We are seeking a Staff Product Support Engineer / Product Specialist - Hadoop Operations SME who will be responsible for designing, optimizing, migrating, and scaling Hadoop and Spark-based data processing systems. This role involves hands-on experience with Hadoop and other core data operations, focusing on building resilient, high-performance distributed data systems.
You will collaborate with customer engineering teams to deliver high-throughput Hadoop, Nifi, and Spark applications and solve complex data challenges in migration, upgrades, reliability, and optimise post-migration system performance.
This role requires flexibility to work in rotational shifts, based on team coverage needs and customer demand. Candidates should be comfortable supporting operations in a 24/7 environment and be willing to adjust their working hours accordingly.
We’re looking for someone who perform:
- Design and optimise distributed Hadoop-based applications, ensuring low-latency, high-throughput performance for big data workloads.
- Troubleshooting: Provide expert-level support for data or performance issues in Hadoop, Nifi, and Spark jobs and clusters.
- Data Processing Expertise: Work extensively with large-scale data pipelines using Hadoop, Nifi, and Spark's core components.
- Performance Tuning: Conduct deep-dive performance analysis, debugging, and optimization of Nifi, Impala, and Spark jobs to reduce processing time and resource consumption.
- Cluster Management: Collaborate with DevOps and infrastructure teams to manage Nifi, Impala, and Spark clusters on platforms like Hadoop/YARN, Kubernetes, or cloud platforms (AWS EMR, GCP Dataproc, etc.).
- Migration: Provide dedicated support to ensure the stability and reliability of the new ODP Hadoop environment during and after migration. Promptly address evolving technical challenges and optimise system performance after migration to ODP.
Good to have:
- Master’s degree and Experience working with scripting languages (Scala, Python, Bash, PowerShell).
- Familiarity with virtual machine technologies and multi-node environment (50+ nodes).
- Proficient with Linux, NFS, and Windows, including application installation, scripting, and working with the command line.
- Working knowledge of application, server, and network security management concepts. Certification on any of the leading Cloud providers (AWS, Azure, GCP ) and/or Kubernetes.
- Knowledge of databases like MySQL and PostgreSQL.
- Be involved with and work on other support-related activities - Performing POC & assisting with Onboarding deployments of Acceldata & Hadoop distribution products.
Top Skills
Aws Emr
Bash
Gcp Dataproc
Hadoop
Impala
Kubernetes
Linux
MySQL
Nifi
Postgres
Powershell
Python
Scala
Spark
acceldata Campbell, California, USA Office
2105 S Bascom Ave, Campbell, CA, United States, 95008
Similar Jobs
Edtech • Kids + Family • Social Impact • Software
The Instructional Specialist, Multilingual Learning designs instructional frameworks, ensures content aligns with language standards, coaches internal teams, and leads research on multilingual education trends.
Top Skills:
Digital Authoring ToolsGoogle Workspace
.png){kind=logo}
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
The Lead Technical GRC Analyst will manage NBCUniversal's security governance framework, while supporting IT cyber governance processes, conducting technical assessments, and collaborating with stakeholders to ensure effective security measures.
Top Skills:
Active DirectoryArcherAzure AdCisIso 27001M365Microsoft Defender For CloudNistOnetrustServicenow GrcSlack
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
This role involves leading the design and implementation of SAP SuccessFactors solutions, managing technical teams, and establishing governance standards to align technology strategy with business goals.
Top Skills:
APIsBusiness Application StudioBw/4HanaFieldglassIntegration SuiteJava ConnectorsRestful Abap ProgrammingSacSap BtpSap BwSap Successfactors
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
