About the role:
The Data Storage and Compute team is part of the Data Platform team and is responsible for building the underlying storage, resource management and table management infrastructure (with Apache YARN, Kubernetes, Spark, Hudi, Parquet) for the entire company.

• Design systems architecture and deliver software and tools to advance the reliability of infrastructure including availability, performance, efficiency and scaling.
• Provide technical leadership, influence and partner with fellow engineers to architect, design and build infrastructure that can stand the test of scale and availability, while reducing operational overhead.
• Drive efficiencies in systems and processes through automation: capacity planning, configuration management, performance tuning, monitoring and root cause analysis.
• Participate in periodic on-call rotations and be available for escalations.
• Collaborate with platform, product and security engineering teams, and enable successful use of infrastructure and foundational services, and manage upstream and downstream dependencies
• Partners with stakeholders and leads team efforts to build and maintain backend services and solutions to support user-facing products, downstream services, or infrastructure tools and platforms used across Uber

About the Team:
Uber is committed to delivering safer and more reliable transportation across our global markets. To accomplish this, Uber relies heavily on making data-driven decisions at every level, from forecasting rider demand during high traffic events to identifying and addressing bottlenecks in our driver-partner sign-up process. Over time, the need for more insights has resulted in over 100's of petabytes of analytical data that needs to be cleaned, stored, and served with minimum latency through our multilayered Data Platform architecture.
Since the beginning of Data engineering at Uber, we have worked to develop a Big Data platform that ensures data reliability, scalability, and ease-of-use, and are now focusing on increasing our platform's speed and efficiency [Uber's Big Data Platform: 100+ Petabytes with Minute Latency].
This team manages more than 20,000+ node clusters to serve exabyte storage that supports scheduling compute resources for 500k+ data process applications daily. We implement and support a storage platform that manages Uber's exabyte Lakehouse that provides ingestion and query engine API's that facilitate management of 10K+ tables and millions of queries a week. Additionally, our team builds and manages self service table management services that allow for data encryption, data deletions, data sorting and other kinds of use-cases for all of Uber.
Minimum qualifications:

• Masters or equivalent in Computer Science, Engineering, Mathematics or related field with 2-years of experience OR 4-years of full-time Software Engineering work experience.

• Highly proficient in one of the following programming languages: Java, Go, C/C++. Good scripting skills and ability to pick up new ones.
• Systematic problem solving approach and knowledge of algorithms, data structures and complexity analysis.
• A love for coding, debugging and attacking hard problems.
• Good working knowledge of networking and RPC stack (TCP/IP, HTTP, Thrift).
• Good working knowledge of distributed caching, storage and indexing systems.
• Grit, drive and a strong sense of ownership coupled with a penchant for collaboration

Technical skills:
Preferred:

• Under the hood experience with Apache YARN, Kubernetes, Apache Spark, Hudi, Hadoop HDFS or similar systems such as Apache HBase is a strong plus
• Experience with highly available/fault tolerant, replicated data storage systems, large scale data processing systems or enterprise/cloud storage systems is also a strong plus
• 3+ years of experience building and managing distributed systems. Sound understanding of distributed system fundamentals.