Senior Systems Engineer
Quantcast operates some of the largest custom-developed data processing infrastructure in the world, storing and processing tens of petabytes of data daily! This includes a fault tolerant space efficient distributed file system (QFS), along with a custom map/reduce implementation that is four times faster than open source alternatives.
Low-probability failures are common occurrences in any system that contains thousands of nodes.
Our platform teams own the development and operation of software systems that not only identify problems efficiently across the stack (network, hardware, OS, services), but also auto-correct and compensate for them. The team also leads a custom resource allocation system that has been used to achieve meaningful service co-tenancy across the compute infrastructure: scaling the distributed processing and storage platforms to become a highly available service that supports analytics and modeling across the company effectively! Are you ready to help build and scale our highly distributed systems?
Responsibilities
- Mentor and grow the more junior engineers on the team
- Build and lead operational excellence through automation, monitoring, and incident analysis
- Improve and maintain the services and development of tools that support and scale distributed storage and compute platform
- Scale and improve our platform and make it more elastic and fault tolerant
Requirements
- BS in computer science or equivalent experience
- 5+ years experience with large scale distributed systems; HDFS, Hadoop, Cassandra, etc.
- Experience with configuration management (Puppet, Chef), databases (MySQL, PostgreSQL), and Amazon Web Services
- Linux system administration/automation experience
- Comfortable working in an event driven environment
- Excellent communication and interpersonal skills
Quantcast is an advertising technology company and the creator of an innovative intelligent audience platform that empowers brands, agencies and publishers to know and grow their audiences online. Our solutions are leveling the playing field for our customers when it comes to effectively reaching audiences online and helping them power a thriving free and open internet for everyone. Founded in 2006 and headquartered in San Francisco, Quantcast has employees in 20 offices across 10 countries.
We are committed to building an inclusive and diverse environment where everyone can be their authentic self. We work to ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.