Software Engineer - Core Engine Internals
Sorry, this job was removed at 12:02 p.m. (PST) on Tuesday, January 18, 2022
By clicking Apply Now you agree to share your profile information with the hiring company.
At Sisu, we're building a software platform that empowers people to make better decisions using data. Based on years of cutting-edge research at Stanford, Sisu enables users to quickly and comprehensively understand what’s driving their key metrics, so they never miss a window of opportunity to act.
Sisu is a fundamental rearchitecture of how users interact with data. Instead of relying on users to tell us what they want to see, as in traditional dashboarding systems, Sisu predicts users’ intent based on their metrics of interest, and past behaviors. We take this kind of predictive analysis for granted in consumer applications (e.g., Netflix, Google, Facebook), but the hardest interface design and ranking problems lie in making private data found in everyday organizations actionable. The key problem we’re solving as a first step in this direction is to help identify what’s driving change in key metrics like revenue, retention, and churn among the enormous set of factors like user demographics, campaigns, and acquisition channels. To do so, we combine statistical analysis and machine learning at scale to provide users personalized, real-time diagnoses of changes in their metrics via an explainable, interpretable user interface.
As a Core Engine Internals Engineer, you'll help us shape the future of the Sisu Engine - a parallel, distributed, cloud-native dataflow engine for high performance ML model execution, written in Rust. We're looking for database internals engineers experienced with query planning and optimization, access methods, and data layout. You'll play a critical role in helping us advance the performance and efficiency of our system—both with low-level optimizations, such as honing concurrency and reducing memory allocations, as well as high-level architecture, such as maximizing distributed computing throughput and scheduling work fairly. Your contributions to the speed and scalability of our system will directly go into one of the key differentiators for our product against the status quo. Success in this area will be the foundation on which future Sisu features depend.
Responsibilities
- Design, engineer, and operate Sisu's scalable parallel data processing engine, including distributed query processing, columnar execution, and streaming updates
- Develop novel and scalable systems combining hardware-accelerated data processing with machine learning and statistical hypothesis testing
- Technical leadership
- Shape the engineering culture through mentorship
- Build out our data plane
- Implement new analyses in our performance-critical data processing engine
- Optimize and scale data ingest and workload distribution
- Actively participate in customer POCs to optimize customer experience and adoption.
- Work collaboratively with multiple departments such as Sales, Marketing, and G&A.
Key Qualifications
- A passion for database systems, storage systems, distributed systems, and/or performance optimization
- Substantial experience building production-grade data-intensive systems, especially database internals and/or large-scale parallel data processing systems internals
- Substantial experience in low-level systems programming, including but not limited to SIMD vectorization, memory management, and general mechanical sympathy
- Strong understanding of data-related CS fundamentals including distributed systems and distributed computing, fault tolerance and consensus protocols, and query optimization
- Strong interest in or production experience writing large-scale systems in Rust
- Optional: PhD in databases or distributed systems
Read Full Job Description