Databricks Logo

Databricks

Staff Software Engineer - Database Engine Internals

Reposted 2 Months Ago
Be an Early Applicant
In-Office
Mountain View, CA, USA
192K-260K Annually
Senior level
In-Office
Mountain View, CA, USA
192K-260K Annually
Senior level
The role involves designing and implementing next-gen database systems focusing on query optimization, execution, and efficient storage structures.
The summary above was generated by AI

P-188

Our mission at Databricks is to radically simplify the whole data lifecycle from ingestion to ETL, BI, and all the way up to ML/AI with a unified platform. To achieve this goal, we believe the data warehouse architecture as we know it today will be replaced by a new architectural pattern, Lakehouse (CIDR 2021 paper), open platforms that unify data warehousing and advanced analytics. The new architecture will help address several major challenges, including data staleness, reliability, total cost of ownership, data lock-in, and limited use-case support.

A critical part of realizing this vision is the next generation (decoupled) query engine and structured storage system that can outperform specialized data warehouses in relational query performance, yet retain the expressiveness and of general purpose systems such as Apache Spark™ to support diverse workloads ranging from ETL to data science.

As part of this team, you will be working in one or more of the following areas to design and implement these next gen systems that leapfrog state-of-the-art:

  • Query compilation and optimization
  • Distributed query execution and scheduling
  • Vectorized execution engine
  • Data security
  • Resource management
  • Transaction coordination
  • Efficient storage structures (encodings, indexes)
  • Automatic physical data optimization

What we look for:

  • A passion for database systems, storage systems, distributed systems, language design, or performance optimization
  • Experience working towards a multi-year vision with incremental deliverables
  • Motivated by delivering customer value and impact
  • 8+ years of experience working in a related system (preferred)
  • Optional: PhD in databases or distributed systems

Pay Range Transparency

Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles.  Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Based on the factors above, Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above. For more information regarding which range your location is in visit our page here.


Local Pay Range
$192,000—$260,000 USD

About Databricks

Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and 70% of the Fortune 500 — rely on the Databricks Data + AI Platform to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. To learn more, follow Databricks on LinkedIn, X, YouTube, and Instagram.
Benefits
At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here.

Our Commitment to Diversity and Inclusion

At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.

Compliance

If access to export-controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone.

HQ

Databricks San Francisco, California, USA Office

160 Spear Street, San Francisco, CA, United States, 94105

Databricks Mountain View, California, USA Office

351 E Evelyn Ave Mountain View, CA USA, Mountain View, United States, 94041

Similar Jobs

2 Months Ago
In-Office
San Francisco, CA, USA
192K-260K Annually
Senior level
192K-260K Annually
Senior level
Big Data • Machine Learning • Software • Analytics • Big Data Analytics
Design and implement next-generation database engine systems focusing on query compilation, distributed execution, optimization, and storage management.
Top Skills: SparkDatabase SystemsDistributed SystemsPerformance OptimizationVectorized Execution Engine
7 Minutes Ago
In-Office
San Jose, CA, USA
109K-219K Annually
Entry level
109K-219K Annually
Entry level
Artificial Intelligence • Cloud • Information Technology • Consulting
Supports product strategy and roadmaps for HPE CX Switching and Self-Driving Network products. Translates customer needs, market trends, competitive insights, and emerging technologies into product requirements and recommendations. Partners with engineering, cloud, AI, sales, customers, and technical teams across the product lifecycle. Develops market, pricing, cost, competitive, and business-case analyses, defines product positioning, and provides data-driven recommendations to leadership.
Top Skills: AIAPIsAristaCiscoCloud TechnologiesEthernetHpe Aos-CxHpe NetworkingIpv4Ipv6JuniperLayer 2 NetworkingLayer 3 NetworkingLinuxPythonRouting ProtocolsVlan
7 Minutes Ago
In-Office
San Jose, CA, USA
116K-253K Annually
Mid level
116K-253K Annually
Mid level
Artificial Intelligence • Cloud • Information Technology • Consulting
Designs, develops, tests, debugs, and integrates compiler and systems software enhancements. Leads software engineering projects, collaborates with internal and outsourced development teams, resolves complex technical issues, and provides guidance to less-experienced engineers. The role requires hybrid work averaging two days per week in an HPE office.
Top Skills: C++CompilersDatabasesDebuggingNetworkingOperating SystemsSoftware TestingTesting Scripts

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account