Databricks Logo

Databricks

Senior Software Engineer - Database Engine Internals

Reposted 2 Months Ago
Be an Early Applicant
In-Office
San Francisco, CA, USA
166K-225K Annually
Senior level
In-Office
San Francisco, CA, USA
166K-225K Annually
Senior level
Design and implement next-generation query engine and structured storage systems focused on performance optimization and data management.
The summary above was generated by AI

P-97

Our mission at Databricks is to radically simplify the whole data lifecycle from ingestion to ETL, BI, and all the way up to ML/AI with a unified platform. To achieve this goal, we believe the data warehouse architecture as we know it today will be replaced by a new architectural pattern, Lakehouse (CIDR 2021 paper), open platforms that unify data warehousing and advanced analytics. The new architecture will help address several major challenges, including data staleness, reliability, total cost of ownership, data lock-in, and limited use-case support.

A critical part of realizing this vision is the next generation (decoupled) query engine and structured storage system that can outperform specialized data warehouses in relational query performance, yet retain the expressiveness and of general purpose systems such as Spark™ to support diverse workloads ranging from ETL to data science.

As part of this team, you will be working in one or more of the following areas to design and implement these next gen systems that leapfrog state-of-the-art:

  • Query compilation and optimization
  • Distributed query execution and scheduling
  • Vectorized execution engine
  • Data security
  • Resource management
  • Transaction coordination
  • Efficient storage structures (encodings, indexes)
  • Automatic physical data optimization

What we look for:

  • A passion for database systems, storage systems, distributed systems, language design, or performance optimization
  • Experience working towards a multi-year vision with incremental deliverables
  • Motivated by delivering customer value and impact
  • 5+ years of experience working in a related system (preferred)
  • Optional: PhD in databases or distributed systems

Pay Range Transparency

Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles.  Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Based on the factors above, Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above. For more information regarding which range your location is in visit our page here.


Local Pay Range
$166,000—$225,000 USD

About Databricks

Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and 70% of the Fortune 500 — rely on the Databricks Data + AI Platform to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. To learn more, follow Databricks on LinkedIn, X, YouTube, and Instagram.
Benefits
At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here.

Our Commitment to Diversity and Inclusion

At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.

Compliance

If access to export-controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone.

HQ

Databricks San Francisco, California, USA Office

160 Spear Street, San Francisco, CA, United States, 94105

Databricks Mountain View, California, USA Office

351 E Evelyn Ave Mountain View, CA USA, Mountain View, United States, 94041

Similar Jobs

2 Months Ago
In-Office
Mountain View, CA, USA
166K-225K Annually
Senior level
166K-225K Annually
Senior level
Big Data • Machine Learning • Software • Analytics • Big Data Analytics
Design and implement next-gen database systems focused on performance optimization, query execution, and efficient data storage as part of a team at Databricks.
Top Skills: SparkDelta LakeMlflow
4 Minutes Ago
Hybrid
5 Locations
124K-280K Annually
Senior level
124K-280K Annually
Senior level
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Leads custom application development, architecture, testing, modernization, and delivery across cloud-native and on-premise environments. Translates business requirements into scalable technical solutions, coordinates software planning and releases, reviews code, applies CI/CD and automated testing, advises clients on technology decisions, coaches engineers, and advances AI-enabled engineering practices.
Top Skills: .NetAPIsAutomated TestingCi/CdCloud-Native TechnologiesJavaMicroservicesPythonReact Native
4 Minutes Ago
Hybrid
5 Locations
77K-202K Annually
Senior level
77K-202K Annually
Senior level
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Designs, develops, tests, modernizes, and maintains custom applications across cloud-native and on-premise environments. Translates business requirements into technical solutions, builds APIs and integrations, applies AI-native engineering practices, supports CI/CD and release activities, and performs debugging and code reviews. Collaborates with clients, contributes to architecture and project planning, estimates complex workstreams, and mentors junior engineers while ensuring high-quality software delivery.
Top Skills: APIsC#.NetCi/CdCloud ComputingJavaJavaScriptLow-Code PlatformsMicroservicesPythonSource ControlSwiftui

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account