Staff Software Engineer - Data Processing & Execution Platform

What do we Need:

As a Staff Software Engineer, you will be the technical lead for the data processing and execution platform — a set of services that take user-initiated operations (data commits, transformations, workflows) and execute them reliably across shared compute infrastructure. This is a distributed, event-driven system with multiple services coordinating through asynchronous message passing.
This is no ordinary engineering role - this is working on a product with real world meaning - a product that helps define the way scientists works, helping bring medicine to market quicker, finding cures for diseases faster
This is LUMA
**This is a remote position, located in Mass. There will be some travel required to the office in Boston for team meetings and working sessions.**
In this role, you will:

• Define and drive system architecture for event-driven data processing services, leading the migration of key components to a Node.js/TypeScript and Python ecosystem
• Contribute to and improve engineering standards, patterns, and best practices for distributed systems, observability, and reliability across a multi-service platform coordinated through Kafka, GraphQL and RESTful APIs
• Architect and implement asynchronous data processing pipelines for high-volume scientific data, with attention to at-least-once delivery guarantees, backpressure, and graceful degradation
• Guarantee the scalability, maintainability, and security of software solutions running on Kubernetes and AWS
• Take ownership of existing services, develop a deep understanding of their behavior, and make pragmatic decisions about how to evolve them

We are looking for candidates with 12+ years experience in engineering preferably in a SaaS environment, and ideally with a degree in Computer Science, Software Engineering, or equivalent.

The Key Skills we are looking for:

• Language-agnostic engineering mindset — you pick up new languages and codebases quickly, reason about distributed system design independent of any single runtime, and can be productive in an unfamiliar stack within weeks
• Advanced working experience in Asynchronous processing
• Strong proficiency in Node.js/TypeScript and Python for building production backend services
• Proven experience designing and implementing distributed, event-driven systems coordinated through message buses, GraphQL and RESTful APIs
• Experience implementing automated testing platforms, unit tests, and integration tests
• Advanced working experience with large data processing platforms such as Spark, Databricks, or Snowflake
• Experience managing state across multiple stores (e.g. a message bus, a relational database, and a cache) and the consistency challenges that come with it
• Hands-on experience with AWS in production environments and a solid understanding of Kubernetes for orchestrating workloads
• Proficiency with CI/CD tools such as GitHub Actions
• Knowledge of Agile software development practices
• Setting technical direction, leading cross-team initiatives, and leveling up other engineers through mentoring and architectural guidance

You may also have experience in:

• Scala or other JVM languages (helpful for understanding existing services during migration)
• Message-based architectures using Kafka (consumer groups, partitioning, delivery semantics)
• Background in complex data pipelines, schema management, and incremental processing
• Exposure to AWS and/or GCP or designing systems portable across multiple cloud providers
• Building scalable distributed systems using Kubernetes and other cloud-native technologies
• Experience within Life Sciences or R&D data management

Why this role is interesting:

You're not maintaining a system in steady state. You're taking a complex, production-proven execution platform and leading its evolution — migrating to a new tech stack while keeping it running reliably for customers doing drug discovery. The system processes real scientific data through multi-step pipelines on shared compute infrastructure, with all the distributed systems challenges that implies: multi-tenancy, cancellation, crash recovery, and coordinating state across multiple data stores. If you enjoy understanding how complex systems actually work, regardless of what language they happen to be written in, and then making them better - this is the role.