Senior Engineer, Cloud (Observability Lead)

Illumio is the leader in ransomware and breach containment, redefining how organizations contain cyberattacks and enable operational resilience. Powered by the Illumio AI Security Graph, our breach containment platform identifies and contains threats across hybrid multi-cloud environments – stopping the spread of attacks before they become disasters.
Recognized as a Leader in the Forrester Wave™ for Microsegmentation, Illumio enables Zero Trust, strengthening cyber resilience for the infrastructure, systems, and organizations that keep the world running. 

Our Engineering team is driven by a culture that thrives on visionary leadership, autonomy, and ownership, creating a dynamic synergy that drives us forward in the ever-evolving landscape of cybersecurity. 

When you join our team, you become part of the leader in Zero Trust Segmentation. You'll work with a cutting-edge technology stack that spans operating systems, distributed applications, and immersive UI/visualization tools.  

We're shaping the future of cybersecurity. And together, we will continue to build world-class products—led by people with different perspectives, backgrounds, and a commitment to innovation in a time when the world faces its greatest cybersecurity threats in history. 

We are seeking a Senior Engineer for our Cloud team with a strong focus on observability to join our engineering team as the Observability Lead. In this role, you will champion initiatives to enhance our production systems the reliability, visibility, and operational readiness. of our production systems. You will collaborate closely with engineers to catalog services, improve logging practices, reduce log noise, and integrate additional metrics across all applications. Additionally, you will develop runbooks, build dashboards, and manage PagerDuty configurations and escalation workflows.

• Serve as an advocate for observability practices within the engineering team, promoting operational best practices and reliability. 

• Catalog all production services, documenting critical details for operational visibility and management. 

• Collaborate with engineering teams to develop and implement a comprehensive observability plan, ensuring metrics are integrated into all services. 

• Enhance logging practices where needed, reduce log noise, and ensure meaningful insights are captured. 

• Add and refine metrics across applications to improve operational visibility and performance tracking. 

• Develop detailed runbooks for critical alerts and incidents, facilitating efficient response processes. 

• Build and maintain dashboards that offer insights into SLAs, performance, and business metrics for engineering and product teams. 

• Set up and manage PagerDuty alerts, define on-call duties, and establish incident escalation paths. 

• Continuously improve alerting, logging, and monitoring processes to enhance service reliability and reduce unnecessary noise. 

• Proven experience in a DevOps or observability-focused role, concentrating on production service management and operational excellence. 

• Prior experience working with microservices in a production environment is a must. 

• At least 5+ years of experience managing large numbers of instances in public clouds like AWS, Azure, GCP, etc. 

• Strong expertise in observability practices and tools (e.g., Prometheus, Grafana, Datadog). 

• Experience enhancing logging, reducing log noise, and integrating critical metrics into services. 

• Proficiency in building and managing dashboards and monitoring tools. 

• Expertise in setting up and managing PagerDuty alerts, with on-call rotation and escalation management knowledge. 

• Strong collaboration skills to work closely with engineering teams, advocating for observability best practices. 

• Familiarity with cloud platforms (AWS, GCP, Azure) and modern CI/CD processes. 

• Automation scripting or coding experience (Python, Go, or similar). 

• Knowledge of infrastructure-as-code tools (e.g., Terraform, CloudFormation). 

• Excellent problem-solving skills and attention to detail in managing complex systems. 

Compensation:

$ 164,000 USD - $ 189,000 USD

The pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include responsibilities of the job, education, location, experience, knowledge, skills, abilities, and internal equity, alignment with market data, or applicable laws. 

At Illumio we offer a wide range of benefits to our eligible team members. Our benefit programs vary by location and can include Medical, Dental, Vision Coverage – Health and Dependent Savings Accounts – Life and Disability Programs – Paid Parental Leave – Voluntary Benefit Programs – Company Sponsored Wellness Program – Wellness Reimbursement Program - Retirement Savings – Equity Opportunities – Paid time off and Paid Holidays – Employee Incentive Program. #LI-KD1 #LI-ONSITE

Our Commitment: 

Illumio believes that an environment of unique backgrounds, experiences, viewpoints, and individual contributions drives our success and makes us stronger together. We are dedicated to creating and maintaining a diverse culture and emphasizing inclusion and belonging.