Senior individual contributor who designs and delivers test architecture, reusable automation framework patterns, and Kubernetes-based test environments. Owns CI/CD quality gate integration, contract testing, test data strategy, and E2E frameworks to ensure scalable, reliable testing across a containerized microservices platform while driving adoption across product teams.
Job Description
Agilent is building a Kubernetes-based software platform spanning instruments, applications, and shared services across multiple product lines. We are embedding AI into how we build and verify software. We are looking for a Lead Test Framework Architect to lead the adoption of AI in our software test lifecycle, and to build the test frameworks, evaluation methods, and governance that let us ship AI-accelerated software safely in regulated, GxP / 21 CFR Part 11 environments.
This is a hands-on senior individual-contributor role. AI accelerates software delivery, but that acceleration exposes weaknesses downstream without a strong automated-testing control system. Building that control system - and leading the organization to adopt it - is the core of this role.
Responsibilities:
- Lead AI SDLC adoption for quality engineering. Own the strategy for AI-assisted test authoring, execution, triage, and maintenance. Prove value through reference workflows, then scale adoption across product teams.
- Build AI-native test frameworks. Design workflows that generate unit, contract, integration, and E2E suites from specifications and intent. Verify that AI-generated tests exercise specified intent, not implementation. Integrate GitHub Copilot (enterprise standard) and complementary tools (e.g., Qodo, Diffblue, Playwright AI agents, self-healing frameworks) into CI/CD quality gates.
- Test our AI features. Build evaluation and regression frameworks for LLM- and agent-powered product capabilities: non-deterministic evaluation, hallucination and grounding scoring, drift detection, and AI observability using OpenTelemetry GenAI conventions.
- Own the regulatory bridge. Integrate AI-assisted testing into validated environments using risk-based Computer Software Assurance (CSA), GAMP 5 (2nd Edition) and the ISPE GAMP AI Guide, and - for diagnostics software - IEC 62304. Preserve audit trails, traceability, and data integrity for AI-generated artifacts.
- Establish human-in-the-loop governance. Define risk-scaled review checkpoints, acceptance criteria, and rollback triggers for AI-generated code and tests.
- Define platform-wide test architecture standards for a Kubernetes-based microservices platform: testability-by-design, test environment strategy (ephemeral namespaces, environment-as-code), CI/CD quality gate architecture, contract testing for microservices and APIs, test data strategy (synthetic data, isolation, AI/ML data requirements), and E2E frameworks that work reliably in containerized environments.
- Instrument and report on impact. DORA delivery metrics augmented with AI-specific measures (coverage delta, defect-escape rate, mean-time-to-triage, eval-suite pass rates). Lead the change effort - champions, enablement, playbooks - that drives durable adoption.
Required
- 10+ years of software engineering, with 3+ years in test architecture or quality engineering architecture roles.
- Demonstrated daily, hands-on use of AI-assisted development and testing tools (GitHub Copilot, Claude Code, Cursor, or equivalent) with the ability to speak concretely to effective usage patterns and failure modes.
- Deep hands-on Kubernetes expertise for test environment design: ephemeral namespace provisioning, container-based test infrastructure, environment-as-code.
- Strong experience designing multi-level test automation frameworks: unit, contract, integration, E2E, and performance testing in distributed systems.
- CI/CD pipeline architecture and quality gate integration (GitHub Actions, Jenkins, or equivalent).
- Contract testing experience (Pact or consumer-driven contract testing) for microservices and API-first architectures.
- Demonstrated ability to deliver reusable test frameworks adopted across multiple product teams — reference implementations, not just guidelines.
- Working knowledge of testability-by-design and the ability to influence how software is architected to be inherently more testable.
- Experience leading adoption and change in an established engineering organization, with a track record of defensible outcome metrics.
Preferred
- Experience testing AI/ML, LLM, or agent-based systems: evaluation frameworks (deepeval, promptfoo, LangSmith, Langfuse, Maxim AI, or equivalent), non-deterministic regression, hallucination and grounding scoring, drift detection.
- Fluency in regulated software validation: 21 CFR Part 11, GxP / GAMP 5, CSA vs. CSV, and IEC 62304 for medical device or diagnostics software.
- Context engineering for LLMs: prompt/spec design, RAG, MCP servers, project-memory patterns.
- Observability tooling in containerized environments, ideally with OpenTelemetry GenAI Semantic Conventions.
- Performance and load testing at platform scale (k6, Gatling, Locust, or equivalent).
- Infrastructure-as-code and GitOps applied to test environment provisioning.
- Background in scientific instrument software, laboratory platforms, or life sciences / diagnostics.
We offer (depending on the country you are based in):
- An independent job in collaboration with good colleagues, in a growth-orientated organization
- A true commitment to work/life balance
- An opportunity to learn and grow
- Agilent Result Bonus, Stock Purchase Plan, Life Insurance, Pension, Healthcare, Employee Assistance Program, Holiday, Company activities
Additional Details
This job has a full time weekly schedule.Our pay ranges are determined by role, level, and location. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. During the hiring process, a recruiter can share more about the specific pay range for a preferred location. Pay and benefit information by country are available at: https://careers.agilent.com/locationsAgilent Technologies Inc. is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, disability or any other protected categories under all applicable laws.Travel Required: 25% of the TimeShift: DayDuration: No End DateJob Function: R&DAgilent Technologies Santa Clara, California, USA Office
5301 Stevens Creek Boulevard, Santa Clara, CA, United States, 95051
Similar Jobs
Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Manages full-cycle accounting for multiple client companies, including bookkeeping, invoicing, bank reconciliations, tax filings and SII submissions, annual accounts, financial reporting, cash-flow and treasury statements, and regulatory reports. Supports audits and prepares documentation for tax authorities. The role requires accurate, timely client communication, multitasking, attention to detail, and fluent Spanish and English in a hybrid Madrid environment.
Top Skills:
ExcelSii
Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Manages full-cycle accounting and client portfolios, including bookkeeping, invoicing, bank reconciliations, tax filings, SII submissions, annual accounts, financial reporting, regulatory reports, and audit support. Prepares client reports covering cash flow, payments, financial statements, and tax positions. Requires accuracy, client focus, strong organization, deadline management, teamwork, and fluent English in a fast-paced, multicultural environment.
Top Skills:
Sii
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads global R&D environmental, health, and safety programs with primary authority in industrial hygiene. Oversees exposure assessments, hazard controls, laboratory safety, containment verification, risk registers, investigations, metrics, governance, and enterprise partnerships. Leads a global EH&S community of practice, influences senior stakeholders across sites, supports vendor programs and M&A due diligence, and develops organizational capability. The role requires up to 30% domestic and international travel and includes periodic laboratory and manufacturing-site access.
Top Skills:
CorityEnablonIntelex
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
