Similar Jobs
Clarity Innovations is a trusted national security partner, dedicated to safeguarding our nation’s interests and delivering innovative solutions that empower the Intelligence Community (IC) and Department of Defense (DoD) to transform data into actionable intelligence, ensuring mission success in an evolving world.
Our mission-first software and data engineering platform modernizes data operations, utilizing advanced workflows, CI/CD, and secure DevSecOps practices. We focus on challenges in Information Warfare, Cyber Operations, Operational Security, and Data Structuring, enabling end-to-end solutions that drive operational impact.
We are committed to delivering cutting-edge tools and capabilities that address the most complex national security challenges, empowering our partners to stay ahead of emerging threats and ensuring the success of their critical missions. At Clarity, we are people-focused and set on being a destination employer for top talent, offering an environment where innovation thrives, careers grow, and individuals are valued. Join us as we continue to lead innovation and tackle the most pressing challenges in national security.
Role
The Principal Software Engineer is a senior individual contributor responsible for designing and implementing data transformation pipelines and driving the technical evolution of the Unified Data Model (UDM) team's analysis infrastructure. This role requires deep Java expertise, with a specific focus on transitioning legacy data pipelines to modern NiFi-based workflows and integrating with enterprise data services.
Responsibilities
- Serve as the team's Java subject matter expert, providing technical guidance, code reviews, and architectural input on Java-based components.
- Design and implement custom Apache NiFi processors to support UDM data processing and routing requirements.
- Lead and assist in the migration of existing analysis pipelines to NiFi-based workflows, ensuring continuity, correctness, and performance parity.
- Collaborate with data transport, platform, and infrastructure engineers to align pipeline design with enterprise standards.
- Troubleshoot complex integration issues across data pipeline stages, including format conversion, schema validation, and service connectivity.
- Contribute to technical documentation, architecture decision records, and pipeline design artifacts.
- Mentor junior and mid-level engineers on Java best practices, NiFi patterns, and integration design.
- Deliver within an Agile/Scrum framework, actively participating in sprint planning, backlog refinement, and technical reviews.
Requirements
Clearance & Compliance
- Active Secret security clearance (Clearable to TS/SCI).
- Ability to operate in classified and constrained environments in accordance with all applicable security protocols.
Technical Skills
- Expert-level proficiency in Java, including design patterns, concurrency, performance tuning, and enterprise integration patterns.
- Strong understanding of data integration patterns, schema validation, and format transformation (JSON, XML, Avro, Protobuf, or similar).
- Experience integrating data pipelines with enterprise services, including APIs, message brokers, or data warehouses.
- Familiarity with version control, CI/CD pipelines, and DevSecOps practices.
Experience
- 10+ years of professional software engineering experience with a strong emphasis on Java development and data pipeline engineering.
- Prior experience in DoD, defense contracting, or other classified program environments strongly preferred.
- Track record of leading technical migrations or modernization efforts on production data systems.
- Experience deploying Java applications into containerized environments
Preferred Qualifications
- Experience with deploying Java applications in Kubernetes environments.
- Experience with deploying Java applications using GitOps (e.g., Flux, ArgoCD) methodologies.
- Experience integrating with Intelligence Community data flow systems.
- Experience working in cross-functional Agile teams as a technical lead or senior individual contributor.
- Hands-on experience designing and implementing Apache NiFi flows, custom processors, and controller services.
- Demonstrated experience migrating or re-architecting legacy data pipelines to modern workflow frameworks.
What you need to know about the San Francisco Tech Scene
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
