The AI Engineer will design, prototype, and deploy RAG pipelines and agentic workflows to process unstructured data for Government and Military clients. Responsibilities include collaborating on architecture and delivering technical reports while ensuring systems are scalable and reliable.
The Mission
At Unstructured, we are obsessed with transforming messy, unstructured data into a format that LLMs can actually use. Our Public Sector team has recently secured several high-impact contracts that demand more than just "off-the-shelf" solutions. We are looking for an AI Engineer who thrives at the intersection of R&D and production-grade software engineering.
You won’t just be building notebook demos; you will be architecting, prototyping, and shipping novel multimodal data processing, RAG, and agentic systems that solve critical problems for Government and Military clients. Your work will bridge the gap between one-off custom builds and a repeatable, scalable product roadmap.
What You’ll Do Day-to-Day
You will be a high-agency individual contributor, owning the lifecycle of AI solutions from initial research to AWS deployment.
50% Building & Shipping: Design and implement production-grade RAG pipelines and agentic workflows using Python. You’ll build systems that handle real-world "messy" data (PDFs, scanned docs, images, full motion video) and ensure they are performant and scalable.
30% Research & Experimentation: Stay at the bleeding edge. You’ll evaluate new models (LLMs, embedding models, object detection), prototype approaches for SBIR/government deliverables, and run experiments to prove what actually works.
20% Strategy & Collaboration: Partner with the team to document architectures, contribute to technical reports for contract deliverables, and participate in pre-sales calls to architect solutions for complex client needs.
The Ideal Candidate
We are looking for a self-directed engineer who excels in high-stakes, ambiguous environments. You are likely a strong fit if your professional background reflects the following:
Systems-First Engineering: You prioritize building reliable, scalable systems over experimental scripts. You have a track record of moving AI models out of notebooks and into production environments where latency, cost, and accuracy are treated as first-class citizens.
Technical Resourcefulness: You are comfortable working in restricted or air-gapped environments. When commercial APIs aren’t an option, you have the expertise to deploy, fine-tune, and optimize open-source models to achieve the mission objective.
Autonomous Problem Solving: You can take a high-level government requirement and translate it into a technical roadmap. You don't require constant oversight to identify the right tool for a job, whether it’s a specific vector database or a custom multimodal pipeline.
A "Generalist" Mindset: While you specialize in AI, you understand the full stack. You are as comfortable discussing embedding strategies as you are configuring AWS GovCloud infrastructure or debugging a FastAPI endpoint.
Must-Haves
- Proven experience deploying Production RAG pipelines against real-world, messy datasets.
- Deep expertise in Agentic system design (tool-use, multi-agent orchestration).
- Strong Python engineering skills—writing clean, scalable, and maintainable code
- Experience operating within AWS/GovCloud environments.
Nice-to-Haves
- Experience fine-tuning NLP or object detection models.
- Familiarity with LLM evaluation frameworks (hallucination detection, drift monitoring).
- Knowledge of government security standards and working in different classification environments and on-prem
- Security Clearance: Existing Secret/TS clearance or eligibility is a significant plus.
Your Technical Toolkit
- Languages: Python (expert-level), SQL
- LLM & Agentic Frameworks: LangChain, LangGraph, CrewAI, or similar orchestration frameworks
- RAG Stack: Retrieval with vector databases (Pinecone, Weaviate, Chroma, pgvector), graph databases (Neo4J), Elasticsearch, BM25, and Sentence-Transformers; NLP enrichment with spaCy, GLiNER, and Transformers; optimization using embedding models, reranking pipelines, and DSPy
- Evaluation & Observability: RAGAS, DeepEval, Arize Phoenix, and synthetic annotations
- Cloud & Infrastructure: AWS, SageMaker, Bedrock, S3, Lambda, Docker, and FastAPI
- Data Processing: Complex pipelines for unstructured and multimodal data, including PDFs, scanned documents, images, and audio.
Why Join Us
Opportunity to work on a dynamic team and work on cutting-edge machine learning projects.
Collaborative and innovative work environment with a focus on learning and growth.
Impactful role in shaping the company's direction and driving innovation in unstructured data processing.
Competitive compensation package, including benefits and stock options.
unstructured.io San Francisco, California, USA Office
San Francisco, CA, United States
Similar Jobs
.png)
Big Data • Machine Learning • Software • Analytics • Big Data Analytics
Develop and productionize GenAI applications for federal-sector customers, including RAG, multi-agent systems, Text2SQL, and fine-tuning solutions. Advise customers, lead production rollouts, present technical concepts, collaborate with product and engineering teams, and influence the AI product roadmap. The role requires production machine learning deployment experience, strong data science expertise, customer-facing communication, U.S. citizenship, Secret clearance eligibility, and periodic customer travel.
Top Skills:
SparkAWSAzureDatabricks Intelligence PlatformDspyFine-TuningGCPGenaiHuggingfaceLangchainMulti-Agent SystemsPandasPyTorchRagScikit-LearnText2Sql
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Leads employee communications programs, events, giveaway campaigns, and cultural initiatives. Responsibilities include developing content, managing event logistics, coordinating vendors and global distribution, maintaining the employee events calendar, and triaging communications. The role requires independently managing complex workstreams, writing for varied audiences, and partnering across internal and external teams to deliver large-scale employee experiences.
Artificial Intelligence • Cloud • Consumer Web • Productivity • Software • App development • Data Privacy
Serve as a strategic finance partner to Product leadership, building financial models, forecasts, business cases, and scenario analyses for growth initiatives, pricing, investments, and resource allocation. Analyze SaaS metrics such as ARR, retention, churn, and unit economics, translating complex data into recommendations for senior leaders. Collaborate across Product, Finance, Data Science, and Marketing while improving analysis through automation, standardized processes, and AI-enabled workflows.
Top Skills:
Ai ToolsData Visualization PlatformsSaaSSQL
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
