As a Senior Software Engineer, you'll build AI features focused on LLMs and RAG systems, collaborating across teams to ensure robust performance.
About Glue
Glue is not just another team chat app. We're building the first platform for agentic team chat—a workspace where AI agents and humans collaborate as peers. Our platform supports leading AI models (GPT-5, Claude, Gemini, and open-source alternatives) and integrates with thousands of apps through the Model Context Protocol (MCP), enabling teams to direct actions across their entire tech stack without leaving chat.
We recently raised $20M in Series A funding. Co-founded by David Sacks and Evan Owen, Glue is redefining how teams communicate and get work done in the AI era.
Opportunity
We’re looking for an AI/ML Engineer with a strong foundation in large language models (LLMs) and retrieval-augmented generation (RAG) to help us build smart, useful, and scalable AI features. You’ll work across the stack—from prototyping prompts and pipelines to shipping production systems—and play a key role in evaluating model performance and behavior. You'll collaborate across engineering, product, and design to ship experiences that feel magical—but are grounded in robust infrastructure and measurable outcomes.
About You
- Have shipped across the stack to production—from data ingestion and model integration to serving and monitoring—and understand what it takes to build resilient ML-powered features.
- Have dealt with issues like latency, cost, edge cases, and prompt brittleness, and you’ve built systems that people rely on
- Understand the nuances of RAG systems, including text chunking, hybrid retrieval (semantic + keyword), vector store tuning, and relevance optimization.
- A self-starter, a builder, a doer. You know how to architect a project the “right way”, but also know when to ask about requirements and make tradeoffs depending on the maturity of the project
- Care about details but know when to avoid getting lost in the weeds and prioritize the immediate outcomes needed.
- Want to be part of a scrappy, early team who is building something ambitious and exciting.
What will help you succeed
- 5+ years of engineering experience, ideally including ML, NLP, or applied AI experience
- 2+ years hands-on with LLMs, transformers, and production-grade generative AI systems
- 2+ years of experience with a server-side language such as Go, TypeScript, Python, etc.
- Experience building RAG systems using frameworks like LangChain, LlamaIndex, or custom stacks
- Experience with multi-modal inputs, fine-tuning open models, or orchestrating agent-style systems
- Familiarity with evaluation techniques, tooling (e.g., Braintrust, Promptfoo, etc.) and quality metrics
- Independent and self motivated—maintaining side projects or libraries a major plus
Benefits
We offer a competitive salary in addition to significant equity, a generous healthcare package, and whatever equipment you need to excel at your job.
Flexible vacation policy—we’re moving quickly but want a sustainable culture.
This role is based in our San Francisco headquarters.
Glue is committed to providing equal employment opportunities for all applicants and employees. Glue doesn’t discriminate on the basis of any protected characteristic, including race, color, ancestry, national origin, religion, creed, age, disability, sex, gender, sexual orientation, gender identity, gender expression, medical condition, genetic information, family care or medical leave status, marital status, domestic partner status, military and veteran status, or any other characteristic protected by US federal, state or local laws, or the laws of the country or jurisdiction where you work.
Glue San Francisco, California, USA Office
San Francisco, California, United States
Similar Jobs
Financial Services
Designs, builds, and operates secure, scalable cloud and GPU infrastructure platforms for enterprise AI/ML workloads. Leads architecture, production coding, Kubernetes and container operations, CI/CD, infrastructure automation, performance optimization, and reliability efforts. Partners with AI/ML and platform teams to support distributed multi-GPU training and inference. Provides technical leadership while advancing responsible AI-assisted engineering, secure SDLC practices, automation, and operational excellence.
Top Skills:
BcmC#Ci/CdCloud InfrastructureCudaDistributed SystemsDockerGoGpu InfrastructureJavaKubernetesLinuxMicroservicesMlflowNvidia DcgmNvidia DriversPythonRay.IoSlurm
Automotive • Big Data • Information Technology • Robotics • Software • Transportation • Manufacturing
Leads model distillation, parameter-efficient fine-tuning, reinforcement-learning alignment, dataset development, quantization-aware training, and evaluation for multimodal AI models deployed on automotive edge hardware. Sets architectural direction for optimization pipelines, selects foundation models, manages continuous improvement loops, and ensures compact models retain reasoning, vision, language, accuracy, and safety performance after compression.
Top Skills:
DeepspeedDirect Preference Optimization (Dpo)Hugging FaceLoraMegatronPyTorchQloraQuantization-Aware TrainingRayReinforcement Learning From Human Feedback (Rlhf)
Financial Services
Designs, develops, and troubleshoots secure, scalable ML software systems and services. Builds production code, architecture artifacts, APIs, and distributed microservices; analyzes data to improve applications and system architecture. Uses AI-assisted development tools responsibly, validates generated outputs, and guides peers on secure engineering practices. Requires Python and ML framework expertise, cloud-native technologies, public cloud experience, distributed systems knowledge, and Cassandra or equivalent NoSQL experience. Preferred experience includes GPU workloads, model serving, distributed inference, model compression, and edge deployment.
Top Skills:
AWSC++CassandraDockerGCPGpuJavaKubernetesNoSQLPythonPyTorchTensorFlowTensorflow ServingTorchserveTriton Inference Server
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
