Own and evolve the core platform for model APIs and agentic products: build high-throughput, low-latency distributed services, data pipelines, SDKs, context retrieval, observability, and collaborate with ML/product teams to meet strict latency and reliability SLIs.
About Boson AI: At Boson AI, we are not just building AI solutions; we are pioneering the future of enterprise AI. Driven by a passion for cutting-edge AI research, particularly in the transformative areas of large language models and agentic systems, our mission is to tackle the most complex real-world problems for businesses and unlock significant value. We are a dynamic and collaborative team of researchers and engineers who thrive on pushing the boundaries of what's possible, dedicated to delivering high-quality, reliable products that seamlessly integrate into the fabric of enterprise workflows and set new industry standards.
About the Role: Build and operate the core platform behind Boson's model APIs and agentic products. You'll own the infrastructure that every Boson agent runs on — API serving, state management, data pipelines, context retrieval, and execution runtime — and make it fast, reliable, and easy for product teams to build on.
Responsibilities
- Own and evolve the core platform infrastructure: API serving layer, state management, policy enforcement engine, and execution runtime for agentic workflows.
- Design and operate high-throughput, low-latency distributed services that back our model API products — including request routing, load management, rate limiting, and multi-tenant isolation.
- Build and maintain downstream data pipelines (ETL/ELT) for API logs, usage analytics, and billing — ensuring data correctness, freshness, and queryability at scale.
- Develop production-grade internal SDKs and libraries with clean APIs, strong type safety, and clear contracts that product teams can build on confidently.
- Architect context and memory systems for conversational workloads — low-latency retrieval, caching, and integration with vector stores and retrieval pipelines.
- Instrument end-to-end observability: define SLIs/SLOs, build structured logging and tracing, and drive reliability improvements across the platform.
- Collaborate closely with ML and product teams to integrate model serving, voice runtime, and tooling infrastructure under tight latency and quality constraints.
Qualifications
- 3+ years building and operating backend systems at scale — you've owned services that other teams depend on in production.
- Strong distributed systems fundamentals: concurrency, fault tolerance, consistency tradeoffs, capacity planning.
- Hands-on experience with data pipeline infrastructure (Kafka/Kinesis, Spark/Flink, Airflow, or similar) for log processing, analytics, or ETL workloads.
- Track record of designing APIs and frameworks adopted by other engineering teams — you care about developer experience and long-term maintainability.
- Proficiency in at least one systems language (Go, Rust, Java, C++) or Python in a performance-sensitive context.
- Comfortable working across the stack: cloud infrastructure (AWS/GCP), containerized deployments (K8s), CI/CD, and production oncall.
Bonus point
- Experience with LLM serving, agentic orchestration patterns (ReAct, planner-executor), or RAG pipelines.
- Familiarity with emerging agent integration protocols (MCP, A2A) or orchestration frameworks (LangChain, LlamaIndex).
- Background in real-time media systems (audio/video streaming, low-latency signaling).
- Experience building high-stakes platform services (payments, identity, core data) where correctness and auditability are non-negotiable.
Boson AI Santa Clara, California, USA Office
Santa Clara, CA , United States, 95054
Similar Jobs
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Develops and maintains full-stack client-side, server-side, and desktop software for mission-critical satellite systems. Designs APIs and web services, integrates capabilities across engineering teams, and optimizes applications for scalability, performance, and reliability. Responsibilities include troubleshooting, upgrades, testing, code reviews, documentation, and continuous improvement. The role requires Java experience, software lifecycle leadership, CI/CD expertise, a bachelor's degree, an active U.S. Secret clearance, and onsite work in El Segundo, California.
Top Skills:
BashCi/CdConfluenceCoverityCSSGitlabGrpcHibernateHTMLJavaJava EeJava SwingJavafxJavaScriptJenkinsJIRAJspKotlinMavenMicroservicesMicrosoft Sql ServerOracle LinuxQuarkusRed HatRelational DatabasesRestSonarqubeSpring BootSQLTailwindThymeleafXML
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Develop and maintain requirements, architectures, algorithms, interfaces, and designs for satellite command-and-control software. Lead software development in scaled Agile and DevSecOps environments, integrate software components, support customer demonstrations, and perform Product Owner, Technical Lead, Scrum Master, Release Train Engineer, or Integration and Test responsibilities. Manage development schedules, integration activities, and cost baselines while supporting high-performance ground systems.
Top Skills:
ActivemqAgileC++CcsdsCi/CdDevOpsDockerGitlabJavaKafkaKubernetesNoSQLPythonSQL
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Leads the design, development, and operation of highly available, multi-tenant data platform services on Kubernetes, including Postgres, streaming systems, and key-value stores. Defines distributed-systems architecture, failover, disaster recovery, observability, scalability, and SLOs. Builds operators, controllers, automation, and self-service APIs; drives major technical initiatives; partners across engineering; leads reliability efforts; and mentors engineers. The role requires deep hands-on software development and operation of stateful systems at scale.
Top Skills:
AksAuroraAWSAzureCi/CdCloud SqlCniCrossplaneCsiCustom Resource DefinitionsEksElasticacheEtcdGCPGitopsGkeGoIamInfrastructure As CodeKafkaKubernetesKubernetes ControllersKubernetes OperatorsMskMtlsNatsPersistent StoragePoddisruptionbudgetsPostgresRabbitMQRdsRedisService MeshStatefulsetsTerraformTopology Spread ConstraintsValkey
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
