Owns the architecture, infrastructure, evaluation, and quality improvement of multi-channel AI marketing agents. The role develops measurable agent evaluation systems, feedback loops, SEO and GEO optimization, model strategies, and supporting data pipelines. It also involves hiring and leading the AI team as the startup grows, while partnering directly with the founder.
Founding Senior AI Engineer — Agent Quality &
Evaluation
On behalf of an early-stage startup · San Francisco, on-site · Full-time
The build: Our client is building multi-channel
marketing agents across SEO, content, social, and search — and they want their
first senior AI hire to own the quality and intelligence of those agents end to
end. You'll work directly with the founder, shape agent architecture and eval
systems, and build the AI team as it scales.
This is a first-senior-hire role. You set the standard for
what "good output" even means — and make it measurable.
What you'll do
- Improve
agent output and content quality across Reddit, LinkedIn, X, and long-form
articles — with evaluation systems that make quality measurable.
- Improve
SEO and generative engine optimization (GEO) performance.
- Design
feedback loops so agents learn from real-world results and performance
signals at scale.
- Own AI
infrastructure, agent architecture, model strategy, and the data pipelines
behind them.
- Hire
and lead the AI team as the org grows.
Requirements
What we're looking for
- 3+
years building production ML or AI systems, with recent hands-on shipping
of LLM products or agents.
- Proven
eval design and outcome-signal learning in domains like search, ranking,
or recommendations.
- Strong
TypeScript (Next.js) and Python — you ship end to end and can do the
data/eval work yourself.
- Sharp
judgment about output quality, and the ability to make it measurable and
actionable.
- Ownership
experience in a startup or small team.
- Search
or ranking background — especially Google or AI answer engines. Bonus:
leading AI lab or top research program, marketing/GTM exposure, or
RL/recommender systems experience.
Similar Jobs
Artificial Intelligence • Software • Biotech • Pharmaceutical
Build and operate production AI systems for FDA regulatory intelligence, including evaluation pipelines, document ingestion and retrieval, multi-agent workflows, data pipelines, cloud infrastructure, and secure multi-tenant services. Collaborate with former FDA experts to translate domain judgment into product requirements and evaluation cases. Own architecture, reliability, observability, cost, security, and rapid 0-to-1 delivery across the stack.
Top Skills:
AlembicAmazon EcrAmazon RdsAmazon S3Anthropic ApiApplication Load BalancerAws EcsAws FargateAws Secrets ManagerClaude Agent SdkCloudflareDockerFastapiGithub ActionsGraphragKnowledge GraphsLanggraphMcpOpentofuPostgresPymupdfPythonPython-DocxReactSQLSqlalchemyTailwind CssTerraformTypescriptVercel Ai SdkVite
Artificial Intelligence • Machine Learning • Software • Generative AI
Build and optimize foundational AI infrastructure for large-scale training, reinforcement learning, rollout, and inference workloads. Responsibilities include profiling bottlenecks, optimizing GPU and distributed execution, developing scheduling and load-balancing systems, modeling performance and capacity, improving fault tolerance, and creating benchmarks and observability tools. The role partners closely with AI researchers and systems engineers to improve throughput, reliability, utilization, latency, and cost across complex GPU clusters.
Top Skills:
C++Cluster SchedulersContainer RuntimesCudaDeepspeedFsdpGoInfinibandJaxKubernetesLinuxMegatronNcclNvlinkPythonPyTorchRayRcclRdmaRoceRustSglangStorage SystemsTensorrt-LlmTorch.DistributedTritonVllm
Artificial Intelligence • Software • Automation
Own major technical initiatives from ambiguous customer and product problems through architecture, implementation, iteration, and production delivery. Build AI-native systems using LLMs, agents, evaluations, retrieval, enterprise context, and workflow automation. Set technical direction, make architectural tradeoffs, review work, unblock engineers, and translate customer signals into technical plans. Ship meaningful production work, clarify technical ownership, and reduce execution bottlenecks in a fast-moving startup environment.
Top Skills:
Ai AgentsAi SystemsInfrastructureLlmsRetrievalWorkflow Automation
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
