Intern will create evaluation frameworks for AI agents, benchmark models, and develop automated assessments for data-focused tasks in AI systems.
Research Engineer Intern, Evaluations & Benchmarks
Location: San Francisco (Hybrid)
About TensorStax:
TensorStax is building fully autonomous AI systems to manage and optimize mission-critical data infrastructure. Our research integrates reinforcement learning and language models to enhance reasoning over large-scale data lakes and warehouses, detect failures in pipelines, and autonomously construct and optimize data workflows with high precision.
We are looking for a Research Engineer Intern to design evaluation frameworks and benchmarks that assess the autonomy, adaptability, and reliability of AI agents in data engineering environments. This role is ideal for candidates passionate about AI evaluations, language model benchmarking, and autonomous data systems.
What You’ll Do:
- Develop evaluation environments to test AI agents' ability to reason, plan, and act autonomously within mission-critical data pipelines.
- Design benchmarks to assess model capabilities in failure detection, pipeline optimization, and agentic decision-making in data workflows.
- Implement automated assessment frameworks for language model-based agents operating over data lakes and warehouses.
- Work with synthetic and real-world datasets to create robust testing environments for AI-driven data automation.
- Collaborate with research engineers to refine reward shaping strategies, guiding models toward more efficient and agentic behaviors in data-intensive tasks.
What We’re Looking For:
- Experience in language model research, with a focus on benchmarking LLMs in mission-critical domains.
- Strong background in AI evaluation methodologies, reinforcement learning, and RLHF techniques.
- Familiarity with benchmarking language models for structured and unstructured data tasks.
- Proficiency in Python and experience with ML frameworks like PyTorch or JAX.
- Hands-on experience with data lakes, warehouses, and data engineering tools (Snowflake, BigQuery, dbt, Spark, Kafka).
- High agency—proactive, resourceful, and comfortable working in a fast-paced research environment with minimal supervision.
- Attention to detail—ability to design rigorous, reproducible experiments and evaluations.
Bonus Points:
- Contributions to open-source AI benchmarks (e.g., SweBench, BIRD, SPIDER).
- Contributions to open-source agentic frameworks.
- Experience developing custom RL environments for AI evaluation.
- Strong understanding of ETL, ELT, and data transformation pipelines.
Benefits:
- Competitive internship stipend.
- 100% employer-covered health, dental, and vision insurance (for eligible interns).
- Access to Bay Club or Equinox in San Francisco.
- Opportunity to work at the cutting edge of AI evaluations and autonomous data engineering research.
Top Skills
BigQuery
Dbt
Jax
Kafka
Python
PyTorch
Snowflake
Spark
TensorStax San Francisco, California, USA Office
San Francisco, CA, United States
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
As a Full Stack Software Engineer, you will develop tools for cloud management, build APIs and UIs, and maintain cloud systems.
Top Skills:
AWSCi/CdCloudFormationGoGrpcJavaScriptKubernetesLinuxReactRestful ApisTerraformTypescript
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Lead AI-driven insights and predictive analytics for enterprise business applications through data manipulation and visualization, enhancing revenue forecasting and automation.
Top Skills:
AWSAzureGCPPythonRSnowflakeSQLTableau
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Support Alliances and national reseller partners by delivering technical enablement, training, POC/POV guidance, and partner-led sales support to grow CrowdStrike adoption and revenue while advising on integrations, SOC operations, and modern attack methodologies.
Top Skills:
APIsAvAWSAzureBashCloudCrowdstrikeDnsEdrEmail ProtectionFirewallForensicsGCPHipsHttps/Ssl/TlsIdsIncident ResponseIp RoutingIpsIt Process AutomationLinuxmacOSMalware AnalysisOrchestrationPowershellPythonSaaSSandbox AnalysisSIEMSoc OperationsSwitchingVirtualizationWindows
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
