Top Tech Jobs & Startup Jobs in San Francisco Bay Area, CA

Applies statistics, experimentation, data analysis, and machine learning to complex datasets and AI model evaluation. Designs analyses, develops and validates models, performs Python and Pandas data transformations, creates expert prompts and datasets, identifies quantitative errors, and provides structured feedback. Produces reproducible code, analytical reports, and clear documentation while collaborating with stakeholders and reviewing peer contributions.

Create reinforcement learning environments to evaluate AI systems solving complex software engineering tasks with MCP tools. Responsibilities include designing reproducible environments, deterministic verification, golden reference solutions, bug fixes, feature implementation, codebase refactoring, and performance optimization across software codebases.

Part-time remote contractor who evaluates and improves AI-generated software engineering content. Responsibilities include prompt authoring, rubric-based model evaluation, technical documentation, fact-checking, data annotation, research, and detailed feedback on AI outputs. The role requires strong software engineering knowledge, technical writing, analytical reasoning, and attention to detail. Three or more years of professional software engineering experience is preferred, along with experience reviewing technical artifacts such as architecture documents, RFCs, design specifications, or code reviews.

Designs vulnerable and secure application fixtures addressing OWASP LLM risks, creates deterministic tests and security manifests, develops answer keys for AI and scanner evaluation, and improves AppSec assessment pipelines. The role also evaluates traditional and agent-based security architectures, documents technical findings, and collaborates on peer reviews and actionable security recommendations.