Top Tech Jobs & Startup Jobs in San Francisco Bay Area, CA

Evaluates AI-generated data science and quantitative reasoning outputs; develops prompts, datasets, reference solutions, and objective evaluation benchmarks. Identifies statistical errors, flawed methodologies, and unsupported conclusions while providing structured feedback to improve model accuracy. Responsibilities include dataset validation, statistical and quantitative assessment, quality assurance, technical documentation, and clear communication of analytical findings.

Develop and evaluate backend security coding tasks for AI systems. Responsibilities include implementing backend features, fixing bugs, refactoring code, optimizing performance, identifying and remediating vulnerabilities, conducting security-focused code reviews, and creating deterministic automated tests and verifiers. The role requires strong backend engineering, application security, debugging, testing, algorithms, and software architecture skills. Work is remote, flexible, part-time, and contractor-based.

Develop and debug production-grade full-stack applications across front-end and back-end systems. Build features, APIs, databases, and data models; troubleshoot end-to-end issues; analyze unfamiliar codebases; translate implementations across programming languages; and review AI-generated and developer-written code for correctness, security, scalability, performance, and maintainability.

Develop and evaluate coding tasks for AI systems, including bug fixes, feature development, refactoring, optimization, reproducible environments, reference solutions, automated testing, and deterministic evaluation. The role also involves debugging complex codebases, reviewing contributions, documenting technical decisions, and refining evaluation criteria. Candidates should have strong software engineering fundamentals, backend or API development experience, open-source contributions, and proficiency in at least one listed programming language.

Develop coding tasks and reference solutions for evaluating AI systems. Build deterministic verifiers, test suites, scalable backend services, and APIs. Debug complex defects, assess code quality, refactor legacy implementations, optimize performance, and evaluate AI-generated technical reasoning. Document requirements and verification criteria while independently applying software engineering judgment across algorithms, system design, testing, and reliability.

Build and maintain software engineering infrastructure for AI development and evaluation workflows. Develop rubrics, review and label preference data, create scalable pipelines, document findings, navigate complex codebases, and improve evaluation quality and consistency. Collaborate with researchers and engineers in an ambiguous, fast-paced environment while applying strong judgment, debugging, communication, and system-design skills.

Perform contract drafting, review, redlining, and simulated negotiations while evaluating AI-generated legal responses. Identify contractual risks, assess legal accuracy using structured rubrics, and provide actionable feedback to improve AI systems for contract review. Collaborate with legal, product, engineering, and research teams on datasets, evaluation guidelines, and quality standards.

Designs challenging data science and machine learning tasks, builds reproducible environments and verified reference solutions, and develops automated tests and evaluation frameworks for AI-generated code. The role analyzes AI agent performance, identifies bugs, leakage, overfitting, and methodological weaknesses, documents technical requirements, and iteratively improves task reliability and complexity.

Evaluates AI-generated software for correctness, quality, efficiency, security, and maintainability. Designs programming challenges, reference solutions, test cases, and evaluation criteria. Reviews technical reasoning related to debugging, architecture, and system design, then provides structured feedback and documentation. The role requires strong programming fundamentals, software testing and code review experience, analytical problem-solving ability, and clear technical communication.

Prepare and review individual U.S. income tax returns, identify complex tax edge cases, validate product outputs against professional tax software, and collaborate with engineers and founders to improve tax product functionality, roadmap, and user experience. The role requires substantial tax-season experience, strong federal tax knowledge, and expertise with professional tax preparation platforms.