Red team conversational AI models and agents by probing jailbreaks, prompt injections, misuse cases, bias, and multi-turn manipulation. Annotate failures, classify vulnerabilities, apply testing frameworks and benchmarks, and produce reproducible reports, datasets, and attack cases. The role requires native English and Finnish fluency, prior AI or cybersecurity red teaming experience, strong communication, structured analysis, and adaptability across projects.
This role is for one of our clients
Compensation: $48 - $62 per hour
Fluent Language Skills Required: English & Finnish. Native fluency in English and Finnish is required for this position.
Why This Role Exists
We are assembling a red team for this project - human data experts who probe AI models with adversarial inputs, surface vulnerabilities, and generate the red team data that makes AI safer for our customers.
This project involves reviewing AI outputs that touch on sensitive topics such as bias, misinformation, or harmful behaviors. All work is text-based, and participation in higher-sensitivity projects is optional and supported by clear guidelines and wellness resources. Before being exposed to any content, the topics will be clearly communicated.
Requirements
What You’ll Do
- Red team conversational AI models and agents: jailbreaks, prompt injections, misuse cases, bias exploitation, multi-turn manipulation
- Generate high-quality human data: annotate failures, classify vulnerabilities, and flag systemic risks
- Apply structure: follow taxonomies, benchmarks, and playbooks to keep testing consistent
- Document reproducibly: produce reports, datasets, and attack cases customers can act on
Who You Are
- You bring prior red teaming experience (AI adversarial work, cybersecurity, socio-technical probing)
- You’re curious and adversarial: you instinctively push systems to breaking points
- You’re structured: you use frameworks or benchmarks, not just random hacks
- You’re communicative: you explain risks clearly to technical and non-technical stakeholders
- You’re adaptable: thrive on moving across projects and customers
Nice-to-Have Specialties
- Adversarial ML: jailbreak datasets, prompt injection, RLHF/DPO attacks, model extraction
- Cybersecurity: penetration testing, exploit development, reverse engineering
- Socio-technical risk: harassment/disinfo probing, abuse analysis, conversational AI testing
- Creative probing: psychology, acting, writing for unconventional adversarial thinking
What Success Looks Like
- You uncover vulnerabilities automated tests miss
- You deliver reproducible artifacts that strengthen customer AI systems
- Evaluation coverage expands: more scenarios tested, fewer surprises in production
Similar Jobs
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Engage high-potential and prospective VIP players across communication channels, build relationships, evaluate VIP program eligibility, and optimize player engagement and onboarding strategies. The role supports acquisition efforts for new state launches, gathers customer feedback, and requires adaptability, travel, nights, weekends, and state gaming licensure.
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Manage prediction-market operations, including market creation, configuration, launches, settlement, member support, surveillance, KPI reporting, and liquidity coordination. Monitor activity for anomalies and escalate potential trade-practice violations. Collaborate with engineering, regulatory, analytics, product, clearinghouse, and ecosystem partners to support compliant, reliable market operations and new feature launches. Shift coverage may be required for real-time surveillance.
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Build and manage relationships with high-value players through communications, events, promotions, and issue escalation. The VIP Host will drive revenue, foster loyalty, gather player feedback, monitor market trends, and follow responsible gaming policies. This role requires Spanish fluency, strong customer-facing or account management experience, analytical decision-making, willingness to work nights and weekends, travel flexibility, and the ability to obtain required state gaming licenses.
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine
