Patrianna Logo

Patrianna

AI Quality and Automation Specialist

Posted 11 Days Ago
Be an Early Applicant
Remote
Hiring Remotely in Poland
Entry level
Remote
Hiring Remotely in Poland
Entry level
Build and maintain automated API, integration, and end-to-end tests for AI agents and critical workflows. Evaluate agent task completion, response quality, tool usage, permissions, and changes to models, prompts, and context. Integrate quality checks into CI/CD, convert production defects into regression tests, use AI to explore workflows and reproduce bugs, and share testing methods with AI and QA teams.
The summary above was generated by AI
About Patrianna

Patrianna is a fast-scaling product development company headquartered in Gibraltar, with a dynamic, global team powering our growth. We operate at the intersection of technology and entertainment, building innovative solutions that shape the future of social gaming and deliver outstanding experiences to millions of players worldwide.

We're driven by speed, ambition, and bold ideas. At our core, we're product creators and problem-solvers who thrive in a high-performance environment. We're looking for exceptional talent—smart, adaptable, and motivated individuals eager to make an impact, scale business functions at pace, and continuously improve.

Whether you're a domain expert or an agile thinker who thrives in change, at Patrianna you'll have the freedom to innovate, take ownership, and help us lead the next wave of gaming innovation.

Join us—and be part of something extraordinary.

The Role

You'll make sure our AI agents and critical workflows actually work by building smart automated testing and evaluation, then using AI to test AI.

What you will be doing
  1. You'll build and maintain API, integration and end-to-end regression tests that keep our critical workflows rock solid.

  2. You'll evaluate AI agents on task completion, response quality, tool use and permissions, and measure how model, prompt and context changes affect results.

  3. You'll build quality checks into our CI/CD pipelines and turn every production defect into a regression case, so the same bug never bites twice.

  4. You'll put agents to work exploring workflows, generating tests and reproducing bugs, then rigorously check what they find.

  5. You'll share the tools and methods that work with the wider AI team and our central QA department, raising the bar for everyone.

What we are looking for
  1. Strong test automation and programming skills in TypeScript or Python

  2. Solid hands-on experience with web apps, APIs, Git and CI/CD

  3. A structured approach to judging AI output against clear, measurable criteria

  4. A knack for reproducing, isolating and pinning down bugs fast, with a sharp eye for root cause

  5. A direct communicator who documents decisions and shares what works

  6. Bonus: agentic browser testing, LLM evaluation frameworks, Supabase, Vercel or GCP

Why you will love it
  1. You'll be at the cutting edge. AI agent evaluation is new territory, and you'll help define how it's done here.

  2. Real ownership. We act like one of three team members: accountable, trusted and close to the impact.

  3. Progress over perfectionism. We set achievable deadlines, ship, learn and improve.

  4. Genuinely open culture. Conversations happen in open Slack channels, outcomes live in Monday.com, and calls are on Google Meet with cameras on because we like feeling connected.

  5. No fluff. If it can be said in 20 words, we won't use 1,000.

  6. Great people. We want to work with the best, and we have a strict no ass-hole policy.

Equal Opportunities

We hire based on skills, drive, and ideas — nothing else. Your background, gender, age, race, ethnicity, disability, sexual orientation, religion, neurodiversity, or educational path will never be a barrier to joining us. We also welcome candidates from non-traditional career journeys and value diverse perspectives that challenge conventional thinking. Diversity fuels our innovation, collaboration, and growth, and we're committed to creating an environment where everyone can contribute their best work and thrive.

Similar Jobs

Yesterday
Remote or Hybrid
Senior level
Senior level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Provides senior Avaloq development services for banking and wealth management transformation initiatives. Designs, configures, enhances, integrates, and migrates Avaloq solutions; analyzes requirements; troubleshoots complex issues; and collaborates with clients, architects, and technical specialists. The role also contributes to testing, code quality, solution design, maintainable development practices, and knowledge sharing under an independent B2B contract.
Top Skills: AvaloqAvaloq Banking SuiteAvaloq Core PlatformAvaloq ScriptPl/SqlVersion Control
Yesterday
Remote or Hybrid
Entry level
Entry level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Provides Avaloq business analysis and functional consulting for banking and wealth management transformation initiatives. Responsibilities include gathering requirements, documenting functional specifications and user stories, facilitating stakeholder workshops, supporting solution design, testing, implementation, and platform enhancements. The role requires translating business needs into Avaloq solutions, improving processes, and collaborating autonomously with clients, technical specialists, and project teams under a B2B contract.
Top Skills: Avaloq Banking Suite
Yesterday
Remote or Hybrid
Entry level
Entry level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Analyze, transform, and validate large datasets for banking and regulatory reporting projects. Investigate data quality issues, develop ETL transformations, document mappings and findings, and collaborate with data engineers, business analysts, and stakeholders. Support data preparation for corporate client, financial product, ESG, and management reporting using SQL, Python, and PySpark.
Top Skills: ETLGitGitJupyter NotebookPandasProphecyPysparkPythonSQL

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

  • Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Google, Apple, Salesforce, Meta
  • Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
  • Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
  • Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account