Design and implement supervised fine-tuning pipelines, create benchmarks, analyze and improve production LLMs, run experiments and hyperparameter searches, build Python tooling for training/evaluation, and document research results.
Why Join This Team
Appen’s GenAI research team advances how frontier models are evaluated, improved, and deployed in production environments.
The purpose of this role is to design and implement research and engineering workflows that strengthen model performance, create new benchmarks, and improve production models without regressing on core characteristics.
This role provides hands on ownership of training and evaluation pipelines, benchmark development, and model improvement initiatives that directly influence deployed systems.
Your Impact
- Design and implement a lightweight supervised fine tuning training pipeline using open source LLMs.
- Create new benchmarks to evaluate frontier models across defined scientific and performance criteria.
- Analyze production models to identify measurable areas for improvement.
- Improve model performance through targeted retraining and hyperparameter search.
- Deploy improved models while maintaining core model characteristics and avoiding regression.
- Build Python tooling to automate training, evaluation, benchmarking, and experimentation workflows.
- Implement structured evaluation methods, including rubric based scoring and LLM as a judge workflows.
- Document experimental design, benchmark methodology, and performance results with clarity and precision.
- Iterate rapidly in a research driven environment to increase model quality and reliability.
What You Bring
- Current enrollment in or recent completion of a Master’s or PhD in Computer Science, AI, Machine Learning, Computer Engineering, or a closely related technical field.
- Strong experience working with large language models, including supervised fine tuning, prompt engineering, or model evaluation.
- Hands on experience building machine learning pipelines or research infrastructure.
- Experience improving model performance through retraining or hyperparameter tuning.
- Proficiency in Python and comfort working with machine learning frameworks and open source model ecosystems.
- Familiarity with cloud environments such as AWS or Azure.
- Strong technical problem solving ability, including use of LLMs as development aids for building and iteration.
- Ability to work independently with minimal hand holding.
- Strong written communication skills for summarising research and drafting technical documentation.
- Ability to collaborate effectively in a remote research environment.
Additional Details
- Duration: June-August
- Schedule: Full-time
- Work Type: Remote
Why You'll Love Working Here
At Appen, we foster a culture of innovation, collaboration, and excellence. We value curiosity, accountability, and a commitment to delivering the highest quality AI solutions for frontier models.
You’ll work on complex challenges that shape the future of AI across industries and geographies, alongside talented people in a culture that values humility over ego. You’ll have the flexibility to deliver in a way that works for you and your team, supported by tools, resources and development opportunities to continue to build your capability over time.
About Appen
Appen has been a leader in AI training data for over 30 years. We specialise in human generated data to train, fine tune, and evaluate models across generative AI, large language models, computer vision, and speech recognition. Our AI assisted data annotation platform and global crowd of more than 1 million contributors in over 200 countries support model pre training, supervised fine tuning, evaluation and benchmarking, safety and red teaming, and multilingual global expansion.
Top Skills
Python,Large Language Models,Open Source Llms,Supervised Fine Tuning,Prompt Engineering,Hyperparameter Tuning,Machine Learning Frameworks,Aws,Azure
Similar Jobs
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
Responsible for managing workplace operations and employee experience across offices, overseeing facilities, vendor relationships, budgeting, and promoting workplace culture through events and initiatives.
Top Skills:
AuditboardGoogle SuiteIroncladOraclePigmentSlack
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
In this role, you'll develop corporate development strategy, execute M&A transactions, manage investor relationships, and support fundraising activities.
Top Skills:
Data AnalysisFinancial Modeling
Artificial Intelligence • Fintech • Payments • Business Intelligence • Financial Services • Generative AI
The Senior Analyst in Merchant Risk Operations executes complex risk reviews, builds systems and dashboards, and collaborates with stakeholders to streamline processes and improve operational efficiency.
Top Skills:
LookerSigmaSQLTableau
What you need to know about the San Francisco Tech Scene
San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.
Key Facts About San Francisco Tech
- Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Google, Apple, Salesforce, Meta
- Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
- Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
- Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine

