Prolific is building the human data infrastructure that powers the next generation of AI systems. As frontier AI labs scale their use of human-generated data for training, evaluation, and alignment, the way we measure quality, performance, and operational efficiency becomes increasingly important.
The Role
As an Applied Scientist, you will design and prototype AI/ML methods that improve data quality, scale human judgement, and support robust AI evaluation workflows. You will work on applied problems such as quality modelling, judgement aggregation, evaluation design, LLM-assisted review, and reliability testing for AI systems.
Ideal for someone with deep scientific judgement, strong applied ML skills, and a practical bias toward methods that work in real customer and product contexts. This is not a pure research role or a production ML engineering role. You will turn ambiguous problems into clear methodologies, benchmarks, models, and prototypes that product and engineering teams can adopt.
What You'll Be Doing
- Prototype AI/ML methods to improve human data quality, judgement aggregation, and AI evaluation workflows.
- Design experiments, benchmarks, and reliability tests to measure whether new methods improve quality, efficiency, or customer outcomes.
- Apply classical ML, statistics, LLMs, and agentic techniques where they create practical value.
- Use modern AI tools to accelerate prototyping, experimentation, and iteration.
- Partner closely with product and engineering to translate scientific methods into scalable platform capabilities.
- Communicate technical assumptions, trade-offs, and recommendations clearly across technical and non-technical teams.
What We are looking for
- PhD or MSc in Computer Science, Mathematics, Statistics, Machine Learning, or a related field.
- 3+ years of applied ML, AI research, or data science experience with demonstrated real-world impact.
- Experience with human-in-the-loop AI systems, including RLHF, annotation pipelines, data quality modelling, judgement aggregation, benchmarks, or AI evaluation.
- Fluency with modern LLM and agentic techniques, such as Retrieval-Augmented Generation (RAG), LLM-as-judge, multi-agent workflows, synthetic data generation, and automated quality review.
- Strong Python skills and the ability to quickly build, test, and iterate on working prototypes.
- Good judgement on when to use simple statistical methods, classical ML, LLMs, or agentic approaches.
- Ability to translate ambiguous product or customer problems into clear hypotheses, experiments, metrics, and reusable methodologies.
- Strong cross-functional communication and experience partnering with product and engineering teams.
Why Prolific is a great place to work
We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioural data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.
We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviours into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breadth and the best of humanity.
Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research. Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture.
Links to more information on Prolific
Benefits
External Handbook
Website
Youtube
Privacy Statement
By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.
