Quality
High-quality data is attainable at any scale with our tools and know-how.
Set the bar high
Data quality is vital to your projects. No matter how fast you need to scale, our platform is designed to deliver guaranteed quality on any amount of data.
Want to know how it works? Read on.
Get better data with our proven approach to quality
- Intelligent crowd selection
- Crowd training and testing
- Benchmark tasks to monitor quality
- Post-verification and spot checks
- State-of-the-art aggregation tools
- Platform-wide anti-fraud system
Start with the right crowd
We apply extensive filters and checks to match crowd performers to your projects.
Crowd targeting
Target users on the right devices, with the right language and location, and with the right skills and interests for your tasks.
Behavioral checks
Automatically monitor the time spent on tasks and patterns of user behavior to prevent cheating.
Accuracy rules
Define unique skills to measure accuracy on your specific task and filter performers by skill level.
Leverage performer quality with smart ranking
We use ML algorithms to analyze each performer's relevant skills and quality in similar tasks across the platform — and pre-select the best individuals for your project.
Establish quality benchmarks
Measure performance with a combination of quality control options.
- Training and testingUse practice tasks to teach users a skill for your project
and verify their mastery with an exam. - Advanced solutions Use our Crowd-Kit library to evaluate accuracy in complex
tasks such as text generation and image segmentation. - Benchmark tasks (honeypots) Add control tasks with correct answers to automatically
monitor individual quality while labeling is in progress. - Automatic benchmarks (coming soon) Benchmark tasks can be generated by experienced users
and verified with our advanced technologies.
Harness the wisdom of the crowd
Toloka uses cutting-edge aggregation techniques to maximize the power of the crowd and deliver high-performing data.
Categorical Aggregations
For classification tasks with overlap, standard categorical aggregations can give you the most accurate labels.
Non-categorical Aggregations
For complex open-ended tasks, we have a collection of cutting-edge solutions to measure accuracy in any scenario.
No room for fraud
Our real-time anti-fraud system is built into every level of the platform to ensure that the people on the other end are real people. Our dedicated anti-fraud team works non-stop to detect and prevent cheating in any form.
Rely on our crowd expertise
For complex scenarios, combine the power of the crowd with trained experts to achieve high-quality results.
- — Train and test performers to do specific tasks and assign skill levels.
- — Dynamic pricing based on skill level motivates performers and improves quality.
- — Expert crowd performers can review and validate tasks completed by other people.
To learn more, see our documentation and knowledge base.