General criteria

Dynamic overlap

Dynamic overlap (also known as incremental relabeling or IRL) saves your budget by not setting a fixed overlap for all tasks in the pool. An algorithm determines how many Tolokers should complete the same task in order to get a reliable result, based on the Toloker skill levels and the consistency of their responses. It works well for tasks with response options and an overlap of 2–5 (like classification tasks). Learn more about dynamic overlap

Toloker selection: training — exam — retry

Each Toloker has their own strengths. Some people are good at finding typos, some like selecting objects in images, others do a great job transcribing audio, and some people are talented secret shoppers. But they all need to practice before completing a new task. Your project should have a training pool and an exam (a control task pool) — this way, you can choose the best Tolokers. A "retry pool" will help Tolokers who make mistakes improve their skills and get a second chance to complete tasks.

Bonus program and dynamic pricing

You can issue bonuses to Tolokers for completing tasks well or increase their payment depending on the skill level. This encourages Tolokers to try harder on tasks

If your projects meet these requirements, we'll be happy to partner with you and recommend your services to requesters.

See also

Contact support

Last updated: February 15, 2023

Introduction
Getting started
Important tips
Useful recommendations
Working with Toloka
Projects
Pools
Tasks
Results
Project analysis
Toloka settings
Task interface
Template Builder
HTML/CSS/JS editor
Help and support
FAQTroubleshootingSupportGlossary