Dynamic overlap (also known as incremental relabeling or IRL) saves your budget by not setting a fixed overlap for all tasks in the pool. An algorithm determines how many Tolokers should complete the same task in order to get a reliable result, based on the Toloker skill levels and the consistency of their responses. It works well for tasks with response options and an overlap of 2–5 (like classification tasks). Learn more about dynamic overlap
Each Toloker has their own strengths. Some people are good at finding typos, some like selecting objects in images, others do a great job transcribing audio, and some people are talented secret shoppers. But they all need to practice before completing a new task. Your project should have a training pool and an exam (a control task pool) — this way, you can choose the best Tolokers. A "retry pool" will help Tolokers who make mistakes improve their skills and get a second chance to complete tasks.
You can issue bonuses to Tolokers for completing tasks well or increase their payment depending on the skill level. This encourages Tolokers to try harder on tasks
If your projects meet these requirements, we'll be happy to partner with you and recommend your services to requesters.Contact support
Last updated: February 15, 2023