Improve data labeling quality with automated control tasks

Toloka Team
by Toloka Team
Image

Subscribe to Toloka News

Subscribe to Toloka News

How it works

Once you've requested control tasks, the platform's algorithms analyze your pool and select the tasks that are best suited for quality control. These tasks are then labeled by experienced performers and uploaded back to your pool. Launching a pool that contains control tasks allows you to closely monitor percentages of correct answers. This way, you can block any Toloker whose work you aren't satisfied with, leading to an improved dataset quality overall.

How to request control tasks

  • Create a new project and upload the data to the main pool using Smart mixing. You will need no fewer than 100 tasks uploaded to your pool.
  • On the top of pool overview page click Prepare control tasks.
Image
  • You will see the new beta feature offering control tasks from our experts. Click Try it for free.
Image

A few things to keep in mind:

  • You will get a notification once your control tasks are ready. Time to delivery and elapsed time will be displayed throughout.
  • We recommend that you refrain from editing pool settings and instructions while your control task is being prepared - we won't be able to accommodate these changes.
  • This new function is accessible primarily in object classification tasks.

If you have a different type of project and can't see this option but would like to use it, please contact us on Slack or write to our Support team.

How to use control tasks in your projects

With control tasks in your pool, you can track how well performers are doing on your project and use that information to improve labeling quality. The important thing to remember is that control tasks alone don't affect quality. To see actual improvements, you need to set up quality control rules. For example, you can set up a rule to block Tolokers who perform poorly, or grant/revoke access to tasks based on their skill level. You can also use this information to reward diligent performers, offering them an additional incentive to keep up the good work. Learn more about control tasks and quality control rules here.

Get started

We appreciate your feedback

Toloka values your opinion - we work 24/7 to offer you the highest possible labeling quality. Please share your feedback about this new feature and help us improve our product. You can reach us on Slack or by emailing our Support team.

Article written by:
Toloka Team
Toloka Team
Updated: 

Recent articles

Have a data labeling project?

Take advantage of Toloka technologies. Chat with our expert to learn how to get reliable training data for machine learning at any scale.
Fractal