Task markup is available only for training pools and pools uploaded with “smart mixing”.
Using markup, you can add correct task responses and hints, as well as change task types. You can turn a general task into a control task by adding the correct answer, or into a training task by adding the correct answer and a hint.
There are two ways to create control tasks:
Automatic markup by Toloka experts.
Manual markup, which allows you to create both control tasks and training tasks.
To save time, you can opt for having your control tasks edited by Toloka experts. In this case, markup is performed within the pool.
You can set up automatic markup if the following conditions are met:
Markup is only available for control tasks.
By default, markup is supported for object classification projects created with the following templates:
For other types of projects, you can request this feature by contacting support.
The pool must contain more than 100 tasks.
At least one required output field must be set up in the task interface.
You can get control tasks only once for each project. This does not apply to cases when task markup failed.
Control task markup performed by Toloka is only available in the interface. You can't access this feature through the API.
Don't make changes to the project or pool while tasks are being edited. Markup will continue without these changes taken into account, which may result in discrepancies.
Make sure the pool isn't running.
Make sure that the task instructions are correct.
Click Edit in the Pool tasks section.
Go to the General tab.
In the We can label control tasks for you section, click Try it for free.
Click Order. Your pool data will be sent for markup to expert Tolokers. For better quality, we use algorithms that select tasks that can be used as control tasks.
Wait until the markup is completed. This process usually takes about 6 hours. The estimated time of markup completion is indicated in the Edit tasks section.
After the markup is completed, the tasks are uploaded back to the pool. In the Edit tasks section on the Control tasks tab, you'll see the following message: “Control tasks are ready”. You can edit and delete the control tasks you received.
If the markup process failed for some reason, contact support. Go to the tab for control tasks or to the pool page and click Contact us. After the problem is solved, you'll be able to request control task markup again.
Last updated: February 15, 2023