Task markup

Restriction

Task markup is available only for training pools and pools uploaded with “smart mixing”.

Using markup, you can add correct task responses and hints, as well as change task types. You can turn a general task into a control task by adding the correct answer, or into a training task by adding the correct answer and a hint.

Why this is important

There are two ways to create control tasks:

How to set up automatic markup

To save time, you can opt for having your control tasks edited by Toloka experts. In this case, markup is performed within the pool.

You can set up automatic markup if the following conditions are met:

  • Markup is only available for control tasks.

  • By default, markup is supported for object classification projects created with the following templates:

    • Image classification
    • Hand gesture classification
    • Clickbait or not?
    • Sentiment analysis & Content moderation
    • Product search relevance
  • For other types of projects, you can request this feature by contacting support.

  • The pool must contain more than 100 tasks.

  • At least one required output field must be set up in the task interface.

  • You can get control tasks only once for each project. This does not apply to cases when task markup failed.

  • Control task markup performed by Toloka is only available in the interface. You can't access this feature through the API.

How does it work?

Alert

Don't make changes to the project or pool while tasks are being edited. Markup will continue without these changes taken into account, which may result in discrepancies.

  1. Make sure the pool isn't running.

  2. Make sure that the task instructions are correct.

  3. Click Edit in the Pool tasks section.

  4. Go to the General tab.

  5. In the We can label control tasks for you section, click Try it for free.

  6. Click Order. Your pool data will be sent for markup to expert Tolokers. For better quality, we use algorithms that select tasks that can be used as control tasks.

  7. Wait until the markup is completed. This process usually takes about 6 hours. The estimated time of markup completion is indicated in the Edit tasks section.

  8. After the markup is completed, the tasks are uploaded back to the pool. In the Edit tasks section on the Control tasks tab, you'll see the following message: “Control tasks are ready”. You can edit and delete the control tasks you received.

Note

If the markup process failed for some reason, contact support. Go to the tab for control tasks or to the pool page and click Contact us. After the problem is solved, you'll be able to request control task markup again.

What's next

See also

Troubleshooting

How many control tasks do I need to add?
How long does the labeling performed by experts take?
What should I do if the labeling process failed?
Contact support

Last updated: February 15, 2023

Introduction
Getting started
Important tips
Useful recommendations
Working with Toloka
Projects
Pools
Tasks
Results
Project analysis
Toloka settings
Task interface
Template Builder
HTML/CSS/JS editor
Help and support
FAQTroubleshootingSupportGlossary