Project 1. Does the image contain a specific object?

In this project, you ask the Tolokers if a photo contains traffic signs.

  1. Create a project
  2. Create a pool
  3. Upload tasks
  4. Get the results

Create a project

In the interface:
  1. Choose a template:

    1. Click the + Create project button.

    2. In the window that opens, find the Image classification template and click Select.

  2. Provide general information:

    1. In the Name to show Tolokers field, enter Are there traffic signs in the picture?.

    2. In the Description for Tolokers field, enter Look at the picture and answer whether or not it shows any traffic signs.

    3. Save the changes.

  3. Edit the task interface in the editor you selected:

    Template Builder
    HTML/CSS/JS editor
    1. The task interface describes how the elements should be arranged in the task.

      The template has pre-configured validation. The Toloker won't be able to submit a response without selecting one of the options.

      For more information, see the Template Builder Help:

    2. On the Configuration panel, replace lines 19 to 28 in the code:

              "label": "What is the cat's mood?",
              "options": [
                  "label": "Good",
                  "value": "ok"
                  "label": "Bad",
                  "value": "bad"
      Copied to clipboard


              "label": "Are there traffic signs in the picture?",
              "options": [
                  "label": "Yes",
                  "value": "ok"
                  "label": "No",
                  "value": "bad"
      Copied to clipboard
    3. Click Show specifications to see the input and output data fields.

      Input data fields are created from the code on the Example of input data tab.

      The output data fields depend on the components that use data.output and values supported by it.

      Learn more about input and output data fields in the Template Builder Help.
      • Input data field: image — A link to an image.

        Change the data type to string to add links to your files.

      • Output data field: result — string for saving the Toloker's response.
      What are input and output data?

      Input data is types of objects that are passed to the Toloker for completing the task. For example, this could be a text, an image, or geographic coordinates.

      Output data is types of objects that you receive after the task is completed. For example, this could be one of several response options, typed text, or an uploaded file.

      If you add interface elements to the task template, the corresponding fields in the Data specification block will be created automatically.

    1. Edit the HTML block in the Task interface.

      1. After the line with the image, enter the question:

        <div>Are there <b>traffic signs</b> in the picture?<div>
      2. Change the labels on the response options: GoodYes, BadNo:

        {{img src=image width="100%" height="400px"}}
        <div>Are there <b>traffic signs</b> in the picture?<div>
        {{field type="radio" name="result" value="OK" label="Yes" hotkey="1"}}
        {{field type="radio" name="result" value="BAD" label="No" hotkey="2"}}
        {{field type="radio" name="result" value="404" label="Loading error" hotkey="3"}}
        Copied to clipboard
    2. Leave the JS, CSS, and Data specification blocks unchanged.

      Learn more about Specifications in Input and output data.

    3. Click the Preview task button to view the task.

      Note. The project preview shows one task with standard data. You can define the number of tasks to show on the page later.
    4. In the window that opens, check if the task options work correctly. In the lower-right corner, click Submit.

    5. Exit preview mode. In the lower-left corner, click ExitExit . If there were errors when testing, check the code blocks that you entered.

  4. Save the changes.

  5. In the Instructions for Tolokers field, enter the instructions and add an image.

    1. Instructions:
      Look at the image and answer whether there are any traffic signs in it.
      If there are, click Yes.
      If there aren't, click No.
      For example, there are traffic signs in the image, so the correct answer is Yes.
      Copied to clipboard
    2. To add an image, click and provide a link to the image you want to use as an example.
    3. Save the changes.

  6. In the upper-right corner, click Finish

    Learn more about working with the project in the Project section.

Create a pool

  1. Open the page of the project titled Are there traffic signs on the picture?.

  2. Click the Add a pool button.

  3. Specify the Pool name.

  4. (optional) Add a Private comment. This information is available only to you.

  5. In the Audience block in the Tolokers section, filter Tolokers by language:

    1. Click Add filter.

    2. Add the Region by phone number and Languages filters and select Tolokers from Russia, Ukraine, Kazakhstan, and Belarus who indicated in their profiles that they know Russian.

    Tasks in pools will be automatically available in the web version of Toloka and the mobile app. If you want to change the default settings and limit the visibility of the task for any of the versions, add the Client filter and select the desired value: Toloka web version or Toloka for mobile.

  6. (optional) In the Speed/quality balance section, specify the desired quality level. Improving quality may reduce the speed of task completion because the pool will be available to fewer Tolokers. Learn more in the Speed/quality balance section.

  7. In the Price block, find the Price per task suite field and specify the price. For example, 0.01.

  8. In the Quality control block, set up quality control settings for the pool:

    1. Click Add a quality control rule.

    2. Find the Rules block in the list and choose Control tasks.

    3. Set a rule for control task: if the number of responses to the control questions is ≥ 3 and correct responses (%) to the control questions is < 60, then ban the Toloker on project for 10 days. Specify the Control task as a reason.

      Learn more in Quality control.

  9. In the Task overlap section, enter 3 in the The number of Tolokers to complete each task field.

  10. In the Additional settings block, enter 600 in the Time per task suite field.

  11. Click Create a pool.

Upload tasks

Download the sample upload file. You can find it on the pool page. Use it to prepare your own file with tasks.

  1. Click Upload. In the window that opens, you can download the file template.
    Use sample data

    If you want to see what your project will look like after the launch, but you don't have any labeling tasks yet, you can upload ready-made sample data to the pool.

    Click Use sample data next to Attach the prepared file with data. This lets you avoid any additional actions with files.

    Once you've finished working with the sample data and everything looks good, prepare your data and upload it to the pool.

  2. In the window that opens, configure the file upload settings:

    1. Choose Smart mixing.

    2. In the General tasks field, specify 9.

    3. In the Training tasks field, specify 0.

    4. In the Control tasks field, specify 1.

    5. Click Upload.

    6. In the window that opens, choose the file with tasks to upload and click Open.

    7. In the window that opens, check the number of tasks and click Add.

  3. Create a control task.

    1. Click Edit.

      Note. If you selected something else instead of smart mixing, click Edit. If this button is missing, delete the file and upload it again.
    2. In the window that opens, click Create control tasks.

    3. In the window that opens, turn on the result option in the Create control task section.

    4. Choose the correct answer to the question.

    5. Click Save and go to next.

    6. Click Are there traffic signs in the photo? to exit the task markup mode.


    In small pools, control tasks should be 10-20% of all tasks. Include different versions of correct responses in equal amounts. View the distribution of responses on the Edit tasks page, Control tasks tab.

  4. Click to start the pool.


    The tasks will be completed by real Tolokers in Toloka. Recheck your project setup before you start the pool.

Get the results

  1. Next to the Download results button, click .

  2. Choose Dawid-Skene aggregation model from the list. Learn more about Result aggregation based on the Dawid-Skene model.

  3. In the window that opens, click Yes.

  4. At the top of the page, click View the list of operations.


    To track the progress, refresh the page from time to time. Aggregation takes from 5 to 20 minutes. You can start designing another project meanwhile.

  5. When the operation is complete, download the results file. To do this, click Download in the Files column.

  6. Use the results file in project 2.

What's next