Project 1. Does the image contain a specific object?

In this project, you ask the Tolokers if a photo contains traffic signs.

Create a project

In the interface:

1. Choose a preset:

   1. Click Create a project.

      

   2. Click Do it myself.

   3. Select the Image classification preset.

2. In the General information section, add the project name and description:

   1. In the Name to show Tolokers field, enter Are there traffic signs in the picture?.

   2. In the Description for Tolokers field, enter Look at the picture and answer whether or not it shows any traffic signs.

   3. Optionally add a Private comment.

3. Edit the task interface in the editor you selected:

   Template Builder

   HTML/CSS/JS editor

   1. The task interface describes how the elements should be arranged in the task.

      The template has pre-configured validation. The Toloker won't be able to submit a response without selecting one of the options.

      For more information, see the Template Builder Help:

      • Setting up conditions.
      • Image classification template.

   2. On the Configuration panel, replace lines 19 to 28 in the code:

      "label": "What is the cat's mood?",
      "options": [
          {
              "label": "Good",
              "value": "ok"
          },
          {
              "label": "Bad",
              "value": "bad"
          },
      Copy text

      with:

      "label": "Are there traffic signs in the picture?",
      "options": [
          {
              "label": "Yes",
              "value": "ok"
          },
          {
              "label": "No",
              "value": "bad"
          },
      Copy text

   3. Click Show specifications to see the input and output data fields.

      Input data fields are created from the code on the Example of input data tab.

      The output data fields depend on the components that use data.output and values supported by it.

      Learn more about input and output data fields in the Template Builder Help.

      • Input data field: image — A link to an image.

        Change the data type to string to add links to your files.

      • Output data field: result — string for saving the Toloker's response.

      What are input and output data?

4. In the Instructions for Tolokers editor, enter the instructions Tolokers will see when they start doing your tasks. You can add text, tables, and images to your instructions.

   Instructions text:

   Look at the image and answer whether there are any traffic signs in it.
   If there are, click "Yes".
   If there aren't, click "No".
   For example, there are traffic signs in the image, so the correct answer is "Yes".
   Copy text

   Tip

   If you want to add a task examples in the instruction, complete it yourself in the preview mode. Take screenshots, upload them to photo hosting or cloud storage and insert image links in the instructions by clicking the button on the toolbar.

5. In the upper-right corner, click Save.

   Learn more about working with the project in the Project section.

Create a pool

1. Open the page of the project titled Are there traffic signs on the picture?.

2. Click Create new pool on the project page.

3. Select the value in the Pool type drop-down list.

   Pool types
   Note

   If the price per task suite is zero, you must select the pool type.

4. Set the Pool name (visible only to you) field. Only you will see this pool name on the project page.

5. Specify the pool description which will be displayed instead of the project description in the task list for Tolokers. By default, Tolokers see the description from the project settings. To use a different description, uncheck the Use project description box and set Public description. If necessary, click + Private comment to add a private project description that only you will see.

6. Click Create.

7. At the Select the audience for your task step, set up filters to select Tolokers for your pool.

   1. Use the Languages filter to select Tolokers with knowledge of the language.

   2. Clear My tasks may contain shocking or pornographic content if your project has none of those.

   3. Tasks in pools will be automatically available in the web version of Toloka and the mobile app. If you want to change the default settings and limit the visibility of the task for any of the versions, add the Client filter and select the desired value: Toloka web version or Toloka for mobile.

8. (optional) In the Speed/quality balance section, specify the desired quality level. Improving quality may reduce the speed of task completion because the pool will be available to fewer Tolokers. Learn more in the Speed/quality balance section.

9. At the Set the task price and overlap step, set up how much a single task will cost for you.

   1. In Price per task suite, $, set the amount of money to pay per task suite done by one Toloker. For example, 0.01.

   2. In the Overlap field, define how many Tolokers must do each task.

10. At the Setup quality control step, set quality control rules for more accurate results:

    1. Click Add a quality control rule.

    2. Find the Rules block in the list and choose Control tasks.

    3. Set a rule for control task: if the number of responses to the control questions is ≥ 3 and correct responses (%) to the control questions is < 60, then ban the Toloker on project for 10 days. Specify the Control task as a reason.

       Learn more in Quality control.

11. At the Add optional pool settings step, specify the Time per task suite, sec.

    The time should be long enough to read the instructions and wait for task data to load. For example, 600 seconds.

12. At the Prepare and upload data step, upload your task data.

    1. Create the tasks for Tolokers:

    2. To download a template, click one of the buttons:

       • Template in XLSX
       • Template in TSV
       • Template in JSON

    3. Open the downloaded file, and replace the sample data with your information.

    Use sample data

    1. Click Drop file here or select, and upload the file you’ve just made.

    2. Click Continue.

    3. In the window that opens, configure the file upload settings:

       1. Choose Smart mixing.

       2. In the General tasks field, specify 9.

       3. In the Training tasks field, specify 0.

       4. In the Control tasks field, specify 1.

       5. Click Combine tasks into suites.

13. Create a control task.

    1. In the window that opens, click Create control tasks.

    2. In the window that opens, turn on the result option in the Create control task section.

    3. Choose the correct answer to the question.

    4. Click Save and go to next.

    5. Click Are there traffic signs in the photo? to exit the task markup mode.

    Note

    In small pools, control tasks should be 10–20% of all tasks. Include different versions of correct responses in equal amounts. View the distribution of responses on the Edit tasks page, Control tasks tab.

14. At the Double-check your project and try out tasks step, check how the task will look from the Toloker's point of view.

    Note

    This step will be enabled after you complete the previous steps. You can skip this step by clicking Do it later.

15. Click to start the pool.

Get the results

1. Next to the Download results button, click .

2. Choose Dawid-Skene aggregation model from the list. Learn more about Result aggregation based on the Dawid-Skene model.

3. In the window that opens, click Yes.

4. At the top of the page, click View the list of operations.

   Note

   To track the progress, refresh the page from time to time. Aggregation takes from 5 to 20 minutes. You can start designing another project meanwhile.

5. When the operation is complete, download the results file. To do this, click Download in the Files column.

6. Use the results file in project 2.

What's next

• Create Project 2 to select objects.

Last updated: March 10, 2023