Project 1. Does the photo contain a certain object?

In this project, you will ask performers if there are shoes in the photo.

  1. Create a project
  2. Create a pool
  3. Upload a TSV file with tasks.
  4. Get the results

Create a project

To create a project, open Toloka for requesters.

  1. Choose a template:

    1. Click Create project.

    2. Select the Image classification template.

  2. Provide general information:

    1. In the Name for performers field, enter Are there shoes in the photo?.

    2. In the Description for performers field, enter Look at the picture and answer if there are shoes in it..

    3. Optionally add a Private comment.

    4. Save the changes.

  3. Edit the task interface in the editor you selected:

    Template Builder
    HTML/CSS/JS editor
    1. The task interface describes how the elements should be arranged in the task.

      The template has pre-configured validation. The performer won't be able to submit a response without selecting one of the options.

      For more information, see the Template Builder Help:

    2. On the Configuration panel, replace lines 19 to 28 in the code:

              "label": "What is the cat's mood?",
              "options": [
                {
                  "label": "Good",
                  "value": "ok"
                },
                {
                  "label": "Bad",
                  "value": "bad"
                },
      Copied to clipboard

      with the following:

              "label": "Are there shoes in the image?",
              "options": [
                {
                  "label": "Yes",
                  "value": "ok"
                },
                {
                  "label": "No",
                  "value": "bad"
                },
      Copied to clipboard
    3. Click Show specifications to see the input and output data fields.

      Input data fields are created from the code on the Example of input data tab.

      The output data fields depend on the components that use data.output and values supported by it.

      Learn more about input and output data fields in the Template Builder Help.

      • Input data field: image — A link to an image.

        Change the data type to string to add links to your files.

      • Output data field: result — string for saving the performer's response.
      What are input and output data?

      Input data is types of objects that are passed to the performer for completing the task. For example, this could be a text, an image, or geographic coordinates.

      Output data is types of objects that you receive after the task is completed. For example, this could be one of several response options, typed text, or an uploaded file.

      If you add interface elements to the task template, the corresponding fields in the Data specification block will be created automatically.

    1. In the Task interface, edit the HTML block.

      • After the image line, add the question:

        <div>Are there <b>shoes</b> in the picture?<div>
      • Change the labels on the response options: GoodYes, BadNo:

        {{img src=image width="100%" height="400px"}} <div>Are there <b>shoes</b> in the picture?<div>
        
        {{field type="radio" name="result" value="OK" label="Да" hotkey="1"}}
        {{field type="radio" name="result" value="BAD" label="Нет" hotkey="2"}}
        {{field type="radio" name="result" value="404" label="Ошибка загрузки" hotkey="3"}}
        Copied to clipboard
    2. Leave the JS, CSS, and Data specification blocks unchanged.

      Learn more about the Specification parameters in Input and output data.

    3. Click to see the performer's view of the task.

      Note. The project preview shows one task with standard data. You can define the number of tasks to show on the page later.
      1. In the window that opens, check if the task options work correctly. In the lower-right corner, click Submit.

      2. Exit preview mode.

        In the lower-left corner, click Exit. If there were errors when testing, check the code blocks that you entered.

  4. Save the changes.

  5. In the Instructions for performers field, enter the instructions and add an image.

    1. Instructions:
      Look at the picture and determine if there are shoes in the picture.
      If there are, click Yes.
      If there aren't, click No.
      For example, there are shoes in the photo, so the correct answer is Yes.
      Copied to clipboard
    2. To add an image, click and provide a link to the image you want to use as an example.

    3. Save the changes.

  6. To go back to the Projects page, click Finish editing.

Learn more about working with the project in Project.

Create a pool

To create a pool:

  1. Open the page of the project titled Are there shoes in the photo?.

  2. Click the Add pool button.

  3. Specify the pool name.

  4. (optional) Enter the Private description.

    1. Turn on the Add a private description option.

    2. In the Private description field, enter the pool description. This information is available only to you.

  5. In the Price per task suite section, go to the Price in US dollars field and specify 0.01. You can see more examples of pricing here.

  6. In the Performers section, filter performers by language:

    1. Click Add filter.

    2. Find the Performer profile block in the list and choose the Languages skill.

    3. In the ? field specify =.

    4. In the Value field, specify Russian.

  7. In the Quality control section, specify quality control settings for the pool:

    1. Click Add quality control rule.

    2. Find the Rules block in the list and choose Control tasks.

    3. Set a rule for control tasks: If the number of responses to control questions is greater than or equal to 3 and the percentage of correct responses to control questions is less than 60, then restrict the performer's access to the project for 10 days. Specify the Control task as a reason.

      Learn more in Quality control.

  8. In the Overlap section, go to the Overlap field and enter 3.

  9. (optional) In the Speed/quality section, specify the percentage of top-rated users who will get access to the pool. Learn more in Speed/quality balance.

  10. In the Parameters section, go to Time to submit assignments, in seconds and specify 600.

  11. Click Save.

Upload a TSV file with tasks.

Prepare your own task file. Check out the example in the demo TSV file. In the Toloka interface under the pool name, there are links to TSV file templates for regular, control, and training tasks.

To upload tasks:

  1. Click Upload. In the window that opens, you can also download a sample TSV file by clicking Sample file for uploading tasks.

  2. In the window that opens, configure the file upload settings:

    1. Choose Smart mixing.

    2. In the Main tasks field, specify 9.

    3. In the Training tasks field, specify 0.

    4. In the Control tasks field, specify 1.

    5. Click Upload.

    6. In the window that opens, choose the TSV file with tasks to upload and click Open.

    7. In the window that opens, check the number of tasks and click Add.

  3. Create a control task.

    1. Click Edit.

      Note. If you selected something else instead of smart mixing, click Edit. If this button is missing, delete the file and upload it again.
    2. In the window that opens, click Create control tasks.

    3. In the window that opens, in the column on the left, turn on the result option.

    4. Choose the correct answer to the question.

    5. Click Save and go to next.

    6. Click Are there shoes in the photo? to exit the task markup mode.

    Note.

    In small pools, control tasks should be 10-20% of all tasks. Include different versions of correct responses in equal amounts. See the distribution of responses on the Edit tasks page, Control tasks tab.

  4. Click to start the pool.

    Attention.

    The tasks will be completed by real performers in Toloka. Recheck your project setup before you start the pool.

Get the results

To get the results:

  1. Next to the Download results button, click .

  2. Choose the Dawid-Skene aggregation model. Learn more about Result aggregation based on the Dawid-Skene model.

  3. In the window that opens, click Yes.

  4. At the top of the page, click View operations list.

    Note.

    To track the progress, refresh the page from time to time. Aggregation takes from 5 to 20 minutes. You can start designing another project meanwhile.

  5. When the operation is complete, download the TSV file with the results. To do this, click Download in the Files column.

  6. Use the TSV file with the results in project 2.

What's next

  • Create Project 2 to search for similar items in an online store.