Quick start

Imagine you have hundreds of pictures of cats, and you need to find out whether the cat in each picture is in a good or bad mood. It's rather difficult to solve this problem with a computer, but you can get internet users to do it for you by creating a task for them in Toloka.

First you need to decide what the task will look like and how you will set it up in Toloka:

  • The task will show an image and a choice of responses: “good”, “bad”, and “loading error” (if the image didn't load).

  • Since the task is in English, you need to select only English-speaking users.

  • To prevent users from guessing, add a restriction: no more than two “fast responses”. A response is considered too fast if it was chosen in 2 seconds or less.

Next you need to create a task and check whether the settings work correctly (make sure your task is displayed correctly, and the filters and quality control settings work the way you intended). Try completing the task and  make sure that you can get the responses.

When your task is ready, publish it in the production version of Toloka. You can get the responses as  users complete the tasks.

Create tasks

  1. Register in the sandbox and get an OAuth token in the requester interface: https://sandbox.toloka.yandex.com/requester/profile.

  2. Create a project. The project should include:

    • Input data parameters for the tasks (the input_spec key). In our example, the input data is the URL for accessing the cat images.

    • Response parameters (the output_spec key). The user's response is returned as a string.

    • Description of the task interface. The task contains an image sized 100x400 pixels (the {{img}} component) and response buttons (the {{field type=\"radio\"}} component). Next you can configure how Toloka interface elements are displayed (the timer and task buttons). For more information about configuring the task interface, see the Requester's guide.

    In response to the  request, you will receive a JSON object for the created project. This object contains the project ID in the id key.

    Request
    POST /api/v1/projects
    Host: https://sandbox.toloka.yandex.com
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "public_name":"What is the cat's mood?",
       "public_description":"Look at the picture and determine the cat's mood",
       "public_instructions":"Choose the best option",
       "task_spec":{
          "input_spec":{
             "image":{
                "type":"url",
                "required":true
             }
          },
          "output_spec":{
             "result":{
                "type":"string",
                "required":true
             }
          },
          "view_spec":{
             "assets":{
                "script_urls":[
                   "$TOLOKA_ASSETS/js/toloka-handlebars-templates.js"
                ]
             },
             "markup":"{{img src=image width=\"100%\" height=\"400px\"}}\n\n{{field type=\"radio\" name=\"result\" value=\"OK\" label=\"Хорошее\" hotkey=\"1\"}}\n{{field type=\"radio\" name=\"result\" value=\"BAD\" label=\"Плохое\" hotkey=\"2\"}}\n{{field type=\"radio\" name=\"result\" value=\"404\" label=\"Ошибка загрузки\" hotkey=\"3\"}}\n\n",
             "script":"exports.Task = extend(TolokaHandlebarsTask, function (options) {\n  TolokaHandlebarsTask.call(this, options);\n}, {\n  onRender: function() {\n    // DOM element for the task is generated (available via #getDOMElement()) \n  },\n  onDestroy: function() {\n    // Task is completed. Global resources can be released (if used)\n  }\n});\n\nfunction extend(ParentClass, constructorFunction, prototypeHash) {\n  constructorFunction = constructorFunction || function () {};\n  prototypeHash = prototypeHash || {};\n  if (ParentClass) {\n    constructorFunction.prototype = Object.create(ParentClass.prototype);\n  }\n  for (var i in prototypeHash) {\n    constructorFunction.prototype[i] = prototypeHash[i];\n  }\n  return constructorFunction;\n}\n",
             "styles":"",
             "settings":{
                "showSkip":true,
                "showTimer":true,
                "showTitle":true,
                "showFinish":true,
                "showSubmit":true,
                "showMessage":true,
                "showFullscreen":true,
                "showInstructions":true
             }
          }
       },
       "assignments_issuing_type":"AUTOMATED"
       
    }
  3. Create a pool with the task settings. In response to the  request, you will receive a JSON object for the created pool. This object contains the pool ID in the id key.

    Request
    POST /api/v1/pools
    Host: https://sandbox.toloka.yandex.com
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "project_id":"2405",
       "private_name":"Cat pool 1",
       "may_contain_adult_content":true,
       "will_expire":"2017-11-25T12:37:39",
       "reward_per_assignment":0.02,
       "assignment_max_duration_seconds":60,
       "mixer_config":{
          "real_tasks_count":5,
          "golden_tasks_count":0,
          "training_tasks_count":0
       },
       "defaults":{
          "default_overlap_for_new_task_suites":5
       }
    }
  4. Add filters to the pool for selecting users by language.

    Request
    PUT /api/v1/pools/14025
    Host: https://sandbox.toloka.yandex.com
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    {
       "project_id":"2405",
       "private_name":"Cat pool 1",
       "may_contain_adult_content":true,
       "will_expire":"2017-11-25T12:37:39",
       "reward_per_assignment":0.02,
       "assignment_max_duration_seconds":60,
       "filter":{
          "and":[
            {
              "category":"profile",
              "key":"languages",
              "operator":"IN",
              "value":"RU"
            }
          ]
       },
       "mixer_config":{
          "real_tasks_count":5,
          "golden_tasks_count":0,
          "training_tasks_count":0
       },
       "defaults":{
          "default_overlap_for_new_task_suites":5
       }
    }
  5. Add a quality control rule (the quality control key). The settings for quality control rules depend on the type of task. Restricting fast responses is useful for almost any type of task. The settings shown below will block access to the  pool if the user responds too quickly on two tasks in a row.

    Request
    |||UNTRANSLATED_CONTENT_START|||PUT /api/v1/pools/14025
    Host: https://sandbox.toloka.yandex.com
    Authorization: OAuth <OAuth-токен>
    Content-Type: application/JSON
    
    {
       "project_id":"2405",
       "private_name":"Cat pool 1",
       "may_contain_adult_content":true,
       "will_expire":"2017-11-25T12:37:39",
       "reward_per_assignment":0.02,
       "assignment_max_duration_seconds":60,
       "filter":{
         "and":[
           {
             "category":"profile",
             "key":"languages",
             "operator":"IN",
             "value":"RU"
           }
          ]
       }, 
       "quality_control":{
          "configs":[
           {
             "collector_config":{
                "type":"ASSIGNMENT_SUBMIT_TIME",
                "parameters":{
                   "history_size":10,
                   "fast_submit_threshold_seconds":3
                }
             },
             "rules":[
                {
                   "conditions":[
                      {
                         "key":"total_submitted_count",
                         "operator":"EQ",
                         "value":2
                      },
                      {
                         "key":"fast_submitted_count",
                         "operator":"GTE",
                         "value":2
                      }
                   ],
                   "action":{
                      "type":"RESTRICTION",
                      "parameters":{
                         "scope":"PROJECT",
                         "duration_days":1,
                         "private_comment":"Более двух быстрых ответов"
                      }
                   }
                }
             ]
          }
          ]
       },
       "mixer_config":{
          "real_tasks_count":5,
          "golden_tasks_count":0,
          "training_tasks_count":0
       },
       "defaults":{
          "default_overlap_for_new_task_suites":5
       }
    }|||UNTRANSLATED_CONTENT_END|||
  6. Upload the input data for tasks (image URLs).

    Request
    POST /api/v1/tasks
    Host: https://sandbox.toloka.yandex.com
    Authorization: OAuth <OAuth token>
    Content-Type: application/JSON
    
    [
          {
             "input_values":{
                "image":"https://toloka.yandex.com/api/proxy/photos/9.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.com/api/proxy/photos/8.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.com/api/proxy/photos/7.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.com/api/proxy/photos/6.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          },
          {
             "input_values":{
                "image":"https://toloka.yandex.com/api/proxy/photos/5.jpg"
             },
             "pool_id":"14025",
             "overlap":5
          }
      ]

    Now you can view the task on the pool page in the sandbox. Check whether it is displayed correctly and  start the pool.

    Request
    |||UNTRANSLATED_CONTENT_START|||POST /api/v1/pools/14025/open
    Host:https://sandbox.toloka.yandex.com
    Authorization: OAuth <OAuth-токен>|||UNTRANSLATED_CONTENT_END|||

Check the tasks

After starting the pool, make sure that all your settings are working and the tasks are displayed correctly:

  1. Register as a user in the Toloka sandbox.

  2. Log in to the sandbox with the requester's username and follow the Add trusted users link on the Users page.

  3. Click the Add user button and enter the username of the created account.

  4. Perform the tasks in the sandbox under the username of a trusted user.

Get responses

After completing at least one task, you can get the responses:

Request
GET /api/v1/assignments?pool_id=14025
Host: https://sandbox.toloka.yandex.com
Authorization: OAuth <OAuth token>

Responses are returned in the following format:

{
  "id": "566ec2b0ff0deeaae5f9d4fe",
  "task_suite_id": "566ec2b0ff0deeaae5f9d4ff",
  "pool_id": "14025",
  "mixed": true,
  "user_id": "566ec2b0ff0deeaae5f9d500",
  "status": "ACCEPTED",
  "created": "2015-12-15T14:52:00",
  "submitted": "2015-12-15T15:10:00",
  "public_comment": "Well done!",
  "owner": {
    "id": "ac1e4701364b4ccef8a4fe10a8980cff",
    "myself": true
  },
  "tasks": [{
    "id": "57a333ea-2827-4c1c-ab1f-8ab1bfe4ee7e",
    "input_values": {
      "image": "https://toloka.yandex.com/api/proxy/photos/9.jpg"
    }
  }],
  "solutions": [{
    "output_values": {
      "result": "OK"
    }
  }]
}

When the project is set up to your satisfaction (tasks are displayed correctly, and user filters and quality control rules work properly), you can move it to the production version of Toloka. You can do this in the requester interface (see the Requester's guide) or using the API for the Toloka production version (send a request to https://toloka.yandex.com):