Creating a pool

If you already have a pool and you want to create an identical one, clone the existing pool. If not, create a new pool.

New pool

A pool contains the settings of a set of task that are sent out for completion. The task interface is described in the project.

Note

You can first test the pool settings in the sandbox and then move them to the Toloka production version along with the project.

To create a pool:

  1. Click Create new pool on the project page.

  2. Select the value in the Pool type drop-down list.

    Pool types
    Note

    If the price per task suite is zero, you must select the pool type.

  3. Set the Pool name (visible only to you) field. Only you will see this pool name on the project page.

  4. Specify the pool description which will be displayed instead of the project description in the task list for Tolokers. By default, Tolokers see the description from the project settings. To use a different description, uncheck the Use project description box and set Public description. If necessary, click + Private comment to add a private project description that only you will see.

  5. Click Create.

  6. Follow the steps in the pool wizard:

    Note

    After each step click Save to apply changes or Cancel to discard them.

    1. Select the audience for your task

      Select Tolokers using filters. Tasks will only be shown to matching Tolokers, rather than to all of them.

      Click the Add filter button and set the filter parameters.

    2. Setup quality control

      Quality control lets you get more accurate responses and restrict access to tasks for cheating users. Quality control consists of rules. All rules work independently.

      To learn more, see the Quality control overview section.

    3. Set the task price and overlap

      In the Price per task suite, $ field, set the amount of money to pay per task suite done by one Toloker.

      Pay attention to the Toloker interest at this price indicator. It shows how your price meets expectations of Tolokers. If the interest is low, you can raise price per task suite to have a bigger number of Tolokers in the project. If the interest is high, you can reduce price to save money.

      In the Overlap field, define how many Tolokers must do each task.

      You can also set up dynamic pricing and dynamic overlap.

    4. Prepare and upload data

      Upload tasks to the pool. This step will be enabled after you complete the previous steps.

    5. Add control tasks for checking performance

      Add control tasks to monitor the quality of task completion:

      1. Enter correct task responses.

      2. Select the fields to use.

      3. Click Save and go to next or Skip task.

      4. Click Save all control tasks.

      Note

      This step will be enabled after you complete the previous steps. You can skip this step by clicking Do it later.

    6. Double-check your project and try out tasks

      At this step, check how the task will look from the Toloker's point of view.

      Note

      This step will be enabled after you complete the previous steps. You can skip this step by clicking Do it later.

    7. Add optional pool settings

      Set up advanced pool settings.

      Note

      This step will be enabled after you complete the previous steps. You can skip this step by clicking Use default settings.

After all the steps, you'll see the Set up is finished and your pool is ready for labeling tip on the pool page.

Tasks in pools will automatically be available in the web version of Toloka and the mobile app. If you want to change the default settings and limit the visibility of the task for any of the versions, add the Client filter and select the desired value: Toloka web version or Toloka for mobile.

How to work via Toloka API

To create a pool using Toloka API, send a POST request with the information about the pool:

curl -X POST 'https://toloka.dev/api/v1/pools' \
-H 'Content-Type: application/json' \
-H 'Authorization: ApiKey PlaceYourRealApiKey_Here' \
-d '{"project_id":"83859","private_name":"First pool","will_expire":"2030-01-01T00:00:00.000Z","reward_per_assignment":0.05,"assignment_max_duration_seconds":300,"auto_accept_solutions":false,"defaults":{"default_overlap_for_new_task_suites":1}}'

Refer to the Create pool section of the Toloka API documentation for more details about the request, its parameters, and possible responses. You will find examples of the requests for Toloka-Kit and other code samples there.

Pool parameters

ParameterOverviewFound at the step
My tasks may contain shocking or pornographic contentWhether the tasks have some content which can be marked as adult, including shocking or pornographic content. Tasks with such content are only given to Tolokers who agree to complete these types of tasks.

If you aren't sure whether the tasks have such content, enable this option.
Select the audience for your tasks
Filters

By setting the filters, you choose Tolokers that meet your requirements to complete your task. There are filters by various criteria, and filters by skills that you can create yourself.

You can filter Tolokers by language, country, city, and so on. Learn more

If you give training tasks to Tolokers, you can add a skill to the pool to choose Tolokers who already completed training. Learn more about skills

Select the audience for your tasks
Speed/quality balanceA setting for choosing Tolokers for your tasks. If you want to get responses very quickly, the quality may be doubtful. If you want very high quality, be prepared to wait longer. Find a balance that meets your needs. Learn more.Select the audience for your tasks
Review task responses manuallyTurn on this option to review the completed tasks manually.

Set the time period when you can check the tasks and reject them if the results are poor.

To learn more about manual review, see the Manual review page.
Set up quality control
TrainingThis field is only available if you have training pools in the project.

Training pool that will be linked to the main one.
Set up quality control
Level required, %This field is only available if you have filled in the Training field.

Percentage of correct responses in training tasks (from 0 to 100) required to be admitted to the pool tasks. The calculation is based on the first response the Toloker gave in each task.

The minimum required level that you can set is 5. Tolokers who complete training with apercentage below this level won't have access to tasks.
Set up quality control
Review period in days

The number of days for reviewing and accepting tasks (max: 21). The Toloker will see the deadline for checking the tasks:

  • In the task information on the Toloka main page.
  • In the history of completed tasks.
Set up quality control

Price per task suite, $
Dynamic overlapClick Show dynamic pricing & overlap to go to the parameter. You can use this parameter if you specify fixed values for the input data fields. It lets you save your budget and avoid setting fixed overlap for all pool tasks.

You set the range, the service analyzes the responses, their consistency, the level of Tolokers' skills and, if necessary, increases the overlap within the range you specified.
Set the task price and overlap
Pool priority within the projectNumber from 0 to 100. Allows you to rank a pool within a project. First, a Toloker is assigned tasks from a pool with higher priority.Add optional pool settings
Time per task suite, secThe time allowed for completing a task suite, in seconds. Uncompleted tasks are redistributed to other Tolokers.

We recommend giving at least 60 seconds per task suite (including the time for page loading and sending responses).

If you set the interval too long, the tasks will stay open for a long time. If it is too short, the tasks won't be completed and paid for.
Add optional pool settings
Keep pool open after completion, secThe interval during which the pool will remain open from the moment all tasks are completed. Minimum — 0, maximum — 259200 seconds (three days).

For example, you can use this parameter if you need an open pool where you regularly upload new tasks.
Additional settings
Pool closing dateThe date the pool closes. The pool closes automatically on this date, even if the tasks aren't marked up.Add optional pool settings

Metafields
General tasksThe number of general tasks per suite when using smart mixing.Prepare and upload data
Control tasksThe number of control tasks per suite when using smart mixing.Prepare and upload data
Training tasksThe number of training tasks per suite when using smart mixing.Prepare and upload data
Min general tasksThe minimum number of general tasks per suite when using smart mixing.Prepare and upload data
Min control tasksThe minimum number of control tasks per suite when using smart mixing.Prepare and upload data
Min training tasksThe minimum number of training tasks per suite when using smart mixing.Prepare and upload data
Number of tasks per suiteThe number of tasks per suite when using the Set manually method.Prepare and upload data
Allow partial task suites
Note

This option is available only if the Smart mixing method is selected.

The setting determines the output in the last task suite if it has less than the required number of general tasks.

By default, the option is enabled.
Prepare and upload data
Keep task order from uploaded data
Note

This option is available only if the Smart mixing method is selected.

Option disabled (default value)

The uploaded tasks are grouped in suites and given to Tolokers in random order. Within the suite, the task order is always random.

Example

Option enabled

Tasks will be grouped on suites in the order they are listed in the task file.

Example

If the pool has an overlap, the next task is distributed only when the previous task is completed by the necessary number of Tolokers.

Use this parameter to:

  • Speed up collection of task responses for majority vote check.

  • Assign tasks by priority.

    Put important tasks in the beginning of the file. They will be completed faster and with the necessary overlap.

Prepare and upload data

Project moderation

Warning

Before launching the project, see the What tasks can't be placed in Toloka? page.

Moderation begins when you launch the first pool of a new project in which the pools have not been started before.

This usually takes about 1 hour. In some cases, the moderation process may take longer. Note that tasks are not available to Tolokers during moderation. Once it's finished, you'll get a message with the results.

If your project successfully passes moderation and you didn't pause the pool, Tolokers will start your tasks.

Otherwise, you'll see the Your project didn't pass moderation label under the project name. In this case, you won't be able to launch pools and Tolokers won't see tasks from the project.

If your project is rejected, press the Contact support button on the pool page or on the project page to find out why and get specific recommendations.

Use the recommendations to fix the task content and let us know when it's ready to check. If everything is OK, your project will be approved, and Tolokers will be able to start your tasks. If not, we'll tell you what else you need to do. Also you can clone the rejected project, make changes and launch it again.

How Tolokers see pools

Tolokers can't see pools inside the project and select specific tasks (except for field tasks, where they can select points on the map). They also can't see pool settings, the number of tasks, or completion progress.

If at least one pool from the project is available to the Toloker, then this project is displayed on the main page as a card with the name, description, and price for the task. After selecting a project, the Toloker is issued a task from the pool. When they complete it, another one is issued, and so on.

A single project card is displayed if the pools in it differ only by name, quality control, or filters. In the latter case, the project card only includes tasks from the pools that the Toloker has access to.

Tolokers see one project card

Several cards are displayed if the pools differ in these settings:

  • Acceptance type
  • Public description
  • Task price

This is useful if you want to combine tasks in pools by subject, duration of audio recordings, or difficulty of evaluation.

Tolokers see several project cards

Cloning a pool

To clone a pool, click Drop-down button → Clone at the top of the pool page or in the list of pools on the project page.

If you need to change the pool settings, open edit mode.

What's next

For developers

Troubleshooting

What overlap should I set?
I want my project to be completed by a specific number of people. How can I specify that in the project settings?
Can I change overlap after the pool is started?
What is the right time limit for the task completion?
How many tasks should be in a suite?
Should I create a skill for every pool?
Contact support

Last updated: August 3, 2023

Introduction
Getting started
Important tips
Useful recommendations
Working with Toloka
Projects
Pools
Tasks
Results
Project analysis
Toloka settings
Task interface
Template Builder
HTML/CSS/JS editor
Help and support
FAQTroubleshootingSupportGlossary