Price monitoring

  1. Prerequisites
  2. Choose a preset
  3. Create a project
  4. Create a pool
  5. Upload data
  6. Start labeling
  7. See the results

In this tutorial, you will learn how to run a price monitoring project in Toloka. We will use a project preset designed specifically for this type of data labeling.

Price monitoring is intended for field tasks in the Toloka mobile app. The preset is suitable for collecting information about products from stores and retail outlets. A Toloker chooses a point on the map where they should go to check something and take photos.

Use this preset when you need to:

  • Check if prices are up-to-date at specific retail outlets.
  • Compare competitors' pricing and stocking policies.
  • Monitor promotions and special offers.
  • Run out-of-stock (OOS) control.
Note.

You may need additional projects for your task, such as dataset pre-check or checking Tolokers' responses. Learn more about this in Decomposition of the task.

Prerequisites

Before you begin:

  • Make sure you are registered in Toloka as a requester.
  • Top up your Toloka account. If you are unsure about the budget, you can do that later in this tutorial. Toloka will display the budget estimate for your project.

Choose a preset

We recommend starting with a project preset for easier configuration and better results.

  1. Follow this link, or create a project manually:

    1. In the main menu, choose the Projects tab, and click Create a project.
    2. Select the Photos of product and price tag preset.
  2. Click Choose solution in the pop-up tab.

Create a project

Set up how your tasks will look for Tolokers. Tolokers are people around the world who get paid for completing your tasks.

  1. Under General information, add the project name and description.

    • Name to show performers: In 2–5 words, state the general idea of the project.
    • Description for performers: In a couple of sentences, explain what you expect Tolokers to do. This is just an overview. You will write instructions later.
  2. In the Task interface section, set up what your tasks will look like.
    Note.

    This tutorial uses Template Builder, but you can use the HTML/JS/CSS editor for the same purpose.

    1. Copy the code of the example and paste it to the Config section of your project. This code has validation and task layout pre-configured.
    2. You can edit the code. For example, to change the description, replace the sample text with your value in the content property.
    3. A Toloker will be able to submit a response when they are within 50 meters of the specified location. To change this condition, replace the value of the max property with the desired distance:
      Note. To learn about other properties of the Config section, their possible values and the impact on the task interface, see the Template Builder Help.
    4. Raw task data is stored in the XSLX, TSV, or JSON format. The labeling results are presented in a TSV file. The Data specification section determines which parameters these files might contain.

      Click Show specifications and check the values:

      • Input data: Parameters in the file with raw task data.
      • Output data: Parameters in the file with labeling results.

      Input data and Output data match the task interface you set up in Template Builder. Check that there are fields for all data types you use for your tasks, and for the ones you want to see in the results file.

    5. In the Settings for displaying field tasks section, configure the settings which affect the task display on the map.

      • The Title format and Short description format are used to help a Toloker distinguish one task from another when they select a task on the map. These fields contain links to the input data fields to show the name of the point and its coordinates. You can leave these fields unchanged.
      • The Map provider for tasks field sets which map a Toloker will use when performing your tasks.
  3. Under Instructions for performers, add the instructions Tolokers will see when they start doing your tasks. You can add text, tables, and images to your instructions.

    Check the sample text of the instructions, and update it to fit your project.

    Field task instructions should be easy to read on a mobile phone screen.

    Tip.

    When writing instructions, remember that most performers don’t know anything about your tasks beforehand. Make sure your instructions are as clear as possible, but not too wordy. For successful data labeling, try to strike a balance between covering all the essentials and keeping it short. Learn more in our knowledge base.

  4. To save your data and continue, click Create a project.

Create a pool

A pool is a set of tasks sent out to Tolokers at the same time. One project can have many pools. When creating a pool, you set up pricing, audience filters for Tolokers, and quality control.

  1. Click Create new pool.
  2. Under General information, set the Pool name.
  3. Under Audience, set up filters to select Tolokers for your pool.

    1. Clear My tasks may contain shocking or pornographic content if your project has none of those.
    2. To select Tolokers based on their language, location, age, gender, and other parameters, click the Add filter button.

    3. Tasks in pools are available in the web version of Toloka and the mobile app by default. Make your tasks available in the mobile app only: add the Client filter and select the Toloka for mobile option.
    4. Use the Speed/quality balance slider to change the number of Tolokers who can see your tasks. Move the slider to the right to exclude Tolokers with lower ratings from participating in your project.
  4. Under Quality control, set quality control rules for more accurate results.
    1. Click the Non-automatic acceptance toggle, and specify the number of days for checking the task in the Review period in days field (for example, 7).

      What is non-automatic acceptance (assignment review)?

      The non-automatic acceptance option allows you to review completed assignments before accepting them and paying for them. If the Toloker didn't follow the instructions, you can reject the assignment.

    2. Click Add a quality control rule → Results of assignment review, and enter the following values:

      This means that if more than 35% of a Toloker's responses are rejected, the Toloker is banned and can't access your tasks for 15 days. The rule takes effect after 3 responses of the Toloker are reviewed.

    3. Add the Processing rejected and accepted assignments rule.

      This means that if you reject assignments during the review, they’ll be sent for re-completion to another Toloker.

  5. Click Show advanced settings. In the Additional settings, specify Time per task suite, sec. It should be long enough to get to the place, find the specified point and product, and upload the photos. For field tasks, this time is usually 86,400 seconds (24 hours).
  6. In Price, set up how much a single task will cost for you.

    1. In Price per task suite, set the amount of money to pay per task suite done by one Toloker. A task suite is a page with a number of tasks.

      For this type of project, a task suite must contain only one task. You will set the number of tasks per suite later in this tutorial.

    2. In the Overlap field, define how many Tolokers must do each task.

      For field tasks, it is usually 1. This means that each task will have 1 response.

  7. To save the settings and continue, click Create pool.

Upload data

At this step, upload your task data to Toloka.

  1. Click Upload data.
  2. Create the tasks for Tolokers:
    1. To download a template, click one of the buttons:
      • Template in XLSX
      • Template in TSV
      • Template in JSON

      For this type of project, the file with tasks must contain seven parameters:

      • INPUT:name: A string with the store name.
      • INPUT:image: A string with the link to the product photo.
      • INPUT:address: A string with the store address.
      • INPUT:product: A string with the product name.
      • INPUT:coordinates: A string with the coordinates of the point that the Toloker should go to.
      • AI:latitude, AI:longitude: The latitude and longitude from the INPUT:coordinates parameter presented separately. The values of INPUT:coordinates, AI:latitude and AI:longitude should have the same accuracy, that is have the same number of digits after the decimal separator.
      INPUT:name	INPUT:image	INPUT:address	INPUT:product	INPUT:coordinates	AI:latitude	AI:longitude
      Store-name-1	https://yastatic.net/s3/toloka/p/toloka-requester-page-project-preview/877a55555a5f199dff16.jpg	Address-1	Product-1	53.947516,27.669428	53.947516	27.669428
      Store-name-2	https://tlk.s3.yandex.net/dataset/cats_vs_dogs/dogs/048e5760fc5a46faa434922b2447a527.jpg	Address-2	Product-2	53.947517,27.669429	53.947517	27.669429
      Store-name-3	https://tlk.s3.yandex.net/dataset/cats_vs_dogs/dogs/05334365c060421ab25264166bbb4fd1.jpg	Address-3	Product-3	53.947518,27.669428	53.947518	27.669428
    2. Open the downloaded file, and replace the sample values with your data. You can use a service like Google Maps to get the coordinates.
    3. Click Drop file here or select, and upload the file you’ve just made.
    4. Click Continue.
  3. Tasks are shown to Tolokers in suites. A suite is a single page with multiple tasks. Define how many tasks to include per suite:

    1. Go to the Set manually tab.
    2. In this type of project, the task suite must contain only one task:
  4. Click Combine tasks into suites.

Start labeling

  1. Make sure you have topped up your account.
  2. To send the tasks to Tolokers and begin the labeling process, click Start labeling.

  3. In the pop-up panel, review the budget and click Launch.

See the results

Track the labeling progress on the pool page. You can start the review when the first results are received.

After the time period specified in step 4.1 of creating the pool, all responses are automatically accepted, regardless of their quality.

  1. Go to the pool, and click Review assignments.
  2. Choose an assignment.
  3. Check the responses, and click Accept or Decline. For rejected responses, enter a comment to specify the reason.
    Note. To learn about other ways of review, see the Reviewing Tolokers' responses section.
  4. After checking all the assignments, click Download results.

    You will get the TSV file with the labeling results.

  5. To download the files Tolokers attached to the tasks, click the arrow next to the Download results button. Choose Download attachments from the drop-down menu.

Troubleshooting

Can I limit the source of the photo to camera-only in a field task so that the Toloker can't upload a photo from anywhere else?

The accept property of the field.media-file component adds different buttons for four types of uploads. To prohibit using any sources except of the camera, use the "photo": true property only, like it is done in the example.

If you allow Tolokers to use the mobile device gallery, you can prohibit adding images without geotags to limit uploading photos taken from the Internet. To do that, specify the "requiredCoordinates": true property.

To learn more about setting up the field.media-file component, see the Template Builder Help.

How do I check the task display in the mobile Toloka app?

To check the task's look-and-feel on the mobile phone screen, you can use the following buttons:

  • Preview task on the project settings page.
  • Preview on the pool settings page.

You can also use the mobile version of the sandbox. Write to support to get access to it.