All questions on one page
- If you aren't using Toloka yet and need advice
- If you have a problem working with Toloka
-
In the section contents, select the stage where you encountered the problem and find the appropriate solution in the list. If not found, select the last item Other questions, fill out the form and wait for a response from the support team.
Registration and getting started
- Check the number you entered. You might have mistyped it.
- Check the number in Yandex ID. It must be set as your main number. If a different number is specified in Yandex ID, an SMS code will be sent to that number.
- Log out of your requester account.
- Go to the Toloka home page.
- Click Join.
- Follow the system instructions.
You can also use the recommendations in Yandex ID Help. If this doesn't help, contact support.
You can't change the account type from “Requester” to “Toloker”.
If you want to complete tasks, you need to register in Toloka once again, with a different username, but as a Toloker.
If you see this warning, make sure you entered the phone number correctly.
Please note that you can only have one requester account per phone number (see the User Agreement).
If your Yandex ID is linked to the wrong number, change it. For more information about linking a phone number, see Yandex ID Help.
If you don't remember your credentials in Toloka, use Restoring access.
If you deleted your Toloka account, create a new Yandex ID and register in Toloka.
To change your phone number, go to Yandex ID and change your main number.
Sandbox
- Try exporting the project without pools.
- Select an exported project and don't select pools in the window that opens.
- If the error persists, refresh the token.
- Go to Yandex ID.
- The user isn't registered in the sandbox. The account you registered in the Toloka production version won't work. Make sure you have specified the correct Toloker's account as a trusted one.
- The Toloker's account is authorized via social networks. Register a new user in Yandex.
To test your task, add yourself as a trusted Toloker in the Sandbox. To do this, on the Users page, click Add trusted users. Enter the username for the Toloker's account (case-sensitive).
To do this, you will need a Sandbox version of the Toloka app. Write to support to get it.
You won't be able to update a previously exported project. If you clone a project from the sandbox again, its current copy is added. The old project doesn't change.
Use your requester account to preview the pool and check whether it's possible to submit a task. If this fails, most likely there is an error in your project.
The tasks themselves are not exported, only the project configuration and the settings of the selected pool. You can download the completed tasks from the pool in the Sandbox and import them to the exported pool.
To download only the control tasks (if you completed them in the interface), go to Mark up, then click Control tasks and Download.
Possible reasons:
Setting up a customizable project
Instructions
- Put your instructions inside the task, but make sure that it doesn't clutter the interface.
- Use a side window for your instructions so that the user can quickly expand or collapse them.
- Hide the instructions in an expandable section or add hints for the individual interface elements.
If the tags or attributes disappear after you save the instructions (for example, checked="true"
), it means that they are not supported. For the full list of tags that can be used in the instructions, see the Guide.
Configuring the task interface
- See how this is implemented in the “ Search for information online ” template.
- To solve the second problem, you can add another validation like this:
if (solution.output_values.url && solution.output_values.check) {return {task_id: this.getTask().id,errors: {'url': {code: ''Insert a link or check the box if the site doesn't exist'}}}}
Copied to clipboard - Click the “gear button” in project editing mode.
- In the field that opens on the left, enter the links and press Enter.
- with checkboxes
- with a dropdown list
- with radio buttons
- Click the “gear button” to open the settings.
- Delete
$TOLOKA_ASSETS/js/toloka-handlebars-templates.js
. - Decompose the task: First select images with the items you need, then select areas in them.
- Select an arbitrary area in the image. For example, put a square in the upper-right corner.
Mention this in your instructions for reviewers.
- Ask the Toloker to skip the task and report it in a personal message. Messages are reviewed by the requester. If the selectable object is missing, the task is deleted from the pool (by resetting the overlap).
- Add the “No object” checkbox to the interface and make sure that your JS checks that either the object is selected or the checkbox is selected.
For control purposes, add information about the value of this checkbox to the task interface.
To the component that inserts the image, add the parameters: real-size=true
and screenshot=true
.
onKey: function(key) { var el = this.getDOMElement().querySelector(".image-annotation-editor__shape-polygon"); if (key === 'D') { el.click(); el.classList.add('image-annotation-editor__shape_active') }
Copied to clipboard
If you need to further modify the area-selection editor, use this library.
You can't check loading of audio files in the preview mode, but you can do it in the sandbox if you do your task. To do this, register in the sandbox as a Toloker and add the Toloker username to your trusted list on the Users page. For more information, see this post.
If none of the Tolokers can submit the assignment, the most likely reason is JS validation. Run JS validation again.
Export your project to the sandbox and try to complete the task in the sandbox yourself.
To check the task's look-and-feel on the mobile phone screen, enable the emulation mode in Chrome or Safari and edit the CSS block.
You can also use the mobile version of the sandbox. Write to support to get access to it.
{{field type="checkbox" name=(concat "result." @index ) label=(concat "checkbox – " @index) size="L"}}
Copied to clipboard
Try to disable extensions in your browser. They might block iframe loading.
To pass a label
in the input data, enter the input field name into the label.
For example, if you have the asd
input field with the string type, the component would look like: {{field type="checkbox" name="like" label=asd hotkey="q"}}
.
If you want to pass different label values in different tasks or the number of checkboxes may differ, use concatenation.
You can hide text in an expandable section by using CSS styles, both in the task itself and in the instructions. You can see the sample code here.
{{field type="button-clicked" name="ads" label="Click me" href=name_escape action=true}}
Copied to clipboard
The names of the output fields must differ: each checkbox must have its own unique name. For more information about this component, see here.
You can use JavaScript to add assignment validation depending on the checkbox. An example is provided in the “Search for information online” template.
You can see an example in the comments for this project. The example includes the output data format and libraries to be added.
The issue is probably in the JS block. Try deleting its content, then test the Submit button in the preview mode.
You can't deselect a radio button. You can only select another radio button as a different response option.
In this case, you need to leave the field optional in the output data and set up validation depending on the Toloker response. You can see how this is implemented in the “Text classification” template.
Add sources="CAMERA"
to the attributes of the image loading component. This disables adding photos from the gallery.
setSolution: function(solution) { var secondScale = this.getDOMElement().querySelector('.second-scale'); if(secondScale) { secondScale.style.display = solution.output_values.grammar === 'no' ? 'block' : 'none'; } TolokaHandlebarsTask.prototype.setSolution.call(this, solution); },
Copied to clipboard
You can create a selection and drop-down list with category selection. See how it is implemented on this page (Dropdown list tab).
Learn more about the template's JS extension here.
To avoid conflict between the Vue markup and the Handlebars syntax, disable the "toloka-handlebars-templates" library and inherit from the Task/TaskSuite classes.
The “Side-by-side image comparison” template uses a component rather than an HTML tag. This means that you should enclose your proxy in curly brackets like this example: {{img src=(proxy image)}}
.
.task
or .task-suite
element. For example, to use a black background:You can also assign a class to the interface block with the image and set the background for this block only..task-suite { background-color: #000000; } .task { background-color: #000000; }
Copied to clipboard
You can check the link format using regular expressions. To do this, add the link validation JavaScript code with regexp
to the task template.
For example: var regexp = /^(https://www.myurl.com/).{4,200}$/
.
input
field with the string type in the output data. Make the field mandatory. Then add the Text input field field in the task interface (in the HTML block) and specify the field name in the name
attribute:{{field type="textarea" name="input" width="270px" rows=5}}
Copied to clipboard
You can use JavaScript to add assignment validation depending on a checkbox. An example is provided in the “Search for information online” template.
Pass an array of strings as the input field. For example, as shown in the screenshot:
{{#each words}} {{field type="radio" name="result" value=this label=this}} {{/each}}
Copied to clipboard
<input type=""range"" list=""rng"" class=""res""> and include the following in onRender in your JS: onRender: function() { // Generated DOM element for the task (available via #getDOMElement()) //Adding auxiliary variables var $root = $(this.getDOMElement()); var _this = this; var solution = TolokaHandlebarsTask.prototype.getSolution.apply(this, arguments); $root.on('change', '.res', function(){ var range_result = $(this).val() _this.setSolutionOutputValue('result', range_result); return solution; }) }
Copied to clipboard
You can't expand the HTML window. To expand the JS and CSS fields, click any area within the field.
Enclose the input field in double curly brackets {{text}}
.
Enclose the input field in triple curly brackets {{{input_field}}}
.
For more information about using the component, see the Requester's guide.
To change the number of output fields dynamically, use the recommendations from this page.
We didn't intend this window for any sophisticated development. Usually, the content is prepared in a third-party prettified editor, and the resulting code is pasted to the window prior to the update.
However, in the context of TolokaHandlebars editability, there are no differences between our window and a third-party editor.
To display the text in the input field with HTML tags, use the <pre>
tag. For example:<pre>{{text}}</pre>
.
.task { max-width: 800px; } pre { white-space: pre-wrap; }
Copied to clipboard
You need to break down your task for Toloka. If you have a 30-minute task intended for one person and containing all the phrases, the best approach is to divide it into individual phrases, which you can give to different Tolokers. The template for voice recording tasks doesn't require that the user installs a separate voice recorder app.
Refer to our step-by-step guide for creating a voice recording task.
In the input data, you can pass the phrase for the Toloker to record ("enable navigation"), and the speech speed (normal speed). This is the data you need to save in the TSV file. See the Guide to learn about creating a TSV file and its structure.
The preview contains 4 tasks per suite by default. You can specify the number of tasks for your project when you upload the task file. Learn more about the preview.
For example, if you want one person to say the same phrase 10 times, create 10 tasks in a suite. The cost is specified per task suite. To define how many people should say a particular phrase, use the overlap in the pool. By the way, don't forget to set up filters in your pool. This way you can select only the Tolokers who speak a certain language and use mobile devices: client = mobile Toloka.
You can delegate review of the voice recordings to other Tolokers by creating a separate project. Find brief instructions on how to do this here.
Input and output data
You can find an example of the task template for selecting image groups at this link. The input and output data, as well as a fragment of the instructions, are in the comments to the project.
To validate the input data format, you can use the output field type, specifying the acceptable or minimum/maximum values. For example, create an output field for the taxpayer number with the “string” type and enter its minimum and maximum length (like 10 and 12). To use a more sophisticated validation in the template, use RegExp.
To enter a date, you can add a calendar to the task interface. See an example of a calendar.
You can't use a range as a fixed value.
Pool settings
Filters
- Has a skill value lower than the one specified in the Level required field.
- Didn't complete any tasks linked to training during this period.
- Upload the task file using Smart mixing.
- Specify
student
as the correct answer to the question. Don't take other questions into account (leave the fields empty or unselected). - Add the Control tasks rule to the pool:
if the percentage of correct control answers = 100, then set the skill value Student = 1
.
Yes, you can do that. In the filters, select . Please note that the profile data is entered by the user when they register in Toloka. We recommended that you use the filters Region by phone number and Region by IP.
Yes, of course — you can use the same skill for different projects. But most often, a skill is intended for a specific project. If the Toloker completes a certain task well, this doesn't mean that they will complete other ones successfully. Another disadvantage is that if you filter by skills that were set long ago, you will artificially limit the number of available Tolokers.
If you mean multiple different projects, you can't do that.
You can merge all the projects into one and use History size in the quality control rules. See examples in the Control tasks post.
You can use Aggregation by skill, but you'll need to list all the possible values, which is probably not the best choice. Perhaps you'll find another method of aggregation helpful.
In the main Toloka version, you can only assign a skill to users who have completed at least one of your tasks. There is no option to assign a skill to an arbitrary user. To limit the audience of users who will see your project, use filters. For example, specify the city, date of birth, gender, or some other parameters of your target Tolokers.
Tasks from an open pool are available to every user that matches your pool filters. You can restrict access, like by using a skill.
Your users will need to be trained again.
Tasks in pools are automatically available in the web version of Toloka and the mobile app. Check the pool settings. You might have the Client = Toloka web version
filter on.
If the user mismatches your preset filter, they can't see the task. You can only remove the restricting filter from the pool. You can test the task in the Sandbox by adding the desired user to your trusted list.
You can do that. To select Tolokers for the pool, use filters.
Tasks in pools are automatically available in the web version of Toloka and the mobile app. Check the pool settings. You might have the Client = Mobile Toloka
filter on.
You can assign a skill to these people based on their performance in the previous pools. Use this skill as a filter in the new pool.
Specify this skill as a filter, but leave the value field empty (this is equivalent to absence of the skill).
If the user already has a given skill, you can't add the same skill to them from the task review interface. You can open the user's profile and edit the skill value.
If it's a public or training skill, they see it and they get a message about it.
There is no such option. If the skill is public, the Toloker sees it in their profile.
Requesters can't see the full details about specific Tolokers. So you can't see information like the date of birth, gender, last name, or first name. However, you can use filters by date of birth and gender (in the pool settings). This way you can select a group of Tolokers without accessing the personal information of individual Tolokers. This decreases the risk of user de-anonymization.

Quality control
Always use one or more ways to control quality of answers.
Counting fast responses makes sense for most tasks.
If the user has to choose between options (for example, by selecting checkboxes), check the answers using majority vote or control tasks.
If the user has to provide a response as a text or link or upload a photo, the best way to control quality is by reviewing assignments. You can outsource task acceptance to Tolokers. Create a task with a question (for example, “Is this phrase translated correctly?”) and possible responses (for example, “yes”/“no”). Set up overlap and majority vote check.
If a task is more like an opinion poll (for example, choosing nice pictures from a set), majority vote is not a good way to control quality. Make
The settings for quality control rules depend on the type of tasks. General recommendations: