Review results
If you review Toloker responses yourself and don't want Tolokers who made a lot of mistakes to do your tasks in the future, set up this quality control rule.
When to use
Set the Toloker's skill based on their responses.
Block access for Tolokers who give incorrect responses.
Use this rule if you don't want a Toloker who often makes mistakes to complete tasks in your new pools or projects.
To have rejected tasks redistributed to other Tolokers, set up the Recompletion of rejected assignments rule.
Rule settings
The If and then fields in this rule are mandatory. If you don't fill in at least one field, you won't be able to save the rule.
| Field | Overview |
|---|---|
| Recent values to use | How many recent responses from the Toloker to use. If this field is not filled in, the calculation includes only task responses from the pool to which the rule is applied. If the field is filled in, the corresponding number of responses is used. The rule takes into account responses from both the current pool and other pools where this field is filled in. Learn more about how this field works. |
| If | A condition for performing the action in the then field:
To add multiple conditions, click |
| then | Action to perform for the condition:
|
.| Field | Overview |
|---|---|
| Recent values to use | How many recent responses from the Toloker to use. If this field is not filled in, the calculation includes only task responses from the pool to which the rule is applied. If the field is filled in, the corresponding number of responses is used. The rule takes into account responses from both the current pool and other pools where this field is filled in. Learn more about how this field works. |
| If | A condition for performing the action in the then field:
To add multiple conditions, click |
| then | Action to perform for the condition:
|
Examples of rules
You understand that the Toloker didn't complete your tasks well enough. You can block this Toloker's access to tasks that you're going to do yourself.
You can also assign a skill that you can use to either block access to tasks for Tolokers, or to encourage those who did tasks well.
- Blocking for incorrect responses
- Attention. The assignments submitted by banned Tolokers will be taken into account if they are not rejected manually using assignment review They can be reassigned by setting up the Recompletion of assignments from banned users rule.
If more than 50% of Toloker's responses are rejected, the Toloker is blocked and can't complete your tasks for 15 days.
The Toloker is blocked after the first rejected response, because the number of the reviewed tasks is not specified.
- Setting the skill
-
After 3 tasks are reviewed, the skill is set to the percentage of correct responses. Use the skill value to set access to other pools with filters.
Example of filter settings
Troubleshooting
Calculate the skill for each pool separately. The current skill value is the value of the skill in the pool the user completed last. This option is convenient if:
The pools are intended for different groups of Tolokers (for example, there are filters by city or country).
Pools are started one by one and you don't want to take into account the responses in the previous pools to calculate the skill in the current pool.
This calculation method is used by default when adding a quality control rule to a pool. For the control tasks block, leave the Recent control task responses to use field empty.
Calculate skill based on all tasks in a project This option is good if the pools are small and you don't need to have skill calculated for each pool.
This option is available only for skills on control tasks. To use it, fill in the Recent control task responses to use field in pool quality control rules.
It is better to use one skill in a project. You can choose the way to calculate the skill:
Yes, of course — you can use the same skill for different projects. But most often, a skill is intended for a specific project. If the Toloker completes a certain task well, this doesn't mean that they will complete other ones successfully. Another disadvantage is that if you filter by skills that were set long ago, you will artificially limit the number of available Tolokers.
You can deny access to the pool if the Toloker's responses are too fast, if they don't match the majority vote, or if the Toloker makes too many mistakes in control tasks. Tasks completed by such Tolokers can be given to other Tolokers.
No. After sending a task, the Toloker can't make any changes to it. You can add tasks that were completed incorrectly to a new pool.
