Control tasks contain samples of correct responses. They allow you to monitor the quality of task completion without having to check all the responses. Control tasks simplify the verification process, help identify Tolokers' qualifications, and save your time.
To save time, you can opt for having your control tasks edited by Toloka experts. In this case, markup is performed within the pool.
Use control tasks to assign a skill to Tolokers based on their responses and ban Tolokers who submit incorrect responses.
Don't use it if:
The If and then fields in this rule are required. If you don't fill in at least one field, you won't be able to save the rule.
|Recent control and training task responses to use||The number of the Toloker's last responses to control tasks.|
If this field is not filled in, the calculation includes only control task responses in the pool to which the rule applies.
If the field is filled in, the corresponding number of control task responses is used. The rule takes into account responses from both the current pool and other pools where this field is filled in.
Learn more about how this field works.
A condition for performing the action in the then field:
To add multiple conditions, click .
Action to perform for the If condition:
Purpose: filter out Tolokers who often make mistakes.
Ban Tolokers based on control tasks and the percentage of correct responses
Set a skill and deny access to Tolokers with a low skill level
Ban Tolokers if their percentage of correct responses to control tasks is less than 40%
The assignments submitted by banned Tolokers will be taken into account if you don't reject them using manual review. They can be reassigned by setting up the Recompletion of assignments from banned users rule.
Both rules work independently:
If the Toloker gives at least 3 responses to the control tasks, the percentage of correct answers is assigned as the skill value. This is useful if you want to prevent Tolokers with a low skill level from completing your tasks.
If the Toloker gives at least 3 responses to the control tasks and the percentage of correct answers is less than 60%, they lose access to the project.
The calculation uses up to 10 of the Toloker's recent responses to the project's control tasks.
Skills help identify how well Tolokers do your tasks. You can ban Tolokers with low skill levels from completing tasks in your pool or project.
If the Toloker completes 3 control or training tasks, they get a skill. Use the skill value to set access to other pools with filters.
If the percentage of correct responses in the control tasks is less than 40%, the Toloker loses access to the project for 30 days.
This rule doesn't take into account responses in the training tasks for banning.