Quality control lets you get more accurate responses and restrict access to tasks for cheating users. Quality control consists of rules. All rules work independently.
To set up quality control:
Go to the pool editing page.
If you already have a pool with the appropriate quality control settings, you can copy it along with the audience settings. To do this, go to Tolokers filter and click Copy settings from... and then Add Quality Control Rule.
Under Quality Control, choose the rules you want to use.
If you aren't sure which quality control rules you need, use a preset with default settings.
Make settings for the rules you added. For more information, see the List of rules.
Click Save.
If you already have a pool with the quality control settings you need, you can copy it along with the audience settings:
In the Audience block, click the Copy from another pool button.
Select a project and pool.
Click the Copy audience filters and quality control settings button.
To keep track of how often Tolokers make mistakes:
Control tasks: Use them to assign a skill to Tolokers based on their control task responses and ban Tolokers who submit incorrect responses.
Majority vote: Quality is based on matching the response from the majority of Tolokers who complete the same task.
Results of checking: Evaluate Tolokers based on the number of accepted and rejected responses.
To protect your project from robots and cheaters:
Fast responses: Control the minimum time that must be spent per task suite.
Skipped assignments: Restrict access to your pool tasks for Tolokers who skip multiple assignments in a row.
To attract a variety of Tolokers:
Earnings: Limit the amount each Toloker can earn in the pool per day.
Submitted assignments: Limit how many assignments each Toloker can submit in the pool per day.
To allow recompletion of certain assignments:
Recompletion of assignments from banned users: Send completed assignments to other Tolokers to redo them if the Toloker was banned.
Processing of rejected and accepted assignments: Send rejected assignments to other Tolokers to redo them.
Add tasks to the pool.
Learn more about how to set up a pool:
Last updated:Â February 15, 2023