To set up selective majority vote check, you need a project, a pool, and a file with tasks.
This function allows you to set up majority vote check for only a portion of tasks.
Depending on how many tasks the Toloker completed in N days, overlap for certain tasks will be higher or lower than in the pool settings.
Use selective majority vote check if:
There are few or no control tasks in the project.
You have experienced Tolokers whom you trust.
In the open pool, the verification tasks that haven't gained enough overlap are assigned to a Toloker.
If there are no tasks of this kind in the open pool, they are transferred there from archived and closed pools that belong to the same project.
If in the project there are no verification tasks with an overlap that is less than the specified value, a random task from the current open pool is assigned as a verification task.
To add selective majority vote check:
Upload tasks to the pool using smart mixing.
Switch to pool editing mode.
Under Smart mixing settings, click Selective increase in overlap for general tasks.
Specify how many days the calculation should take into account, Tolokers, and overlap for majority vote tasks.
Set the intervals of completed tasks and the number of review tasks for each interval.
Don't use selective majority vote check if:
There are many questions and many response options in the task.
Tolokers need to attach a file to their assignment.
Tolokers need to transcribe text or select objects in a photo, or other tasks where the Tolokers can't possibly provide the same responses.
Use selective majority vote check together with the fast responses and skipped assignments quality control rules. This way, you will eliminate dishonest Tolokers who skip tasks or complete them too quickly, which directly affects the number of completed tasks and overlap.
Learn more about how to set up overlap:
Last updated:Â June 23, 2023