Selective majority vote check
Before you start
To set up selective majority vote check, you need a project, a pool, and a file with tasks.
This function allows you to set up majority vote check for only a portion of tasks.
Depending on how many tasks the performer completed in
N days, overlap for certain tasks will be higher or lower than in the pool settings.
- Reducing overlap
The pool's overlap is set to 4. To speed up pool completion and reduce data labeling costs, we'll reduce overlap for every third, fifth, and tenth task.
Performer A completed 15 tasks in the last 10 days. Performer B completed 30 tasks. Performer C completed 60 tasks.
For performer A, overlap will be set to 3 for every third task; for performer B — for every fifth task; for performer C — for every tenth task.
This way we can reduce overlap for more experienced performers.
- Increasing overlap
The pool's overlap is set to 2. This project has experienced performers who perform tasks well. We want to increase overlap for new performers and check the experienced performers' responses from time to time. To do this, we will increase overlap for every first, third, fifth, and tenth task.
Performer A completed 5 tasks in the last 10 days. Performer B completed 15 tasks. Performer C completed 25 tasks. Performer E completed 40 tasks.
For performer A, overlap will be set to 5 for every task; for performer B — for every third task; for performer C — for every fifth task; for performer E — for every tenth task.
This way we will increase overlap for inexperienced users and save money on the responses of experienced performers.
- When to use
- Use selective majority vote check if:
There are few or no control tasks in the project.
- You have experienced performers whom you trust.
To add selective majority vote check:
Upload tasks to the pool using smart mixing.
Switch to pool editing mode.
Under Smart mixing settings, click .
Specify how many days the calculation should take into account, performers, and overlap for majority vote tasks.
Set the intervals of completed tasks and the number of review tasks for each interval.
Tips and recommendations
- Don't use selective majority vote check if:
There are many questions and many response options in the task.
Users need to attach a file to their assignment.
Users need to transcribe text or select objects in a photo, or other tasks where the performers can't possibly provide the same responses.