What is overlap?
Overlap is the number of Tolokers who should complete each task in the pool.
You can set overlap in the pool settings:
Tips and recommendations
| Task type | Recommended overlap |
|---|---|
| Most tasks have automatic acceptance | from 3 to 5 |
| Simple task | 3 |
| Reviewed task | 1 |
You can change overlap after the pool is started. To do this, open the pool editing mode and set the new overlap value. You don't need to restart the pool. Updating the settings is usually fast, but if there are many tasks, it may take several minutes.
Advanced overlap types
-
Dynamic overlap lets you change overlap depending on:
- How well the Tolokers do on the task.
- How well Tolokers' responses match each other.
This type of overlap helps you save money without reducing the quality of data labeling.
-
Selective majority vote verification allows you to set up majority vote check for only a portion of tasks.
This type of overlap helps you save money and speed up pool completion.
For developers
Troubleshooting
Yes, you can do that. Set up dynamic overlap (incremental relabeling, IRL).
You can use overlap to let multiple Tolokers do the same task. The overlap value is set up in the pool settings.
The progress bar shows the number of task suites including the overlap. If the overlap is greater than one, the number of task suites is different from the total number of tasks.
overlap = 3 in the pool and response threshold = 3 in the majority vote?In this case, if you don't have 3 identical responses for your task (response threshold), no user would be considered a good or poor Toloker, because the system can't see which of the Tolokers made an error.
But if you set response threshold = 2 with overlap = 3, then two users with the same responses are considered good Tolokers, but the third user, who gives a different response, is a poor Toloker.