It may be tempting to set a higher price hoping that it will make performers be more conscientious. However, practice shows that pricing impacts how fast a task is taken up, but it does not directly influence the quality of results. The only way to actually influence quality with money is by using a smart motivation scheme, where the task price changes depending on the quality of individual performers.
This controversy leads to the concept of performance-based payment. Instead of having a fixed price for each approved task, payment can be connected to the quality of the work done by the performer. There are various ways to calculate individual quality, but most of them rely on checks that are hidden from the performer. This means that to maximize their earnings they need to complete every assigned task carefully, thereby increasing the overall quality of data.
Here's an example of how a bonus of $0.50 can be awarded to performers with different quality. Note that in all three cases quality is reasonable, meaning that the performers have successfully passed through basic quality filters.
Fixed payment | Accuracy bonus | Total | |
---|---|---|---|
Correct on 7 out of 10 | $1 | 0.7 x $0.5 | $1.35 |
Correct on 5 out of 10 | $1 | 0.5 x $0.5 | $1.25 |
Correct on 10 out of 10 | $1 | 1.0 x $0.5 | $1.50 |
In Toloka, you can configure different prices for different skill values by using the dynamic pricing pool setting.