Basic pricing
it's helpful to look at the task from the performer's point of view:
- Think about how long it will take someone to read the instructions, figure out what to do, and complete it.
- Consider what your task looks like compared to other requesters’ tasks on the platform.
- Register as a performer and check the prices for similar projects. Pricing varies across different types of tasks, depending on the expected time and effort involved. For instance, classifying basic images is much faster and easier than creating content, and this difference is reflected in the task prices.
- Don’t forget about the law of supply and demand: the fewer performers available for your task (perhaps because it requires specific skills), the higher the price needs to be.
- But even careful analysis doesn’t always help predict demand — the optimal price may depend on individual factors in the project. The easiest solution is to make a test run with a small amount of data and see how performers behave on your task.
It may be tempting to set a higher price hoping that it will make performers be more conscientious. However, practice shows that pricing impacts how fast a task is taken up, but it does not directly influence the quality of results. The only way to actually influence quality with money is by using a smart motivation scheme, where the task price changes depending on the quality of individual performers.
Smart pricing
This controversy leads to the concept of performance-based payment. Instead of having a fixed price for each approved task, payment can be connected to the quality of the work done by the performer. There are various ways to calculate individual quality, but most of them rely on checks that are hidden from the performer. This means that to maximize their earnings they need to complete every assigned task carefully, thereby increasing the overall quality of data.
Here's an example of how a bonus of $0.50 can be awarded to performers with different quality. Note that in all three cases quality is reasonable, meaning that the performers have successfully passed through basic quality filters.
Fixed payment | Accuracy bonus | Total | |
|---|---|---|---|
Correct on 7 out of 10 | $1 | 0.7 x $0.5 | $1.35 |
Correct on 5 out of 10 | $1 | 0.5 x $0.5 | $1.25 |
Correct on 10 out of 10 | $1 | 1.0 x $0.5 | $1.50 |
- Performers need to be fully aware of the pricing strategy, so that they see a clear connection between effort and earnings.
- Regardless of bonuses, there always should be a reasonable fixed payment for all approved tasks. The reasoning here is both ethical and practical: if the fixed price is too low, it may scare away decent performers.
- The maximum bonus should be large enough to motivate performers to put more effort into your task. In practice, anything above thirty percent of the fixed payment is a reasonable starting point for the maximum bonus.
In Toloka, you can configure different prices for different skill values by using the dynamic pricing pool setting.