Pricing

Subscribe to Toloka News

Subscribe to Toloka News

Basic pricing

  • Consider how long it will take someone to read the instructions, figure out what to do, and complete it.
  • The fewer performers available for your task (perhaps because it requires specific skills), the higher the price needs to be.
  • Register as a performer and check the prices for similar projects. Pricing varies across different types of tasks, depending on the expected time and effort involved. For instance, classifying basic images is much faster and easier than creating content.
  • Optimal price may depend on individual factors in the project. Test run projects with a small amount of data and see how performers behave on your task.

Pricing impacts how fast a task is taken up, but it does not directly influence the quality of results. The only way to actually influence quality with money is by using a smart motivation scheme, where the task price changes depending on the quality of individual performers.

Smart pricing

Performers try to maximize earnings while minimizing effort and time spent. However, it is highly undesirable for requesters who want their data to be labeled with the best quality possible.

This can be solved with performance-based payment. Instead of having a fixed price for each approved task, payment can be connected to the quality of the work done by the performer. There are various ways to calculate individual quality, but most of them rely on checks that are hidden from the performer. This means that to maximize their earnings they need to complete every assigned task carefully, thereby increasing the overall quality of data.

Here's an example of how a bonus of $0.50 can be awarded to performers with different quality. Note that in all three cases quality is reasonable, meaning that the performers have successfully passed through basic quality filters.

Fixed paymentAccuracy bonusTotal
One complex task$10.7 x $0.5$1.35
Several smaller tasks$10.5 x $0.5$1.25
Correct on 10 out of 10$11.0 x $0.5$1.50

Important points to keep in mind when using performance-based pricing:

  • Performers need to be fully aware of the pricing strategy, so that they see a clear connection between effort and earnings.
  • Regardless of bonuses, there always should be a reasonable fixed payment for all approved tasks. The reasoning here is both ethical and practical: if the fixed price is too low, it may scare away decent performers.
  • The maximum bonus should be large enough to motivate performers to put more effort into your task. Anything above thirty percent of the fixed payment is a reasonable starting point for the maximum bonus.

In Toloka, you can configure different prices for different skill values by using dynamic pricing pool setting.