Toloka documentation

Quality of instructions

After completing a task, Tolokers rate different aspects of it. In particular, they indicate whether the instructions were clear to them. Based on these ratings, an instructions quality indicator is generated.


If no Toloker has yet given a rating, the indicator value will be missing.

How to calculate

Qi=Hi×CSIQ_{i} = {H_{i}}\times{CSI}


  • QiQ_{i} is the instructions quality indicator.

  • HiH_{i} is the availability of instructions (0 means available and 1 means not available).

  • CSICSI is the Tolokers' rating, which shows how satisfied they are with the instructions.

How to estimate

  • A good indicator is > 4.5.

  • A low indicator is < 4.5 or missing.

See also