Toloka documentation

Quality of instructions

    After completing a task, Tolokers rate different aspects of it. In particular, they indicate whether the instructions were clear to them. Based on these ratings, an instructions quality indicator is generated.


    If no Toloker has yet given a rating, the indicator value will be missing.

    How to calculate

    Qi=Hi×CSIQ_{i} = {H_{i}}\times{CSI}


    • QiQ_{i} is the instructions quality indicator.

    • HiH_{i} is the availability of instructions (0 means available and 1 means not available).

    • CSICSI is the Tolokers' rating, which shows how satisfied they are with the instructions.

    How to estimate

    • A good indicator is > 4.5.

    • A low indicator is < 4.5 or missing.