Work with results
Once the pool is finished, it’s time to check the results.
Monitor statistics
Remember to check project statistics and pool statistics and track metrics like time spent, budget, and quality.
Analyze which aspects you could improve.
Download the results

There are two ways to aggregate results in the Toloka interface.

1. Aggregation using the Dawid-Skene method:

This type of aggregation works well if the task overlap was >1 and multiple responses can be compared with each other. In the pool, click Download results -> Dawid-Skene aggregation model. You can monitor the process and download the finished file on the page with the list of operations. Each response is listed with a percentage based on the confidence.

2. Aggregation by skill:

Aggregates responses based on the level of trust in the performer, which is indicated by the skill value. This works well if the pool used dynamic overlap, or if you set up a skill that measures the quality of responses and the output fields have a limited set of possible values.

You can also use other aggregation methods. To learn more, take a look at our presentation on aggregation.
Yandex.Toloka News
Receive information about platform updates, partners, training materials, and other news.
Файлы cookies
Для персонализации сервисов Яндекс использует файлы cookies. Продолжая использование сайта, вы соглашаетесь с этим. Подробности о файлах cookies и об обработке ваших данных в Политике конфиденциальности.
Wed Oct 21 2020 22:16:40 GMT+0300 (Moscow Standard Time)