You can aggregate responses in two ways:
Yes — Aggregate responses by skill (
No — Aggregate responses using the Dawid-Skene method (
The Dawid-Skene aggregation model works with control and training tasks as well as with general tasks. Responses specified for control tasks might not match the actual responses to the task if Tolokers often submit an incorrect response to it.
The Dawid-Skene aggregation model automatically evaluates
|L|²parameters for each Toloker, where
Lis the number of different aggregation values. Note that these parameters are determined automatically and are only used in calculations.
Because the Dawid-Skene method evaluates
|L|²parameters for each Toloker, we don't recommend using it when the Toloker labels
< |L|²tasks. In this case, the quality of aggregation may be poor.
Aggregation via the interface and via the API works differently. When using the Dawid-Skene aggregation model, you can't aggregate more than one field via the API. You can only do that via the interface.
|POST||/aggregated-solutions/aggregate-by-pool||Starts aggregating responses to all completed tasks in the pool.|
|POST||/aggregated-solutions/aggregate-by-task||Starts aggregating responses to a single task.|
|GET||/aggregated-solutions/<operation_id>||Gets aggregated responses.|