GoldenSet

toloka.client.collectors.GoldenSet | Source code

GoldenSet(
self,
*,
uuid: Optional[UUID] = None,
history_size: Optional[int] = None
)

Collects control and training task statistics for a Toloker.

Use control tasks to assign a skill to Tolokers based on their responses and block Tolokers who submit incorrect responses.

It is better not to use this collector if:

  • There are a lot of response options.
  • Tolokers need to attach files to assignments.
  • Tolokers need to transcribe text.
  • Tolokers need to select objects on a photo.
  • Tasks don't have a correct or incorrect responses. For example, you ask about Toloker preferences.

The collector can be used with conditions:

The collector can be used with actions:

Parameters description

ParametersTypeDescription
uuidOptional[UUID]

The ID of a collector. Note that when you clone a pool, both pools start using the same collector, because it is not cloned. Usually, it is not an intended behavior. For example, in this case one collector gathers history size from both pools.

history_sizeOptional[int]

The maximum number of recent control or training tasks used to calculate the statistics. If history_size is omitted, all Toloker's control or training tasks in the pool are counted.

Examples:

The example shows how to accept all assignments if more than 80% of responses to control tasks are correct.

new_pool = toloka.client.pool.Pool()
new_pool.quality_control.add_action(
collector=toloka.client.collectors.GoldenSet(history_size=5),
conditions=[
toloka.client.conditions.GoldenSetCorrectAnswersRate > 80,
toloka.client.conditions.GoldenSetAnswersCount >= 5,
],
action=toloka.client.actions.ApproveAllAssignments()
)

Last updated: August 28, 2023

Toloka-Kit
OverviewGetting API keyQuick start
Recipes
Reference
toloka.client
toloka.async_client
toloka.autoquality [autoquality]
toloka.metrics
toloka.streaming