GoldenSet

toloka.client.collectors.GoldenSet | Source code

GoldenSet(
    self,
    *,
    uuid: Optional[UUID] = None,
    history_size: Optional[int] = None
)
Copy text

Collects control and training task statistics for a Toloker.

Use control tasks to assign a skill to Tolokers based on their responses and block Tolokers who submit incorrect responses.

It is better not to use this collector if:

• There are a lot of response options.
• Tolokers need to attach files to assignments.
• Tolokers need to transcribe text.
• Tolokers need to select objects on a photo.
• Tasks don't have a correct or incorrect responses. For example, you ask about Toloker preferences.

The collector can be used with conditions:

• TotalAnswersCount — The number of completed control and training tasks.
• CorrectAnswersRate — The percentage of correct responses to control and training tasks.
• IncorrectAnswersRate — The percentage of incorrect responses to control and training tasks.
• GoldenSetAnswersCount — The number of completed control tasks.
• GoldenSetCorrectAnswersRate — The percentage of correct responses to control tasks.
• GoldenSetIncorrectAnswersRate — The percentage of incorrect responses to control tasks.

The collector can be used with actions:

• RestrictionV2 blocks access to projects or pools.
• ApproveAllAssignments accepts all Toloker's assignments.
• RejectAllAssignments rejects all Toloker's assignments.
• SetSkill sets Toloker's skill value.
• SetSkillFromOutputField sets Toloker's skill value using an output field.

Parameters description

Examples:

The example shows how to accept all assignments if more than 80% of responses to control tasks are correct.

new_pool = toloka.client.pool.Pool()
new_pool.quality_control.add_action(
    collector=toloka.client.collectors.GoldenSet(history_size=5),
    conditions=[
        toloka.client.conditions.GoldenSetCorrectAnswersRate > 80,
        toloka.client.conditions.GoldenSetAnswersCount >= 5,
    ],
    action=toloka.client.actions.ApproveAllAssignments()
)
Copy text

Last updated: August 28, 2023