toloka.client.collectors.GoldenSet | Source code
GoldenSet( self, *, uuid: Optional[UUID] = None, history_size: Optional[int] = None )
How performer answers on control tasks
Use control tasks to assign a skill to performers based on their responses and ban performers who submit incorrect responses.
Don't use it if:
- You have a lot of response options.
- Users need to attach a file to their assignment.
- Users need to transcribe text.
- Users need to select objects in a photo.
- Tasks don't have a correct or incorrect response. For example: "Which image do you like best?" or "Choose the page design option that you like best".
Used with conditions:
- TotalAnswersCount - The number of completed control and training tasks.
- CorrectAnswersRate - The percentage of correct responses in training and control tasks.
- IncorrectAnswersRate - The percentage of incorrect responses in training and control tasks.
- GoldenSetAnswersCount - The number of completed control tasks
- GoldenSetCorrectAnswersRate - The percentage of correct responses in control tasks.
- GoldenSetIncorrectAnswersRate - The percentage of incorrect responses in control tasks.
Used with actions:
- RestrictionV2 - Block access to projects or pools.
- ApproveAllAssignments - Approve all replies from the performer.
- RejectAllAssignments - Reject all replies from the performer.
- SetSkill - Set perfmer skill value.
- SetSkillFromOutputField - Set performer skill value from source.
The number of the performer's last responses to control tasks.
How to approve all assignments if performer doing well with golden tasks.
new_pool = toloka.pool.Pool(....) new_pool.quality_control.add_action( collector=toloka.collectors.GoldenSet(history_size=5), conditions=[toloka.conditions.GoldenSetCorrectAnswersRate > 90], action=toloka.actions.ApproveAllAssignments() )