toloka.client.collectors.GoldenSet | Source code
GoldenSet( self, *, uuid: Optional[UUID] = None, history_size: Optional[int] = None )
Collects control and training task statistics for a Toloker.
Use control tasks to assign a skill to Tolokers based on their responses and block Tolokers who submit incorrect responses.
It is better not to use this collector if:
- There are a lot of response options.
- Tolokers need to attach files to assignments.
- Tolokers need to transcribe text.
- Tolokers need to select objects on a photo.
- Tasks don't have a correct or incorrect responses. For example, you ask about Toloker preferences.
The collector can be used with conditions:
- TotalAnswersCount — The number of completed control and training tasks.
- CorrectAnswersRate — The percentage of correct responses to control and training tasks.
- IncorrectAnswersRate — The percentage of incorrect responses to control and training tasks.
- GoldenSetAnswersCount — The number of completed control tasks.
- GoldenSetCorrectAnswersRate — The percentage of correct responses to control tasks.
- GoldenSetIncorrectAnswersRate — The percentage of incorrect responses to control tasks.
The collector can be used with actions:
- RestrictionV2 blocks access to projects or pools.
- ApproveAllAssignments accepts all Toloker's assignments.
- RejectAllAssignments rejects all Toloker's assignments.
- SetSkill sets Toloker's skill value.
- SetSkillFromOutputField sets Toloker's skill value using an output field.
The ID of a collector. Note that when you clone a pool, both pools start using the same collector, because it is not cloned. Usually, it is not an intended behavior. For example, in this case one collector gathers history size from both pools.
The maximum number of recent control or training tasks used to calculate the statistics. If
The example shows how to accept all assignments if more than 80% of responses to control tasks are correct.
new_pool = toloka.pool.Pool(....) new_pool.quality_control.add_action( collector=toloka.collectors.GoldenSet(history_size=5), conditions=[ toloka.conditions.GoldenSetCorrectAnswersRate > 80, toloka.conditions.GoldenSetAnswersCount >= 5, ], action=toloka.actions.ApproveAllAssignments() )