MajorityVote

toloka.client.collectors.MajorityVote | Source code

MajorityVote(
    self,
    *,
    uuid: Optional[UUID] = None,
    answer_threshold: Optional[int] = None,
    history_size: Optional[int] = None
)

Counts correct responses determined by the majority vote method.

A response chosen by the majority is considered to be correct, and other responses are considered to be incorrect. Depending on the percentage of correct responses, you can either increase a Toloker's skill value, or to block the Toloker.

The collector can be used with conditions:

TotalAnswersCount — The number of completed tasks by the Toloker.
CorrectAnswersRate — The percentage of correct responses.
IncorrectAnswersRate — The percentage of incorrect responses.

The collector can be used with actions:

RestrictionV2 blocks access to projects or pools.
ApproveAllAssignments accepts all Toloker's assignments.
RejectAllAssignments rejects all Toloker's assignments.
SetSkill sets Toloker's skill value.
SetSkillFromOutputField sets Toloker's skill value using an output field.

Parameters description

Parameters	Type	Description
`uuid`	Optional[UUID]	The ID of a collector. Note that when you clone a pool, both pools start using the same collector, because it is not cloned. Usually, it is not an intended behavior. For example, in this case one collector gathers history size from both pools.
`answer_threshold`	Optional[int]	The number of Tolokers considered the majority.
`history_size`	Optional[int]	The maximum number of recent Toloker's responses to calculate the statistics. If it is omitted, calculation is based on all collected responses.

Examples:

The example shows how to reject all Toloker's responses if they significantly differ from the majority. The rule is applied after collecting at least 10 responses.

new_pool = toloka.client.pool.Pool()
new_pool.quality_control.add_action(
    collector=toloka.client.collectors.MajorityVote(answer_threshold=2),
    conditions=[
        toloka.client.conditions.TotalAnswersCount > 9,
        toloka.client.conditions.CorrectAnswersRate < 60,
    ],
    action=toloka.client.actions.RejectAllAssignments(public_comment='Too low quality')
)

Last updated: August 28, 2023

Toloka-Kit

Overview Getting API key Quick start

Recipes

Reference

toloka.client

toloka.async_client

toloka.autoquality [autoquality]

toloka.metrics

toloka.streaming