MajorityVote

toloka.client.collectors.MajorityVote | Source code

MajorityVote(
self,
*,
uuid: Optional[UUID] = None,
answer_threshold: Optional[int] = None,
history_size: Optional[int] = None
)

Counts correct responses determined by the majority vote method.

A response chosen by the majority is considered to be correct, and other responses are considered to be incorrect. Depending on the percentage of correct responses, you can either increase a Toloker's skill value, or to block the Toloker.

The collector can be used with conditions:

The collector can be used with actions:

Parameters description

ParametersTypeDescription
uuidOptional[UUID]

The ID of a collector. Note that when you clone a pool, both pools start using the same collector, because it is not cloned. Usually, it is not an intended behavior. For example, in this case one collector gathers history size from both pools.

answer_thresholdOptional[int]

The number of Tolokers considered the majority.

history_sizeOptional[int]

The maximum number of recent Toloker's responses to calculate the statistics. If it is omitted, calculation is based on all collected responses.

Examples:

The example shows how to reject all Toloker's responses if they significantly differ from the majority. The rule is applied after collecting at least 10 responses.

new_pool = toloka.client.pool.Pool()
new_pool.quality_control.add_action(
collector=toloka.client.collectors.MajorityVote(answer_threshold=2),
conditions=[
toloka.client.conditions.TotalAnswersCount > 9,
toloka.client.conditions.CorrectAnswersRate < 60,
],
action=toloka.client.actions.RejectAllAssignments(public_comment='Too low quality')
)

Last updated: August 28, 2023

Toloka-Kit
OverviewGetting API keyQuick start
Recipes
Reference
toloka.client
toloka.async_client
toloka.autoquality [autoquality]
toloka.metrics
toloka.streaming