Use quality control rules to restrict access to the tasks for the Tolokers who try and breach them.
Connect the Toloka-Kit library to your script.
import toloka.client as tolokaReplace the sample API key with your own one.
toloka_client = toloka.TolokaClient('PlaceYourRealApiKey_Here', 'PRODUCTION');Create a pool and set the default overlap value for all the tasks which will be uploaded without additionally specifying it. Use the default_overlap_for_new_task_suites value of the Defaults class for that.
Refer to the Create pool recipe for more information on how to create a pool and what parameters you can use.
new_pool = toloka.pool.Pool(
project_id='133047',
private_name='Pool with quality control rules',
will_expire=datetime(2030, 1, 1),
reward_per_assignment=0.05,
assignment_max_duration_seconds=60*5
)Now, add some quality control rules to the pool.
Each of the quality control rules has three main components:
collector which specifies what information to collect before the system applies some conditions and restrictions,conditions which sets conditions that the system considers before applying some actions,action which defines the actions that the system applies based on the collector and conditions data.Set up the Fast responses quality control rule using the add_action() method to create the AssignmentSubmitTime collector class object with the appropriate conditions and actions.
new_pool.quality_control.add_action(
collector=toloka.collectors.AssignmentSubmitTime(history_size=5, fast_submit_threshold_seconds=20),
conditions=[toloka.conditions.FastSubmittedCount > 1],
action=toloka.actions.RestrictionV2(
scope='POOL',
duration=10,
duration_unit='DAYS',
private_comment='Fast responses',
)
)Refer to the Fast responses section for more information on how to set up the Fast responses quality control rule and what parameters it should have for the correct settings.
Set up the Majority votes quality control rule using the add_action() method to create the MajorityVote collector class object with the appropriate conditions and actions.
new_pool.quality_control.add_action(
collector=toloka.collectors.MajorityVote(answer_threshold=2),
conditions=[
toloka.conditions.TotalAnswersCount > 9,
toloka.conditions.CorrectAnswersRate < 60,
],
action=toloka.actions.RejectAllAssignments(public_comment='Too low quality')
)Refer to the Majority vote section for more information on how to set up the Majority vote quality control rule and what parameters it should have for the correct settings.
Set up the Skipped assignments quality control rule using the add_action() method to create the SkippedInRowAssignments collector class object with the appropriate conditions and actions.
new_pool.quality_control.add_action(
collector=toloka.collectors.SkippedInRowAssignments(),
conditions=[toloka.conditions.SkippedInRowCount > 3],
action=toloka.actions.RestrictionV2(
scope='POOL',
duration=15,
duration_unit='DAYS',
private_comment='Skips too many task suites in a row',
)
)Refer to the Skipped assignments section for more information on how to set up the Skipped assignments quality control rule and what parameters it should have for the correct settings.
Set up the Control tasks quality control rule using the add_action() method to create the GoldenSet collector class object with the appropriate conditions and actions.
new_pool.quality_control.add_action(
collector=toloka.collectors.GoldenSet(history_size=5),
conditions=[
toloka.conditions.GoldenSetCorrectAnswersRate > 80,
toloka.conditions.GoldenSetAnswersCount >= 5,
],
action=toloka.actions.ApproveAllAssignments()
)Refer to the Control tasks section for more information on how to set up the Control tasks quality control rule and what parameters it should have for the correct settings.
See the Collectors section of the Toloka-Kit reference for the complete list of the available collectors used to create quality control rules and their possible values.
This actually creates a pool in Toloka.
new_pool = toloka_client.create_pool(new_pool)The create_pool() request will return the Pool class object. You can use its attributes to print the information you need.
print(new_pool.id)You should get an output with the created pool ID which looks like this.
1444049import toloka.client as tolokatoloka_client = toloka.TolokaClient('PlaceYourRealApiKey_Here', 'PRODUCTION')new_pool = toloka.pool.Pool( project_id='133047', private_name='Pool with quality control rules', will_expire=datetime(2030, 1, 1), reward_per_assignment=0.05, assignment_max_duration_seconds=60*5)new_pool.quality_control.add_action( collector=toloka.collectors.AssignmentSubmitTime(history_size=5, fast_submit_threshold_seconds=20), conditions=[toloka.conditions.FastSubmittedCount > 1], action=toloka.actions.RestrictionV2( scope='POOL', duration=10, duration_unit='DAYS', private_comment='Fast responses', ))new_pool.quality_control.add_action( collector=toloka.collectors.MajorityVote(answer_threshold=2), conditions=[ toloka.conditions.TotalAnswersCount > 9, toloka.conditions.CorrectAnswersRate < 60, ], action=toloka.actions.RejectAllAssignments(public_comment='Too low quality'))new_pool.quality_control.add_action( collector=toloka.collectors.SkippedInRowAssignments(), conditions=[toloka.conditions.SkippedInRowCount > 3], action=toloka.actions.RestrictionV2( scope='POOL', duration=15, duration_unit='DAYS', private_comment='Skips too many task suites in a row', ))new_pool.quality_control.add_action( collector=toloka.collectors.GoldenSet(history_size=5), conditions=[ toloka.conditions.GoldenSetCorrectAnswersRate > 80, toloka.conditions.GoldenSetAnswersCount >= 5, ], action=toloka.actions.ApproveAllAssignments())new_pool = toloka_client.create_pool(new_pool)print(new_pool.id)Last updated: February 7, 2023