Toloka documentation

AutoQuality

toloka.autoquality.optimizer.AutoQuality | Source code

AutoQuality(
    self,
    toloka_client: TolokaClient,
    project_id: str,
    base_pool_id: str,
    training_pool_id: str,
    exam_pool_id: Optional[str] = None,
    exam_skill_id: Optional[str] = None,
    label_field: str = 'label',
    n_iter: int = 10,
    parameter_distributions: Dict = ...,
    score_func: Callable = default_calc_scores,
    ranking_func: Callable = default_calc_ranks,
    create_autoquality_pool_func: Callable = _create_autoquality_pool_default
)

This class implements a tool to help set up quality control for Toloka project.

To use toloka.autoquality install toloka-kit via pip install toloka-kit[autoquality]

Parameters Description

Parameters Type Description
toloka_client TolokaClient

TolokaClient instance to interact with requester's account

project_id str

Toloka project ID

base_pool_id str

Template Pool for autoquality pools

training_pool_id str

Training Pool ID

exam_pool_id Optional[str]

Exam Pool ID

exam_skill_id Optional[str]

Skill for filtering by exam perfomance

label_field str

Output field name

n_iter int

Number of an autoquality pools

parameter_distributions Dict

Parameter distributions

score_func Callable

Callable to calculate pool scores

ranking_func Callable

Callabale to ranking pools based on their scores

create_autoquality_pool_func Callable

Callable to create autoquality pool

run_id str

ID of autoquality run

Examples:

aq = AutoQuality(
  toloka_client=toloka_client,
  project_id=...,
  base_pool_id=...,
  training_pool_id=...,
  exam_pool_id = ...,
  exam_skill_id = ...
)
aq.setup_pools()
aq.create_tasks(aq_tasks)
aq.run()
aq.best_pool_params

Methods Summary

Method Description
archive_autoquality_pools Archive all pools created by AutoQuality.setup_pools
create_tasks Add tasks to autoquality pools.
run Run autoquality process.
setup_pools Create autoquality pools with sampled quality control parameters.