toloka.autoquality.optimizer.AutoQuality | Source code
AutoQuality( self, toloka_client: TolokaClient, project_id: str, base_pool_id: str, training_pool_id: str, exam_pool_id: Optional[str] = None, exam_skill_id: Optional[str] = None, label_field: str = 'label', n_iter: int = 10, parameter_distributions: Dict = ..., score_func: Callable = default_calc_scores, ranking_func: Callable = default_calc_ranks, create_autoquality_pool_func: Callable = _create_autoquality_pool_default)This class implements a tool to help set up quality control for Toloka project.
To use toloka.autoquality install toloka-kit via pip install toloka-kit[autoquality]
| Parameters | Type | Description |
|---|---|---|
toloka_client | TolokaClient | TolokaClient instance to interact with requester's account |
project_id | str | Toloka project ID |
base_pool_id | str | Template Pool for autoquality pools |
training_pool_id | str | Training Pool ID |
exam_pool_id | Optional[str] | Exam Pool ID |
exam_skill_id | Optional[str] | Skill for filtering by exam perfomance |
label_field | str | Output field name |
n_iter | int | Number of an autoquality pools |
parameter_distributions | Dict | Parameter distributions |
score_func | Callable | Callable to calculate pool scores |
ranking_func | Callable | Callabale to ranking pools based on their scores |
create_autoquality_pool_func | Callable | Callable to create autoquality pool |
run_id | str | ID of autoquality run |
Examples:
aq = AutoQuality( toloka_client=toloka_client, project_id=..., base_pool_id=..., training_pool_id=..., exam_pool_id = ..., exam_skill_id = ...)aq.setup_pools()aq.create_tasks(aq_tasks)aq.run()aq.best_pool_params| Method | Description |
|---|---|
| archive_autoquality_pools | Archive all pools created by AutoQuality.setup_pools |
| create_tasks | Add tasks to autoquality pools. |
| run | Run autoquality process. |
| setup_pools | Create autoquality pools with sampled quality control parameters. |
Last updated: August 28, 2023