ZeroBasedSkill

crowdkit.aggregation.classification.zero_based_skill.ZeroBasedSkill | Source code

ZeroBasedSkill(
    self,
    n_iter: int = 100,
    lr_init: float = 1.0,
    lr_steps_to_reduce: int = 20,
    lr_reduce_factor: float = 0.5,
    eps: float = 1e-05
)

The Zero-Based Skill aggregation model aka ZBS.

Performs weighted majority voting on tasks. After processing a pool of tasks, re-estimates workers' skills through a gradient descend step of optimization of the mean squared error of current skills and the fraction of responses that are equal to the aggregated labels.

Repeats this process until labels do not change or the number of iterations exceeds.

It's necessary that all workers in a dataset that send to 'predict' existed in answers the dataset that was sent to 'fit'.

Parameters Description

Parameters	Type	Description
`n_iter`	int	A number of iterations to perform.
`lr_init`	float	A starting learning rate.
`lr_steps_to_reduce`	int	A number of steps necessary to decrease the learning rate.
`lr_reduce_factor`	float	A factor that the learning rate will be multiplied by every `lr_steps_to_reduce` steps.
`eps`	float	A convergence threshold.

Examples:

from crowdkit.aggregation import ZeroBasedSkill
from crowdkit.datasets import load_dataset
df, gt = load_dataset('relevance-2')
result = ZeroBasedSkill().fit_predict(df)

Methods Summary

Method	Description
fit	Fit the model.
fit_predict	Fit the model and return aggregated results.
fit_predict_proba	Fit the model and return probability distributions on labels for each task.
predict	Infer the true labels when the model is fitted.
predict_proba	Return probability distributions on labels for each task when the model is fitted.

Crowd-Kit

Overview

Reference

Aggregation

Datasets

Learning

Metrics

Postprocessing