Wawa

crowdkit.aggregation.classification.wawa.Wawa | Source code

Wawa(self)

Worker Agreement with Aggregate.

This algorithm does three steps:

Calculate the majority vote label
Estimate workers' skills as a fraction of responses that are equal to the majority vote
Calculate the weigthed majority vote based on skills from the previous step

Parameters Description

Parameters	Type	Description
`labels_`	Optional[Series]	Tasks' labels. A pandas.Series indexed by `task` such that `labels.loc[task]` is the tasks's most likely true label.
`skills_`	Optional[Series]	workers' skills. A pandas.Series index by workers and holding corresponding worker's skill
`probas_`	Optional[DataFrame]	Tasks' label probability distributions. A pandas.DataFrame indexed by `task` such that `result.loc[task, label]` is the probability of `task`'s true label to be equal to `label`. Each probability is between 0 and 1, all task's probabilities should sum up to 1

Examples:

from crowdkit.aggregation import Wawa
from crowdkit.datasets import load_dataset
df, gt = load_dataset('relevance-2')
result = Wawa().fit_predict(df)

Method	Description
fit	Fit the model.
fit_predict	Fit the model and return aggregated results.
fit_predict_proba	Fit the model and return probability distributions on labels for each task.
predict	Infer the true labels when the model is fitted.
predict_proba	Return probability distributions on labels for each task when the model is fitted.

Crowd-Kit

Reference

Aggregation

Datasets

Learning

Metrics

Postprocessing