Wawa

crowdkit.aggregation.classification.wawa.Wawa | Source code

Wawa(self)

The Worker Agreement with Aggregate (Wawa) algorithm consists of three steps:

  1. calculates the majority vote label;
  2. estimates workers' skills as a fraction of responses that are equal to the majority vote;
  3. calculates the weigthed majority vote based on skills from the previous step.

Parameters description

ParametersTypeDescription
labels_Optional[Series]

The task labels. The pandas.Series data is indexed by task so that labels.loc[task] is the most likely true label of tasks.

skills_Optional[Series]

The workers' skills. The pandas.Series data is indexed by worker and has the corresponding worker skill.

probas_Optional[DataFrame]

The probability distributions of task labels. The pandas.DataFrame data is indexed by task so that result.loc[task, label] is the probability that the task true label is equal to label. Each probability is in the range from 0 to 1, all task probabilities must sum up to 1.

Examples:

from crowdkit.aggregation import Wawa
from crowdkit.datasets import load_dataset
df, gt = load_dataset('relevance-2')
result = Wawa().fit_predict(df)

Methods summary

MethodDescription
fitFits the model to the training data.
fit_predictFits the model to the training data and returns the aggregated results.
fit_predict_probaFits the model to the training data and returns probability distributions of labels for each task.
predictPredicts the true labels of tasks when the model is fitted.
predict_probaReturns probability distributions of labels for each task when the model is fitted.

Last updated: March 31, 2023

Crowd-Kit
Overview
Reference
Aggregation
Datasets
Learning
Metrics
Postprocessing