GoldMajorityVote

crowdkit.aggregation.classification.gold_majority_vote.GoldMajorityVote | Source code

GoldMajorityVote(self)
Copy text

Majority Vote when exist golden dataset (ground truth) for some tasks.

Calculates the probability of a correct label for each worker based on the golden set. Based on this, for each task, calculates the sum of the probabilities of each label. The correct label is the one where the sum of the probabilities is greater.

For Example: You have 10k tasks completed by 3k different workers. And you have 100 tasks where you already know ground truth labels. First you can call fit to calc percents of correct labels for each workers. And then call predict to calculate labels for you 10k tasks.

It's necessary that:

1. All workers must done at least one task from golden dataset.
2. All workers in dataset that send to predict, existed in answers dataset that was sent to fit.

Parameters Description

Examples:

import pandas as pd
from crowdkit.aggregation import GoldMajorityVote
df = pd.DataFrame(
    [
        ['t1', 'p1', 0],
        ['t1', 'p2', 0],
        ['t1', 'p3', 1],
        ['t2', 'p1', 1],
        ['t2', 'p2', 0],
        ['t2', 'p3', 1],
    ],
    columns=['task', 'worker', 'label']
)
true_labels = pd.Series({'t1': 0})
gold_mv = GoldMajorityVote()
result = gold_mv.fit_predict(df, true_labels)
Copy text

Methods Summary