accuracy_on_aggregates

crowdkit.metrics.workers.accuracy_on_aggregates.accuracy_on_aggregates | Source code

accuracy_on_aggregates(
    answers: DataFrame,
    aggregator: Optional[BaseClassificationAggregator] = ...,
    aggregates: Optional[Series] = None,
    by: Optional[str] = None
)

Accuracy on aggregates: a fraction of worker's answers that match the aggregated one.

Parameters Description

Parameters	Type	Description
`answers`	DataFrame	a data frame containing `task`, `worker` and `label` columns.
`aggregator`	Optional[BaseClassificationAggregator]	aggregation algorithm. default: MajorityVote
`aggregates`	Optional[Series]	aggregated answers for provided tasks.
`by`	Optional[str]	if set, returns accuracies for every worker in provided data frame. Otherwise, returns an average accuracy of all workers.
`Returns`	-	Union[float, pd.Series]

Crowd-Kit

Overview

Reference

Aggregation

Datasets

Learning

Metrics

Postprocessing