Crowd-Kit Documentation

Overview
Aggregations
- Aggregation
  - Classification
    - DawidSkene
    - GLAD
    - GoldMajorityVote
    - MMSR
    - MajorityVote
    - Wawa
    - ZeroBasedSkill
  - Embeddings
    - ClosestToAverage
    - HRRASA
    - RASA
  - Image segmentation
    - SegmentationEM
    - SegmentationMajorityVote
    - SegmentationRASA
  - Pairwise
    - BradleyTerry
    - NoisyBradleyTerry
  - Texts
    - ROVER
    - TextHRRASA
    - TextRASA
    - TextSummarization
      - Overview
      - fit_predict
  - utils
- Datasets
  - get_datasets_list
  - load_dataset
- Metrics
  - Data
  - Workers
    - accuracy_on_aggregates
- Postprocessing
  - entropy_threshold

get_accuracy

Parameters Description

crowdkit.aggregation.utils.get_accuracy | Source code

get_accuracy(
    data: DataFrame,
    true_labels: Series,
    by: Optional[str] = None
)

Parameters Description

Parameters	Type	Description
`data`	DataFrame	Workers' labeling results. A pandas.DataFrame containing `task`, `worker` and `label` columns.
`true_labels`	Series	Tasks' ground truth labels. A pandas.Series indexed by `task` such that `labels.loc[task]` is the tasks's ground truth label.

Returns:

workers' skills. A pandas.Series index by workers and holding corresponding worker's skill
Return type:

Series

Was the article helpful?