consistency
crowdkit.metrics.data._classification.consistency
| Source code
consistency(
answers: DataFrame,
workers_skills: Optional[Series] = None,
aggregator: BaseClassificationAggregator = ...,
by_task: bool = False
)
Consistency metric: posterior probability of aggregated label given workers skills
calculated using standard Dawid-Skene model.
Parameters Description
Parameters | Type | Description |
---|---|---|
answers |
DataFrame | A data frame containing |
workers_skills |
Optional[Series] | workers skills e.g. golden set skills. If not provided, uses aggregator's |
aggregator |
BaseClassificationAggregator | aggregation method, default: MajorityVote |
by_task |
bool | if set, returns consistencies for every task in provided data frame. |
-
Returns:
Union[float, pd.Series]
-
Return type:
Union[float, Series]