Skill estimation is a crux of crowdsourcing – if we know or estimate the accuracies of performers, we can aggregate their answers more efficiently. However, skill estimation is often challenging because worker assignments are sparse and irregular due to the arbitrary and uncontrolled availability of workers. In this talk, we will reduce the skill-estimation problem to a statistical problem of matrix completion. Next, we will establish necessary and sufficient conditions under which efficient skill recovery is possible. Finally, we propose practical algorithms that carefully estimate the skills of the performers given their responses. Importantly, we will show that it is possible to estimate these skills even when workers do not satisfy strong assumptions made by the conventional crowdsourcing models. This observation enables estimators that perform well even when some workers are random or malicious.


Alex Olshevsky
Boston University
Associate Professor
Don't miss
Mon Aug 02 2021 12:22:09 GMT+0300 (Moscow Standard Time)