AI Camp 2021: Practical crowdsourcing for ML

Overview

AI stands on three pillars: algorithms, hardware and training data. While the first two have already become commodities on the market, the latter - reliable labelled data - is still a bottleneck in the industry. Need to add twice as much data to the training set to improve your model? Want to validate the accuracy of a new classificator in an hour? Or maybe you are building a human-in-the-loop process with 90% of cases processed automatically and the trickiest 10% of cases fine-tuned by people in real time. You can do it all with crowdsourcing , but only with crowdsourcing done right.

In this talk, we discuss how the new generation of methods and tools allows to collect high quality human labelled data on a large scale and why every ML specialist should know how to use crowdsourcing.

• Understand the applicability, benefits and limits of the crowdsourcing approach.
• Integrate an on-demand workforce into your processes and build human-in-the-loop processes.
• Control the quality and accuracy of data labeling to develop high performing ML models.
• Understand the full-cycle crowdsourcing project

Speaker

Daria Baidakova

TolokaDirector of Educational Programs at Toloka