The practice of crowdsourcing

Overview

Many data-intensive applications that use ML/AI techniques depend on humans providing the initial dataset, enabling algorithms to process the rest or for other humans to evaluate such algorithms' performance. There are, however, practical issues with the adoption of human computation at scale in the real world. It remains difficult to build systems and data processing pipelines that require crowd computing. In this talk, we discuss practical considerations for designing and implementing tasks that require the use of humans and machines in combination with the goal of producing high-quality labels.

Speaker

Omar Alonso

InstacartOmar is an Engineering Manager at Instacart where he works on the intersection of information retrieval, knowledge graphs, and human computation. He spent a decade at Microsoft working on Bing and related properties. He is the co-organizer of DESIRES, a new information retrieval conference with a focus on system implementation and experimental design. He is also a co-chair of the crowdsourcing and human computation track for WWW 2021.Profile link