WSDM 2023 Crowd Science Workshop

CANDLE: Collaboration of Humans and Learning Algorithms for Data Labeling.

Call for papersMar 3, 2023, 05:30 UTC


Crowdsourcing has been used to produce impactful and large-scale datasets for Machine Learning and Artificial Intelligence (AI), such as ImageNET, SuperGLUE, etc. Since the rise of crowdsourcing in the early 2000s, the AI community has been studying its computational, system design, and data-centric aspects at various angles at such workshops as CSS, CrowdML, DCAI, and HILL.

We welcome studies on developing and enhancing crowdworker-centric tools that offer task matching, requester assessment, and instruction validation, among other topics. We are also interested in exploring methods that leverage crowdworkers as a resource for improving the recognition and performance of machine learning models. Thus, we invite studies of active learning techniques, methods for joint learning from noisy data and from crowds, novel approaches for crowd-computer interaction, repetitive task automation, and role separation between humans and machines. Moreover, we invite works on designing and applying such techniques in various domains, including e-commerce and medicine.


Ujwal Gadiraju
Delft University of TechnologyAssistant ProfessorProfile link


Dmitry Ustalov
TolokaHead of ResearchProfile link
Saiph Savage
Northeastern UniversityAssistant ProfessorProfile link
Niels van Berkel
Aalborg UniversityAssociate ProfessorProfile link
Yang Liu
University of California, Santa CruzAssistant ProfessorProfile link

Similar conferences

Don't miss out

Be sure to attend our informative workshops,
tutorials, and webinars.