Two F-words in peer review (fraud and feedback)

Overview

We discuss two major challenges in peer evaluation: (1) Fraud, where a group of participants form a coalition, get assigned to evaluate each other, and then give highly positive reviews; (2) Feedback, where opinions given by evaluatees (e.g., authors of papers) about evaluators (e.g., reviewers of papers) are biased by the evaluation given to them (e.g., whether the paper was accepted or rejected). We propose practical solutions with strong theoretical guarantees, and discuss important and impactful open problems.

Speaker

Nihar B. Shah

Carnegie Mellon UniversityAssistant Professor in the Machine Learning and the Computer Science Departments