Let's say you want to improve a computer vision algorithm that is used for self-driving cars. You give performers a set of photos of streets and ask them to outline every traffic light with a bounding box. After the labeling is finished you receive noisy labels. Why? Because the task didn’t make allowances for details such as photos without any traffic lights and you didn’t consider how to check the quality of results provided by thousands of anonymous performers.
You can decompose this task and try again with a pipeline like this:
- Check whether the image contains traffic lights or not (classification)
- Outline each traffic light with a bounding box (object segmentation)
- Check whether traffic lights are outlined correctly (classification)
To ensure quality, the second and third projects are completed by different performers. The third project checks whether the results of the second project are correct, and performers are only rewarded for correct results.