How Toloka helped train Yandex autonomous vehicles

About the company

Yandex began developing its autonomous vehicle system in early 2017 and it has grown to 200 self-driving cars, which drove more than 4 million autonomous miles on city streets in Russia, Israel, and the US.

Task

Yandex uses Toloka to label tens of thousands images needed to train neural network to detect surrounding objects in the cities of Russia.

An important task for the creator of a self-driving vehicle is to train it to extract information about its surroundings from the data it receives from sensors. During the ride, the car records everything it sees around it. This data is uploaded to the cloud, where the preliminary analysis is completed, and then it goes to post-processing, which includes labeling the data. The labeled data is sent to the machine learning algorithms, the result is returned to the vehicle, and the cycle repeats, improving the quality of object detection through multiple iterations.

Many different objects are encountered in the city, and all of them need to be labeled. This task requires certain skills and takes a lot of time, and tens of thousands of images are needed to train the neural network. You can use open datasets, but those are created abroad, so the images don't match the reality of Russian roads. You can buy labeled images at a starting price of $4 each, but it's about 10 times cheaper to do the labeling in Toloka.

Because you can embed any interface in Toloka and send data via the API, the developers added their own visual editor, which has layers, transparency, selection, zoom, and classification. This increased the speed and quality of the data labeling by a long way.

In addition, the API allows you to automatically split tasks into simpler ones and then piece the results together. For example, before labeling an image, you can select what objects there are in it. This will make it clear which classes to use for labeling the image.

After that, the objects in the image can be classified. For example, you can offer tolokers a selection of images of people, and ask them to specify if they see pedestrians, cyclists, motorcyclists, or someone else.

When a toloker has finished labeling an image, it needs checking. Verification tasks are offered to other tolokers for that purpose.

In addition to "Tolokers", neural networks can also be used to perform labeling. Some networks have already learned to do this task as well as people do. But the quality of their work also needs to be evaluated. That's why tasks have a mix of images labeled by Tolokers and by a neural network.

This way, Toloka is integrated directly into the training of neural networks and becomes part of the general machine learning pipeline.