Knowledge base

Hands-on tutorials

Launch projects successfully by following our step-by-step instructions. Use our datasets for practice projects, or run projects on your own datasets and adapt the settings to your needs. You can also run the projects programmatically using the Toloka-Kit.

Image & Video Data

Text Data

Audio Data

Field Data Collection

Image classification

The challenge: classify a set of photographs according to what animal is shown in the picture – a cat or a dog

There’s a set of real-life photos of cats and dogs. Performers are asked to look at the photos and decide what animal is in the picture.

Learn more

Object detection

The challenge: get a set of contours defined by an array of points that represent road signs in each photo

There’s a set of real-life photos of roads. Performers are asked to outline every traffic sign in each photo.

Learn more

Side-by-side

The challenge: find out which icon people prefer and determine the top icon out of the set

There’s a set of 3 icons in 3 pairs. Performers are asked to look at the icons and choose the one they prefer. The results are then aggregated to obtain the top icon.

Learn more

Image collection

The challenge: collect a set of categorized, real-life photos of pets — cats and dogs

Performers are asked to send a picture of their cat or dog (or a random photo if they don’t have a cat or a dog) and select the appropriate label for their picture.

Learn more

Video collection

The challenge: collect a set of video recordings of people imitating popular emoji gestures.

There’s a set of emoji combinations, and performers are asked to record videos showing these emojis with their hands. Each video must meet certain quality criteria.

Learn more

Text classification

The challenge: classify news article headlines according to whether they are clickbait or not

There’s a set of news article headlines. Performers are asked to read a headline and decide whether it’s clickbait.

Learn more

Sentiment analysis

The challenge: classify reviews by putting them into two categories – “Positive” or “Negative”

There’s a set of customer reviews, and performers are asked to read a review and decide whether it’s positive or negative.

Learn more

Survey

The challenge: collect information about how people manage stress, and if they are prepared to get a meditation app to do so.

There is a survey where performers answer questions about their stress levels and stress management techniques, meditation practices, and habits relating to paid apps.

Learn more

Audio classification

The challenge: classify audio recordings according to the speaker’s gender

There’s a set of audio recordings from different people. Performers are asked to listen to the recordings and decide whether the speaker is a man or a woman.

Learn more

Audio collection

The challenge: collect audio recordings of texts

There’s a set of texts, and performers are asked to read the texts aloud and record themselves. Recordings like these are used for training voice assistants.

Learn more

Field task

The challenge: collect a set of real-life photos of metro station entrances

Performers are asked to visit a specific metro entrance and take a few pictures of it. These pictures can be used to decide which entrances need cleaning.

Learn more

Keys to clean and accurate training data

Our methodology based on years of research and unique industry expertise can help you successfully tap into the wisdom of the crowd on a large scale. If you want to efficiently use the knowledge of thousands of people to get clean and accurate data for your ML needs, follow our tips for each of these essential steps.

1. Decomposition

Break your task down into steps until each separate level is clear enough for any performer to handle.

2. Instructions

The more comprehensive the instructions, the more accurate the results.

3. Interfaces

A good interface makes it easy for users to perform the same repeated actions quickly and correctly.

4. Quality control

Carefully plan and configure a quality control system to ensure high-quality results.

5. Pricing

Find the optimal price based on speed and quality.

6. Results

After the pool is finished, aggregate the results and check statistics.

Toloka 101

Join one of our regular webinars to learn more about Toloka, watch a live demo based on a real use case, and ask questions.

Useful resources

API

Integrate on-demand global crowdforce & build fully automated ML pipelines.

Python library

We have an open-sourced library with a client that covers all API functionalities.

Public datasets

Use our datasets for your projects or collect your own data that meets your needs.