Toloka documentation

Text classification

For this type of project, you can use the Clickbait or not? preset.

This preset helps you classify any kind of text for training classifiers and NLP models.

Take a look at the example: the labeling interface includes a text block, and radio buttons for categories. Note that validation, keyboard shortcuts, and task layout are already configured in this example.

Components used in the example

  • view.text: The text that you want to classify.

    Show code
    {
  "type": "view.text",
  "label": "Headline",
  "content": {
    "type": "data.input",
    "path": "headline"
  }
}

  • field.button-radio-group: Adds buttons for selecting an answer option.

    Show code
    {
  "type": "field.button-radio-group",
  "validation": {
    "type": "condition.required"
  },
  "label": "Is this headline clickbait?",
  "options": [
    {
      "label": "Clickbait",
      "value": "yes"
    },
    {
      "value": "no",
      "label": "Not clickbait"
    }
  ],
  "data": {
    "type": "data.output",
    "path": "result"
  }
}

  • condition.required: Checks if at least one option is selected.

    Show code
    {
  "type": "condition.required",
  "hint": "Select an option"
}

  • plugin.toloka: Customizes the task layout.

    Show code
    {
  "type": "plugin.toloka",
  "layout": {
    "kind": "scroll",
    "taskWidth": 500
  }
}

  • plugin.hotkeys: Keyboard shortcuts.

    Show code
    {
  "1": {
    "type": "action.set",
    "data": {
      "type": "data.output",
      "path": "result"
    },
    "payload": "yes"
  },
  "2": {
    "type": "action.set",
    "data": {
      "type": "data.output",
      "path": "result"
    },
    "payload": "no"
  },
  "type": "plugin.hotkeys"
}

Note

You can add media files (audio files, videos, images) from your own server, Yandex Disk, or a cloud storage like Yandex Cloud, Google Cloud, or Amazon AWS.

If this preset doesn't meet your needs, see other examples in this section.

Add a description

To add a detailed description to the task, use the view.text component.

Show code
{
  "type": "view.text",
  "content": "Read the headline and tell if it is clickbait."
}

Add a response field

To let Tolokers leave comments about the task or their response, add a text field using field.textarea.

Show code
{
  "type": "field.textarea",
  "label": "Comments",
  "placeholder": "Enter text",
  "data": {
    "type": "data.output",
    "path": "comment"
  }
}

Add a layout

To enhance Toloker's experience, you can highlight different types of data with colors using view.alert. In this example, the description is highlighted with a blue border, and the buttons are highlighted with a yellow one.

Show code
{
  "type": "view.alert",
  "theme": "info",
  "content": {
    "type": "view.text",
    "content": "Look at the text and tell if it is clickbait."
  }
},
{
  "type": "view.text",
  "label": "Headline",
  "content": {
    "type": "data.input",
    "path": "headline"
  }
},
{
  "type": "view.alert",
  "theme": "warning",
  "content": {
    "type": "field.button-radio-group",
    "validation": {
      "type": "condition.required",
      "hint": "Select an option"
    },
    "label": "Is this headline clickbait?",
    "options": [
      {
        "label": "Clickbait",
        "value": "yes"
      },
      {
        "label": "Not clickbait",
        "value": "no"
      }
    ],
    "data": {
      "type": "data.output",
      "path": "result"
    }
  }
}

Other options for buttons

Decide whether a Toloker can select only one or multiple answer options:

If there are several possible answers to the question, use field.checkbox-group checkboxes.

Show code
{
  "type": "field.checkbox-group",
  "label": "This headline is:",
  "options": [
    {
      "label": "Clickbate",
      "value": "clickbate"
    },
    {
      "label": "Too long",
      "value": "long"
    },
    {
      "label": "Gramatically incorrect",
      "value": "bad-grammar"
    },
    {
      "label": "None of this",
      "value": "none"
    }
  ],
  "data": {
    "type": "data.output",
    "path": "input_data"
  },
  "validation": {
    "type": "condition.required",
    "hint": "Select at least one option"
  }
}

The field.button-radio-group component is displayed as solid buttons. It's better to use these buttons if the question has 2–4 short answer options.

If there are more answer options, or they are long, it's better to use field.radio-group, as in the example.

Show code
{
  "type": "field.radio-group",
  "label": "This headline is:",
  "options": [
    {
      "label": "Clickbate",
      "value": "clickbate"
    },
    {
      "label": "Too long",
      "value": "long"
    },
    {
      "label": "Grammatically incorrect",
      "value": "bad-grammar"
    },
    {
      "label": "None of this",
      "value": "none"
    },
    {
      "label": "The text hasn't loaded",
      "value": "404"
    }
  ],
  "data": {
    "type": "data.output",
    "path": "input_data"
  },
  "validation": {
    "type": "condition.required",
    "hint": "Select an option"
  }
}

Add conditions

The helper.if component displays an interface element after a specific response is selected.

Show code
{
  "type": "helper.if",
  "condition": {
    "type": "condition.equals",
    "data": {
      "type": "data.output",
      "path": "first-q"
    },
    "to": "no"
  },
  "then": {
    "type": "field.checkbox-group",
    "label": "2. Why don't you like it?",
    "options": [
      {
        "label": "It's clickbate",
        "value": "clickbate"
      },
      {
        "label": "It's too long",
        "value": "long"
      },
      {
        "label": "It's grammarly incorrect",
        "value": "bad-grammar"
      },
      {
        "label": "Other",
        "value": "other"
      }
    ],
    "data": {
      "type": "data.output",
      "path": "input_data"
    },
    "validation": {
      "type": "condition.required",
      "hint": "Select an option"
    }
  }
}

Does the text match the search query

Add a button that opens the search results and generate a search query link using the helper.search-query component. To make sure that a Toloker clicked on the link and checked its contents, configure validation, as in the example.

Show code
{
  "type": "condition.link-opened",
  "hint": "Follow the link",
  "url": {
    "type": "helper.search-query",
    "query": {
      "type": "data.input",
      "path": "link"
    },
    "engine": "bing"
  }
}

Side-by-side text and web page comparison

You can display the web page in the built-in window using the view.iframe component. Place the text next to it using layout.side-by-side.

Show code
{
  "type": "view.iframe",
  "maxWidth": 400,
  "fullHeight": true,
  "url": {
    "type": "helper.search-query",
    "query": {
      "type": "data.input",
      "path": "link"
    },
    "engine": "bing"
  }
}

Side-by-side text and mobile web page comparison

This is a more complex example that compares the text with the results of a search query. The following components are added:

Show code
{
  "type": "layout.side-by-side",
  "items": [
    {
      "type": "view.text",
      "content": {
        "type": "data.input",
        "path": "text"
      }
    },
    {
      "type": "view.device-frame",
      "maxWidth": 400,
      "content": {
        "type": "view.iframe",
        "maxWidth": 400,
        "fullHeight": true,
        "url": {
          "type": "helper.search-query",
          "query": {
            "type": "data.input",
            "path": "link"
          },
          "engine": "bing"
        }
      }
    }
  ]
}

In this article: