Easy integration
Open source libraries for seamless integration into fully automated ML pipelines.
Infrastructure for your solutions
- Programmatic access
- Reusable projects
- Unified development environment
- Open-source code
Quick start
# Install the packagespip install -U toloka-kit crowd-kitRich API for true scalability
Toloka has a powerful API that allows ML teams to integrate an on-demand workforce directly into their processes, and to build scalable and fully automated data pipelines. Whether you are using Toloka's preset solutions or customized projects, explore integration with our API and unlock scalability.Toloka-Kit
Our Python toolkit covers all API functionality to give you the full power of TolokaJava-Client (alpha)
Our Java client library provides a lightweight interface to the Toloka API that works in any Java environment
Toloka-Kit sample projects
Check out our samples for detailed instructions on how to create and run each project using our Python library and retrieve the resulting dataset. Sample code is written in Jupyter Notebook, and we use Crowd-Kit for data aggregation.
Image classification
Image collection
Object detection
Side-by-side
Video collection
Check out our paper with Toloka-Kit and Crowd-Kit.
Aggregation methods in Crowd-Kit
Crowd-Kit is a powerful Python library that includes aggregation methods for crowdsourced annotation, relevant metrics, and datasets. You can use Crowd-Kit independently with any labeled data to achieve optimum quality.| Categorical Responses | Textual Responses | Image Segmentation | Pairwise Comparisons |
|---|---|---|---|
| Majority Vote | RASA | Segmentation MV | Bradley-Terry |
| Dawid-Skene | HRRASA | Segmentation RASA | Noisy Bradley-Terry |
| Gold Majority Vote | ROVER | Segmentation EM | |
| M-MSR | |||
| Zero-Based Skill | |||
| Wawa | |||
| GLAD |
To experiment more with Crowd-Kit, see the documentation. Go to GitHub.
Take advantage of Toloka technologies