Question 1

What are adaptive machine learning models?

Accepted Answer

Adaptive machine learning models are implemented in production with a process that automatically samples and re-labels data that is sent to the model for inference. This data is used for regularly evaluating model performance online in real time. When metrics start to decay on the new data, tuning is triggered automatically. This feature is continually running in the background to adapt Toloka models to your data.

Question 2

How can I interact with adaptive ML models?

Accepted Answer

Toloka offers adaptive ML as a remote API with methods available in the model interface. Models are available with two modes of interaction: online (real-time) inference and batch inference.

Online inference is when the model provides a synchronous response for each data item sent for inference. This mode is appropriate for use cases when the response is needed within seconds, like customer support automation or content moderation.
Batch inference is when the model posts results after processing an entire batch of data. This mode works better for use cases that involve gathering a batch of data over time and evaluating it, like social media monitoring and analysis.

Our off-the-shelf models support only online inference mode in the closed beta, but custom models can be set up with batch inference support.

Question 3

What infrastructure does Toloka use to deploy and serve adaptive ML models?

Accepted Answer

Toloka has the capability to deploy and monitor models in any cloud environment of the customer&#x27;s choice, or on the customer&#x27;s premises in some cases. By default, Toloka uses Microsoft Azure infrastructure to provide resilient, compliant, and geographically distributed service hosting.

Question 4

What data do Toloka models train on?

Accepted Answer

Toloka models are trained on very large datasets with multiple sources of data. Some of these include open-source data and public data curated with the help of the Toloka crowd. Toloka also uses proprietary non-public data sources that are gathered and labeled via crowdsourcing with the expertise of our R&D team. We use state-of-the-art technologies to generate high-quality non-standard datasets for search relevance evaluation, e-commerce product categorization, evaluation of semantic similarity, and more.

Question 5

What data can be used for fine tuning and optimization of adaptive machine learning models?

Accepted Answer

There are 3 ways to use Toloka's collection of pre-trained models:

Use the models as-is for standard classification, recognition, or generation tasks.
Fine-tune a model on a pre-labeled dataset that you provide.
Deploy a model with the Adaptivity parameter, which triggers the process to collect and label the fine-tuning dataset on a regular basis.

If you are interested in fine-tuning an algorithm right before deployment, you can label your historical data first on the Toloka data labeling platform.

Question 6

How often do models need to be retrained?

Accepted Answer

Toloka adaptive models support schedule-based or event-based triggers for re-training and fine-tuning. The best approach depends on the scenario. This could include re-training once a day, when accuracy drops below a threshold for new data, or when the "adaptivity" process collects a certain amount of data. We recommend using the default settings designed for each model by our machine learning engineers to best fit the use case and domain.

Question 7

How do adaptive models compare to traditional ML and online machine learning?

Accepted Answer

As approaches to machine learning evolve, new terms are sometimes used to describe dynamic learning processes, such as adaptive learning, continuous learning, and continuous training. While these concepts do not have standard industry definitions, they attempt to differentiate the learning process from both traditional machine learning with static data collection and online learning with streams of real-time data. We use the term adaptive ML because adaptivity ensures that model predictions remain relevant even in fast-changing environments.Our adaptive models eliminate data and model drift in machine learning models by continuously supplying them with new data points. Once set up, predictions of your model are sampled for human annotation on Toloka, so your model becomes re-trained over time on our platform. Our AutoML capabilities allow you to always select the best model based on objective human evaluation.

Adaptive AutoMLAdaptive AutoMLAdaptive AutoML
Tune better models faster

Tune ML models without coding

Cut time to production

Update models with new data

Online and adaptive machine learning: how it works

Need a custom solution tailored to your needs?
Talk to us

Powerful models for relevant solutions
to real-world problems

Explore other models available to you on our model podium at Toloka ML Platform

Discover how adaptive models
can make an impact

Want to see results like this? We can help you find the right solution

Next-level automation: MLOps with the Toloka ML platform

FAQ about
adaptive
auto ML

Adaptive AutoMLAdaptive AutoMLAdaptive AutoMLTune better models faster