Set up a moderation process for items on sale 
to ensure legal and cultural compliance

Accelerate your
e-commerce AI
Talk to our AI expert
Accelerate your
e-commerce AI
Talk to our AI expert


International e-commerce platform based in China
  • Russian website launched in 2019 became the country's most popular online retailer
  • Over 200 million products for sale


Required a scalable and effective product moderation system to comply with local cultural and legal specifics of CIS countries.
  • 200 million items on the marketplace that require moderation
  • Products with legal restrictions pose a special risk to the buyer and the marketplace
  • Moderation involves difficult-to-define variables, such as cultural differences
  • Legal aspects change frequently and moderation must adapt


The method relies on a crowd of Tolokers to moderate items that the algorithms are not sure about.
  1. Prioritize moderation of items that are displayed to users.
  2. The ML model evaluates moderation items to determine if they are acceptable or should be banned from the website, and provides a confidence score for each class prediction.
  3. Items with scores above the threshold are added to the database and passed on to the product service. Items with scores below the threshold are sent to Toloka for human evaluation.
  4. All moderated items are evaluated independently by three annotators and approved when the majority of Tolokers agree on the classification.
  5. After being sent to Toloka, the items are added to the learning set for the ML model to train on and improve the moderation threshold for future moderation.

Business results

  • The daily number of items moderated jumped from 200 to 100,000
  • The cost per moderated item dropped from $0.017 to $0.01
  • It now takes less than 15 seconds to verify one item

“Using instructional and scalable crowdsourcing allows us to estimate tricky things like insults and disgusting content. Getting this knowledge helps us train our ML model so that it also starts to understand. Even more importantly, we use these same annotations to improve our model, optimize it, and make the overall process cheaper. We now spend half as much on each item that needs to be moderated.”

– Elena Gruntova, Product Director, AliExpress

The case was presented at the Data-Driven AI meetup by Elena Gruntova, Product Director, AliExpress. The full video is available here.

Download presentation

Similar success stories

Accelerate your e-commerce AI

Let's talk about the ideal solution for your data needs.