Success Case: Toloka Moderation for Dating platforms

Challenges

A dating platform faced a significant challenge in moderating user-generated content for scam, abuse, and threat context.

User-generated context comprised of

1. User profiles with images and text captions

2. Chats messages

Before client came to Toloka the platform relied on

1. Manual human moderation, which was time-consuming and unable to cover spikes, not always effective and citing client “putting unnecessary burden of watching cruel content daily”

2. Simple keywords lists and filters that did not cover more complex cases and contexts

When the platform reached a whole new level of MAU their community started being more and more toxic and the metrics were falling down.

The platform needed a scalable, ready-to-go and accurate solution to moderate images and texts in real-time, while also improving the accuracy of the moderation model over time.

Solution

To overcome these challenges, the dating platform implemented Toloka moderation service — a machine learning-based solution for image and text moderation. The solution used Toloka a combination of natural language processing (NLP) and computer vision (CV) algorithms to detect and tag inappropriate images and texts.

Toloka models captured from the scratch scam, abuse and threat as pre-defined classes. We were continuously improving model accuracy with human insight of Toloka crowd labelling dataset of user profiles and captions. This helped to identify patterns and contexts that the model may not have detected, and to continuously improve the model's accuracy.

Moreover, the client was able to add phrases specific to their particular community on the go using platform interface.

For personal chats Toloka moderation model analysed not just each message but a bunch of messages to keep the whole context of message.

Business Impact Metrics

The implementation of the machine learning-based moderation solution had a significant impact on the dating platform's business metrics.

1. The service level agreement (SLA) for detecting scammers was reduced by 85%, resulting in a safer and more trusted platform for users.

2. The platform's premium subscription churn rate decreased by 12% after 6 months of successful partnership with Toloka, and the lifetime value of overall users increased by 8%.

3. The client cut costs on moderation by 53% per one item.

Overall, the implementation of machine learning for image and text moderation with human insight was a successful solution to the challenges faced by the dating platform. The platform was able to provide a safer and more secure user experience, while also increasing user engagement and revenue.