Building a lead classification system for 10x client leads

When the average person needs legal advice, they may not know where to turn. With the help of a legal services platform, they can instantly connect to a professional who specializes in their particular problem, from child support cases to criminal law. The platform helps consumers find the legal help they need, and law firms get leads on new clients.

Our client is a Lawtech company with a number of AI-based products, including a lawyer bot that offers free legal advice regarding consumer rights protection, based on data from 100,000 previous consultations. Their legal services aggregation platform connects people to hundreds of professional lawyers registered on the site.

Challenge: Faster lead processing to grow revenue

When consumers submit legal questions on the platform, these requests need to be classified and filtered before passing them on to law firms. They usually entail emotional stories from people who aren't sure how to solve a problem.

The platform relied on in-house experts to score the leads on value, based on whether the situation is a legal request or not, and how much a law firm will pay for the lead. For instance, in a consumer rights case, someone who buys a faulty TV will not compensate a lawyer in the same way as someone who buys a new car that turns out to be a lemon.

As the number of leads grew, they needed to automate and speed up lead processing. The goal was to develop an ML-based product that could handle 3 steps in a split second:

1. Identify a request as a legal issue or not

2. Score the value of a lead by estimating the average cost of a lawyer's services

3. Forward the request to a matching law firm

Solution: AI classification product trained on human feedback

Relying on Toloka's ML expertise, the team set out to find an automated solution. Our engineers developed a model to classify leads as relevant or irrelevant, just like a human expert.

For instance, relevant leads request legal services and have detailed information about the situation, like in this example:

To train the model, Toloka engineers used a dataset of real leads that were previously processed by in-house experts. Since legal questions are subject to different interpretations, they asked experts to check the data. They also incorporated feedback from law firms on the quality of the leads they got from the platform.

The new ML model also identifies lead segments to help law firms adjust their pricing policy. The model differentiates between requests for legal advice (consulting) versus requests for legal services, which have more value to law firms. Valuable leads are automatically forwarded to law firms that specialize in the related legal category.

Impact: Relevant leads increased tenfold

After introducing the new app, the platform sold 10 times more relevant leads to law firms per month than they were able to with manual processing.

Investigations showed that the automated tool performed better than in-house experts for several reasons:

• In-house experts were overly strict in rejecting requests and misclassified a large number of leads.

• Scaling up the volume of requests would require hiring a much larger team of experts. Building the AI tool was cheaper, with the added benefit of unlimited instant scaling.

• Response time is vital for the legal industry, since clients often want to solve their problem or make a decision the same day. Human review takes at least several minutes, whereas AI makes a verdict in less than a second.

This new product has given us a competitive advantage. Our competitors can't process this many leads so fast. So we find a niche with growth potential and make it more and more competitive. It also makes our service more sustainable. We see significant spikes in the workload when laws are changed or introduced, but with this new product we handle these changes easily.

— Client's team

What's next: Plans for expansion

Bolstered by the success of the solution, the company has plans to expand to new geographic regions. The next step is to fine-tune the model using data specific to language variations and legal differences in new regions. Part of the company's plan for entering new markets is to create a model that can translate requests from one language to another and transfer them to the initial model for classification.