Building a lead classification system for 10x client leads
Subscribe to Toloka News
Subscribe to Toloka News
When the average person needs legal advice, they may not know where to turn. With the help of a legal services platform, they can instantly connect to a professional who specializes in their particular problem, from child support cases to criminal law. The platform helps consumers find the legal help they need, and law firms get leads on new clients.
Our client is a Lawtech company with a number of AI-based products, including a lawyer bot that offers free legal advice regarding consumer rights protection, based on data from 100,000 previous consultations. Their legal services aggregation platform connects people to hundreds of professional lawyers registered on the site.
Challenge: Faster lead processing to grow revenue
When consumers submit legal questions on the platform, these requests need to be classified and filtered before passing them on to law firms. They usually entail emotional stories from people who aren't sure how to solve a problem.
The platform relied on in-house experts to score the leads on value, based on whether the situation is a legal request or not, and how much a law firm will pay for the lead. For instance, in a consumer rights case, someone who buys a faulty TV will not compensate a lawyer in the same way as someone who buys a new car that turns out to be a lemon.
As the number of leads grew, they needed to automate and speed up lead processing. The goal was to develop an ML-based product that could handle 3 steps in a split second:
- Identify a request as a legal issue or not
- Score the value of a lead by estimating the average cost of a lawyer's services
- Forward the request to a matching law firm
Solution: AI classification product trained on human feedback
Relying on Toloka's ML expertise, the team set out to find an automated solution. Our engineers developed a model to classify leads as relevant or irrelevant, just like a human expert.
For instance, relevant leads request legal services and have detailed information about the situation, like in this example:
To train the model, Toloka engineers used a dataset of real leads that were previously processed by in-house experts. Since legal questions are subject to different interpretations, they asked experts to check the data. They also incorporated feedback from law firms on the quality of the leads they got from the platform.
The new ML model also identifies lead segments to help law firms adjust their pricing policy. The model differentiates between requests for legal advice (consulting) versus requests for legal services, which have more value to law firms. Valuable leads are automatically forwarded to law firms that specialize in the related legal category.
Impact: Relevant leads increased tenfold
After introducing the new app, the platform sold 10 times more relevant leads to law firms per month than they were able to with manual processing.
Investigations showed that the automated tool performed better than in-house experts for several reasons:
- In-house experts were overly strict in rejecting requests and misclassified a large number of leads.
- Scaling up the volume of requests would require hiring a much larger team of experts. Building the AI tool was cheaper, with the added benefit of unlimited instant scaling.
- Response time is vital for the legal industry, since clients often want to solve their problem or make a decision the same day. Human review takes at least several minutes, whereas AI makes a verdict in less than a second.
This new product has given us a competitive advantage. Our competitors can't process this many leads so fast. So we find a niche with growth potential and make it more and more competitive. It also makes our service more sustainable. We see significant spikes in the workload when laws are changed or introduced, but with this new product we handle these changes easily.
— Client's team
What's next: Plans for expansion
Bolstered by the success of the solution, the company has plans to expand to new geographic regions. The next step is to fine-tune the model using data specific to language variations and legal differences in new regions. Part of the company's plan for entering new markets is to create a model that can translate requests from one language to another and transfer them to the initial model for classification.
Recent articles
Have a data labeling project?
More about Toloka
- Our mission is to empower businesses with high quality data to develop AI products that are safe, responsible and trustworthy.
- Toloka is a European company. Our global headquarters is located in Amsterdam. In addition to the Netherlands, Toloka has offices in the US, Israel, Switzerland, and Serbia. We provide data for Generative AI development.
- We are the trusted data partner for all stages of AI development–from training to evaluation. Toloka has over a decade of experience supporting clients with its unique methodology and optimal combination of machine learning technology and human expertise. Toloka offers high quality expert data for training models at scale.
- The Toloka team has supported clients with high-quality data and exceptional service for over 10 years.
- Toloka ensures the quality and accuracy of collected data through rigorous quality assurance measures–including multiple checks and verifications–to provide our clients with data that is reliable and accurate. Our unique quality control methodology includes built-in post-verification, dynamic overlaps, cross-validation, and golden sets.
- Toloka has developed a state-of-the-art technology platform for data labeling and has over 10 years of managing human efforts, ensuring operational excellence at scale. Now, Toloka collaborates with data workers from 100+ countries speaking 40+ languages across 20+ knowledge domains and 120+ subdomains.
- Toloka provides high-quality data for each stage of large language model (LLM) and generative AI (GenAI) development as a managed service. We offer data for fine-tuning, RLHF, and evaluation. Toloka handles a diverse range of projects and tasks of any data type—text, image, audio, and video—showcasing our versatility and ability to cater to various client needs.
- Toloka addresses ML training data production needs for companies of various sizes and industries– from big tech giants to startups. Our experts cover over 20 knowledge domains and 120 subdomains, enabling us to serve every industry, including complex fields such as medicine and law. Many successful projects have demonstrated Toloka's expertise in delivering high-quality data to clients. Learn more about the use cases we feature on our customer case studies page.