The Client

Our client, a leading IT service provider for a conglomerate of regional banks in a major European country, specializes in delivering innovative solutions that enhance banking productivity. In this particular engagement, their goal was to automate key information extraction from mortgage application documents to accelerate processing time.

The Challenge

Mortgage applications require accurate valuation of real estate properties, which often rely on real estate listings. However, the client faced a major challenge: their existing models could not extract information from these listings.

The reason? Their approach was heavily reliant on structured image processing, and real estate listings varied widely in format, lacking a standardized template. As a result, their previous computer vision-based model failed to comprehend and extract critical details such as price, location, and property characteristics. Attempts to refine this model fell short, as it lacked the necessary semantic understanding to process unstructured text effectively.

The Solution

To address this issue, we introduced the client to vLLM technology-leveraging large language models to extract key data points with zero-shot capabilities. Our approach involved:

1. Proof of Concept Development - We built a working prototype to demonstrate feasibility.
2. Model Evaluation & Selection - By testing multiple LLMs, we identified the best-performing solution for this specific use case.
3. Prompt Engineering - We crafted a robust prompt template that ensured consistent, high-quality extractions.
4. Seamless Integration - Our solution was wrapped in a module that integrated directly with the client’s existing backend.

This transition from a vision-based model to an LLM-driven approach enabled the system to understand and extract relevant information from a variety of listing formats, regardless of structure.

The Results

The implementation of our LLM-based solution delivered significant benefits to the client:

• Successful Market Launch - The client was able to market their improved mortgage processing solution to their own banking clients, providing them with a clear competitive advantage.
• Eliminating Dependency on Training Data - Unlike their previous approach, our solution did not require labeled training data, making it resilient to data source changes-a critical advantage when certain data streams later disappeared.
• Scalability Across Document Types - The success of this project led to the adoption of LLM-based document processing for additional banking-related use cases.

An unexpected but highly valuable outcome was the ability to perform information extraction with zero-shot learning, significantly reducing the need for data annotation and ongoing model retraining.

Client Feedback

The client was extremely satisfied with the results, and the collaboration was renewed for further AI-driven innovations. Their feedback emphasized the efficiency gains and the flexibility of the LLM approach compared to their previous, more rigid methodologies.