How one of the biggest FI-s in Latam uses LLM-s for improving efficiency ratio-s

The Need

Banking in most Latin American countries is a hot commodity, but a large fraction of the population is still unbanked or underbanked. This means that their first experience with banking will be digitally native. 

It also means faster onboarding, smoother user experience, instantaneous money movement , smaller fees and finally a journey that is a lot more personal and data driven than it was ever possible before. 

The FI who was looking for a solution to improve its data quality is one of the oldest in the region, serving over 20 percent of the banked population in Peru.  

They are data rich and came to us with a need to improve the quality of their transaction data, in order to drive internal efficiencies for the bank, from customer call center optimization to the time it takes to underwrite a retail loan application which is their bread and butter. 

Since Ntropy mostly operates in the US and English speaking countries, there was a concern that we were lacking data in the region and hence would not be able to beat existing in-house systems without adding human operations and costs for labeling.

Evaluation

The evaluation took a few weeks, using a few different datasets that were  picked by the bank and a set of different models on Ntropy side. 

We benchmarked 5000 transactions using both our in-house built, smaller models, and the Ntropy’s LLM orchestration stack on top of GPT-4-0314. 

The LLM performed super well out of the box without any country or language specific tuning. 96.53 % of all merchants and websites were covered correctly, 98.84 % of all intermediaries were detected. 

Our in-house specialized  model returned 89% of all merchants and websites correctly, and 77% of intermediaries. The Ntropy LLM stack was chosen as the way forward. 

ROI

As a result of this exercise, the bank is going to save 100s of human hours a month on data quality and cleansing tasks, and can focus on  building new automation workflows on top of previously obscure and underutilized data.  

As a bank, one of the key metrics they care about is the efficiency ratio, which is the percentage of net expenses compared to revenue. As a result of better quality transaction enrichment and the automations built on top of it, this FI will be boosting their efficiency ratio by over 30% in the next few quarters. In layman terms, this means scaling revenue without adding expenses. 

The cost reduction with Ntropy solution vs naked LLM is 110x in this case, with a further 87x reduction in latency  making the implementation highly viable economically even if throughputs increase by orders of magnitude which is the bank’s expectation.

 This also makes further workflow automation with recursive querying a lot more viable. 

Being able to achieve these results,  gives the FI additional leverage, pricing power and opportunity to gain further market share in the region.