Introducing Ntropy AI: the LLM orchestration stack for finance

Today we are excited to announce the launch of Ntropy AI, our LLM orchestration platform that allows anyone in financial services to run high-throughput, high-complexity tasks 1000x faster and cheaper. 

We built this solution for ourselves, to be able to serve our customers in financial services with the best models for cleaning, enriching and categorizing financial data. 

Without this infrastructure, using GPT4-size models to process 100Ms transactions per day at under 500ms latency would be impossible.

At Ntropy, we are excited to make this possible for every developer across many more tasks. 

From SLM-s and transaction categorization to LLM infrastructure

In January 2021 we set out to solve financial intelligence. We started by building the most accurate models for understanding financial data across different sources, formats and languages.

The last decade of innovation has been around building efficient rails for moving money from point A to point B, instantaneously and very cheaply. 

However, the data derived from money movement is low-quality and fragmented, due to legacy rails and heterogenous formats and standards. This makes financial processes operationally expensive and slow , from onboarding to underwriting, from financial management to accounting and reconciliation. 

With language models, we saw a real opportunity to change that by enabling machines to parse and enrich financial data at scale, in milliseconds with human accuracy, yet at a fraction of the cost. All our models were built in-house by us, end to end.

In 2023 everything changed. GPT4, trained on the internet and not specialized for financial data, beat our specialized small-language-models across many hard benchmarks, without any fine-tuning, zero shot. 

Yet, directly serving GPT4 or any model of that size is not viable. The cost and latency are about 3-4 orders of magnitude too high, even with the latest drops. 

Moreover you have to build a whole new orchestration platfrom, retrofitting it to your existing software and machine learning pipelines and the many models, evaluation stacks, training and retraining loops that are now mostly obsolete. The stack for applying language models to FI specific problems and environments needs to be reimagined. We are doing that.

Ntropy 2.0 

Over the last twelve months we have been focused on solving inference cost and latency for our own task, without sacrificing accuracy and reliability. While working on that, we realized that we are onto something much bigger.

We have created the most reliable, fastest and cost-efficient way to run LLM-s on FI tasks.

This infrastructure is already in production with some of our largest customers.

Since the first release, we have had increasing demand from new enterprises across finance and other verticals to leverage it for running LLMs at scale.

Today we are launching our very own LLM orchestration framework for FI-s, Ntropy AI, to meet this demand.

Technical details and benchmarks will follow in our next post that is coming out tomorrow. 

Why Ntropy

We are in the early innings of LLM adoption. Accuracy, latency, reliability, cost, and data security remain barriers to entry. To solve for these fundamentals, the options available today are sub-optimal.

We reduce p99 latencies by 500x lower and average cost per LLM query by 1000x with better accuracy. If you are building in the space and have high volumes, do reach out.

Thank you to our customers, our team and our investors for sticking by us and building with us. Now we can deliver best in class financial intelligence to any application at a cost and speed that no one else can match.

This is a significant milestone on Ntropy’s original mission.

I have never been this excited. Cannot wait to see what you get to build_