Decision Models (e.g. Jev) and Exasol

Running classification over thousands of records requires balancing accuracy, latency, and cost. Using an LLM to ask a yes/no question (e.g. is this email spam?) over 10,000 records is incredibly inefficient. The results are not always trustworthy. Not to mention, the cost of 10,000 calls is not cheap. Cranking up the reasoning adds to costs and makes the entire pipeline even slower. Training a purpose-built classifier is an option if you already have similar data upfront. Not to mention the extra work that comes along with data collection, and data cleaning.

TypeSafe AI introduced a class of frontier models that they call System One Models; these are now also known as Decision Models. ‘Jev’ (their first model) is two orders of magnitude faster and more efficient than any existing LLM on similar levels of intelligence.

After the launch of Jev, we have been seeing a lot of competition from other players since. OpenAI introduced their own version of Jev called Decisions API. Including a bunch of others such as Decider 1, Solar Decide,and d1 from Liquid AI.

There are open-source options available as well such as Kev, Laya, and CLM. These can be self-hosted for complete sovereignty.

We are still in early stages of these purpose-built decision models. Let’s explore how these can add value to analytics and data workflows.

What are Decision Models?

Decision Models are AI models built to hand software a decision rather than a paragraph. You pass it some state, meaning the text or data to be judged, plus a question whose answer space you declare in code, and it returns a value from those declared answers together with a probability distribution across the possible answers.

Focusing the model on just a few choices makes it respond in a fraction of a second.

TypeSafe AI introduced this class as “System One” models.

The name “System One” comes from Daniel Kahneman’s “Thinking, Fast and Slow splits human thought in two” book title. System 1 is fast and intuitive. You see a face and know it is angry before you can say why. System 2 is slow and deliberate. You need to process multiple concepts to divide 400 by 80.

The Three Primitives

Every question you ask with a Decision model is one of these three shapes.

Choice gives each option from a set of options a probability. The option with the highest probability is then declared the winner. The cardinality of this set of options can be up to 255 options.

Score places the content on an ordered scale you define. It needs at least 2 levels and takes at most 10. The returned score is the probability-weighted mean of the level numbers, so it can land between levels: (0 x 0.0) + (1 x 0.70) + (2 x 0.30) gives 1.30.

Noul asks a yes/no question and returns a single number from 0 to 1; the probability that the answer is yes. It returns no separate confidence value, because the probability already is one.

Cost and Speed

TypeSafe AI prices Jev 1.13 at $0.042 per Million Tokens for input tokens and output tokens are so cheap that they don’t even register on the meter, therefore output tokens are free. Their API rate limit is 100,000 tokens per second and 70-500ms end-to-end. Different Decision models have different rate limits and price, but they all are faster and cheaper than any LLM at making calibrated decisions.

This combination of speed and low cost means these Decision Models are perfect for working with Big Data.

An AI model with an excellent performance to price ratio should be paired with a database that also has the best performance to price ratio.

Jev + Exasol for Business

TypeSafe AI’s claim of “Jev can’t hallucinate” makes it an incredible tool for making business decisions. Exasol is used by businesses all over the world to make good business decisions. Users use it for sales analysis, financial reporting, creating business intelligent dashboards and much more.

We targeted two of the common use-cases businesses usually face:

1. Invoice Classification

Invoice classification helps businesses create a better view of their businesses which lets them make good business decisions.

Managing thousands/millions of invoices with SQL is not simple. It means assigning every invoice to categories such as vendor, department, expense type, tax category or accounting/GL code.

Some challenges of invoice classification are:

• Data is often in different languages
• OCR errors
• Ambiguous description
• Missing or inconsistent information

These challenges cannot be solved by just SQL. You usually need people with domain knowledge to define the rules to combat these types of challenges. This requires a whole cross-functional team.

A frontier LLM can replace most of this work and automate this process but, an LLM is designed to create text. It is not well suited to make structured calls. You can put heavy guard rails on it but that just adds complexity to the process because now, you must create and manage an entire harness with the pipeline. This solution is also very costly and inefficient.

Using a Decision Model

A decision model makes sense because invoice classification is about choosing a category from messy business information and not generating text. For a model such as Jev, the appeal is that returns results from a fixed set of choices with accompanying probabilities.

Keeping this in mind we took Jev for a test drive in Exasol by making calls right inside Exasol via Exasol UDFs.

Setting up the Environment

We first are required to set up an account at console.typesafe.ai

Starting from 20^th September, Jev is now available for everyone for free.

After setting up an account, we need to install the typesafe SDK in Exasol. We need to set up a custom SLC (script language container) for this.

The process of setting up a custom SLC with the TypeSafe SDK, loading the dataset, and running the entire pipeline is described here.

Running the pipeline

After setting up everything, we started invoking the UDF with the questions in the parameter. The UDFs can be invoked by a single SQL query, and the entire pipeline runs through that.

An example query is:

EXECUTE SCRIPT TYPESAFE_LAB.REFRESH_AI_INVOICES(
    'Is `invoice.service_line` a service line whose own deliverable is
     artificial intelligence or machine learning work?');

Results

The entire pipeline ran for ~50 seconds. Over 10,000 records, that’s about ~200 calls per second. Each result returned with a confidence score and didn’t hallucinate once. These results at this speed on free text are not possible with a traditional LLM. The confidence score is what makes us decide which bucket each invoice should fall into.

Jev in Exasol helps us quickly classify loose text into sales leads or whatever business advantage you gain from classifying data.

2. Entity Resolution

Entity resolution means working out which records describe the same real-world company. Businesses collect company records from many places, such as the CRM, the ERP vendor list and web forms. The same company often appears several times under slightly different names, and it has to be matched to an official source like a company register. Without this, you get duplicate vendors, split customer history and reports that don’t add up.

Some challenges of entity resolution are:

• Misspellings and typos (Baraset Propertties Limited)
• Different legal suffixes and abbreviations (Ball Mgmt Ltd. vs BALL MANAGEMENT LTD)
• Company names written in another language (Blau Sanitär und Heizung GmbH for BLUE PLUMBING AND HEATING LTD)
• Addresses, VAT numbers and websites written in different formats
• Look-alikes that are not the same company (B.L.C.T. (PHC 15C) LIMITED vs B.L.C.T. (PHC 15A) LIMITED)

SQL handles exact keys well. Once a VAT number or website is normalized, a join finds the match. Names are the problem. String-similarity rules break on translations and on several changes at once, and they happily match look-alikes. Teams usually end up maintaining long lists of hand-written rules and still send a large share of records to people for manual review.

A frontier LLM can read two records and tell whether they are the same company. But the output is text, which then has to be parsed, checked and turned into a decision you can join on. Doing that for every candidate pair, at scale, needs the same kind of harness we described for invoices, and it is just as costly and slow.

Vector similarity is the other common approach: turn every record into an embedding and match the records that sit closest together. Embeddings are good at finding records that look alike, and that is the problem. B.L.C.T. (PHC 15C) LIMITED and B.L.C.T. (PHC 15A) LIMITED differ by one character, so their embeddings are almost identical, yet they are two different companies. The same goes for unrelated companies registered at the same accountant’s address, which share most of their text. A similarity score measures how much two records resemble each other. It doesn’t answer whether they are the same company. You still have to pick a cut-off, and no single cut-off works across names, addresses, VAT numbers and websites. When a pair is matched, the score also can’t tell you which field caused it.

Using a Decision Model

Entity resolution is a yes/no decision on a pair of records, so it fits a decision model well. We asked Jev one main question per pair, “Do these records describe the same registered company?”, and four supporting questions about the name, address, website and VAT number. Every answer comes back as a probability, so Jev doesn’t just say “match”, it also shows which fields made it decide that way. All of this can be used directly in SQL.

Jev judges pairs; it doesn’t search. Comparing every incoming record with every company would mean 1.7 billion pairs. So we used plain SQL for what SQL does well: it narrows each record down to its 5 most likely candidates in about 13 seconds. Jev then makes the decision on those pairs.

Setting up the Environment

The setup is the same as for invoice classification: a TypeSafe account, the custom SLC with the TypeSafe SDK, and the API key stored in an Exasol connection. If you have already set up the invoice use case, there is nothing new to install. You only need to load the dataset. As the master data we used the UK Companies House register (849,123 companies), plus 2,000 messy incoming records generated from it.

To run the pipeline and see the results yourself, follow instructions here.

Running the pipeline

The whole pipeline runs with two SQL statements:

EXECUTE SCRIPT TYPESAFE_LAB.FIND_ENTITY_CANDIDATES();  -- SQL only, ~13 s
EXECUTE SCRIPT TYPESAFE_LAB.RESOLVE_ENTITIES();        -- 10,636 Jev calls, ~46 s

The first statement finds the candidate pairs. The second sends each pair to Jev through a UDF, picks the best match for every record, and groups the duplicates into clusters.

Results

Jev scored 10,636 record pairs in ~46 seconds, roughly 230 calls per second. It matched ~97.7% of the records correctly, compared with 88.2% for a SQL string-matching baseline on the same pairs. The gap was largest where names change:

• Misspellings: 98.9% vs 76.0%
• Several changes at once: 99.7% vs 73.7%
• Translated names: 80.0% vs 2.2%

For finding duplicates among the incoming records, Jev found 99.7% of them, while the baseline found 81.6%.

Where an exact key exists, such as a VAT number or website, SQL matched just as well as Jev, and that is the right tool there. The baseline was also better at rejecting companies that are not in the register (94% vs 88%), because Jev is more willing to call two records the same company. Because every decision comes with a probability, fixing this needs no new calls: raising the threshold from 0.5 to 0.7 brings Jev level with the baseline there and still gets 97.5% right overall. Pairs in the uncertain range can also be sent to a person to review.

Embeddings are still useful one step earlier: they are a good way to find candidates, especially translated names that string matching misses. The decision about each candidate pair is where a decision model comes in.

Conclusion

Using AI models for automations in high-risk environments has taken a giant leap. Decision Models are truly a game changer. The amazing speed they have enables them to make decisions in real time along with cost that no LLM can rival. Their tendency of not hallucinating free text makes them great for making critical decisions.

You can try the above-mentioned examples in Exasol Labs:

github.com/exasol-labs/jev_exasol

Exasol Personal is free to use and can be downloaded here.

Additionally, feel free to add more examples or additional use cases in this repository. We appreciate contributions from our community.