Muddling Mojo: A Script Language Container Recipe for Exasol

Every analytics engineer eventually hits the wall where SQL stops being the right tool. A gnarly parsing routine, a scoring model, a bit of geospatial math – at some point you stop writing CASE WHEN and start writing a UDF. In Exasol, that has many times meant Python.

Motivation

Python is a fine answer. Flexible, vast ecosystem, loved by millions – and I’m one of them; I write it most days, and I’d pick it again tomorrow. But two things nag. It’s slow. And it lets you get away with too much: assign an integer to a variable, then rebind that variable to a string a few lines down, and Python nods along. The failure surfaces somewhere else entirely, at runtime, usually in production. Small mistake, big blast radius. That’s the trade Python makes – writing speed for safety  – and inside a UDF that fires a hundred million times, you pay for it twice. Especially in an analytical database, where it’s not uncommon to touch hundreds of millions of rows with a query. And then you discover that the ultra-fast database becomes something like an ordinary engine.

But here’s the catch: extensibility is the magic word. In an earlier article^[1], I argued that extensibility is a critical criterion for a database and showed what User-Defined Functions buy you, with a Python example. This time we go a step further: instead of extending the database with a function, we extend it with an entire language, one that doesn’t ship in the box. My colleague Marco Nätlitz already walked this path when he built a Script Language Container (SLC) for Rust^[2]. For me, the relatively new language Mojo is an ideal candidate – at least worth a try.

So why do it again, with another language? Three reasons.

First, I wanted to build one myself. Reading someone else’s write-up tells you it works; doing it on your own tells you what it costs.

Second, Mojo has a genuine affinity for Python – familiar syntax and real interoperability with the Python ecosystem, not just a promise of it. For a database whose UDF story is already largely Python, that’s the shortest bridge available.

Third, Mojo’s creators built it for AI workloads: it compiles from a single source to CPUs, GPUs, and accelerators. Whether that matters inside a database is exactly the kind of question a container answers, not an opinion. However, you can’t use the Apple Silicon GPU for a UDF if you run Exasol on Apple Silicon, because the required Metal Framework isn’t available in the container. Other environments I have not considered. You can find more information about the language at mojolang.org.

But what exactly is this „Script Language Container“? Exasol ships UDF support for Python, R, Lua, and Java. Lua is built into the database natively; the others are containerized. But none of those are privileged. The Script Language Container is the actual contract: a container image, a defined protocol for talking to the database, an alias registered in SCRIPT_LANGUAGES, and the database will happily run whatever’s inside. Python is a choice Exasol made, not a limit it imposed. If a language can be packaged into an SLC and speak the protocol, Exasol can run it.

The need, the need for speed

Before we get into how to build a Script Language Container for Mojo and how to create the User Defined Functions (UDF), let me show you where this lands.

The function is deliberately trivial: take values from one column, double and average them. And yes – I know this is a job for plain SQL. That’s the point. A function this small has nowhere to hide, so what we measure is the cost of crossing into the language, not the cost of the algorithm.

First, a table to work on:

CREATE OR REPLACE TABLE BENCHMARK (
	num1 INT,
    	num2 INT
);

INSERT INTO BENCHMARK (num1) VALUES BETWEEN 1 AND <n rows> WITH STEP 1;

With 1M, 10M, 50M, 100M, 500M, 1B rows. The query cache was disabled so that no run could be served from a previous result. The data itself was warm in memory –  I’m measuring execution, not cold I/O.

ALTER SESSION SET QUERY_CACHE = ‘OFF‘

The test queries were the following:

SELECT AVG(DOUBLE_MOJO(NUM1) )FROM BENCHMARK;
SELECT AVG(DOUBLE_PYTHON(NUM1)) FROM BENCHMARK;

All measurements were taken on an Apple MacBook Pro (M4 Max, 128GB) running Exasol Nano – a single laptop, not a cluster. Please don’t read anything about Exasol’s real-world performance into these numbers; the point here is the relative cost of one language versus another on identical hardware. Each measurement is the median of 10 uncached runs. Don’t be misled by the constant distance between the two curves; the y-axis scale is logarithmic.

Mojo is 2.7–8.5× faster than the equivalent Python UDF, with the gap widening as the dataset grows. The more records you need to process, the wider the gap until you reach a certain record count. Response times for the pure SQL statement are consistently around 900ms for the 1B record set. A UDF doesn’t beat native SQL – nothing beats native SQL at doubling a number – but it changes the calculus for cases where you genuinely need a UDF. Work that was too slow to do in-database in Python starts looking possible with a pre-compiled UDF, implemented in Mojo or Rust.

At 1M rows, the advantage is only 2.7× because of container load time. Starting a script-language container costs the same fixed amount whether you process a thousand rows or a billion, and at 1M rows that constant makes up most of the runtime. Interestingly, Mojo’s container is the more expensive of the two to start – roughly 0.17 s against Python’s 0.07 s – and it still wins, because once running it costs 67 ms per million rows against Python’s 582 ms.

That second pair of numbers is the actual result. As the dataset grows, startup is amortized away, and the per-row ratio is all that’s left: 8.71× at 100M rows, 8.66× at 1B. The gap widens with scale and then stops widening, right at the ratio you’d predict.

None of which beats native SQL. The pure SQL statement returns in about 900 ms at 1B rows – some 75× faster than Mojo, and 650× faster than Python. I can’t state it often enough: nothing beats native SQL at doubling a number. But it changes the calculus for the cases where you genuinely need a UDF: work that was too slow to run in-database in Python starts looking possible with a pre-compiled container, whether that’s Mojo or Marco’s Rust.

A single test run indicates that going beyond the above number of records does not increase the Speed-Up factor any more.

Rows	Pure SQL	Python UDF	Mojo UDF	Mojo vs Python	Mojo MS / M Rows	Python MS / M Rows
1M	< 1	0.650	0.239	2.72x	239	650
10M	< 1	5.021	0.762	6.59x	76	502
100M	< 1	56.176	6.453	8.71x	65	562
1B	< 1	581.996	67.239	8.66x	67	582

See the table below for the benchmark results:

All times are in seconds. The pure-SQL column is a bound, not a median. The SQL baseline stayed under one second even at 1B rows; individual medians were not recorded, so it is shown as < 1 seconds rather than plotted as a series.

Big and Bigger

Now that we have a good sense of the speed-ups on non-production hardware, I ran a final test on a six-node demo cluster – machines that are no longer current by today’s standards, but real cluster hardware rather than a laptop. I built tables of 10B and 20B rows and ran the same function, and measured speed-up factors between 15 and 18.

Two caveats before anyone gets excited. These were single runs, not medians of ten. And the cluster changed two variables at once – both the hardware and the data volume – so I can’t cleanly attribute the higher factor to either. My working assumption is that it’s mostly the hardware, since the laptop numbers showed the ratio flattening out rather than climbing. Settling that would take one more run: the same 1B table on the same cluster, as a control.

What the whole set does support is a practical range. At the table sizes most people actually work with –  tens to hundreds of millions of rows – a factor of 6× to 9× is a reasonable expectation, and the cluster results suggest that’s a floor rather than a ceiling on serious hardware. Still very good numbers.

The Foundation

With these results, let’s look at what it takes to add Mojo as a new language to Exasol’s UDF framework. Under the hood, the database and the language container talk via ZeroMQ (ZMQ), a messaging library that sits somewhere between a raw network socket and a full message broker. Instead of a stream of bytes, you send and receive whole messages, and each kind of connection comes with a conversation pattern built in: strict question-and-answer, one-to-many broadcast, or a work pipeline. There’s no server to install and no broker to run; the two sides simply talk to each other. Exasol uses it in its simplest form: a strict request-and-reply conversation over a local socket inside the container, with every message encoded as Protocol Buffers. ZeroMQ is small, fast, and available for almost every language, which makes it a natural backbone for a plugin system.

That’s precisely how Exasol uses it. When a query reaches a UDF, the database starts the client process inside your container and tells it where to connect. From there, the two sides hold a strictly alternating conversation: the container asks, the database answers. Introduce yourself. Here’s the script you’re running. Here are the columns coming in and going out. Give me the next batch of rows. Here are the results. Done.

And that’s the whole opportunity. Exasol has already committed to its half of the conversation, and it doesn’t care who answers – so nothing in the database needs to change for it to learn a new language. All that’s missing is something on the other end that can reply. The rest of this article is about building that something in Mojo: what goes into the container, how it’s assembled, and where it went sideways on the first few attempts.

Creating the Script language container

Inspired by the Rust Sript Language Container I asked the AI coding tool of my choice to do something similar for Mojo – with one constraint: it had to be Mojo-native. No bridge to Rust, no plugin layer, nothing else. That turned out to be a reasonably straightforward brief. Two or three prompts in, the container built and a UDF came back with the right answer.

Which says less about the tool than about the problem. None of the hard parts were actually open questions: ZeroMQ is a C library that Mojo can call directly through its C interop, the protocol is a published protobuf schema rather than something to reverse-engineer, and the Rust container already demonstrated the whole contract in a comparable compiled language. Three constraints, all documented, all pointing the same way.

The result lives in the Exasol Labs GitHub organization:

https://github.com/exasol-labs/language-container-mojo

That’s the starting point for Mojo-based UDFs. But before going further, one thing needs to be said, because it changes how you work.

A Python UDF lives in the database. You write the code inside a CREATE OR REPLACE SCRIPT statement in whatever SQL client you like, and when you want to change it, you edit it the same way and the next query picks it up. A pre-compiled language works nothing like that. The UDF logic lives in the repository above; you write it in a code editor, build the container, and deploy it to the database. Rust and Mojo even differ in the last step: the Rust container loads a shared object at runtime and keeps the container, while the Mojo logic compiles straight into the UDF client of the container, so any change means rebuilding and redeploying the whole container.

The good thing is that the Mojo script language container supports both variants, fully pre-compiled into the container or loading a shared object. For the latter, expect a performance drop in exchange for a bit more flexibility. However, in both variants, you have to upload files to the database – it is just a matter of what you are uploading: a shared object or an entire container. For the rest of the article I stay with the pre-compiled version, where all Mojo UDFs are compiled into the udfclient.

Remember that the database hands the container the UDF source code as plain text and expects it to interpret it? A compiled container simply ignores it. The logic is already inside. Which is the whole bargain in one sentence: you trade flexibility for performance – an edit that used to take seconds now takes a rebuild, and in exchange the function runs up to eight times or more faster.

You only need to edit udf.mojo in the src directory. Every UDF you want to expose lives here, alongside two small functions that wire them up.

Here’s the benchmark function – doubling a number:

fn run_double_mojo(values: List[Int64], nulls: List[Bool]) -> (List[Int64], List[Bool]):

    var out = List[Int64]()
    var out_nulls = List[Bool]()
    for i in range(len(values)):
        if nulls[i]:
            out.append(0); out_nulls.append(True)
        else:
            out.append(values[i] * 2); out_nulls.append(False)
    return (out^, out_nulls^)

Two things in that signature are worth pausing on. First, it takes a batch – a list of values and a parallel list of null flags – not a single row. That is the wire protocol showing through: the database ships column data as packed arrays with a null mask, and that is what arrives here. So the loop lives inside your function, where a Python UDF would be called once per row. Second, out^ is not a typo. The caret transfers ownership of the list to the caller instead of copying it – one of the small Mojo-isms that make the code look like Python right up until it doesn’t.

Two more functions tie things together. The first dispatches to whichever UDF was called (shown here with a second example UDF that this article doesn’t use):

fn run_udf(name: String, values: List[Int64], nulls: List[Bool]) -> (List[Int64], List[Bool]):
    if name == "SUM_POSITIVE":
        return run_sum_positive(values, nulls)
    return run_double_mojo(values, nulls)

And the second declares which names exist, so an unknown UDF is rejected before it ever reaches the dispatcher:

fn is_known(name: String) -> Bool:
    return name == "DOUBLE_MOJO" or name == „SUM_POSITIVE"

That’s all the code. Next, build the container with Docker – one command that writes the finished container into the out directory:

docker build -no-cache  -target artifact --output type=local,dest=./out .

The build took about 55 seconds. Subsequent builds without the „-no-cache“ option took considerably less time.

The container then has to reach the database, which means uploading it into BucketFS, Exasol’s built-in file store. The documentation describes the normal route. On an Exasol Nano running in Docker, there’s a shortcut: copy the archive into the slc directory of the mounted volume bound to /exa and unpack it there.

One more step, and it’s the one everyone forgets: the database still has no idea the container exists. SCRIPT_LANGUAGES maps an alias to a container URL, and until you set it, CREATE SCRIPT ... MOJO will simply fail.

ALTER SESSION SET SCRIPT_LANGUAGES =
   'PYTHON3=builtin_python3 MOJO=localzmq+protobuf:///__builtin__/slc/mojo?lang=mojo#exaudf/mojoudfclient';

With the alias in place, you can create the UDF:

CREATE OR REPLACE MOJO SCALAR SCRIPT DOUBLE_MOJO(val INT)
RETURNS INT AS

    -- No code here; the body will be ignored

The empty body is not an oversight. Whatever you write there is ignored, because the Mojo implementation is already compiled into the UDF client – the script exists only to give the function a name and a signature.

If you’d rather not rebuild the client every time the logic changes, there is a second route: load a shared object at runtime and point the script at it. You have to compile your UDF by yourself with the mojo command and instruct the compile to create the shared object. Something which was automated in the container build before.

CREATE OR REPLACE MOJO SCALAR SCRIPT "MOJO"."DOUBLE_MOJO_SO" ("val" DECIMAL(36,0)) RETURNS DECIMAL(36,0) AS
%udf_object /exaudf/double_ext.so

Registering the container itself works differently on Exasol Personal and Exasol Enterprise; the repository linked at the end of the article has the instructions for both.

The first check is as simple as:

SELECT DOUBLE_MOJO(21);

Done! We now have a new language option for UDFs in Exasol.

Wait, there is one more thing

So far this has all been about speed. But back at the start, I mentioned Mojo’s affinity with Python, and that is more than a syntax resemblance; Mojo can call into the Python ecosystem through an embedded CPython interpreter. Here is a UDF that does exactly that:

from python import Python

fn run_py_scale(values: List[Int64], nulls: List[Bool]) raises -> (List[Int64], List[Bool]):

    var transform = Python.import_module("pyudf.transform")   
    var out = List[Int64]()
    var out_nulls = List[Bool]()
    for i in range(len(values)):
        if nulls[i]:
            out.append(0); out_nulls.append(True)
        else:
            var r = transform.scale(values[i])                   
            out.append(Int(r)); out_nulls.append(False)

    return (out^, out_nulls^)

That conversion is also the catch. Every call to transform.scale() crosses from Mojo into CPython and back, and it happens once per row – which is precisely the cost the benchmark measured. A UDF written this way will perform much closer to the Python numbers than the Mojo ones. The interop is real, but it is not free.

This gives a usable rule for where to draw the line. Mojo should own the inner loop; call Python once per batch, or not at all inside it. If the functionality is there, start a UDF from scratch and write it entirely in Mojo. When porting an existing Python UDF, move the hot loop into Mojo and leave the rest – but check that “the rest” isn’t running per row, or you have moved nothing. And where a UDF’s real cost is an external API call, use whichever language is more convenient, because the network latency dwarfs both.

If a needed Python package isn’t in the container, add it to requirements.txt, rebuild and redeploy the container.

Finally

Extensibility has always been the quiet strength of Exasol’s UDF framework. Four languages ship out of the box – Python, Java, R, and Lua – with only Lua running natively inside the database while the other three are served by containers. Mojo now makes number five or six, with Rust. So why bother adding a language at all?

The strongest reason has nothing to do with speed. It’s about where the work happens. When a job is too slow to run in-database, it gets moved out – into a service, a pipeline, a batch job scheduled somewhere else – and suddenly you’re shipping a billion rows across a network and back again. Having the right language available inside the database collapses that whole detour into a single query. That isn’t an eight time or better improvement. That’s an architecture with one less moving part.

The second reason is fit. Languages are good at different things, and the right one tends to make the solution smaller.

The third is that people already have code – and code you don’t have to rewrite is code that already works. This is where Mojo earns its place: the syntax is close enough to Python to feel familiar, and it interoperates with Python directly. You keep your existing logic and rewrite only the parts where performance actually matters.

Performance is real, too, but keep it in proportion. A compiled UDF doesn’t beat native SQL, and it isn’t meant to. What it does is move the boundary of what’s worth doing in a UDF at all – and it charges you for the privilege: a rebuild-and-redeploy cycle where Python only ever asked you to re-run a CREATE SCRIPT.

The best part is that none of this required a new Exasol. The Script Language Container framework has been sitting there the whole time, waiting for someone to write the other end of the conversation. Mojo is one answer; Rust is another. And you don’t have to wait for Exasol to build it – you can do it yourself. That, more than anything else here, is what extensibility actually means.

If you want a starting point, the language container repository is here:

https://github.com/exasol-labs/language-container-mojo

Happy Exasoling – and afterward, reward yourself with a Mojito