There are four basic features that contribute to Exasol’s speed, which I call the “four degrees of awesome”
Exactly how do you make the fastest database in the world?
A few years ago I was looking at new technologies for a consultancy company. During the course of this I came across Exasol’s Loading...TPC-H benchmark figures. I couldn’t get them out of my head – here was a company I had never heard of that made a product that was many times faster than all the big-name databases I knew about. How was it possible that they could be faster than companies like Oracle and IBM who spend zillions of dollars on R&D?
I ended up cold-calling Exasol and asking them to talk me through it. The basic ideas were patiently explained to me and I was so impressed that, to cut a long story short, I ended up working for them.
I found out that there are four basic features that contribute to Exasol’s speed, which I call the “four degrees of awesome” – each of the features is present to some extent in other products on the market, but it’s the combination of all the four together that make Exasol so special.
1. Clustered
It seems like only yesterday that there were products (Vectorwise, for example) which could run on one server only. Their marketing talked about the evils of database clusters – hard to manage, prone to error etc. Nobody (not even Vectorwise’s new owners) believe that these days.
With the clustered approach, instead of buying a bigger server when you want to increase the power of your database, you just add more servers to the cluster. Just like when you want to dig a ditch quicker, you employ more people to dig rather than doubling the size of your shovel.
2. Columnar
Columnar storage is particularly well adapted to analytical queries – both because it avoids the need to scan whole rows when you likely only need certain columns, but also because it is possible to compress the data more efficiently.
3. Loading...In-memory
This is the most important contributor to the speed of Exasol. Memory is fast – disk is slow – let’s do our processing in-memory if we can.
It’s not just a question of bolting a ton of memory onto a database – the database needs to make intelligent use of the memory – this is well illustrated by our CEO’s set of articles for Wired magazine which I recommend you to read if you want to fully appreciate this “magic”.
4. Engineering
The power of German engineering is something of cliché. But the best clichés persist because they are fundamentally true. Exasol reminds me very much of the large German car companies. Their architecture is clean and their design principles are clear. The best engineers are hired and best practices are followed. There is a well-communicated management vision and a commitment to continuous improvement. Also, no opportunity is ever lost to reduce unnecessary weight and drag or to increase the power of the engine.
Put these four elements together, allow the product to mature through five major releases over more than a decade – that’s how you make the fastest database in the world.