In-memory Database

The Importance of In-DB Tools

04 Oct 2018 | Share

In-DB Tools

In-DB tools are of growing importance to organisations.

Have you ever struggled to work with big data in Alteryx? And when I say big data, I mean hundreds of millions or even billion of rows… I have!

This happened almost three years ago, when I was still new to using Alteryx. I got involved in a customer project that caused me some headaches… My only regret? I really wish I could have used the In-Database tools back then. But they were not supported (yet) for most databases and that caused a real hassle…

I remember the frustration on the first day of the project: getting the data into Alteryx was the first challenge, as the data volume was very large. After some attempts, the decision was made to only extract a small subset of the data as .yxdb file. Meaning no more fun with big data…

Long story short, the project ended up being a prototype Alteryx workflow which then was handed over to the customer and deployed by them into production.

If In-DB tools were available back then, I could have enjoyed my very first “big data project” in 2015.

The advantages of using Exasol & Alteryx together

Alteryx In-DB tools started supporting Exasol from version 11.3, so the functionality has been available for over one year now.

As you can imagine the difference can be significant, particularly if you regularly use live connections to a database in your day-to-day work and if big data is involved. Having a high performant database like Exasol means this work is not just possible – it’s actually fun and allows you to analyze your data much faster.

I compared a normal input trying to import 825M rows into Alteryx but my laptop crashed after a short time of trying to load the data. When using the In-DB Input tool, the same process took less a second.

 

in-db-tools

What does this mean for you? You can run end-to-end data analytics with Alteryx and Exasol. You start by using Alteryx to load data into Exasol, then prepare and blend data in Alteryx, go through your analysis and write the results back into the database. At this point you could connect your BI tool to Exasol to for visual analytics and subsequent reporting.

Why do In-DB tools make such a difference?

If you want to focus on extracting insights from your data for analysis, a fast engine for ELT (extract – load – transform) allows you to speed up the data preparation process and leaves more time and resources for the value-added analysis and visualization steps.

Building your Alteryx workflows using exclusively In-DB tools gives you the full benefit of the high performance of Exasol.

In-DB tools avoid unnecessary data transfers between the database and Alteryx: all data remains in the database and extensive load times to get the data into Alteryx can be avoided. The processing power of the database is fully utilized, leaving Alteryx to do the tasks it was designed for: data prep and blending, analytics and data enrichment, all supported by visual workflows.

The SQL queries run in Exasol. The heavy lifting, i.e. the processing of calculations and analytical functions on massive data volumes, are done in the database rather than locally in Alteryx.

Predictive models and Machine Learning

A common use case of Alteryx in combination with Exasol are machine learning and predictive applications. Start by using Alteryx to train your predictive models on a subset of data, then deploy the models into production with Exasol, using R.

This allows you and your organization to use Alteryx as a development tool for data analysts and data scientists, supported by Exasol for productionizing your analytics applications with the fastest performance speeds.

To hear and see more, meet us at the Alteryx Inspire Europe 2018 conference for a live demo or request a demo here.

White Paper: Business Technology Predictions 2019 

Free White Paper

white paper download

Pre-register to save your copy of the upcoming Business Technology Predictions 2019 White Paper.

Considering implementing an in-memory database?

Free White Paper