Slow databases are SO 1990s
I recently heard a remarkable sentence from a friend of mine.
He was trying to analyse some data for his company, using a big database with a short famous name. His queries were constantly hanging and I asked him what he was intended to do about that.
“Tonight I’ll get the DBAs to build me some cubes on last month’s data”
I have four problems with that short sentence:
- Last Month’s data
- Tonight – this isn’t the 1990’s anymore. Heavy duty computation shouldn’t be something that has to wait until everyone’s gone home. And you know what happens the next day? You get the results back, you think of another query that you want to run and sometimes that has to wait yet another night. It’s like playing chess by post.
- DBAs – I love DBAs – I used to be one in fact. But analysis shouldn’t have to wait on a technician doing something technical. Wouldn’t it be good if the system were so easy to use and so self-tuning that you didn’t need a team of technical wizards to sustain it?
- Cubes – nobody wants cubes – they are just a means of getting some kind of a result out of bad databases. Wouldn’t it be better if you just submitted queries and didn’t need to worry about how to make them run fast, because the system just, you know, worked?
- Last month’s data – oh boy, I’ll bet the competition are so not scared of the kind of lame insights you’re likely to get using last month’s data. It’s like trying to drive a car by looking out the back window.
And that’s actually the root of the issue here. My friend’s company has competitors who are running ad-hoc queries against raw data that is bang up-to-date on database systems that pretty much take care of themselves. They’re also not spending a small fortune on technicians or cluttering up their databases with copies of data in different formats to speed up specific queries.
Those competitors are asking harder questions and getting faster answers. Their staff are spending more of their time better serving their customers and less time just trying to make a database work. This isn’t a question of convenience or technology for the sake of it – these other companies will financially out-perform my friend’s company if they are better at finding customers and better at serving their customers.
Meanwhile my friend’s company is stuck in the 1990s where everything had to wait, everything was technically difficult and you were always working with past data.
It’s nice to listen to 1990’s music once in a while (“Ace of Base” anybody?) and the clothes are hilarious but you don’t have to live there. Time has moved on and so should you – take Exasol for a spin for a sneak peek at how data analysis is done nowadays.