Web analysis in real time

Webtrekk GmbH analyses user behaviour at websites. With the support of Intel®Xeon®X5460 processors, this company can analyse and graphically display large data volumes in real time.

How do users behave at a website? How can websites be optimised, so that visitors stay longer and become customers? These are the sorts of questions to which Webtrekk GmbH provides the answers. Web analysis entails large data volumes, because an individual dataset is created for each visit to the website. As the previous database had reached its limits, Webtrekk opted for a data warehouse solution from EXASOL, based on Quad-Core Intel® Xeon® X5460 processors. Now, for the first time, due to the higher computational performance, the company is able to calculate and visualise subsets of a larger data volume in real time, with its newly developed tool Webtrekk Q3. In addition, administration of the system has become considerably easier.

Challenges

• Analysis of user behaviour at websites entails very large data volumes. Webtrekk's previous database system was no longer powerful enough to analyse and graphically display this data in an appropriate length of time.
• The new solution should be able to use all of the web analysis' raw data for segmentation of the website visitors, without editing it in advance.
• The goal is to analyse the data pertaining to the behaviour of website visitors in real time.

Solution

• For web analysis, Webtrekk GmbH uses tools such as the self-developed Q3. To gather and to analyse the data on which the analysis is based, the company uses the data warehouse system EXASolution from EXASOL.
• This platform runs on servers with energy-efficient 45nm Quad-Core Intel Xeon X5460 processors, which operate at a clock speed of 3.16 GHz.
• Tools such as Intel® C++ Compiler or Intel® VTune Performance Analyzer ensure that the database management system EXASolution achieves the highest possible performance with the Quad-Core processors.

The situation

Webtrekk GmbH is based in Berlin, has 17 employees, and specialises in web analysis, which involves using tools such as Webtrekk Q3 to examine the behaviour of visitors to commercial websites. To this end, Webtrekk uses the so-called 'pixel method', in which the counter impulse is realised by means of loading a hidden pixel. The pixel method is considered the most precise method for gathering user data at a website. 
Here, the focus is on online marketing, improvement of the conversion rate (visitor becomes customer), and behavioural targeting. With the latter, the individual users are identified (e.g. 'new customer from Berlin'), and their click behaviour at the website is recorded and analysed. On this basis, the websites'
operators can present individualised content or advertising to the corresponding user in the future. In this manner, Webtrekk also helps companies to optimise their websites and online shops. Webtrekk GmbH's customers include Esprit, Map 24, Heinrich Bauer Verlag, and DIE ZEIT, for example.

"In website analysis, we currently process around 50 billion datasets per year. For this vast amount of data, we need a powerful database management system," explains Christian Sauer, CEO at Webtrekk GmbH. But it had become increasingly obvious that analysis of the large volume of data was pushing the existing MySQL database to its limits. For segmentation of customers according to certain properties, e.g. visitors from Berlin with DSL access or first-time visitors, the company was unable to use all the available raw data. Instead it was necessary to summarise the data in an additional step in advance, so as to be able to process it. Webtrekk had also become dissatisfied with the performance with regard to response time. "So our goal was a data warehouse solution which can analyse, segment, and graphically display such a complex data volume in real time," says Christian Sauer.

The solution
Since September 2008, Webtrekk bases its web analysis on the database management system EXASolution from EXASOL. This uses so-called 'in-memory cluster' technology, and combines it with efficient compression algorithms in order to process approximately 50 billion datasets per annum. The cluster comprises five nodes, each of which is equipped with two Intel Xeon Quad-Core X5460 processors.
These operate at a clock speed of 3.16 GHz, and are manufactured with 45-nanometer process technology. Due to the tiny structures, more transistors fit on a single chip, and less charge (power)is required in order to trigger the switching process in the transistor (power flows or doesn't flow). Hence, the same computational operations can be realised more quickly, andrequire much less energy (up to 40 percent less).
Four additional tracking servers record the individual websites'datasets and transfer them cyclically to the data warehousesystem EXASolution. Here, a dedicated report server acquiresthe data and visualises it. The requests to the data warehousesystem are controlled by an access server.
"After an extensive benchmark test, we decided on the solution from EXASOL with Intel Xeon processors. The high performance of the system, the priceperformance ratio, and the service from EXASOL totally convinced us," explains Christian Sauer.
In just eight weeks, Webtrekk converted its database management system with the support of EXASOL experts. Alongside complete replacement of the database, the company also had to adapt all queries and applications which operate with the database, so as to accommodate the EXASOL solution. "We were able to simplify many queries, but some of them had to be completely restructured, as there are a number of traditional indices which EXASolution doesn't support," remarks the Webtrekk CEO. "But EXASOL was then very quick to implement any missing functions." The Webtrekk Q3 application, which works with the database, was also adapted by Webtrekk to accommodate the new read / write performance of the EXASOL solution.

Higher performance, more data

The new data warehouse solution offers several advantages for Webtrekk. For instance, the better system performance provides the possibility to process all raw data for the segmentation. It is no longer necessary to pre-calculate and to summarise in advance. Thus, not only is it possible to make qualified statements on user behaviour at websites, but the functionality also increases.
Furthermore, the optimised performance of the database on which the system is based also enables Webtrekk to conduct so-called multivariate tests for optimisation of websites with its new tool Q3. 
This involves redesigning parts of the website, e.g. applying a different colour or navigation method, so as to test how users react to the respective variants. The layout of the site then can be adjusted according to the users' click behaviour.
Special tools provided by Intel help to improve the performance of the sophisticated data warehouse solution: Intel C++ Compiler optimises and parallelises software automatically, so as to achieve maximum performance with the Multi-Core Intel processors. Intel VTune Performance Analyzer detects performance bottlenecks in programs by determining the exact response times as each function is called. This is highly beneficial to Webtrekk.
"With the EXASOL / Intel solution, the higher performance means that the datasets can now also be analysed in real time," as Christian Sauer points out. Data subsets can thus be quickly filtered and graphically displayed, to answer website operators' key questions such as: Where do our visitors come from? How do they navigate through the site? How long do they stay? Who becomes a customer? When is which product purchased? When do we have the highest turnover?
The parameters for analysis are practically unlimited. Once the important questions have been answered, the website can be optimised and personalised with the aid of Webtrekk Q3. Here, the key phrase is 'behavioural targeting'. Once the individual users have been identified, and their click behaviour at the website has been analysed, the website operator can present them with individualised content or advertising platforms on the basis of the segmented data.

High-performance system processes around 50 billion datasets per year

Conclusion
Webtrekk CEO Christian Sauer is highly satisfied with the new database managementsystem based on Quad-Core Intel Xeon X5460 processors. "We have achieved allof our goals. On the basis of this powerful system, we can now use our tool Q3 toprocess the mass of raw data, as well as to calculate and graphically display subsetsof this data in real time. The simplified administration is another bonus." In contrastto the previous solution, the administration of the database cluster is now realisedcentrally by means of a web interface, via which new databases can be added, orbackups made.