Xing and Exasol

Intelligent data analysis for the XING business network

The XING business network is an integral part of daily communications management for many business people and includes important communications tools such as an email programme and web browser. More than 10.8 million members worldwide use XING to manage their contacts, stay up-to-date on industry news in their groups, exchange views, expand networks, get information about events, or just log on.


Challenges

Large volume of data made statistical analysis very complex and was beyond the capacity of the previous solution

Solutions

Exasol integrated into existing infrastructure and ETL (Extract Transform Load) processes were converted consecutively into the new solution

Benefits

The company‘s important questions can now be answered quickly with targeted data analyses

Compared to the previous system, the performance has increased significantly; queries that were previously unthinkable are now possible. We are well-equipped for the future because the system is almost limitlessly scalable.

Dr Ulrich Fricke Data warehouse and reporting team leader at XING AG
0 MIL.

members worldwide

0

employees

0 BIL.

data sets per year processed

0

weeks it took to integrate Exasol into XING’s existing infrastructure

Background

XING AG in Hamburg, Germany, operates the XING platform with 363 employees and offers business and career contacts worldwide in 16 languages. Intelligent data analysis allows XING to display relevant information to its members. Besides networking functions, XING also offers a variety of e-recruiting features and services that help members find suitable job openings or employees. In addition, there is the XING company profile: companies can professionally put themselves in the limelight here. With XING Advertising & Best Offers, an attractive business target group can be reached with targeted ads or exclusive offers

  • Industry Social Media
  • Application Data Warehouse Modernization
  • Database Exasol
Case study background details

The journey

You can use our analytics database with our private cloud, any public cloud, on premises – or as part of a hybrid deployment.

The challenge

Meteoric growth of online platform

XING Enterprise Groups makes it possible for businesses and organisations to start their own professional communities on XING. All of these offers require the ability to recognise respective target groups and find out more about their needs. The multitude of customers and products results in a very large volume of data. As a result, the statistical analysis of the data is very complex.

The company has grown quickly, and along with it the amount of data that make more complex business intelligence tasks possible. It has far exceeded the capacity of an earlier-installed solution. The time to process data for one day has partially exceeded the 24 hour mark.

A database system was put into operation by XING in 2006 but could no longer fulfil the requirements, and the processing time of individual analysis processes lasted a disproportionately long time.

“For internal analysis, we now process about 10 billion data sets per year, or about ten terabytes. By 2009, our database solution was already reaching its limits. For this reason we looked around for a high-performance database management system,” explains Dr Ulrich Fricke, data warehouse and reporting team leader at XING AG. “The system was not upgradable or scalable, and we couldn‘t perform clustering,” recalls Fricke. The BI department at XING searched for a new solution that could achieve significantly higher performance with lower administrative expense.

The solution

A high-performance scalable database management system

XING started the search for a suitable, high-performance solution with a selection process. With the small number of candidates, a “proof of concept” was carried out. The product EXASolution from EXASOL came out the winner of this selection process and offered above all high performance, low administrative expense, simple integration into the existing infrastructure, as well as a good price/performance ratio score. A contract was signed between EXASOL and XING at the beginning of 2009.

“We chose the solution from EXASOL after an extensive benchmark test. The high performance of the system, the price/performance relationship and the service had us completely convinced,” said Dr Ulrich Fricke.

Since then, the data volume has already increased five times over. EXASolution has taken this swift increase in information in stride and is well-placed for continued data growth. In April 2011, EXASOL received the best marks for data warehousing appliances in the TPC H Benchmark in all five categories for 100 GB to 10 TB of queried data volume. Query performance is one of the essential performance criteria for analytical databases because increasing demands from users for response time with a concurrently increasing data volume are core challenges for BI environments. The good cost/performance ratio shown by EXASOL in the benchmark also addressed a crucial trigger point of many current systems.

Since May 2009, XING has relied on the EXASolution database management system from EXASOL as the basis for internal analyses. At the same time, the so-named Loading...In-Memory Cluster Technology plays a decisive roll because when combined with efficient compression algorithms, the system is well-positioned to process XING‘s large data volume. The InMemory-Technology allows for the highest data throughput and with it high performance when accessing data. Query processing takes place completely in the RAM, eliminating the need for slow access to hard drives.

The system for XING consists of multiple individual computers (nodes), each having two processors. The individual servers are connected in a cluster. The data is automatically divided within the cluster so that the calculations use all hardware resources optimally. EXASolution then saves all the information in columns and not in lines. The database system can independently determine what the ideal system configuration should be. This system was expanded at the start of 2011 by a factor of 2 in order to meet increasing demands.

The benefits for XING

For XING, there was an additional attribute of EXASOL that was decisive in the selection: EXASOL customers only pay for what they actually need. Unlike other conventional systems, with EXASolution, distributed data storage is already integrated, and an expensive storage area network (SAN) for external storage is not needed. By using standard components, the EXASOL cluster hardware is much more economical than special hardware or “all-in-one” hardware. In addition, EXASOL uses compression algorithms that enable efficient usage of the main memory, and system costs are kept low as a result. EXASolution is also a self-learning system that adjusts to individual user preferences. Indices are automatically laid out, data is compressed, and SQL instructions from the query optimiser are tuned. XING can thus seamlessly process millions of queries per month, which are composed of the daily highly parallelised and complex loading and transformation processes, as well as interactive user queries.

In only four weeks, XING integrated the EXASolution into their existing infrastructure for data evaluation with help from the EXASOL experts. The immediate effects were quickly apparent, particularly the significant improvement in the analysis as well as in the development of processes for data processing. In the second step, the existing ETL (Extract Transform Load) processes were converted consecutively to the new database management system. At the same time, the data model, metadata management, and business logic were expanded and further optimised. XING uses direct contact with EXASOL employees to get feedback on future expansions of the functionality and to get answers to concrete questions on the use and administration of the system.

The migration to a data warehouse solution from EXASOL has helped XING take a big step forward – and for more than two years has gone along with each additional increase without a problem. With improved system performance, it is also possible to pose queries that before had no chance of being answered. Pre-calculation and aggregation are no longer necessary. “Due to the improved performance of the EXASOL solution, previously impossible evaluations of data sets are now feasible,” says Dr Ulrich Fricke. The company‘s important questions can now be answered quickly with targeted data analyses.

Start your Journey

Get in touch today

We’d love to hear from you if you’ve got a burning question for one of our experts – or if you simply want to talk through how to take your project forward with us, or one of our partners.

Contact Us