Dynamically scalable solution processes polystructured data in real time
Nuremberg, 2 October 2012 – EXASOL and xplosion interactive were awarded third place at IDG’s Computerwoche Loading...Big Data Award 2012 for the project “Loading...Hadoop dynamics for retargeting with EXASolution”. The combination of the Loading...in-memory database and handmade retargeting enables advertisers and e-commerce operators to use their online advertising in a more targeted and cost-effective way. With the new xData Platform, even enormous amounts of data for controlling offers on websites can be processed intelligently and made usable. The solution provides online retailers with significant added value. Product offers can be selected on multiple levels so as to provide potential customers with ideal product recommendations. To specifically control the supply of advertisements (frequency cap) the users are only shown a limited number of ads that could be interesting for them. Last but not least, conversion-based calculation forms such as per order or per subscription (cost per order) lead to fair, success-orientated collaboration between those involved.
How does dynamic, personalised retargeting work?
While users browse in online shops, xplosion interactive profiles their buying and searching behaviour and uses this information as the basis for personal product suggestions. Once the users leave the site, they will consequently be shown personalised banner advertisements. Using complex Loading...data science methods, areas of interest and affinities are analysed and deter-mined. This increases the quality and relevance of the advertisements that are generated in real time. By clicking on the banner ad, the potential customer is taken back to the provider’s product page and encouraged to make a purchase.
The new challenge: Shaping polystructured data
“More than 60 of our customers, including brands such as SportScheck, Baur and opodo.de, use xplosion and thus generate more than 2 billion data points per month. This means that the amount of raw data increases monthly by 0.5 TB; in total over 8 TB of data are used for analyses,” says Nils Grabbert, Director of Data Science at xplosion. Processing and using these enormous, heterogeneous amounts of data intelligently was the challenge that xplosion interactive and EXASOL met successfully. With the dynamically scalable xData Platform solution, which acts as a central data storage for real time data mining systems and reports, they are able to process polystructured data from various sources. “Polystructured” data are all corporate data that are already structured, e.g. key figures; are semi-structured, e.g. XML files or are unstructured, e.g. blog entries or comments.
In the first step, the new platform permanently gathers all polystructured data via the messaging system Apache Kafka into the Hadoop Distributed File System (HDFS). Here they are saved in their original form as raw data. The data that is to be used in further analyses is extracted using metadata. Originally unstructured data becomes semi-structured data that is then turned into structured data so as to enable analyses in real time. The dynamic metadata administration thereby simplifies the complex processes enormously. New fields can be added directly to the metadata. Once added, the entries are automatically updated throughout the entire system.
Overall architecture with a source for all data and connected systems
The column-based database EXASolution saves all information from the retargeting application’s web log data as a central data warehouse. Connecting the polystructured data to the Hadoop framework is another key part of the solution. The structure of this raw data memory is the single point of truth in the overall architecture. This manages and controls all connected planning systems such as the data warehouse or the operative data unit. The data that reflects the current behaviour of the website users can thus be combined and analysed with data from the users’ purchasing history.
“The challenge in the era of big data is in making data in various and even unknown structures usable for analysis,” says Steffen Weissbarth, CEO of EXASOL AG. “With xplosion interactive we have managed to develop a dynamically scalable solution that evaluates polystructured data from various sources in real time. xplosion’s customers can therefore continuously improve their retargeting campaigns, significantly increase turnover and generally redesign their e-business to make it more successful,” says Weissbarth.
EXASOL AG, based in Nuremberg, develops and markets the high-performance database EXASolution, based on Loading...in-memory technology, which was designed specifically for data warehouse applications and business intelligence solutions. This allows very large volumes of data to be analysed and evaluated in the shortest space of time. Thanks to the high performance and low administrative maintenance, EXASolution supports businesses not just with valuable decision-making fundamentals from their data, but also by reducing the total cost of ownership. In April 2011, EXASOL AG was named by Gartner as a “Cool Vendor” in the category Data Management and Integration 2011. In 2012 the solution was included in the Magic Quadrant “Data Warehouse Database Management Systems”.
About xplosion interactive GmbH
Hamburg-based xplosion interactive is an international specialist for individual, tailor-made quality solutions for retargeting. With brand-orientated campaigns that are targeted to user interests, xplosion interactive provides a new and intelligent retargeting approach which is billed on a performance basis. Customers include bonprix, Heine and SportScheck. The company was founded in 2008 and is headed by Markus Berg and Daniel Neuhaus.
For more information, please visit http://xplosion.de.
About EXASOL AG
EXASOL AG, based in Nuremberg in Germany, develops a data management system that enables the rapid analysis and evaluation of data. By using the solution from EXASOL, business processes are optimised and secure foundations for decision-making are created, which give companies from all industries a sustainable competitive edge.
EXASolution is a relational, high-performance database that was specifically developed for enterprise warehouse applications and business intelligence solutions. The database is based on in-memory technology and is used for time-critical analyses, comprehensive data research, planning and reporting. By integrating geodata, big data sources and unstructured data, EXASolution opens up additional analysis dimensions that allow for even more efficient analyses, particularly ad hoc ones. The easy-to-manage database can be integrated effortlessly into existing IT infrastructures and requires a minimal amount of administrative maintenance with low investment and operation costs (TCO). As an appliance EXASolution, is available in three versions. EXACloud, based on EXASolution, is offered as datawarehousing as a services. This highly efficient data management solution allows EXASOL customers to benefit from complete data access at any time and a transparent price model. Companies such as XING, Sony Music, Olympus, media control, Zalando, stayfriends, Coop, IMS Health, Semikron, Webtrekk, econda and xplosion rely on EXASOL’s “made in Germany” technology. For further information, please visit www.EXASOL.com, follow us on Twitter or “Like” us on facebook.