Big Data in Real-Time
Uğur CANDAN
SAP Turkey - Chief Operating Officer
@ugurcandan
ugurcandan.net
Youtube
in-memory database

© 2011 SAP AG. All rights reserved.

2
© 2011 SAP AG. All rights reserved.

3
© 2011 SAP AG. All rights reserved.

4
© 2011 SAP AG. All rights reserved.

5
© 2011 SAP AG. All rights reserved.

6
© 2011 SAP AG. All rights reserved.

7
© 2011 SAP AG. All rights reserved.

8
© 2011 SAP AG. All rights reserved.

9
© 2011 SAP AG. All rights reserved.

10
© 2011 SAP AG. All rights reserved.

11
© 2011 SAP AG. All rights reserved.

12
© 2011 SAP AG. All rights reserved.

13
© 2011 SAP AG. All rights reserved.

14
© 2011 SAP AG. All rights reserved.

15
© 2011 SAP AG. All rights reserved.

16
© 2011 SAP AG. All rights reserved.

17
© 2011 SAP AG. All rights reserved.

18
© 2011 SAP AG. All rights reserved.

19
© 2011 SAP AG. All rights reserved.

20
© 2011 SAP AG. All rights reserved.

21
Technology today requires tradeoff
A breakthrough in today’s information processing architecture is needed

DEEP
Complex & interactive questions
on granular data

OR

HIGH SPEED
Fast response-time,
interactivity

DEEP
Complex & interactive questions
on granular data

HIGH SPEED
BROAD

Fast response-time,
interactivity

Big data,
many data types

SIMPLE
No data preparation,
no pre-aggregates,
no tuning

© 2011 SAP AG. All rights reserved.

REAL -TIME
Recent data, preferably realtime

SIMPLE
No data preparation,
no pre-aggregates,
no tuning

22
SAP HANA Platform – More than just a database

Any Apps

SAP Business Suite

Any App Server

Supports
any Device

and BW ABAP App Server

SQL

MDX

R

JSON

Open Connectivity

SAP HANA Platform
SQL, SQLScript, JavaScript
Spatial

Search

Text Mining

Stored Procedure
& Data Models

Application & UI
Services

Business Function
Library

Predictive
Analysis Library

Database
Services

Planning Engine

Rules Engine

Integration Services

Transaction

Unstructured

Machine

HADOOP

Real-time

Locations

Other Apps

SAP HANA Platform Converges Database, Data Processing and Application Platform
Capabilities & Provides Libraries for Predictive, Planning, Text, Spatial, and Business
Analytics to enable business to operate in real-time.
© 2011 SAP AG. All rights reserved.

23
Dünyanın en büyük in-memory veritabanı sistemi – Santa Clara, CA
250 HANA sunucusu | 250TB Ana Bellek | 10,000 x86 Core

© 2011 SAP AG. All rights reserved.

24
Breakthrough solutions from startups & ISVs
A single platform powering next generation of applications

nexvisionix

DRIVING ADOPTION

RECENT PROJECTS



Platform to imagine new generation of applications



Industry solutions - Healthcare, Capital Markets



Simple consumption model – lowering barriers to entry



Consumer and enterprise applications



Rapid commercialization of innovation



www.startups.saphana.com (700+ Startups & ISVs)

© 2011 SAP AG. All rights reserved.

25
Predictive Analytics & Machine Learning
Transforming the Future with Insight Today

Hadoop/ Sybase IQ,
Sybase ASE, Teradata

SAP HANA

KNN
classification

Regression

Main Memory
C4.5
decision tree

K-means
Virtual Tables

SQL Script
Optimized Query Plan

Spatial, Machine,

Text Analysis

Real-time data

PAL
R-scripts

ABC
classification
Weighted score
tables

Associate
analysis:
market basket

R-Engine

Spatial Data

Unstructured
HANA Studio/AFM,
Apps & Tools

Accelerate predictive analysis and
scoring with in-database algorithms
delivered out-of-the-box.
Adapt the models frequently
© 2011 SAP AG. All rights reserved.

Execute R commands as part of
overall query plan by transferring
intermediate DB tables directly to R
as vector-oriented data structures

Predictive analytics across multiple
data types and sources.
(e.g.: Unstructured Text, Geospatial,
Hadoop)
26
Innovation Previously Infeasible
Predict and analyzes game player behavior in real-time

Real-time insights, analysis,
and consumer engagement
for increased revenue and
decreased churn

© 2011 SAP AG. All rights reserved.

27
Simplicity Previously Unachievable
eBay Early Signal Detection System powered by Predictive Analytics

Automated signal
detection system to
proactively respond to
real-time market
dynamics

© 2011 SAP AG. All rights reserved.

28
Product: Agile Datamart
Yodobashi - POS Data Analizi
Business Challenges

250 million POS

 Lack of real-time insights into POS data make it difficult to create effective,
tailored sales promotions and marketing campaigns

sales order line items

 Need shorter response time for customer segmentation to plan sales
campaigns

10-12 minute

Technical Challenges

sales campaign
planning (not
possible before)

 Inability to process big data (billions) POS records quickly because of high
latency and static reporting
 Shop floor staff not able to access relevant information on-the-fly, with iPad
Benefits

100,000x faster
sales analysis – from
3 days to 2-3
seconds

© 2011 SAP AG. All rights reserved.

 Real-time insights into POS data improve customer satisfaction and
merchandising
 Dynamic personalized offerings while customer is at store or on web site

29
12,000 Staff with
3,200 pure scientist,
650,000 patients/year,
1,4 B€ revenue

500,000 data
points from each
cancer patient.
Instant patient data
analysis during
treatment
Mitsui Knowledge Industry
Healthcare industry – Cancer cell genomic analysis
408,000x faster
than traditional diskbased systems in
technical PoC

216x faster DNA
analysis results from 2,5 days to 20
minutes

© 2011 SAP AG. All rights reserved.

31
Thank you

WHY SAP Real Time Data Platform - RTDP

  • 1.
    Big Data inReal-Time Uğur CANDAN SAP Turkey - Chief Operating Officer @ugurcandan ugurcandan.net
  • 2.
    Youtube in-memory database © 2011SAP AG. All rights reserved. 2
  • 3.
    © 2011 SAPAG. All rights reserved. 3
  • 4.
    © 2011 SAPAG. All rights reserved. 4
  • 5.
    © 2011 SAPAG. All rights reserved. 5
  • 6.
    © 2011 SAPAG. All rights reserved. 6
  • 7.
    © 2011 SAPAG. All rights reserved. 7
  • 8.
    © 2011 SAPAG. All rights reserved. 8
  • 9.
    © 2011 SAPAG. All rights reserved. 9
  • 10.
    © 2011 SAPAG. All rights reserved. 10
  • 11.
    © 2011 SAPAG. All rights reserved. 11
  • 12.
    © 2011 SAPAG. All rights reserved. 12
  • 13.
    © 2011 SAPAG. All rights reserved. 13
  • 14.
    © 2011 SAPAG. All rights reserved. 14
  • 15.
    © 2011 SAPAG. All rights reserved. 15
  • 16.
    © 2011 SAPAG. All rights reserved. 16
  • 17.
    © 2011 SAPAG. All rights reserved. 17
  • 18.
    © 2011 SAPAG. All rights reserved. 18
  • 19.
    © 2011 SAPAG. All rights reserved. 19
  • 20.
    © 2011 SAPAG. All rights reserved. 20
  • 21.
    © 2011 SAPAG. All rights reserved. 21
  • 22.
    Technology today requirestradeoff A breakthrough in today’s information processing architecture is needed DEEP Complex & interactive questions on granular data OR HIGH SPEED Fast response-time, interactivity DEEP Complex & interactive questions on granular data HIGH SPEED BROAD Fast response-time, interactivity Big data, many data types SIMPLE No data preparation, no pre-aggregates, no tuning © 2011 SAP AG. All rights reserved. REAL -TIME Recent data, preferably realtime SIMPLE No data preparation, no pre-aggregates, no tuning 22
  • 23.
    SAP HANA Platform– More than just a database Any Apps SAP Business Suite Any App Server Supports any Device and BW ABAP App Server SQL MDX R JSON Open Connectivity SAP HANA Platform SQL, SQLScript, JavaScript Spatial Search Text Mining Stored Procedure & Data Models Application & UI Services Business Function Library Predictive Analysis Library Database Services Planning Engine Rules Engine Integration Services Transaction Unstructured Machine HADOOP Real-time Locations Other Apps SAP HANA Platform Converges Database, Data Processing and Application Platform Capabilities & Provides Libraries for Predictive, Planning, Text, Spatial, and Business Analytics to enable business to operate in real-time. © 2011 SAP AG. All rights reserved. 23
  • 24.
    Dünyanın en büyükin-memory veritabanı sistemi – Santa Clara, CA 250 HANA sunucusu | 250TB Ana Bellek | 10,000 x86 Core © 2011 SAP AG. All rights reserved. 24
  • 25.
    Breakthrough solutions fromstartups & ISVs A single platform powering next generation of applications nexvisionix DRIVING ADOPTION RECENT PROJECTS  Platform to imagine new generation of applications  Industry solutions - Healthcare, Capital Markets  Simple consumption model – lowering barriers to entry  Consumer and enterprise applications  Rapid commercialization of innovation  www.startups.saphana.com (700+ Startups & ISVs) © 2011 SAP AG. All rights reserved. 25
  • 26.
    Predictive Analytics &Machine Learning Transforming the Future with Insight Today Hadoop/ Sybase IQ, Sybase ASE, Teradata SAP HANA KNN classification Regression Main Memory C4.5 decision tree K-means Virtual Tables SQL Script Optimized Query Plan Spatial, Machine, Text Analysis Real-time data PAL R-scripts ABC classification Weighted score tables Associate analysis: market basket R-Engine Spatial Data Unstructured HANA Studio/AFM, Apps & Tools Accelerate predictive analysis and scoring with in-database algorithms delivered out-of-the-box. Adapt the models frequently © 2011 SAP AG. All rights reserved. Execute R commands as part of overall query plan by transferring intermediate DB tables directly to R as vector-oriented data structures Predictive analytics across multiple data types and sources. (e.g.: Unstructured Text, Geospatial, Hadoop) 26
  • 27.
    Innovation Previously Infeasible Predictand analyzes game player behavior in real-time Real-time insights, analysis, and consumer engagement for increased revenue and decreased churn © 2011 SAP AG. All rights reserved. 27
  • 28.
    Simplicity Previously Unachievable eBayEarly Signal Detection System powered by Predictive Analytics Automated signal detection system to proactively respond to real-time market dynamics © 2011 SAP AG. All rights reserved. 28
  • 29.
    Product: Agile Datamart Yodobashi- POS Data Analizi Business Challenges 250 million POS  Lack of real-time insights into POS data make it difficult to create effective, tailored sales promotions and marketing campaigns sales order line items  Need shorter response time for customer segmentation to plan sales campaigns 10-12 minute Technical Challenges sales campaign planning (not possible before)  Inability to process big data (billions) POS records quickly because of high latency and static reporting  Shop floor staff not able to access relevant information on-the-fly, with iPad Benefits 100,000x faster sales analysis – from 3 days to 2-3 seconds © 2011 SAP AG. All rights reserved.  Real-time insights into POS data improve customer satisfaction and merchandising  Dynamic personalized offerings while customer is at store or on web site 29
  • 30.
    12,000 Staff with 3,200pure scientist, 650,000 patients/year, 1,4 B€ revenue 500,000 data points from each cancer patient. Instant patient data analysis during treatment
  • 31.
    Mitsui Knowledge Industry Healthcareindustry – Cancer cell genomic analysis 408,000x faster than traditional diskbased systems in technical PoC 216x faster DNA analysis results from 2,5 days to 20 minutes © 2011 SAP AG. All rights reserved. 31
  • 32.