Learn how existing open source technologies like Apache Kylin, Spark, and Mondrian can be used to increase the value of your analytics investment. As we enter what some have called The Golden Age of Analytics, there are still some fundamental challenges that plague even the largest and most sophisticated cloud analytics adopters. Chief among these is the challenge of scale, often reflected in limitations of concurrency, multi-tenancy, distributed query performance, and all manner of latencies. Other less obvious, but equally crucial, challenges of scale and performance have to do with IT and end-user productivity. In other words, there have been few technological advances that enable the quick deployment of big data analytics and the rapid creation of business value from the data being analyzed. This presentation will consider a few of these systemic challenges and suggest some ways that they can be addressed with available open source technology such as Apache Kylin, Apache Spark, and Apache Mondrian. Presenter: Kaige Liu is a Senior Solutions Architect at Kyligence, where he works on building the next-generation big data analytics platform. Previously, he worked on the OpenStack and Bluemix team at IBM, focusing on cloud computing and virtualization technology. Kaige loves the open source community and is an active Apache Kylin committer.

1. Addressing the Systemic
Shortcomings of Cloud
Analytics
George Demarest
Senior Solution Architect, Kyligence
Kai Liu
Senior Director of Marketing, Kyligence

2. Agenda
 Benefits and Challenges of Cloud Analytics
 Introduction to Apache Kylin and Kyligence
 Case Study
 Drawing

3. Later on…
Drawing for an Amazon Gift Card
All attendees will receive a Kyligence T-shirt

4. © Kyligence Inc. 2020, Confidential.
Why Analytics in the Cloud?
 Benefits
• Fast, Easy Deployment
• Flexible Cost Model
• Most Data Lives There
• New Innovations are Cloud First

5. © Kyligence Inc. 2020, Confidential.
What Could Possibly Go Wrong?
 Challenges
• Lack of Cloud Native Thinking
• Failure to Rationalize BI Semantic Layers
• Lack of Data Modeling and Performance
Strategy
• Losing Track of Security and Governance

6. How Can
We Get
There?
With Apache Kylin and Kyligence
• Distributed Analytics at Cloud Scale
• Open Source OLAP/Precomputation
Platform
• AI-Assisted Query Optimization
• Self-Service, Automated Data Modeling
• Promotes Enterprise-grade Security and
Access

7. © Kyligence Inc. 2020, Confidential.
Apache Kylin
Top Level Project
 The only open-source distributed
OLAP platform
AwardWinning
 InfoWorld’s Bossies 2015 & 2016
(Best of Open Source Software Awards)
Sub-Second Interactive Query
 Large scale, high concurrency, multi-
dimensional, sub-second query latency
1,000+ Organizations
 Adopted by thousands of
organizations globally

8. © Kyligence Inc. 2020, Confidential.
Apache Kylin
Sept. 2013
Oct. 2014
Nov. 2014
Sept. 2015
Nov. 2015
Mar. 2016
Officially
Open Source
Project
Initiated
Apache
Incubator Project
InfoWorld
Best Open Source
Big Data Tool Award
Kyligence Inc.
Founded
Apache Top-level
Project
Extreme OLAP Engine for Big Data
Apr. 2019
Real-Time
Data Analysis
Kylin on Parquet
2020

9. © Kyligence Inc. 2020, Confidential.
Kyligence = Kylin + Intelligence
Mission: Develop an AI-Assisted
SQL/OLAP Platform for
Interactive Analytics at
Unprecedented Scale
• Founded in 2016 by the creators of Apache Kylin
• CRN Top-10 big data startups in 2018
• Global Presence: San Jose, Shanghai, Beijing,
Seattle, New York
• VCs: Fidelity International, Shunwei Capital,
Broadband Capital, Coatue, Redpoint, Cisco

10. © Kyligence Inc. 2020, Confidential.
Trusted by Global Fortune 500
BFSI
Telecom
Technology
Manufacturing,
Retail, Etc.

11. © Kyligence Inc. 2020, Confidential.
Kyligence and Microsoft
Kyligence Goes Global with Microsoft
2014 20162015 2017 2018 2019
Co-sell Ready Partner

12. © Kyligence Inc. 2020, Confidential.
Kyligence Azure Reference Architecture

13. Challenges Cloud First Thinking

14. © Kyligence Inc. 2020, Confidential.
The World’s Largest Retailer Has No Oracle DB/DW
Migrating All Oracle Databases to Cloud Services
Purpose-Build Cloud DB
Services

15. © Kyligence Inc. 2020, Confidential.
Kyligence Is Designed for Cloud
Elastic
Stability
Managed
Cost Effective
• Separation of storage and compute
• Unlimited resources
• Minimal maintain efforts
• Focus on business instead of Infra
• High availability, Recovery
• Pay as you go Economics

16. Challenges Semantic Layer

17. © Kyligence Inc. 2020, Confidential.
Data Silos
• Isolated data
• Complex tools and interfaces
• Inefficient communication and
data exchange
• Duplicated maintenance work
• Poor data quality

18. © Kyligence Inc. 2020, Confidential.
MPP DW
Data Mart
Tableau Server
RDBMS
Data Mart
SQL Server
Analysis Service
Excel Tableau Desktop
In-Memory
Aggregations
Data Mart
Cognos
PowerCube
Cognos
Data Platform
RDBMS
Data
Lake
Cloud Storage Data WarehouseHadoop
Semantic Layers with BI Tools

19. © Kyligence Inc. 2020, Confidential.
MPP DW
Data Mart
Tableau Server
RDBMS
Data Mart
SQL Server
Analysis Service
Excel Tableau Desktop
In-Memory
Aggregations
Data Mart
Cognos
PowerCube
Cognos
Do You Really Need Another Database?
Do You Really Need separate Middleware?
Data Platform
RDBMS
Data
Lake
Cloud Storage Data WarehouseHadoop
Rethink: Semantic Layers in BI Tools

20. © Kyligence Inc. 2020, Confidential.
Siloed Business Logic
Incompatible Security Models
Extra IT Infra/Ops Cost
Data Latency
Data Platform
RDBMS
Data
Lake
Cloud Storage Data WarehouseHadoop
Semantic Layers in BI Tools: Out of Control

21. © Kyligence Inc. 2020, Confidential.
Unified Semantic Layer
Unified
Security,
Governanc
e and
Access
Control
Data Platform
RDBMS
Data
Lake
Cloud Storage Data WarehouseHadoop
Excel Tableau Desktop PowerBI
• Transparent to end users
• Standard Access Methods - SQL, MDX, REST API
• Native Connectivity to Enterprise and Cloud Data Sources
Building Unified Semantic Layer

22. © Kyligence Inc. 2020, Confidential.
Kyligence Unified Semantic Layer
 Self-service multidimensional
modeling on massive data
 Defining business dimensions and
indicators flexibly
 Unified management of data life
cycle
 Transparent data access control
 Integrates seamlessly with any BI
tool

23. Challenges Ensuring Optimal
Data Modeling and
Query Performance

24. © Kyligence Inc. 2020, Confidential.
Kyligence AI-Augmented Engine
Model training
Pre-aggregation
Data processing
Auto-Generate
Cube & Index
AI Augmented Engine
Self training
Group wisdom
Queries across the org
ingested as input
分析师
工程师
管理员
Adjust models
Define semantics
Audit
Benefits
- Efficient data modeling
- Intelligent query optimization
- Data to insight from weeks to hours
Analyst
Engineer
Administrator
Recognize and manage most valuable data automatically

25. © Kyligence Inc. 2020, Confidential.
An Intelligent Precomputation Layer
ODBC/JDBC API/SDK
Finance Marketing Sales Customer Checkout
Aggregate Index
10%4% 80%
Data Lake
SQL/MDX
Semantic Services
6%
Distributed
Query Engine
AI-Augmented
Engine
Smart
Pushdown
Metadata
Management
Enterprise
Security
Intelligent Query Routing

26. © Kyligence Inc. 2020, Confidential.
Intelligent Query Routing
Route to Precomputation Layer or Pushdown to Native Engine
ODBC/JDBC API/
SDKF
i
n
a
n
c
e
M
a
r
k
e
ti
n
g
S
a
l
e
s
C
u
s
t
o
m
e
r
C
h
e
c
k
o
u
t
A
g
gr
eg
at
e
I
n
d
e
x
10%4% 80%
D
a
t
a
L
a
k
e
SQL/
MDX
Semantic Services
6%
Distribu
ted
Query
Engine
AI-
Augmente
d
Engine
Smart
Pushd
own
Metada
ta
Manag
ement
Enterpri
se
Security

27. Challenges Complex Security
and Governance

28. © Kyligence Inc. 2020, Confidential.
MPP DW
Data Mart
Tableau Server
RDBMS
Data Mart
SQL Server
Analysis Service
Excel Tableau Desktop
In-Memory
Aggregations
Data Mart
Cognos
PowerCube
Cognos
Security Challenges Multiply
Data Platform
RDBMS
Data
Lake
Cloud Storage Data WarehouseHadoop
Security Challenges Can Multiply

29. © Kyligence Inc. 2020, Confidential.
Is Cloud Safe Enough?
 Network
 Data storage
 Data transfer/access
 Disaster recovery

30. © Kyligence Inc. 2020, Confidential.
Kyligence Provides Enterprise Grade Data Security
Third Party User
Authentication
Multi-tenancy Architecture
Row and column level data
permission control
Data Encryption
Separation of Permissions
Network Security

31. © Kyligence Inc. 2020, Confidential.
Kyligence Has Flexible, Fine-Grained Safety Controls
 Built-in user roles and group management
supports flexible permissions
 Fine-grained table- and cell-level (row + column)
access control
 Docking third-party authentication system via
Kerberos, LDAP
 Centralized permission control transparent to
applications
Area = North China
Region = South
China
Region = East China

32. Case Studies

33. © Kyligence Inc. 2020, Confidential.
Top 10 - Global Investment Bank
Challenges
 Hard to scale
 Bad performance
 Data silos
 High cost
 Difficult to maintain

34. © Kyligence Inc. 2020, Confidential.
Top 10 - Global Investment Bank
Data Lake ApplicationsData Sources
ADLS
Data Semantic Layer
Oracl
e
ETL
Security

35. © Kyligence Inc. 2020, Confidential.
Top 10 - Global Investment Bank
Big Data to Golden Data
• High Scalability and Performance on large scale dataset
and concurrency
• Stable analysis service to support business insight
Business User Friendly
• Seamlessly integration for Excel and
mainstream BI
• Full functional OLAP capability and enrich
semantic layer
Cost Effective
• Scalable architecture for better IT resource planning
• Pays as you go
Dynamic Data Source
Risk Control Devices MonitorTransaction
SQL / MDX
efficiency
cost
Finance KPI ERP KPI
Accounting KPI
……
Marketing KPI
Sales KPI
Business Users Analysts

36. © Kyligence Inc. 2020, Confidential.
Summary
Addressing the Systemic Shortcomings of Cloud Analytics
• Develop Cloud Native Thinking
• Rationalize Your BI Semantic Layers
• Think Kylin/Kyligence – the Analytics
Monster!
• Sub-second query response on petabytes of
data
• AI-assisted precomputation layer makes is
possible
• AI-Assisted Auto-Data Modeling

37. © Kyligence Inc. 2020, Confidential.
Next Steps
https://kyligence.io/
The Winners!
Respond to our
email with your
desired shirt size

38. © Kyligence Inc. 2020, Confidential.
Contact Us
Kyligence Inc
 http://kyligence.io
 info@kyligence.io
 Twitter: @Kyligence
Apache Kylin
 http://kylin.apache.org
 dev@kylin.apache.org
 Twitter: @ApacheKylin

39. Thank You

40. © Kyligence Inc. 2020, Confidential.