You’ve moved your data to the cloud, awesome. Now you’re running into issues of concurrency, scale, and cost overruns. But there’s a better way to run your cloud analytics if you think of cloud resources as commodities to conserve and maximize. Sure, you could run the same query from start to finish every time, or you could speed up this process, and save some cash in the process, by precomputing those queries and storing the response for fast retrieval any time, by any number of analysts. Kyligence Cloud 4’s Spark-Powered Cubing and Indexing feature provides just that - intelligent precomputation, which fundamentally boils down to low-cost, high-performance analytics. Join us for the fourth part of this series exploring the key features of Kyligence Cloud 4. In this webinar you will learn: -About modern, cloud era OLAP and cubing theory -Performance gains you’ll get from intelligent precomputation -How to apply cloud computing and distributed processing -Precomputation strategies and tactics

1. Kyligence Cloud 4 Feature Focus:
Spark-Powered
Cubing and Indexing
Mike Shen
Senior Solutions Architect

2. © Kyligence Inc. 2021
Three Pillars of Modern Cloud Analytics
Distributed Compute Columnar Storage Precomputation

3. © Kyligence Inc. 2021
MPP Queries – Slow and Costly in the Cloud
Common to Greenplum, Snowflake, Synapse, Redshift
Data Volume /
$$$
Resource
Consumption
Runtime Computation
MPP Engines : Runtime Computation
Data
Concurrency

4. © Kyligence Inc. 2021
Precomputation - Faster and Cheaper in the Cloud
An Aggregate Index (a.k.a. Cube)
Precomputation – Compute Once, Query Many Times
Data
Data Volume /
$$$
Resource
Consumption
Runtime Computation
Concurrency
Precomputation

5. © Kyligence Inc. 2021
Explain Precomputation with an Analogy

6. © Kyligence Inc. 2021
 Busy families:
 Full-time working parents with work commutes
 Drive kids to sport or music practices after school
 Minimize outside food like pizza delivery
 Help with weekday dinner preparation
 Objective - fast, reliable, and healthy dinners
 Popular practice in the US
Meal Prep

7. © Kyligence Inc. 2021
 Devote a portion of weekend to prepare
 Plan the meals for an entire week
 Shop for groceries
 Wash and chop the ingredients
 Re-package ingredients into meal sized portions
 On weekdays, cooking is quick and easy
 Repeat every weekend
How Does Meal Prep Work?

8. © Kyligence Inc. 2021
 Predictable Demand
 Family members will come home hungry
 Very little time to prepare dinner
 Resource Management
 Have time on weekends
 More people available to help on weekends
 Net labor saving with consolidated clean up
 Clear Benefits
 Well organized weekday evenings
 Consistent healthy eating
 Economic savings
Key Factors

9. © Kyligence Inc. 2021
 Predictable Demand
 Know the OLAP queries – filters, joins, and aggregations
 Know when these queries will show up, such as scheduled reports
 Resource Management
 Control when to process the source data
 Design optimized indexes
 Control hardware footprint – scale a cluster to suit the workload
 Clear Benefits
 Queries fulfilled at lightning speeds
 Scheduled reports run quickly
 Analysts issue more queries and derive more business insights
 Increased query load does not lead to increased cost
Analogous with Precomputation

10. © Kyligence Inc. 2021
Meal Prep Precomputation Technology
Weekend prepping (heavy lifting) Index building (heavy lifting) Spark build* cluster
Re-packaged ingredients Index files Parquet columnar format
Weekday cooking (payoff) Serve user queries (payoff) Spark query* cluster
* Kyligence separates build and query activities to avoid resource contention, hence different Spark clusters
Direct Comparisons

11. © Kyligence Inc. 2021
Kyligence Cloud 4
The First Cloud-Native OLAP Platform
Cloud-native architecture Elasticity, separate compute from storage
Unified Semantic Model MDX, SQL, REST access transparent to BI users
Cloud immediacy Deploys in minutes, on AWS/Azure marketplace
Enhanced AI engine Auto-modeling, self-tuning cubes/indexes

12. © Kyligence Inc. 2021
Kyligence Architecture
Unified Semantic Layer
Multi-Dimensional Modeling Security & Governance
Acceleration Layer
Aggregate & Table Indexes
AI-Augmented Engine
Any Cloud
Data Warehouse
Data Lake
Streaming Data
Any Data Platform
Any BI Tool
Machine Learning
SaaS & Apps
CRM HCM
SCM
SQL,
MDX,
REST
SQL,
MDX,
REST
SQL,
MDX,
REST

13. © Kyligence Inc. 2021
Cloud Native Architecture
Separation of compute and storage
• Spark compute cluster
• Shared cloud object storage
• Scale and optimize separately for best
results and price performance
• Continuity: build cluster separate from – and
doesn’t disrupt – query cluster
Removed legacy Hadoop requirement
• MapReduce, HDFS, HBase
Spark Cubing Persisted in Parquet format,
Optimized for Cloud Storage
High performance components
• Highly parallel processing for large workloads
• All tasks are executed in memory with Spark
- cubing, querying, modeling, aggregation
• Cubes and indexes saved in Parquet

14. © Kyligence Inc. 2021
Spark Parallel Processing and Cloud Columnar Storage

15. © Kyligence Inc. 2021
Spark Is Best Suited for Parallel Cubing/Indexing Operations

16. © Kyligence Inc. 2021
Precomputing Everything Isn’t Practical
Precomputation
Total Indexes = 2 ^ Number of dimensions - 1

17. © Kyligence Inc. 2021
Precomputing Everything Isn’t Practical
Challenge
• 50 dimensions => 1.2T Indexes
• Today, 100+ dimensions are possible
• Cost is always critical
Solution
• AI-brokered compromise

18. © Kyligence Inc. 2021
Intelligent Optimization: Automatic Index Tuning
User-Guided Intelligent
Operations
• User sets rules, system only recommends the
critical business scenarios that comply
• System continuously learns
Benefits
• Improve build, query, and storage efficiency
• Lower the tuning threshold and optimize
the index without professional coaching
and training
Custom Rules Critical Business Query
Index Auto-Tuning
E
A, E
A, B, E
A, B, C, E
A, B, C, D, E

19. © Kyligence Inc. 2021

20. © Kyligence Inc. 2021
Contact Us
Kyligence Inc
 http://kyligence.io
 info@kyligence.io
 Twitter: @Kyligence
Apache Kylin
 http://kylin.apache.org
 dev@kylin.apache.org
 Twitter: @ApacheKylin

21. © Kyligence Inc. 2020, Confidential.