Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Simplify Data Analytics Over the Cloud


Published on

Integrating and fully utilizing data is a critical prerequisite for ensuring the success of data-driven operations and decision making. This is especially true as more and more corporations begin transforming legacy data warehouses and transitioning to the Cloud. See how Augmented OLAP technology is leading the way in streamlining Big Data analytics on the Cloud with this presentation by Kyligence CEO Luke Han at Big Things Conference 2019. Learn more here:

Published in: Data & Analytics
  • Be the first to comment

Simplify Data Analytics Over the Cloud

  1. 1. Simplify Data Analytics over the Cloud Luke Han | • Co-founder & CEO of Kyligence Inc.
  2. 2. © Kyligence Inc. 2019. About Luke Han • Luke Han • Co-founder & CEO at Kyligence • Co-creator and former PMC Chair of Apache Kylin • Apache Software Foundation (ASF) Member • Microsoft Regional Director • Former eBay Big Data Product Manager Lead
  3. 3. © Kyligence Inc. 2019. Kyligence = Kylin + Intelligence • Founded in 2016 by the original creators of Apache Kylin • CRN - Top 10 Big Data Startups 2018 [US] • Deloitte - China Rising Star 2019 [China] • Dual HQ in Shanghai and San Jose AI-Augmented Data Warehouse 2016 Founded Pre-A RedPoint Cisco 2017 Series A CBC SHUNWEI 2018 Series B 8Roads (Fidelity International) 2019 Series C Coatue Kyligence is the only one who runs the same model as Databricks, Elastic in China based on an Apache Top Level Project
  4. 4. © Kyligence Inc. 2019. Trusted by Global Future 500
  5. 5. © Kyligence Inc. 2019. Apache Kylin – Leading Open Source OLAP for Big Data
  6. 6. © Kyligence Inc. 2019. Apache Kylin Community & Adoptions 1000+ Global Adoptions Leading Open Source OLAP Github Stars JIRA Issues
  7. 7. © Kyligence Inc. 2019. Apache Kylin –managing your Golden Data over Data Lake Presentation Visualization Data Lake Data Source o SemanticLayer o Speed up Analytics using Pre- Calculation o ANSISQL Interface o High Concurrency and High Performance o Batch & Streaming together OLAP Data Mart Hive Impala Spark SQL Drill MapReduce …Spark
  8. 8. © Kyligence Inc. 2019. Use Case: IBM Cognos Replacement Challenge • 1200+ existing cubes to manage • 1000+ jobs to maintain • Time-to-insight over 4 days Job Job Job Job Job Job Job … … Job Job Job 1200+ IBM Cognos Cubes, 1000+ ETL Jobs with dependencies Merchant Topic Daily Cube Region Topic Daily Cube Merchant Topic Monthly Cube 2 Cubes, 1 ETL Job ┄ ┄ Agency Topic Daily Cube Agency Topic Monthly Cube Channel Topic Monthly Cube Region Topic Monthly Cube Merchant Topic Shanghai Cube Merchant Topic Beijing Cube Merchant Topic Zhejiang Cube Merchant Topic Guangdong Cube Sub scenarios Channel Topic Daily Cube Sub scenarios Sub scenarios 2 Kyligence Cubes vs 1200 IBM Cognos Cubes Solution • Using Kyligence replaced IBM Cognos backend but continue keep Cognos Reporting to interactive with Kyligence Result • 1000x improved maintenance efficiency • 10x faster and more stable analytics performance • Time-to-Insight less then 4 hours
  9. 9. © Kyligence Inc. 2019. Data is next Oil The world’s most valuable resource is no longer oil, but data. —“The Economist” China’s Datasphere is expected to grow 30% on average over the next 7 years and will be the largest Datasphere of all regions by 2025 --IDC 175 Zettabytes By 2025 -- IDC
  10. 10. © Kyligence Inc. 2019. Data is moving to Cloud and, Data is moving to Cloud
  11. 11. © Kyligence Inc. 2019. But, the Chaos Happens Again!!!
  12. 12. © Kyligence Inc. 2019. What’s missing there? Intelligent Semantic Layer High Concurrency , High Performance No More Hadoop on Cloud Automation and Automation
  13. 13. © Kyligence Inc. 2019. What’s the must to have of Big Data over the Cloud Cloud Native Elastic Sizing Lower TCO
  14. 14. © Kyligence Inc. 2019. Kyligence Cloud – Simplify Data Analytics over the Cloud Azure AWS Snowflake Kafka Hadoop ANSI SQL Semantic Layer Apache Kylin FinanceMarketing Sales AI-Augmented Engine Index
  15. 15. © Kyligence Inc. 2019. Spark Native – Reduced Hadoop Overhead in the Cloud 1. Read …from S3/ADLS/Snowflake… 2. Build …Build Cube & Index 3. Store …Persistent #2 Data into Cloud Storage: S3/ADLS 4. Serve …Execute SQL parallelly with Cloud Storage
  16. 16. © Kyligence Inc. 2019. Semantic Layer – The Key to Govern Data • The Single Source of Truth • Business KPIs, Metrics, Hierarchies, etc. • Synchronize across BI tools • Flexible business calculations without coding: ---- YTD/MTD/Many-to-Many
  17. 17. © Kyligence Inc. 2019. Semantic Layer & OLAP for Cloud DW FinanceMarketingSales Index More… Landing & Transforming Aggregation & Index ApplicationsSource Azure Blob Storage
  18. 18. © Kyligence Inc. 2019. Demo – Semantic Layer for Snowflake …removed to reduce file size
  19. 19. © Kyligence Inc. 2019. Your car is self-driving now, how about your data?Your car is self-driving…and it is produced automatically…
  20. 20. © Kyligence Inc. 2019, Confidential. How about your Data? What will happen when you arrived office with your auto pilot Tesla?
  21. 21. © Kyligence Inc. 2019. The Reality! • Data Scientists vs SQL Monkey • Automation is the Key • Bias of taste – which is better: Hadoop MPP, Cloud… Rows of women office workers entering data using keypads…
  22. 22. © Kyligence Inc. 2019, Confidential. Automation is the Key “…an approach that automates insights using machine learning and natural-language generation, marks the next wave of disruption in the data and analytics market. Data and analytics leaders should plan to adopt augmented analytics as platform capabilities mature.” Augmented Analytics Is the Future of Data and Analytics Published: 27 July 2017 | Analysts: Carlie Idoine , Cindi Howson, Rita Sallam
  23. 23. © Kyligence Inc. 2019. AI-Augmented – Automation is the Future Pattern recognition Automatic modeling Data preparation Source data Accelerated data AI-Augmented Engine Background Learning Wisdom of the Crowd Enterprise wide businessqueriescan be loggedandimportedinto Kyligence.
  24. 24. © Kyligence Inc. 2019. AI-Augmented Engine – Learn from your analytics history
  25. 25. © Kyligence Inc. 2019. AI-Augmented Engine - Demo …removed to reduce file size
  26. 26. © Kyligence Inc. 2019. High Concurrency – High Performance at Scale Kyligence Data stored in Cloud Storage Index Scale Out Kyligence Nodes for High Concurrency … IndexIndex Azure Blob Storage Kyligence Computing Node at Scale Scale to store PB size Data Provision in the fly Only query node One node support 50+ concurrency Read only Lower cost
  27. 27. © Kyligence Inc. 2019. Elastic Scaling – Handle Peak Time Automatically • Less computation and storage resource utilized • Dynamic on-demand cluster resizing • Use spot instances • Efficiently planning for data growing
  28. 28. © Kyligence Inc. 2019. Security – Support Cell Level ACL Kyligence Kyligence Region: NA Region: APACRegion: EU • Column/Row/Cell Level ACL • LDAP/SSO • Kerberos/HTTPS • Source table ACL sync • SDK
  29. 29. © Kyligence Inc. 2019. Interactive with billions of rows in Excel Interactive with Excel on billions of rows • Perfect work with the Kyligence semantic layer • Seamlessly integration with analysts favorite BI tools
  30. 30. © Kyligence Inc. 2019. WeChat Application – from Cloud to Mobile • Collaboration via WeChat • NO Coding • No iOS/Android Development • Anytime, anywhere • Auto Responsive • beta version released Sales by region Total Sales Amount Total Sales Amount Buyer Country Seller Country Today’s Sales Report Top countries by month Time Range Time Range
  31. 31. © Kyligence Inc. 2019. Use Case: MS SSAS Migration to Azure Result • 1200+ dimensions in 1 cube • Support Project/Column/Cell level data access control Lower TCO • $300/daysaved for cloud infrastructure cost • Less than 1 hour to build cube One of the World’s Largest Cubes Challenge • 300 billion rows of data • 8 billionrows loaded/day • Service unavailable usually • Lack of data access control
  32. 32. © Kyligence Inc. 2019. Take away Kyligence Cloud Simplified Data Analytics in the Cloud • Semitic Layer – the key for data governance and management • AI-Augmented Engine – automation is the future of data analytics • Cloud Native - Cloud is the destination of data and analytics • Spark Native – Reduced Hadoop Overhead in the Cloud • High Concurrency but Lower TCO Try here:
  33. 33. Unleash Big Data Productivity Luke Han |