The use of data lakes continue to grow, and a recent survey by Eckerson Group shows that organizations are getting real value from their deployments. However, there’s still a lot of room for improvement when it comes to giving business users access to the wealth of potential insights in the data lake. While the data management aspect has been fairly well understood over the years, the success of business intelligence (BI) and analytics on data lakes lags behind. In fact, organizations often struggle with data lakes because they are only accessible by highly-skilled data scientists and not by business users. But BI tools have been able to access data warehouses for years, so what gives? In this talk, we’ll discuss: - Why traditional BI tools are architected well for data warehouses, but not data lakes. - Why every organization should have two BI standards: one for data warehouses and one for data lakes. - Innovative capabilities provided by BI for data lakes

1. Arcadia Data. Proprietary and Confidential
A Tale of Two BI Standards:
One for Data Warehouses and One for Data Lakes
October, 2018

2. Arcadia Data. Proprietary and Confidential
2
Featured Speakers
Randy Lea
Chief Revenue officer
Arcadia Data
John Thuma
Direct of Field Innovation
Arcadia Data

3. Arcadia Data. Proprietary and Confidential
3
§ If you have any questions along the way, please type them into the
chat window.
§ If you have audio problems, please chat us for help.
§ A recording of this presentation will be sent to you in a few days.
§ Please live tweet! @arcadiadata
Before We Begin Our Presentation

4. Arcadia Data. Proprietary and Confidential
4
25+ years in Enterprise Analytics
§ 10+ years Sales, Sales Management, Business Consulting
§ 10+ years Product Marketing, Product Management, Business
Development
Companies
§ Teradata – Data Warehouse
§ Aster Data – Data Science Platform
§ Think Big Analytics – Open Source Big Data Consulting
§ Arcadia Data – Visual Analytics and BI Platform Built for Hadoop and
Cloud
4
Quick Background

5. Arcadia Data. Proprietary and Confidential
5
1. Minimize Data Movement
2. Minimize Copies of Data
3. Minimize the Number of Places to Secure Data
4. Leverage the Power of Parallel Processing
5. Visualize Structured and Unstructured Data
6. Visualize Data in Motion
7. Visualize Data from Multiple Data Sources
8. Provide a Self-Service Ad-hoc Discovery Environment
9. Model Data Based on Usage
10. Productionize on the Same Platform as Your Discovery Environment
10 Big Data Considerations for Your Visual Analytics/BI Tool Selection

6. Arcadia Data. Proprietary and Confidential
6
Anyone Remember the 3 V’s?
Volume
Variety
Velocity
6
Why have Many Big Data/Data Lake Initiatives Failed?

7. Arcadia Data. Proprietary and Confidential
7
Companies Focused on the Data Deluge of the 3 V’s – Build a Data
Lake!
7

8. Arcadia Data. Proprietary and Confidential
8
8
What Problem are Companies Faced With Today?
Uncovering Business Value from Their Data Lakes

9. Arcadia Data. Proprietary and Confidential
9
How Traditional BI Nearly Killed Hadoop…
9
RDBMS
≠
Hadoop
https://www.arcadiadata.com/blog/how-traditional-bi-nearly-killed-hadoop/

10. Arcadia Data. Proprietary and Confidential
10
“Data” and “Platforms" Have Changed – Why Haven’t BI Tools?
From To
Data
Platforms
BI Tools
rows and columns and multi-structured
batch and interactive and real-time
small and large volumes
many sources
internal and external
tables and docs, search indexes, events
schema on write and schema on read
commodity hardware
ETL and ELT and ELDT
data warehouses and data lakes
rows and columns
batch
smaller data volumes
limited # sources
mainly internal
tables
schema on write
proprietary hardware
ETL
data warehouses
SQL queries
extracts
cubes
BI servers
small/med scale
Why haven’t
Traditional BI
tools
evolved?

11. Arcadia Data. Proprietary and Confidential
11
Would you use water skis
to ski down a mountain?
Why Not Use Any BI Tool? Architecture Built for a Purpose
Then why would you assume
a data warehouse BI tool
should work on a data lake?

12. Arcadia Data. Proprietary and Confidential
12
The Tale of Two BI Standards

13. Arcadia Data. Proprietary and Confidential
13
It’s All About Architecture and Being Built for a Purpose
13
Data Warehouse Data Lake
BI Standard for
Data Warehouse
(RDBMS)
BI Standard for
Data Lake
(Hadoop, Cloud Object Store)

14. Arcadia Data. Proprietary and Confidential
14
Data Warehouse BI Architecture
14
BI Server Analytic Process
Optimize Physical
Semantic Layer
Secure Data
Load Data
Big Data Requirements
Native Connection
Semi-Structured
Parallel
Real-time
Data Warehouse
(RDBMS)

15. Arcadia Data. Proprietary and Confidential
15
Data Lake BI Architecture
15
BI Server
Data Warehouse
(RDBMS)
Data Lake
(Hadoop, Cloud Object Storage)
Arcadia Data was built
from inception to
run natively on Hadoop and
Cloud
Analytic Process
Optimize Physical
Semantic Layer
Secure Data
Load Data
Big Data Requirements
Native Connection
Semi-Structured
Parallel
Real-time

16. Arcadia Data. Proprietary and Confidential
16
Smart Acceleration and Cost Based Optimization
Data Platform
Consumption Layer
Processing Layer
Queries
Data Coherent Cache
Instant
Results
Analytical Views
Native BI Engine
2
1
100x
Faster
4
Raw Data Storage
Smart
Acceleration3
• Native Parallel Analytics
• Data-Coherent Cache
• Analytical Views
• Smart Acceleration
ü Automatically modeled and
maintained within data
platform
ü Keep logical data models
simple without needing to
target particular cube
structures

17. Arcadia Data. Proprietary and Confidential
17
CACHE
Security Is Important too! Is Your Cached Data Secure?
WEB-BASED
APPS
Raw Data stays in HDFS or Object Storage.
Access to raw data is seamless. Can be
sampled for quick results.
High Concurrency Access for applications.
Data Coherent Cache is used as long as data
has not changed.
Arcadia creates and tracks analytic views over
the data. Queries rewrite to these views in a
cost based manner.
RAW DATA
INCREMENTAL ANALYTICAL VIEWS
CACHE
MULTI-NODE COLUMNAR IN-MEMORY EXECUTION
DATA COHERENT CACHESecure
Access
Rights
Checked at
Every Level
Queries
Queries
Queries

18. Arcadia Data. Proprietary and Confidential
18
The Result: Faster BI Analytics and Higher User Concurrency
18
25 35
88 105
169
427404
644
1440
120
214
366
199
379.107
687
0
200
400
600
800
1000
1200
1400
1 2 5 10 15 30
Completion Time (seconds)
# of Concurrent Jobs
Query 1 Performance Testing - Heavy Query
Arcadia Hive Impala Spark
Customer Benchmark of a Legacy BI Tool Accelerated by Arcadia Data On a Data Lake

19. Arcadia Data. Proprietary and Confidential
19
Data Warehouse BI Tools Treat Hadoop Just Like any other Database
Hadoop Data Lake
Traditional BI Server
BI Deployment Delayed Weeks
Time to Insight/Value in Weeks or Months
Extract
and
Secure
Land / Secure
Data
Performance
Modeling –
Cubes /
Aggregates
Analytical
Discovery
Production
Transform
3NF or Star
Schema
Build
Semantic
Layer
Performance
Modeling
(both places)
Data
Movement

20. Arcadia Data. Proprietary and Confidential
20
Data Warehouse BI Tools Treat Hadoop Just Like any other Database
Hadoop Data Lake
Traditional BI Server
BI Deployment Delayed Weeks
Time to Insight/Value in Weeks or Months
Extract
and
Secure
Land / Secure
Data
Performance
Modeling –
Cubes /
Aggregates
Analytical
Discovery
Production
Transform
3NF or Star
Schema
Build
Semantic
Layer
Performance
Modeling
(both places)
Data
Movement
ELDT

21. Arcadia Data. Proprietary and Confidential
21
Data Warehouse BI Tools Treat Hadoop Just Like any other Database
Hadoop Data Lake
Traditional BI Server
BI Deployment Delayed Weeks
Time to Insight/Value in Weeks or Months
Extract
and
Secure
Land / Secure
Data
Performance
Modeling –
Cubes /
Aggregates
Analytical
Discovery
Production
Transform
3NF or Star
Schema
Build
Semantic
Layer
Performance
Modeling
(both places)
Data
Movement
Extract Load “Discover” Transform
Physical Modeling Based on Usage

22. Arcadia Data. Proprietary and Confidential
22
Arcadia is Native to Hadoop Data Lakes
Arcadia is Changing the Analytic Paradigm due to being Native to
Hadoop
Hadoop Data Lake
Traditional BI Server
ü One security model
ü No movement of data
ü Self-Service Discovery
ü AI-driven performance
modeling
ü Production ReadyTime to Insight/Value in Days
BI Deployment Delayed Weeks
Time to Insight/Value in Weeks or Months
Extract
and
Secure
Land /
Secure Data
Build
Semantic
Layer
Analytical
Discovery
AI-driven
Performance
Modeling
Production
Land / Secure
Data
Performance
Modeling –
Cubes /
Aggregates
Analytical
Discovery
Production
Transform
3NF or Star
Schema
Build
Semantic
Layer
Performance
Modeling
(both places)
Data
Movement

23. Arcadia Data. Proprietary and Confidential
From a Single “Pane of Glass” User Interface – No Code Required
DISCOVER
Build your own
Semantic Layer:
Define Data
Your Way in a
Language You
Understand.
Join Data Your
Way:
Bring Your Own
Data and Join it
to Existing Data
in your Lake.
Alternative Data
EXPLORE
Assisted Visuals Guides you Through the
Visualization / Exploration Process
No Code Required. 100% Drag and Drop.
Customize and Make it Your own
PRODUCTIONIZE
Smart Acceleration speeds up your
Dashboards. Zero Code Required. Data
does not Move Off the Platform.
Secure & Share
your
Dashboards.
Setup Emails
& Alerts based
on Thresholds
Share

24. Arcadia Data. Proprietary and Confidential
24
Data Lake BI Architecture – More than Just Historical Analysis
24
Arc Viz
Streams/Topics
Real-Time Data
Data Warehouse
(RDBMS)
Data Lake
(HDFS, Cloud Object Storage)
Arcadia Data was built
from inception to
run natively on Hadoop and
Cloud

25. Arcadia Data. Proprietary and Confidential
Data Drives Market Disruption
25
Arcadia Data Streaming Visualizations
Data Sources
Historical Visuals
Native Access for Streaming Analytics – Real-Time + Historical
Real-Time
Visuals
Advanced Visualizations
and Semantic Layer
Data Node
KSQL Cluster
Streaming Data
Kafka Cluster
Source Topics
Data Node Data Node
Data Node Data Node
… …
………
……
IoT Dashboard

26. Arcadia Data. Proprietary and Confidential
26
Data Lake BI Architecture – More than Just Historical Analysis
26
Arc Viz
Data Warehouse
(RDBMS)
Data Lake
(HDFS, Cloud Object Storage)
Streams/Topics
Real-Time Data Arcadia Data was built
from inception to
run natively on Hadoop and
Cloud

27. Arcadia Data. Proprietary and Confidential
27
Data Lake BI Architecture – Load, Secure, and Process Data in One Place!
27
Data
Warehouse
Data
Lake
Arcadia Data was built
from inception to
run natively on Hadoop and
Cloud

28. Arcadia Data. Proprietary and Confidential
Role-based Security for Strong Governance Controls
• Integrated with Sentry and Ranger
• Role-based access controls
• Row and column level security
• Complete audit logs

29. Arcadia Data. Proprietary and Confidential
Connect to a Wide Variety of Data Sources, including Big Data

30. Arcadia Data. Proprietary and Confidential
30
Build a Shared Semantic Model for Business Users
• Shared Metadata
• Star & Snowflake schemas
• Modeled Hierarchies
• Geo types
• Search Ontology

31. Arcadia Data. Proprietary and Confidential31
Intuitive and Visual UI that Anyone
Can Use
• Accessed via web-browser
• Easy to compose visuals, dashboards and
apps via drag and drop
• Email integration, scheduled reports, and
alerting based on data
Benefits
• Unlocks big data analytics for business
users and analysts
• Promotes agility and reduces time to insight
• Enables business self-sufficiency and
relieves burden on IT
Drag and Drop Dashboards and Applications – No Coding

32. Arcadia Data. Proprietary and Confidential
32
Instant Visuals: AI-Augmented Visual Recommendations
Visualization Builder Recommended Visualizations
Select data fields, then one click… …shows you which visuals best represent your data

33. Arcadia Data. Proprietary and Confidential
33
Understands Complex
Structures
Easy Self-Service UI Powerful Native SQL
Works Natively with Complex Data Structures
Supports Modern ARRAY, STRUCT, MAP
Complex Types and Nested Schemas
SELECT c.name, sum(i.amount)
FROM customers c, c.orders.items i
GROUP BY 1
Translates Complex Structure
into Intuitive Field Browser
Generates Native SQL for Complex Types
No ETL Needed to Flatten Data Simple Drag & Drop Experience No Flattening at Access Time

34. Arcadia Data. Proprietary and Confidential
34
Summary of Process
Replicate the process – for each report and dashboard
Time consuming and expensive. Ongoing maintenance costs
Manual Physical Database Optimization
Identify poorly
performing dashboard
Traverse
logs for SQL
Build aggregate
structures
Edit each visual to
use aggregates
Schedule &
manage refresh

35. Arcadia Data. Proprietary and Confidential
35
Arcadia Data Smart Acceleration “DBA in a Box”
Pick dashboards
to accelerate
Click on ‘Smart
Acceleration’
Arcadia recommends, creates, and maintains AVs
based on your selected dashboards

36. Arcadia Data. Proprietary and Confidential
36
1. Minimize Data Movement
2. Minimize Copies of Data
3. Minimize the Number of Places to Secure Data
4. Leverage the Power of Parallel Processing
5. Visualize Structured and Unstructured Data
6. Visualize Data in Motion
7. Visualize Data from Multiple Data Sources
8. Provide a Self-Service Ad-hoc Discovery Environment
9. Model Data Based on Usage
10. Productionize on the Same Platform as Your Discovery Environment
10 Big Data Considerations for You Visual Analytics/BI Tool Selection
Did I
Mention
“Google
Like”
Search-
Based Self-
Service?

37. Arcadia Data. Proprietary and Confidential
What Can Search Do for You?
Let’s start with a simple dataset
Year – State – Population

38. Arcadia Data. Proprietary and Confidential
Simply Type in Your Question and Get Your Answer

39. Arcadia Data. Proprietary and Confidential
Simply Type in Your Question and Get Your Answer

40. Arcadia Data. Proprietary and Confidential
Simply Type in Your Question and Get Your Answer

41. Arcadia Data. Proprietary and Confidential
Simply Type in Your Question and Get Your Answer

42. Arcadia Data. Proprietary and Confidential
42
Arcadia Data
42
The Only Visual Analytics and BI
Tool Built from Inception
to Run Natively on Hadoop and
Cloud

43. Data Drives Market Disruption
Arcadia Data. Proprietary and Confidential
43
Thank You
Learn More – Resource Center
https://www.arcadiadata.com/resources
Try Arcadia Instant– Free Download
www.arcadiadata.com/Instant
Read our Blog:
https://www.arcadiadata.com/blog/
Follow Arcadia on Social:
@arcadiadata
New! ESG Technical
Review Report:
Native Business Intelligence (BI)
Performance for Data Lakes
New! See Arcadia Data’s New
Search Based Product in
Action