SlideShare a Scribd company logo
How to Quickly and Easily Draw Value
from Big Data Sources
Moacyr Passador
Senior Sales Engineer
DevelopersAnalystsConsumers Data Scientists
Business Users IT Users
Types of BI Users
The Old BI World
Today’s BI World
Business Users are getting more involved in producing analytical content
The Role Of Business Users In BI Today Has Greatly Evolved
Relational
Databases
MapReduce
& NoSQL
Multi-Dimensional &
Other BI Tools
Cloud
Applications
Departmental
Data
Social Media
Business Users Today Want Direct Access To More Data
To make insightful decisions on their own, business users demand instant access to Data from multiple enterprise sources
&
>100x
More content
creation and
consumption
5-10x
More Content
5-10x
More Content Creators
5-10x
More Sharing
More productive
More content per creator
More producers
More users can create content
More collaborative
Peer-to-peer sharing
&
Adoption Of Self-service Analytics By Business Users Increases Productivity
Introduction to Big Data
What is Big Data, Really?
The Three Vs of Big Data According to Gartner
Volume
• Orders of magnitude bigger than conventional data (Terabytes, Petabytes,
Exabytes)
• Cost-prohibitive or practically impossible to store, manage or analyze in
typical database software
Variety
• Structured, semi-structured, unstructured formats
• Diverse sources - complex event processing, application logs, machine
sensors, social media data
Velocity
• Speed of ingesting incoming data streams
• Processing and real-time analysis of streaming and complex event data
Volume
Variety
Velocity
Four broad categories of Big Data sources and their value
Traditional	sources	
becoming	bigger
Company,	Government,	Financial	sector,	Business	and	
consumer	studies,	Surveys,	Polls
All	business	performance	drivers	– Operational	
efficiency,	Revenue	management,	Strategic	planning
SOURCE
VALUE
Digital	exhaust	
from	interactions
Online	click-stream,	Application	logs,	Call/service	
records,	ID	scans,	Security	cameras
New	revenue	sources,	Consumer	promotions,	Risk	
management,	Fraud	detection
SOURCE
VALUE
Web	2.0	
phenomenon
Content	generated	from	social	media	posts,	tweets,	
blogs,	pictures,	videos,	ratings
Customer	engagement,	Customer	service,	Brand	
management,	Viral	marketing
SOURCE
VALUE
Internet	of	
things
Machine	generated	sensor	data	and	“connected	
device”	communication
Operational	efficiency,	Cost	control,	Risk	avoidance
SOURCE
VALUE
Business Oriented Use Cases for Big Data
SOURCE
VALUE
Data Lakes and MicroStrategy
• Manicured, Often Relational
• Known Data Volumes
• Expected Formats
• Little to No Change
DATA SOURCES ETL DATA WAREHOUSE BI & ANALYTICS
• Complex Structures
• Rigid Transformations
• Extensive Monitoring Required
• Transformed Historical to Read Structures
• Flat, Canned Access to Data
• Report Chaos
• Extensive Data Load Delays
• Inflexible with new sources
Traditional Approaches And Current State Of A DWH
• Increase in Data types
• Rising Data Volumes
• Pressure on the DWH
• Constant change
DATA SOURCES ETL DATA WAREHOUSE BI & ANALYTICS
• Delay in reacting to new sources
• Monitoring is abandoned
• Transformations cant keep pace
• Repaid, Adjust and Redesign ETL
• Reports become invalid
• Delay in updates
• Users seek silos
• Business is disconnected w/ IT
Challenges With Traditional DWHs With Growing Data Demands
A storage repository, usually in Hadoop, that holds a vast amount of raw data in its
native format until it is needed
• Low cost
• Flexible and Loosely governed
• No need to decide up front what data to store or for how long
• Contains a mix of structured, semi-structuredand unstructured data
• Allows for freeform data exploration without having to wait for ETL
“If you think of a datamart as a store of bottled water – cleansed and packaged and
structured for easy consumption – the data lake is a large body of water in a more natural
state. The contents of the data lake stream in from a source to fill the lake, and various
users of the lake can come to examine, dive in, or take samples.” -- James Dixon
What is a Data Lake?
Data is an asset
• Today many organizations discard data due to cost of storage even though business
value may be mined from the data in the future
• A Data Lake allows you to store and process data essentially indefinitely
Unification
• Data Discovery, Data Science and Enterprise BI are treated as silos in many
organizations today
• A Data Lake can help unify these concepts and allow cross team collaboration
Utility
• A Data Lake creates the possibility of answering future questions of your data without
knowing the question in advance
Why Have A Data Lake?
Building the lake is easy / using it is hard
Governance is still key
• Without a meaningful strategy for maintaining
data quality this strategy can quickly fail
• You can quickly create a Data Swamp (dirty)
or a Data Graveyard (useless)
Sandbox strategy
Beware Of The Swamp
• Segment portions of your lake for experiments, testing and data
that may fall outside of your standard governance process
Analytical
Datamarts
Relational
Databases
MDM
ETLand
Governance
Data Warehouse
Structured
Enterprise Data
Operational
Data
Enterprise Metadata
MicroStrategy
Analytics Platform
Enterprise
Applications
Traditional Data Architecture
Data Lake
Enterprise Metadata
Prime
ELTwith
Governance
ETL
Relational Data
Enterprise
Applications
Cloud-based data
Web Logs
Flat Files
Analytical
Datamarts
MicroStrategy
Analytics Platform
In Comes The Data Lake
⎸07062016
Build Analytics and Mobility Applications Using Data Stored with Big Data
Hadoop Vendors SQL on Hadoop NoSQL Databases
Elastic Map Reduce
The MicroStrategy platform empowers organizations to build
applications that leverage big data and Hadoop distributions. All of
the major Hadoop distributions are certified to work with
MicroStrategy and once connected, data stored in Hadoop
becomes just like any other data source.
Users can connect using Hive, Pig, or proprietary SQL-on-Hadoop
connectors like Cloudera Impala or IBM BigInsights.
The MicroStrategy big data engine can natively tap into HDFS,
generating schema on-read and making Hadoop suitable for ad-
hoc querying. It also enables parallel loading of data from HDFS,
resulting in high-performance data loading.
MicroStrategy’s native connectivity saves users from the tedious
process of ETL from HDFS to Hive and helps to overcome the
ODBC overhead associated with Hive.
MicroStrategy’s data wrangling capability lets users cleanse and
refine their big data directly in MicroStrategy’s data discovery tool.
Big DataEnterprise assets
Mobility apps
that source information from
multiple locations, and submit
transactions to your ERP
systems
Analytics applications that
blend data from databases and
big data and deliver insights to
users via reports, dashboards,
and apps
16
MicroStrategy Platform
⎸07062016
MicroStrategy allows organizations toeasily accessandanalyzedatain all
shapes andsizes, from a singleplace. Business and IT users caneasily blend
multiple data-sources includingbig data sources in seconds.From personal
spreadsheets to cloudsources,to evenHDFS,big data access is madequick
and easy withnativeHDFS connectors or viaanyflavor of hiveproducts
includingCloudera,Hortonworks, MapR,Spark andmore.
Batch SQL: Fulfill your batch processing needs with certified
Hive/ODBC drivers from different Hadoop distributors: Cloudera, Hortonworks, MapR, and Amazon EMR
Interactive SQL: Leverage advanced SQL on Hadoop technologies for interactive queries such as
Cloudera Impala, MapR Drill, Apache Spark, IBM BigInsights, Pivotal HAWQ, and Facebook Presto
No SQL: Connect, query, and analyze data from No SQL sources such as HBase and Cassandra
Search: Dynamically search on semi-structured and unstructured data with MicroStrategy’s integration
to Apache Solr and Splunk
17
MicroStrategy Analytics Platform
Distributed File Systems (HDFS, Amazon S3, GFS…)
No SQLBatch SQL Interactive SQL Search
Big Data Analytics for Most Common Scenarios
Big data is just another data source
Product Demonstration
5 Differentiators for Big Data Analytics
“Leverage MicroStrategy's improved data discovery capabilities, as an alternative to
augmenting your BI portfolio with products such as Tableau and Qlik, to lower cost of
ownership and improve enterprise self-serviceviaa single, broader solution.”
Gartner Research Note – December 11, 2015
Enterprise data access with complete data governance
Self-service	data	exploration	and	production	dashboards
User	accessible	advanced	and	predictive	analytics
Analysis	of	semi-structured	and	unstructured	data
Real-time	analysis	from	live	updating	data
1
2
3
4
5
Five Differentiators for MicroStrategy in Big DataAnalytics
The MicroStrategy Analytics Platform enables every business user to get these capabilities
Optimized Access to Your Entire Big Data Ecosystem as If It Were a Single Database
Data Warehouse
Appliances
MapReduce &
NOSQLDatabases
Relational
Databases
Multidimensional
Databases
Columnar
Databases
SaaS-BasedApp
Data
HANA
BigInsights
Parallel Data Warehouse
Elastic Map
Reduce
Analysis Services
Redshift
BringAllRelevant
DatatoDecision
Makers
Distribution
Zendesk
HDFS
Generic Web
Services SOAP REST
User /Departmental
Data
1. Enterprise data access with complete data governance
Stunning Visualizations Instant Query Results Effortless Dashboards No IT Needed
Lightning fast insights, easy for everyone
2. Self-service	data	exploration	and	production	dashboards
User / Departmental
Data
Data Warehouse
Appliances
MapReduce
Databases
Relational
Databases
Multidimensional
Databases
Columnar
Databases
SaaS-Based
App Data
MicroStrategy
Multisource Engine
2 & 3
Join data on-the-fly.
No need to move it to a
staging database first.
Access your entire Big Data ecosystem as if it were a single database
Combine Data from Multiple Sources
2
3
1
2
1
2
3
MicroStrategy Supports Data Discovery at Scale
Start with Departmental teams and grow exponentially to publish to 1000s of users without fear
24
PUBLISH
Team
Department
Enterprise
Value Chain
10s 100s 1,000s 10,000+
MicroStrategy Desktop
Proven Scalability
Built-in clustering, failover, and
comprehensive administrative tools
for performance optimization
In-Memory Performance
Tested sub-second response
times on web and mobile, even
at highest user volume
Advanced Monitoring
Admin tools to automate, report
and alert on system utilization
Content Personalization
Users only see relevant data, and
only access functionality they are
authorized to use
Extend MicroStrategy’s Sophisticated Analytical Capabilities with 3rd party statistics toolkits
25
Industry’s most powerful SQLEngine and 300+ native analytical functions
Projections
RelationshipAnalysis
Benchmarking
TrendAnalysis
Data Summarization
AnalyticalMaturity
What is likely to happen based on past history?
What factors influence activity or behavior?
How are we doing versus comparables?
What direction are we headed in?
What is happening in the aggregate?
Optimization What do we want to happen?
World’s most popular
advancedanalytics tool.
Free, opensource.
More
Specialty Tools
3. Business User Friendly Predictive and Advanced Analytics
Streaming AnalyticsInteractive Search Text Analytics
Quickly	investigate:
• Website	logs
• Application	usage
• Surveys	and	free	form	text	
fields
• Event	and	error	monitoring	
logs
80% of data in most businesses is unstructured and this proportion will keep on rising
4. Analysis	of	semi-structured	and	unstructured	data
Find keyword and event occurrences
in any data
Apply semantic and syntactic
models to text data
Assess rapidly changing data
streams
Extract	relevant	information	to:
• Optimize	search	engine	
marketing
• Understand	sentiment	on	topics
• Get	a	360	degree	view	of	
customers
• Detect	fraud
Analyze	an	array	of	data	from:
• Sensors	and	devices
• Images,	audio,	and	video
• Email	and	document	
management	systems
• Other	operational	and	
transactional	data
New live data update technology
5. Real-time	analysis	from	live	updating	data
2
7
⎸07062016
Native HDFS
Connector
Native Big Data
Wrangling
Support for search
sources
In-memory parallel
architecture
MicroStrategy
Tableau
Qlik
Power BI
IBM Cognos
SAP BOBJ
Oracle OBIEE
MicroStrategy enables organizations to quickly harness the value of big data by deploying analytics at scale
Big Data Analytics: Product Differentiators
28
Product Demonstration
Powerful data preparation for more accurate analysis
Empowering data analysts to deliver deeper insights with intuitive and integrated data wrangling capabilities
Data integration in the hands of every user
Access and combine data from multiple sources “on-the-fly” to drive more productivity
MicroStrategy Prime – Analyze more data in memory
In-memory engine tightly coupled to the underlying DB
MicroStrategy Multi-Source
Effectively navigate data across multiple data sources
MicroStrategy Enabling Technologies for Big Data
Empowering data analysts to deliver deeper insights with intuitive and integrated data wrangling capabilities
31
Streamlined workflows
to parse and preparedata
Hundreds of inbuilt functions
to profile andcleandata
Multi-Table in-memory support
from different sources
Automatically parse and prepare
data with every refresh
Create custom groups
On the fly andwithout coding
Local /URL Files
Hadoop
Data Preparation
NewIntegratedDataPreparationCapability
Salesforce
Twitter/Facebook
Powerful Data Preparation For More Accurate Analysis
Source 1
Source 2
Source 3
Source 4
Public Data
DataBlending
Data Access
Live Connection
In-memory Data
OR
Other BI Tools
SaaS Data
Native HDFS
Access and combine data from multiple sources “on-the-fly” to drive more productivity
Data Integration In The Hands Of Every User
Data Upload
4x Faster
Server
With MicroStrategy 9.x
Serial access to in-memory data
Database
OLAP Cube
With MicroStrategy 10
Multi-threadedaccess to in-memory data
Database
PRIME
Server
2B 2B | 2B . . . 2B
Data Volumes
80x Larger
………Core 1 Core 2 Core 16
CPU
………Core 1 Core 2 Core 16
CPU
Data Interactions
50% faster
Bottleneck
…… Up to 8 parallelthreads
In-memory engine tightly coupled to the underlying DB
MicroStrategy Prime – Analyze More Data In Memory
Seamlessly traverse multiple DBs
End user agnostic
Effectively use aggregates
Automatic navigation
Works with different source types
Move from Hadoop toRelational
Metadata driven
No need towrite SQL
Effectively Navigate Data Across Multiple Data Sources
MicroStrategy Multi-Source
Data lakes are a powerful tool
MicroStrategy is ready to support your hybrid architecture
Meet the needs of multiple user personas
Enterprise BI capabilities combined with ad-hoc data discovery
A scalable solution for all workloads
Highly scalable in memory engine to combine data from multiple sources
MicroStrategy Multi-Source
Effectively navigate data across multiple data sources
Summary
Questions
Thank you
Moa Passador
mpassador@microstrategy.com

More Related Content

What's hot

From Hadoop to Enterprise Data Warehouse
From Hadoop to Enterprise Data WarehouseFrom Hadoop to Enterprise Data Warehouse
From Hadoop to Enterprise Data WarehouseBui Ha
 
The technology of the business data lake
The technology of the business data lakeThe technology of the business data lake
The technology of the business data lakeCapgemini
 
Enabling Cloud Data Integration (EMEA)
Enabling Cloud Data Integration (EMEA)Enabling Cloud Data Integration (EMEA)
Enabling Cloud Data Integration (EMEA)Denodo
 
The principles of the business data lake
The principles of the business data lakeThe principles of the business data lake
The principles of the business data lakeCapgemini
 
Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)
Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)
Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)Denodo
 
Analyst View of Data Virtualization: Conversations with Boulder Business Inte...
Analyst View of Data Virtualization: Conversations with Boulder Business Inte...Analyst View of Data Virtualization: Conversations with Boulder Business Inte...
Analyst View of Data Virtualization: Conversations with Boulder Business Inte...Denodo
 
Performance Acceleration: Summaries, Recommendation, MPP and more
Performance Acceleration: Summaries, Recommendation, MPP and morePerformance Acceleration: Summaries, Recommendation, MPP and more
Performance Acceleration: Summaries, Recommendation, MPP and moreDenodo
 
Data architecture for modern enterprise
Data architecture for modern enterpriseData architecture for modern enterprise
Data architecture for modern enterprisekayalvizhi kandasamy
 
Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...
Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...
Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...Cloudera, Inc.
 
Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...
Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...
Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...Data Con LA
 
Are You Killing the Benefits of Your Data Lake?
Are You Killing the Benefits of Your Data Lake?Are You Killing the Benefits of Your Data Lake?
Are You Killing the Benefits of Your Data Lake?Denodo
 
Data lake benefits
Data lake benefitsData lake benefits
Data lake benefitsRicky Barron
 
Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...
Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...
Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...Denodo
 
Making Big Data Easy for Everyone
Making Big Data Easy for EveryoneMaking Big Data Easy for Everyone
Making Big Data Easy for EveryoneCaserta
 
Creating a Next-Generation Big Data Architecture
Creating a Next-Generation Big Data ArchitectureCreating a Next-Generation Big Data Architecture
Creating a Next-Generation Big Data ArchitecturePerficient, Inc.
 

What's hot (20)

From Hadoop to Enterprise Data Warehouse
From Hadoop to Enterprise Data WarehouseFrom Hadoop to Enterprise Data Warehouse
From Hadoop to Enterprise Data Warehouse
 
The technology of the business data lake
The technology of the business data lakeThe technology of the business data lake
The technology of the business data lake
 
Enabling Cloud Data Integration (EMEA)
Enabling Cloud Data Integration (EMEA)Enabling Cloud Data Integration (EMEA)
Enabling Cloud Data Integration (EMEA)
 
The principles of the business data lake
The principles of the business data lakeThe principles of the business data lake
The principles of the business data lake
 
Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)
Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)
Empowering your Enterprise with a Self-Service Data Marketplace (ASEAN)
 
Analyst View of Data Virtualization: Conversations with Boulder Business Inte...
Analyst View of Data Virtualization: Conversations with Boulder Business Inte...Analyst View of Data Virtualization: Conversations with Boulder Business Inte...
Analyst View of Data Virtualization: Conversations with Boulder Business Inte...
 
Performance Acceleration: Summaries, Recommendation, MPP and more
Performance Acceleration: Summaries, Recommendation, MPP and morePerformance Acceleration: Summaries, Recommendation, MPP and more
Performance Acceleration: Summaries, Recommendation, MPP and more
 
Data architecture for modern enterprise
Data architecture for modern enterpriseData architecture for modern enterprise
Data architecture for modern enterprise
 
SQL Server Disaster Recovery Implementation
SQL Server Disaster Recovery ImplementationSQL Server Disaster Recovery Implementation
SQL Server Disaster Recovery Implementation
 
Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...
Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...
Hadoop World 2011: I Want to Be BIG - Lessons Learned at Scale - David "Sunny...
 
Data Lake,beyond the Data Warehouse
Data Lake,beyond the Data WarehouseData Lake,beyond the Data Warehouse
Data Lake,beyond the Data Warehouse
 
Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...
Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...
Big Data Day LA 2015 - Data Lake - Re Birth of Enterprise Data Thinking by Ra...
 
Are You Killing the Benefits of Your Data Lake?
Are You Killing the Benefits of Your Data Lake?Are You Killing the Benefits of Your Data Lake?
Are You Killing the Benefits of Your Data Lake?
 
Data lake benefits
Data lake benefitsData lake benefits
Data lake benefits
 
DW 101
DW 101DW 101
DW 101
 
Data management
Data managementData management
Data management
 
Taming Big Data With Modern Software Architecture
Taming Big Data  With Modern Software ArchitectureTaming Big Data  With Modern Software Architecture
Taming Big Data With Modern Software Architecture
 
Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...
Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...
Extended Data Warehouse - A New Data Architecture for Modern BI with Claudia ...
 
Making Big Data Easy for Everyone
Making Big Data Easy for EveryoneMaking Big Data Easy for Everyone
Making Big Data Easy for Everyone
 
Creating a Next-Generation Big Data Architecture
Creating a Next-Generation Big Data ArchitectureCreating a Next-Generation Big Data Architecture
Creating a Next-Generation Big Data Architecture
 

Viewers also liked

Ancestry Microstrategy World 2015 Presentation
Ancestry Microstrategy World 2015 PresentationAncestry Microstrategy World 2015 Presentation
Ancestry Microstrategy World 2015 PresentationDavid Sanders
 
Hadoop Integration with Microstrategy
Hadoop Integration with Microstrategy Hadoop Integration with Microstrategy
Hadoop Integration with Microstrategy snehal parikh
 
2. Google Analytics New Interface - Search University 3
2. Google Analytics New Interface - Search University 32. Google Analytics New Interface - Search University 3
2. Google Analytics New Interface - Search University 3Semetis
 
Making Data Visualization & Analytics accessible to Business Users
Making Data Visualization & Analytics accessible to Business UsersMaking Data Visualization & Analytics accessible to Business Users
Making Data Visualization & Analytics accessible to Business UsersHaroen Vermylen
 
Yahoo Microstrategy 2008
Yahoo Microstrategy 2008Yahoo Microstrategy 2008
Yahoo Microstrategy 2008Amr Awadallah
 
MicroStrategy World 2014: Scaling MicroStrategy at eBay
MicroStrategy World 2014: Scaling MicroStrategy at eBayMicroStrategy World 2014: Scaling MicroStrategy at eBay
MicroStrategy World 2014: Scaling MicroStrategy at eBayTim Case
 
Igniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner CableIgniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner CableTim Case
 
The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...
The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...
The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...Hortonworks
 
Business Intelligence Presentation (1/2)
Business Intelligence Presentation (1/2)Business Intelligence Presentation (1/2)
Business Intelligence Presentation (1/2)Bernardo Najlis
 

Viewers also liked (9)

Ancestry Microstrategy World 2015 Presentation
Ancestry Microstrategy World 2015 PresentationAncestry Microstrategy World 2015 Presentation
Ancestry Microstrategy World 2015 Presentation
 
Hadoop Integration with Microstrategy
Hadoop Integration with Microstrategy Hadoop Integration with Microstrategy
Hadoop Integration with Microstrategy
 
2. Google Analytics New Interface - Search University 3
2. Google Analytics New Interface - Search University 32. Google Analytics New Interface - Search University 3
2. Google Analytics New Interface - Search University 3
 
Making Data Visualization & Analytics accessible to Business Users
Making Data Visualization & Analytics accessible to Business UsersMaking Data Visualization & Analytics accessible to Business Users
Making Data Visualization & Analytics accessible to Business Users
 
Yahoo Microstrategy 2008
Yahoo Microstrategy 2008Yahoo Microstrategy 2008
Yahoo Microstrategy 2008
 
MicroStrategy World 2014: Scaling MicroStrategy at eBay
MicroStrategy World 2014: Scaling MicroStrategy at eBayMicroStrategy World 2014: Scaling MicroStrategy at eBay
MicroStrategy World 2014: Scaling MicroStrategy at eBay
 
Igniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner CableIgniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner Cable
 
The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...
The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...
The Modern Data Architecture for Advanced Business Intelligence with Hortonwo...
 
Business Intelligence Presentation (1/2)
Business Intelligence Presentation (1/2)Business Intelligence Presentation (1/2)
Business Intelligence Presentation (1/2)
 

Similar to How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa)

Architecting for Big Data: Trends, Tips, and Deployment Options
Architecting for Big Data: Trends, Tips, and Deployment OptionsArchitecting for Big Data: Trends, Tips, and Deployment Options
Architecting for Big Data: Trends, Tips, and Deployment OptionsCaserta
 
Enterprise Data Lake
Enterprise Data LakeEnterprise Data Lake
Enterprise Data Lakesambiswal
 
BAR360 open data platform presentation at DAMA, Sydney
BAR360 open data platform presentation at DAMA, SydneyBAR360 open data platform presentation at DAMA, Sydney
BAR360 open data platform presentation at DAMA, SydneySai Paravastu
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
 
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsWP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsJane Roberts
 
Introduction to BIG DATA
Introduction to BIG DATA Introduction to BIG DATA
Introduction to BIG DATA Zeeshan Khan
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization Denodo
 
Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)Nathan Bijnens
 
Unlock Your Data for ML & AI using Data Virtualization
Unlock Your Data for ML & AI using Data VirtualizationUnlock Your Data for ML & AI using Data Virtualization
Unlock Your Data for ML & AI using Data VirtualizationDenodo
 
Building a Big Data Analytics Platform- Impetus White Paper
Building a Big Data Analytics Platform- Impetus White PaperBuilding a Big Data Analytics Platform- Impetus White Paper
Building a Big Data Analytics Platform- Impetus White PaperImpetus Technologies
 
Hd insight overview
Hd insight overviewHd insight overview
Hd insight overviewvhrocca
 
Designing modern dw and data lake
Designing modern dw and data lakeDesigning modern dw and data lake
Designing modern dw and data lakepunedevscom
 
Big Data Practice_Planning_steps_RK
Big Data Practice_Planning_steps_RKBig Data Practice_Planning_steps_RK
Big Data Practice_Planning_steps_RKRajesh Jayarman
 
Next Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data WarehouseNext Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data WarehouseDenodo
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Group
 
Data lake-itweekend-sharif university-vahid amiry
Data lake-itweekend-sharif university-vahid amiryData lake-itweekend-sharif university-vahid amiry
Data lake-itweekend-sharif university-vahid amirydatastack
 
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonTraditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonCapgemini
 
Data Lake Overview
Data Lake OverviewData Lake Overview
Data Lake OverviewJames Serra
 
LinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbenchLinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbenchSheetal Pratik
 

Similar to How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa) (20)

Architecting for Big Data: Trends, Tips, and Deployment Options
Architecting for Big Data: Trends, Tips, and Deployment OptionsArchitecting for Big Data: Trends, Tips, and Deployment Options
Architecting for Big Data: Trends, Tips, and Deployment Options
 
Enterprise Data Lake
Enterprise Data LakeEnterprise Data Lake
Enterprise Data Lake
 
BAR360 open data platform presentation at DAMA, Sydney
BAR360 open data platform presentation at DAMA, SydneyBAR360 open data platform presentation at DAMA, Sydney
BAR360 open data platform presentation at DAMA, Sydney
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRobertsWP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
WP_Impetus_2016_Guide_to_Modernize_Your_Enterprise_Data_Warehouse_JRoberts
 
Introduction to BIG DATA
Introduction to BIG DATA Introduction to BIG DATA
Introduction to BIG DATA
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
 
Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)Data Mesh in Azure using Cloud Scale Analytics (WAF)
Data Mesh in Azure using Cloud Scale Analytics (WAF)
 
Unlock Your Data for ML & AI using Data Virtualization
Unlock Your Data for ML & AI using Data VirtualizationUnlock Your Data for ML & AI using Data Virtualization
Unlock Your Data for ML & AI using Data Virtualization
 
Benefits of a data lake
Benefits of a data lake Benefits of a data lake
Benefits of a data lake
 
Building a Big Data Analytics Platform- Impetus White Paper
Building a Big Data Analytics Platform- Impetus White PaperBuilding a Big Data Analytics Platform- Impetus White Paper
Building a Big Data Analytics Platform- Impetus White Paper
 
Hd insight overview
Hd insight overviewHd insight overview
Hd insight overview
 
Designing modern dw and data lake
Designing modern dw and data lakeDesigning modern dw and data lake
Designing modern dw and data lake
 
Big Data Practice_Planning_steps_RK
Big Data Practice_Planning_steps_RKBig Data Practice_Planning_steps_RK
Big Data Practice_Planning_steps_RK
 
Next Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data WarehouseNext Gen Analytics Going Beyond Data Warehouse
Next Gen Analytics Going Beyond Data Warehouse
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2
 
Data lake-itweekend-sharif university-vahid amiry
Data lake-itweekend-sharif university-vahid amiryData lake-itweekend-sharif university-vahid amiry
Data lake-itweekend-sharif university-vahid amiry
 
Traditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A ComparisonTraditional BI vs. Business Data Lake – A Comparison
Traditional BI vs. Business Data Lake – A Comparison
 
Data Lake Overview
Data Lake OverviewData Lake Overview
Data Lake Overview
 
LinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbenchLinkedInSaxoBankDataWorkbench
LinkedInSaxoBankDataWorkbench
 

How to Quickly and Easily Draw Value from Big Data Sources_Q3 symposia(Moa)

  • 1. How to Quickly and Easily Draw Value from Big Data Sources Moacyr Passador Senior Sales Engineer
  • 2. DevelopersAnalystsConsumers Data Scientists Business Users IT Users Types of BI Users The Old BI World Today’s BI World Business Users are getting more involved in producing analytical content The Role Of Business Users In BI Today Has Greatly Evolved
  • 3. Relational Databases MapReduce & NoSQL Multi-Dimensional & Other BI Tools Cloud Applications Departmental Data Social Media Business Users Today Want Direct Access To More Data To make insightful decisions on their own, business users demand instant access to Data from multiple enterprise sources &
  • 4. >100x More content creation and consumption 5-10x More Content 5-10x More Content Creators 5-10x More Sharing More productive More content per creator More producers More users can create content More collaborative Peer-to-peer sharing & Adoption Of Self-service Analytics By Business Users Increases Productivity
  • 6. What is Big Data, Really? The Three Vs of Big Data According to Gartner Volume • Orders of magnitude bigger than conventional data (Terabytes, Petabytes, Exabytes) • Cost-prohibitive or practically impossible to store, manage or analyze in typical database software Variety • Structured, semi-structured, unstructured formats • Diverse sources - complex event processing, application logs, machine sensors, social media data Velocity • Speed of ingesting incoming data streams • Processing and real-time analysis of streaming and complex event data Volume Variety Velocity
  • 7. Four broad categories of Big Data sources and their value Traditional sources becoming bigger Company, Government, Financial sector, Business and consumer studies, Surveys, Polls All business performance drivers – Operational efficiency, Revenue management, Strategic planning SOURCE VALUE Digital exhaust from interactions Online click-stream, Application logs, Call/service records, ID scans, Security cameras New revenue sources, Consumer promotions, Risk management, Fraud detection SOURCE VALUE Web 2.0 phenomenon Content generated from social media posts, tweets, blogs, pictures, videos, ratings Customer engagement, Customer service, Brand management, Viral marketing SOURCE VALUE Internet of things Machine generated sensor data and “connected device” communication Operational efficiency, Cost control, Risk avoidance SOURCE VALUE Business Oriented Use Cases for Big Data SOURCE VALUE
  • 8. Data Lakes and MicroStrategy
  • 9. • Manicured, Often Relational • Known Data Volumes • Expected Formats • Little to No Change DATA SOURCES ETL DATA WAREHOUSE BI & ANALYTICS • Complex Structures • Rigid Transformations • Extensive Monitoring Required • Transformed Historical to Read Structures • Flat, Canned Access to Data • Report Chaos • Extensive Data Load Delays • Inflexible with new sources Traditional Approaches And Current State Of A DWH
  • 10. • Increase in Data types • Rising Data Volumes • Pressure on the DWH • Constant change DATA SOURCES ETL DATA WAREHOUSE BI & ANALYTICS • Delay in reacting to new sources • Monitoring is abandoned • Transformations cant keep pace • Repaid, Adjust and Redesign ETL • Reports become invalid • Delay in updates • Users seek silos • Business is disconnected w/ IT Challenges With Traditional DWHs With Growing Data Demands
  • 11. A storage repository, usually in Hadoop, that holds a vast amount of raw data in its native format until it is needed • Low cost • Flexible and Loosely governed • No need to decide up front what data to store or for how long • Contains a mix of structured, semi-structuredand unstructured data • Allows for freeform data exploration without having to wait for ETL “If you think of a datamart as a store of bottled water – cleansed and packaged and structured for easy consumption – the data lake is a large body of water in a more natural state. The contents of the data lake stream in from a source to fill the lake, and various users of the lake can come to examine, dive in, or take samples.” -- James Dixon What is a Data Lake?
  • 12. Data is an asset • Today many organizations discard data due to cost of storage even though business value may be mined from the data in the future • A Data Lake allows you to store and process data essentially indefinitely Unification • Data Discovery, Data Science and Enterprise BI are treated as silos in many organizations today • A Data Lake can help unify these concepts and allow cross team collaboration Utility • A Data Lake creates the possibility of answering future questions of your data without knowing the question in advance Why Have A Data Lake?
  • 13. Building the lake is easy / using it is hard Governance is still key • Without a meaningful strategy for maintaining data quality this strategy can quickly fail • You can quickly create a Data Swamp (dirty) or a Data Graveyard (useless) Sandbox strategy Beware Of The Swamp • Segment portions of your lake for experiments, testing and data that may fall outside of your standard governance process
  • 14. Analytical Datamarts Relational Databases MDM ETLand Governance Data Warehouse Structured Enterprise Data Operational Data Enterprise Metadata MicroStrategy Analytics Platform Enterprise Applications Traditional Data Architecture
  • 15. Data Lake Enterprise Metadata Prime ELTwith Governance ETL Relational Data Enterprise Applications Cloud-based data Web Logs Flat Files Analytical Datamarts MicroStrategy Analytics Platform In Comes The Data Lake
  • 16. ⎸07062016 Build Analytics and Mobility Applications Using Data Stored with Big Data Hadoop Vendors SQL on Hadoop NoSQL Databases Elastic Map Reduce The MicroStrategy platform empowers organizations to build applications that leverage big data and Hadoop distributions. All of the major Hadoop distributions are certified to work with MicroStrategy and once connected, data stored in Hadoop becomes just like any other data source. Users can connect using Hive, Pig, or proprietary SQL-on-Hadoop connectors like Cloudera Impala or IBM BigInsights. The MicroStrategy big data engine can natively tap into HDFS, generating schema on-read and making Hadoop suitable for ad- hoc querying. It also enables parallel loading of data from HDFS, resulting in high-performance data loading. MicroStrategy’s native connectivity saves users from the tedious process of ETL from HDFS to Hive and helps to overcome the ODBC overhead associated with Hive. MicroStrategy’s data wrangling capability lets users cleanse and refine their big data directly in MicroStrategy’s data discovery tool. Big DataEnterprise assets Mobility apps that source information from multiple locations, and submit transactions to your ERP systems Analytics applications that blend data from databases and big data and deliver insights to users via reports, dashboards, and apps 16 MicroStrategy Platform
  • 17. ⎸07062016 MicroStrategy allows organizations toeasily accessandanalyzedatain all shapes andsizes, from a singleplace. Business and IT users caneasily blend multiple data-sources includingbig data sources in seconds.From personal spreadsheets to cloudsources,to evenHDFS,big data access is madequick and easy withnativeHDFS connectors or viaanyflavor of hiveproducts includingCloudera,Hortonworks, MapR,Spark andmore. Batch SQL: Fulfill your batch processing needs with certified Hive/ODBC drivers from different Hadoop distributors: Cloudera, Hortonworks, MapR, and Amazon EMR Interactive SQL: Leverage advanced SQL on Hadoop technologies for interactive queries such as Cloudera Impala, MapR Drill, Apache Spark, IBM BigInsights, Pivotal HAWQ, and Facebook Presto No SQL: Connect, query, and analyze data from No SQL sources such as HBase and Cassandra Search: Dynamically search on semi-structured and unstructured data with MicroStrategy’s integration to Apache Solr and Splunk 17 MicroStrategy Analytics Platform Distributed File Systems (HDFS, Amazon S3, GFS…) No SQLBatch SQL Interactive SQL Search Big Data Analytics for Most Common Scenarios Big data is just another data source
  • 19. 5 Differentiators for Big Data Analytics “Leverage MicroStrategy's improved data discovery capabilities, as an alternative to augmenting your BI portfolio with products such as Tableau and Qlik, to lower cost of ownership and improve enterprise self-serviceviaa single, broader solution.” Gartner Research Note – December 11, 2015
  • 20. Enterprise data access with complete data governance Self-service data exploration and production dashboards User accessible advanced and predictive analytics Analysis of semi-structured and unstructured data Real-time analysis from live updating data 1 2 3 4 5 Five Differentiators for MicroStrategy in Big DataAnalytics The MicroStrategy Analytics Platform enables every business user to get these capabilities
  • 21. Optimized Access to Your Entire Big Data Ecosystem as If It Were a Single Database Data Warehouse Appliances MapReduce & NOSQLDatabases Relational Databases Multidimensional Databases Columnar Databases SaaS-BasedApp Data HANA BigInsights Parallel Data Warehouse Elastic Map Reduce Analysis Services Redshift BringAllRelevant DatatoDecision Makers Distribution Zendesk HDFS Generic Web Services SOAP REST User /Departmental Data 1. Enterprise data access with complete data governance
  • 22. Stunning Visualizations Instant Query Results Effortless Dashboards No IT Needed Lightning fast insights, easy for everyone 2. Self-service data exploration and production dashboards
  • 23. User / Departmental Data Data Warehouse Appliances MapReduce Databases Relational Databases Multidimensional Databases Columnar Databases SaaS-Based App Data MicroStrategy Multisource Engine 2 & 3 Join data on-the-fly. No need to move it to a staging database first. Access your entire Big Data ecosystem as if it were a single database Combine Data from Multiple Sources 2 3 1 2 1 2 3
  • 24. MicroStrategy Supports Data Discovery at Scale Start with Departmental teams and grow exponentially to publish to 1000s of users without fear 24 PUBLISH Team Department Enterprise Value Chain 10s 100s 1,000s 10,000+ MicroStrategy Desktop Proven Scalability Built-in clustering, failover, and comprehensive administrative tools for performance optimization In-Memory Performance Tested sub-second response times on web and mobile, even at highest user volume Advanced Monitoring Admin tools to automate, report and alert on system utilization Content Personalization Users only see relevant data, and only access functionality they are authorized to use
  • 25. Extend MicroStrategy’s Sophisticated Analytical Capabilities with 3rd party statistics toolkits 25 Industry’s most powerful SQLEngine and 300+ native analytical functions Projections RelationshipAnalysis Benchmarking TrendAnalysis Data Summarization AnalyticalMaturity What is likely to happen based on past history? What factors influence activity or behavior? How are we doing versus comparables? What direction are we headed in? What is happening in the aggregate? Optimization What do we want to happen? World’s most popular advancedanalytics tool. Free, opensource. More Specialty Tools 3. Business User Friendly Predictive and Advanced Analytics
  • 26. Streaming AnalyticsInteractive Search Text Analytics Quickly investigate: • Website logs • Application usage • Surveys and free form text fields • Event and error monitoring logs 80% of data in most businesses is unstructured and this proportion will keep on rising 4. Analysis of semi-structured and unstructured data Find keyword and event occurrences in any data Apply semantic and syntactic models to text data Assess rapidly changing data streams Extract relevant information to: • Optimize search engine marketing • Understand sentiment on topics • Get a 360 degree view of customers • Detect fraud Analyze an array of data from: • Sensors and devices • Images, audio, and video • Email and document management systems • Other operational and transactional data
  • 27. New live data update technology 5. Real-time analysis from live updating data 2 7
  • 28. ⎸07062016 Native HDFS Connector Native Big Data Wrangling Support for search sources In-memory parallel architecture MicroStrategy Tableau Qlik Power BI IBM Cognos SAP BOBJ Oracle OBIEE MicroStrategy enables organizations to quickly harness the value of big data by deploying analytics at scale Big Data Analytics: Product Differentiators 28
  • 30. Powerful data preparation for more accurate analysis Empowering data analysts to deliver deeper insights with intuitive and integrated data wrangling capabilities Data integration in the hands of every user Access and combine data from multiple sources “on-the-fly” to drive more productivity MicroStrategy Prime – Analyze more data in memory In-memory engine tightly coupled to the underlying DB MicroStrategy Multi-Source Effectively navigate data across multiple data sources MicroStrategy Enabling Technologies for Big Data
  • 31. Empowering data analysts to deliver deeper insights with intuitive and integrated data wrangling capabilities 31 Streamlined workflows to parse and preparedata Hundreds of inbuilt functions to profile andcleandata Multi-Table in-memory support from different sources Automatically parse and prepare data with every refresh Create custom groups On the fly andwithout coding Local /URL Files Hadoop Data Preparation NewIntegratedDataPreparationCapability Salesforce Twitter/Facebook Powerful Data Preparation For More Accurate Analysis
  • 32. Source 1 Source 2 Source 3 Source 4 Public Data DataBlending Data Access Live Connection In-memory Data OR Other BI Tools SaaS Data Native HDFS Access and combine data from multiple sources “on-the-fly” to drive more productivity Data Integration In The Hands Of Every User
  • 33. Data Upload 4x Faster Server With MicroStrategy 9.x Serial access to in-memory data Database OLAP Cube With MicroStrategy 10 Multi-threadedaccess to in-memory data Database PRIME Server 2B 2B | 2B . . . 2B Data Volumes 80x Larger ………Core 1 Core 2 Core 16 CPU ………Core 1 Core 2 Core 16 CPU Data Interactions 50% faster Bottleneck …… Up to 8 parallelthreads In-memory engine tightly coupled to the underlying DB MicroStrategy Prime – Analyze More Data In Memory
  • 34. Seamlessly traverse multiple DBs End user agnostic Effectively use aggregates Automatic navigation Works with different source types Move from Hadoop toRelational Metadata driven No need towrite SQL Effectively Navigate Data Across Multiple Data Sources MicroStrategy Multi-Source
  • 35. Data lakes are a powerful tool MicroStrategy is ready to support your hybrid architecture Meet the needs of multiple user personas Enterprise BI capabilities combined with ad-hoc data discovery A scalable solution for all workloads Highly scalable in memory engine to combine data from multiple sources MicroStrategy Multi-Source Effectively navigate data across multiple data sources Summary