New Analytical Architectures          Why Classic Data Warehousing Approaches                         Miss the Mark with B...
“The Future is Data”“Hadoopis the kernel of a new Distributed Data OS”                                     Doug Cutting
Data has Changed > Analytics has Changed   Transactional     > Trailing Indicators                  Communities           ...
“Data” is the Platform                         The World as I See it
Wink EllerNew DataClutch Analytics
My Mountain Bike as a Data Platform                            Data Collection                            Heart Rate      ...
“Personal” Ride Analytics                            …is this a Data Warehouse?
New Data Behaviors (individual actions)   > Content > Time                Behaviors                              Content
New Data More is Better…                                     Meaningful     Guidance                           Massive    ...
“Business”Analytics - Classic “DW”    BUSINESS INTELLIGENCE   OLAP / DATA WAREHOUSE     OLTP / TRANSACTIONS             DA...
Classic “Business” AnalyticsGood for Reporting, Forecasting            What did Happen?                    What will Happe...
New“Personal”Analytics           SELF-SERVICE            GUIDANCE           BEHAVIOURS          DATA.                   An...
“Personal” Analytics“Right Now” is a very important time-frame!                               What is Happening           ...
14
15
“Business”Analytical Architecture Classic “DW” Data Flow - Uni-Directional, Latent,…   Ordering App                       ...
“Personal” Analytical Architecture“New” Data Flow - Iterative, Specialized, Extensible, plug & play Analytics, near real-t...
“Personal Analytics” Data Architecture “New” Data Flow – Detailed View of ComponentsEnd-User Experience    Browser, Tablet...
Let’s get personal…       SALLY LIKES TACOS      HOW DO WE MODEL THIS DATA?
Classic “DW” Data Model
Modeling Social Data“Triples” - Directed (Weighted) Acyclic Graph                        Reach and Influence              ...
Reach and InfluenceAnalyzing Relationships
How important is Social?                                          Shows you who is actively                   Install ghos...
Signals – The Core of New DataMixture of Proprietary and Public Data                      Social                     Perso...
The New “Analytical Application” Architecture“New” Data Flow – Specialized Technology Choices End-User Experience     Brow...
Servicing Multiple Analytical SystemsUsing Shared Analytical Mas- Storage      Self-Service Application A        Self-Serv...
Integrating the Architectures“Personal” Analytics Stack + Classic “DW” Stack                              Only Financial E...
Classic DW Vs. the New AnalyticsThe Shift from “Business” Analytics to “Personal” Analytics                         Classi...
New Signals + New Analytics = NewScenarios         Data                                        New                        ...
Thank You!              casey.l.kiernan@gmail.com        blog: www.the-data-platform.com
Upcoming SlideShare
Loading in …5
×

New Analytical Architectures for Big Data

592 views
504 views

Published on

Why Classic Data Warehousing Architectures miss the mark with the New Analytics

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
592
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Most people see data as a storage issue – persistence / serializatioAnd the associated technologies – oracle,mysql, sql server, and now Hadoop / riak / Hbasebut I don’t see data as a persistence problem – simply – “where do you put it?”I see data lattiss - theirs more data in the network than in databases… – it’s value is comes from how/where the data is used – not how it is stored. I had the opportunity to attend a number of training sessions help by Chris Date – we learned about RI, dead locks,… But one thing he said to me that really stuck – He said that the “data” is in the transaction log, not the database – the database only contains a current snapshot – What is happening is in the transaction log.
  • New Analytical Architectures for Big Data

    1. 1. New Analytical Architectures Why Classic Data Warehousing Approaches Miss the Mark with Big Data March 21, 2013 Casey Kiernan • casey.l.kiernan@gmail.com Blog • www.the-data-platform.com
    2. 2. “The Future is Data”“Hadoopis the kernel of a new Distributed Data OS” Doug Cutting
    3. 3. Data has Changed > Analytics has Changed Transactional > Trailing Indicators Communities > Reach/Influence Personal > Interactive Can the Data Warehouse Architecture adapt?
    4. 4. “Data” is the Platform The World as I See it
    5. 5. Wink EllerNew DataClutch Analytics
    6. 6. My Mountain Bike as a Data Platform Data Collection Heart Rate Data Collection Altitude Data Collection Temperature Speed / Trip Miles Time Guidance Performance Rate of Climb Calories Burned Miles Obtained Total Climbed Elapsed Time Current, Average, Max Values Data Collection Cadence / RPM Data Architecture - on a Local Wireless Network (ANT+ Protocol)
    7. 7. “Personal” Ride Analytics …is this a Data Warehouse?
    8. 8. New Data Behaviors (individual actions) > Content > Time Behaviors Content
    9. 9. New Data More is Better… Meaningful Guidance Massive Data 9
    10. 10. “Business”Analytics - Classic “DW” BUSINESS INTELLIGENCE OLAP / DATA WAREHOUSE OLTP / TRANSACTIONS DATA. Answers the question: What are our most profitable Products?
    11. 11. Classic “Business” AnalyticsGood for Reporting, Forecasting What did Happen? What will Happen? Operational Reporting Tactical Trending Strategic Months WeeksWeeks Months Years Descriptive/Trending Analytics11
    12. 12. New“Personal”Analytics SELF-SERVICE GUIDANCE BEHAVIOURS DATA. Answers the question: Show me a good movie to watch!
    13. 13. “Personal” Analytics“Right Now” is a very important time-frame! What is Happening RIGHT NOW! What did Happen? What will Happen? Operational Reporting Tactical Trending Strategic Months WeeksWeeks Months Years Predictive/Prescriptive Analytics 13
    14. 14. 14
    15. 15. 15
    16. 16. “Business”Analytical Architecture Classic “DW” Data Flow - Uni-Directional, Latent,… Ordering App Data Staging OLAP / Reports Business Financial App Warehouse OLTP to OLAP Facts/Dimensions Analyst Mapping Business Metrics, Master Data Facts & KPI, YTD Reporting Dimensions What are our most Profitable Products? 16
    17. 17. “Personal” Analytical Architecture“New” Data Flow - Iterative, Specialized, Extensible, plug & play Analytics, near real-time[Some components are open-source] Application / UX Analytical Capabilities Scoring/Ranking, Recommendations, Natural Language Processing, Relevancy, Classification, Optimization, Data Analytics Collaborative Filtering, Personalization, Digital Attribution,… Data Analysts What movie should I watch tonight? 17
    18. 18. “Personal Analytics” Data Architecture “New” Data Flow – Detailed View of ComponentsEnd-User Experience Browser, Tablet, Self-Service Application Mobile,… Personalization, Personalized Preferences, State Recommendations App Persistence Published AnalyticsPersistence/Analytics “State” Persistence “Read” Performance Analytics Engines Pluggable Social Signals Mass Data Storage RSS/Facebook/… Behaviors / “Write” Performance Data Scientists 18
    19. 19. Let’s get personal… SALLY LIKES TACOS HOW DO WE MODEL THIS DATA?
    20. 20. Classic “DW” Data Model
    21. 21. Modeling Social Data“Triples” - Directed (Weighted) Acyclic Graph Reach and Influence OBJECT PREDICATE (Score) SUBJECT SALLY LIKES (143) TACOS MARY LIKES (200) TACOS THE_TACO_SHOP MENU_ITEM TACOS SALLY LIKES (125) THE_TACO_SHOP SALLY CITY VENICE BEACH THE_TACO_SHOP CITY VENICE BEACH SALLY FRIEND (187) MARY Collaborative Filtering
    22. 22. Reach and InfluenceAnalyzing Relationships
    23. 23. How important is Social? Shows you who is actively Install ghostery.com watching you surf the web! Lots of people!!!
    24. 24. Signals – The Core of New DataMixture of Proprietary and Public Data Social Personal Content Time
    25. 25. The New “Analytical Application” Architecture“New” Data Flow – Specialized Technology Choices End-User Experience Browser, Tablet, Self-Service Application Mobile,… Personalization, Personalized Preferences, State Recommendations App Persistence Published Analytics Persistence/Analytics Cassandra, Riak,… Hbase Data-Center or Cloud Analytics R, Mahout, Pig Mass Data Storage Hadoop Specialization of Data Technologies 26
    26. 26. Servicing Multiple Analytical SystemsUsing Shared Analytical Mas- Storage Self-Service Application A Self-Service Application B Published Published Persistence Persistence Analytics Analytics Riak Cassandra HBase MySQL Data Scientists Analytics Engine Pluggable Mass Data Storage Analytics Engine Behaviors / “Write”Performance Pluggable Analytics Engine Hadoop / AWS Pluggable p. 27
    27. 27. Integrating the Architectures“Personal” Analytics Stack + Classic “DW” Stack Only Financial Events ($$$) cross the threshold App (and are recorded into) the Data Warehouse Staging App Data Warehouse OLAP / Business OLTP to OLAP Mapping Reports Analyst App “Local” Events stay Local (they are analyzed locally) Not all DATA Belongs in the Data Warehouse! 28
    28. 28. Classic DW Vs. the New AnalyticsThe Shift from “Business” Analytics to “Personal” Analytics Classic DW New AnalyticsScope Enterprise ApplicationAnalytics Trailing:OLAP Predictive: Machine Learning Sentiment Analysis, Recommendations, Personalization,Natural Language Processing, Classification, Clustering, Optimization, Collaborative Filtering, Digital Attribution,…Actionable? Loosely Coupled Tightly Coupled Analytics Embedded in ApplicationData Structures Facts/Dimensions Semantic Data,Graph / Triples, (Requires a DW) Observations, Direct SignalsKnowledge Expert Business Analyst Data ScientistTechnology Stack Vendor Driven ($$$) Open-SourceArchitecture Scale-Up Scale-Out (or in the Cloud)
    29. 29. New Signals + New Analytics = NewScenarios Data New Analytics Signals New Recommendations, Social Scenarios Natural Language Location Customer Processing, Engagement, Relevancy, Personal Customer Loyalty / Classification, Behaviors Attrition / Retention, Optimization, Transactions Fraud, Risk Analysis, Collaborative Content Intent, Customer Filtering, Personalization Digital Time Attribution,…
    30. 30. Thank You! casey.l.kiernan@gmail.com blog: www.the-data-platform.com

    ×