Why Big Data Analytics Needs Business Intelligence Too

1,067 views

Published on

Business and IT are facing the challenge of getting real and urgent value from ever-expanding information sources. Building independent silos of big data analytics is no longer enough. True progress comes only by integrating data from traditional operational and informational sources with the new sources that are becoming available, whether from social media or interconnected machines.

In this April 2014 BrightTALK webinar, Dr. Barry Devlin describes the thinking, architecture, tools and methods needed to achieve a new joined-up, comprehensive data environment.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,067
On SlideShare
0
From Embeds
0
Number of Embeds
24
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Why Big Data Analytics Needs Business Intelligence Too

  1. 1. Copyright © 2014 9sight Consulting, All Rights Reserved Dr Barry Devlin Founder & Principal 9sight Consulting Why Big Data Analytics Needs Business Intelligence Too BrightTALK Webinar 9 April 2014
  2. 2. Dr. Barry Devlin 2 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting Founder and Principal 9sight Consulting, www.9sight.com Dr. Barry Devlin is a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence (BI) and beyond. He is a widely respected consultant, lecturer and author of the seminal “Data Warehouse—from Architecture to Implementation”. His new book, “Business unIntelligence—Insight and Innovation Beyond Analytics and Big Data” (http://bit.ly/BunI-Technics) was published in October 2013. Barry has 30 years of experience in IT, previously with IBM, as an architect, consultant, manager and software evangelist. As founder and principal of 9sight Consulting (www.9sight.com), Barry provides strategic consulting and thought-leadership to buyers and vendors of BI solutions. He is currently developing new architectural models for fully consistent business support— from informational to operational and collaborative work. Based in Cape Town, South Africa, Barry’s knowledge and expertise are in demand both locally and internationally. Email: barry@9sight.com Twitter: @BarryDevlin
  3. 3. Big data analytics began with social media and web logs  Understanding and tracking sentiment – What do you think? How do you react? – Basic analytics and BI activity on a new data source  Real-time insight into and influence on website activities – Why did you abandon your cart? – What would you most likely buy on getting a cross-sell? – Deep, real-time analytics and BI with operational integration 3 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  4. 4. Add the Internet of Things to big data analytics and reinvent businesses  Significant new considerations – Micro-management of supply chains and extension all the way to the consumer – Sourcing and delivery – Completely new business models (usually depending on big data analytics) – Motor insurance – Health monitoring 4 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  5. 5. But wait… it’s not just big data …we also need traditional business data  Traditional business processes – Data created, managed and used in a structured and regulated way – “Process-mediated data” – The legal basis of business  Big data analytics – Data gathered from unreliable sources, often designed for unrelated purposes  Business value of big data depends on linking it to traditional business processes 5 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  6. 6.  Characteristics – Tactical decision making based on reconciled data – Consistency and truth – Separation of operational and informational needs – Vertical and horizontal segmentation of data – Unidirectional data flow  Note: key business needs and technology limitations of the ’80s and ’90s 6 Process-mediated data is the core of BI and layered Data Warehouse since the early ’90s Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting Data marts Enterprise data warehouse Metadata Data warehouse Operational systems “An architecture for a business and information system”, B. A. Devlin, P. T. Murphy, IBM Systems Journal, (1988)
  7. 7. The tri-domain model shows two new types of data / information  Process-mediated data – “Traditional” operational & informational data – Via data entry and cleansing processes  Machine-generated data – Output of machines and sensors – The Internet of Things  Human-sourced information – Subjectively interpreted record of personal experiences – From Tweets to Videos 7 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting Human-sourced information Machine- generated data Process-mediated data Structure/Context Timeliness/ Consistency HistoricalReconciledStableLiveIn-flight [In the context of these domains, “data” signifies well-structured and/or modeled and “information” is more loosely structured and human-centric.]
  8. 8. The modern, REAL logical architecture  Realistic, Extensible, Actionable, Labile  Three interconnected pillars of information – Messages, events, measures and transactions from real world – Metadata is context-setting information  Adaptive process – Business and IT – Information processing – Instantiation, assimilation and reification – ETL, ELT, Virtualization – Workflows and activities – Choreography 8 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting EventsMeasures Messages Transactions Reification Utilization Choreography Organization Instantiation Human- sourced (information) Machine- generated (data) Process- mediated (data) Context-setting (information) Assimilation Transactional (data)
  9. 9. Key characteristics of information pillars  Single architecture includes all types of data/information – Mix/match technology as needed – Relational, NoSQL, CEP, Graph, etc.  Integration of sources and stores – Operational processes gather measures, events, messages and transactions – Assimilation integrates stored information  Data flows as fast as needed and reconciled when necessary – No unnecessary storage or transformations 9 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting EventsMeasures Messages Transactions Human- sourced (information) Machine- generated (data) Process- mediated (data) Context-setting (information) Assimilation Transactional (data) Operational Processes
  10. 10. Process-mediated data: Relational databases evolve to allow de-layering and reintegration  Drivers: Stability, Consistency and Reliability  Relational databases remain core technology – “New” approaches to storage and processing – Columnar (and compressed) to hybrid – Solid-state disk and in-memory – Massively parallel processing – Advantages: – Reduced physical modelling – Faster read and write  Sample offerings: – Upgraded databases: e.g. IBM DB2 BLU, etc. – Appliances: e.g. Actian ParAccel, HP Vertica, SAP HANA, etc. – BI Tools: e.g. Tableau, Qlikview, etc. 10 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting Multi-core MPP
  11. 11. Machine-generated data: NoSQL and streaming take on relational at the extremes  Drivers: Speed, Size and Flexible Structure  NoSQL is the current darling, especially at the extreme of all three drivers  CEP (complex event processing) / Streaming at extreme speed  Relational can address many of these drivers – Even flexible structure (see my relational vs. Hadoop session) 11 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  12. 12. Human-sourced information: Hadoop and/or enterprise content management  Drivers: Soft, Large and Ill-defined data  Hadoop , Hadoop and more Hadoop – Hadoop 2.0 enables more real-time processing  Traditional ECM tools should not be forgotten – Enterprise content management – Soft information needs to be managed 12 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  13. 13. Information processing creates, maintains and mediates access to all information.  Instantiation – Turns measures, events and messages into info. instances – File access, ETL, change capture…  Assimilation – Creation of reconciled and consistent info. sets prior to business use – Key to big data – BI linkage – With context-setting information – ETL, ELT and virtualization  Reification (making the abstract real) – Providing a real-time, consistent, cross-pillar access to info. according to an overarching model – Virtualization 13 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting EventsMeasures Messages Transactions Reification Instantiation Human- sourced (information ) Machine- generated (data) Process- mediated (data) Context-setting (information) Assimilation Transactional (data) Organization
  14. 14. Context-setting information (metadata) is key.  Metadata is two four-letter words! – Information (not data) – Describes all “stuff” (not just data) – Indistinguishable from “business information” by non-IT people (and some IT people)  Context-setting information (CSI) – New image: describes what it is and does – Context-setting information provides the background to each piece of information, to every process component and to all the people that constitute the business – All information is actually context-setting for something else  How to create CSI – Modeling up-front combined with Text Mining on the fly 14 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting Mars Climate Orbiter, lost in 1999, $325M: metadata error
  15. 15. From BI to Business unIntelligence  Rationality of thought and far beyond it  Logic of process, predefined and emergent  Information, knowledge and meaning  The confluence of – Reason and inspiration – Emotion and intention – Collaboration and competition – All that comprises the human and social milieu that is business  Not business intelligence  Business unIntelligence  http://bit.ly/BunI-Technics: 25% discount with code “BIInsights25” 15 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  16. 16. Conclusions  Big data and the Internet of Things only offer background to “real” business  Reconciled and consistent data built via Data Warehouse and BI contains the reality of the business – legally-binding actions and transactions  The emerging architecture consists of three interconnected information pillars based on appropriate technologies 16 Copyright © 2014, 9sight ConsultingCopyright © 2014, 9sight Consulting
  17. 17. Copyright © 2014 9sight Consulting, All Rights Reserved Dr Barry Devlin Founder & Principal 9sight Consulting Thank you Questions? 17

×