Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DI&A Slides: Data Insights and Analytics Frameworks

1,570 views

Published on

This webinar will provide an overview of the standard architecture components needed to perform analytics and derive data insights from within each of the three common database environments. This will include the sandbox environment for initial data assessment and data science modeling, the big data environment for batch analytics that includes critical governance components and the real-time analytics environment for real-time retrieval of data, and lastly, the integration of real-time data sources.

We will also discuss:

- Components for the data scientist sandbox / lab
- Batch analytics with security and metadata
- Data pipelines
- Real-time access and streaming sources

Published in: Technology
  • Be the first to comment

  • Be the first to like this

DI&A Slides: Data Insights and Analytics Frameworks

  1. 1. The First Step in Information Management www.firstsanfranciscopartners.com Produced by: MONTHLY SERIES Brought to you in partnership with: January 5, 2017 Data Insights and Analytics Frameworks
  2. 2. Welcome to the new series § The purpose of our new series is to: − Grow understanding on Data Insights and Analytics − Cover the ins and outs of the Big Data, Analytics, Business Intelligence and reporting universe − Focus on practical, realistic, value − Want to bypass fluff, hype and false promises − Need your feedback − Use Q & A pg 2© 2017 First San Francisco Partners www.firstsanfranciscopartners.com Data Lake vs. Data Warehouse Descriptive, Prescriptive and Predictive Analytics Governing Quality Analytics The Role of a Data Scientist (Interview with a CDS) Analytics, BI and Data Science: What’s the Progression?
  3. 3. Topics for today’s webinar Frameworks defined Enterprise analytics architecture Overview of standard data insights and analytics components Big Data Sandbox Real-time Analytics From Legacy architectures to data insight Key takeaways Q&A pg 3 Frameworks defined Enterprise analytics architecture • Big Data • Sandbox • Real-time Analytics • Legacy Architectures Overview of standard data insights and analytics components Key takeaways Q&A © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  4. 4. Frameworks defined ▪ The structure for delivering and getting value out of your data and an architecture for decision-making, including organizational models ▪ Your enterprise analytics architecture needs to reflect holistic thinking ▪ Your starting point and business needs determine how you progress, not a pre- defined curve ▪ Organizations that can barely deliver an accurate production report are doing predictive analytics – Right? Wrong? pg 4 Predictive Managing Proactive Operating Data Insight and Analytics Maturity © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  5. 5. The Data Insight and Analytics Framework Data Insight and Analytics Framework Data Insight and Analytics Strategy Technology Infrastructure Data Insight and Analytics Operating Model Components “Best Fit” Data Architecture Data Quality Demand Management Presentation Data Wrangling Metadata Management GOVERNANCE ORGANIZATIONAL ALIGNMENT pg 5
  6. 6. Sample enterprise analytics architecture ODS* DM Big Data* Sandbox Security and governance Presentation (visualization, reports, algorithms, queries) Data ingestion Operations scheduling, management, Data Quality, Controls Sources * = Includes Real TimeETL and Movement DW* Metadata Data Lake pg 6© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  7. 7. ODS* DM Big Data* Sandbox Security and governance Presentation (visualization, reports, algorithms, queries) Data ingestion Operations scheduling, management, Data Quality, Controls Sources ETL and Movement DW* Metadata Data Lake Enterprise analytics architecture – Big Data § More than the Big Data “stack” § No longer linear – Production to Access § Arranged by latency, access, intended value, data velocity, data volume and data movement capacity 1. Standard “Big Data” 2. Sandbox 3. Real time 4. Heritage pg 7 1 2*3 4 © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  8. 8. Big Data § Components − Data Sources − Ingestion − Structuring − Analytics and Visualization − Metadata § High Priority Concerns − Metadata – Technical, Business, Lineage, Meaning, Interpretation − Security and privacy - Access and usage must be managed according to risk, permissions, policy, contractual agreements − Data Governance § Oversight of semantics, lineage, quality − Latency, access, usage § Persistent § Type of data structure – Hive, Hbase pg 8 Data Sources Ingestion and Transformation Structuring Data Analytics and Visualization Monitor Technical Metadata Business Metadata © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  9. 9. Use case – Big Data § Telecom − Analyze 500 discrete data elements including support call patterns, late or delinquent payments and other ongoing vital signs via predictive analytics − Identify “churn” prospects and take steps to prevent it § Results − 47% reduction in customer churn, protecting $15 million in revenue − Predictive analytics has spread organically to other parts of the company, including collections pg 9© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  10. 10. Sandbox § Components – Similar to Big Data − Stand alone sandbox is also a relevant component − Ingestion – Batch − Analytics and Visualization - Data Scientist/Analyst only § High Priority Concerns − Data § Discovery § Understanding § Standardization § Usefulness − Security and Privacy § Raw aspect implies no controls § Control the data, not the environment − Data Governance focus, Usage, access − Metadata, provenance and pedigree § Latency, access, usage − Sandbox equals non-persistent, non-production − Self-service − Housekeeping pg 10 Stand Alone Sandbox Data Sources Ingestion and Transformation Analytics and Visualization Monitor Technical Metadata Business Metadata © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  11. 11. Use case – Sandbox § Predictive Maintenance, Manufacturing − Collected machine sensor data − Created a sandbox environment to centralize the parts failure analysis − Combined sensor data with operational data − After finding insights, operationalize the processes in the line of business § Results − Shorter time-to-insight; It took only three weeks (from 6 months) to develop a parts failure prediction algorithm pg 11© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  12. 12. Real-time analytics § Components - similar to regular Big Data − Streaming processing an added component − Real time capability on non-Big Data as well § High Priority Concerns − Speed of ingestion § Real-time Data Ingestion − Data Streaming − Data Messaging − In-memory database for extreme low latency requirements − Security and privacy − Data Governance Focus § Metadata § Compliance − Latency, access, usage − Real-time Data Query § Enterprise Query and Reporting § Fast Query Database to store analytical results § Agents, messaging, new events § Flexible persistency and accessibility § Very low latency, high performance pg 12 Data Sources Ingestion and Transformation Structuring Data Analytics and Visualization Monitor Technical Metadata Business Metadata Streaming Data Processing Real time DW, or ODS © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  13. 13. Use case – Real-time analytics § Health − Automatically sifts through millions of posts on dozens of social media sites, local news reports, medical workers’ social networks and government websites to track instances of disease − Continually plots disease hotspots on a map § Results − Identified a cluster of “mystery hemorrhagic fever” in Guinea over a week before the Ministry of Health of Guinea notified the World Health Organization (WHO), that a day later confirmed the Ebola outbreak pg 13© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  14. 14. Relevance of legacy or traditional data insight § Legacy structures still have relevance − Reporting − Standard BI § Components − Familiar names – ETL, DW, ODS, DM − Many aspects of Big Data technology are not relevant to many data uses § Traditional Concerns − ETL vs. web services pipelines via data layer − Understanding need for traditional uses: § Departmental use § Historical reporting § Operational and ad-hoc reporting − Support of multi-latency, historical, operational, etc., requirements pg 14 ODS* DM Presentation (visualization, reports, algorithms, queries) Sources ETL and Movement DW* © 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  15. 15. Use case – Legacy or traditional data insight § Healthcare − A data warehouse to facilitate information and best practices sharing between thousands of providers and research professionals. − Also deployed predictive analytics and artificial intelligence to derive better insights from Electronic Health Records and improve patient outcome. § Results − 400,000 patient records centralized in a single data warehouse which can scale up to 20 million records. − 42% anticipated improvement in patient outcomes with Artificial Intelligence − 58% anticipated reduction in cost per unit of outcome change pg 15© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  16. 16. Key takeaways § Reference architectures are just that − You may not need a lake, data warehouse or sandbox § Avoid cobbling together technical components § Plan to match your architecture to needs and usage, vs. existing components § Web Services are an important tool − If using services, please consider a distinct data layer pg 16© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  17. 17. Q & A pg 17© 2017 First San Francisco Partners www.firstsanfranciscopartners.com
  18. 18. pg 18 Thank you and Happy New Year! See you Thursday, February 2 for Data Lake vs. Data Warehouse John Ladley @jladley john@firstsanfranciscopartners.com Kelle O’Neal @kellezoneal kelle@firstsanfranciscopartners.com © 2017 First San Francisco Partners www.firstsanfranciscopartners.com

×