Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Self-Service Analytics on Hadoop: Lessons Learned

1,097 views

Published on

Self-Service Analytics on Hadoop: Lessons Learned

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Self-Service Analytics on Hadoop: Lessons Learned

  1. 1. Self-Service Analytics on Hadoop: Lessons Learned June 29, 2016 Drew Leamon Director – Advanced Technology Solutions
  2. 2. Comcast: Shaping the Future of Media and Technology High Speed Internet Video IP Telephony Home Security / Automation Universal Parks Media Properties
  3. 3. Forecast Engineering Design Budget Engineering Analysis: Global Central Analysis Team
  4. 4. Animals are Best Suited in Their Native Habitat
  5. 5. Spreadsheets: The Natural Habitat of Analysts
  6. 6. Evolution of Self Service Analytics SSRS
  7. 7. Self Service: Native Habitat Limitations of the Spreadsheet Native Habitat • 1 Million Row Max Self Service • Not Even Medium Data • Not Collaborative • No Automation • Not Repeatable IT Analyst
  8. 8. Self Service: How We Started Analyst goes to IT, makes request, waited weeks to get results SSRS • 10 TB Storage • 1 Compute Node Not Self Service • 10 TB (Medium Data) • Limited Compute • IT Hand-off • Consultative service • Not self service. IT Analysts
  9. 9. Bigger database still meant building dashboards for team IT Analysts Still Not Self Service • 100s TBs (Large Data) • Data silos • IT Hand-off • Consultative service • Analysts not SQL experts Graduated to Specialized Databases • Clustered Storage • Columnar Compression • Clustered Compute
  10. 10. Datameer, native on Hadoop, enables self-service for big data Analysts True Self Service • PB == Big Data • Data Lake • Excel-like UI • No more waiting for IT Self Service: The New Way • Clustered Storage • Columnar Compression • Clustered Compute • Liberated Data
  11. 11. 11 Multiple Configurations for Big Data
  12. 12. 12 Engineering Analysis IP Telephony Video Research IP Video Engineering X1 Operations Advanced Advertising Web Analytics Enterprise Business Intelligence Network EngineeringMature Evolving On-Boarded On-Deck Expanding Use Cases with Datameer
  13. 13. Use Case #1: Comcast Digital Voice
  14. 14. One Of The Largest IP Telephony Networks
  15. 15. Anonymized Call Detail Records (CDR) Data Set Data complexity from network Data size: TBs/month
  16. 16. Discovered Unusual Patterns Noticed large spikes for high cost areas
  17. 17. Hypothesis: Network Abuse
  18. 18. 30% of this traffic was coming from three accounts. Analysis Shows Traffic Concentration Few Accounts
  19. 19. Ongoing Monitoring of Future Abuse Analyst Scheduled a Tableau Data Extract and built a Tableau dashboard - Now the business can keep an eye out for further abuse.
  20. 20. Result: Future Abuse Prevented and More Abuse detected Analysts empowered Resources saved No IT hand-off Value to organizationAutomated and repeatable
  21. 21. 21 Engineering Analysis IP Telephony Video Research IP Video Engineering X1 Operations Advanced Advertising Web Analytics Enterprise Business Intelligence Network EngineeringMature Evolving On-Boarded On-Deck Expanding Use Cases with Datameer
  22. 22. Use Case #2: Customer Perspective How to measure customer experience from the customer perspective 22
  23. 23. 23 Millions of Viewing Experiences
  24. 24. Improved Customer Experience through Data Analytics 24 Findings / Analysis Best Practices Improved Customer Experience Data driven scheduling Dataflow Automation
  25. 25. Solution: 25 - Build views quickly & aggregate large datasets. - Early visibility of data in Hadoop - Create repeatable processes through automated workflow • Aggregations of large datasets from disparate data sources. - RDBMS, HDFS, APIs • Data Joins / Data Quality Checks / Pipeline between clusters
  26. 26. Result: Data-driven Customer Viewing Experience Enhancements 26 Customer Experience Improved Analysts empowered Capital Spend Directed Intelligently No IT hand-off Value to organizationAutomated and repeatable

×