Big data insights part i


Published on

Big Data insights

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Big data insights part i

  1. 1. Big Data Raji Gogulapati
  2. 2. Government • Street bump Mobile App – City of Boston initiatives • City gets real time information on “bump” data Car Insurance • More granular in pricing • Address more in depth questions Recruiting • How to hire better employees and retain? Simple solutions to Big problems Source: Phil Simon’s Big Data, too big to ignore
  3. 3. Consumer driven trends and technologies Data Science Infonomics Platforms Sandy, Politics, Social Media Big Data Influence
  4. 4. Describe Big Data Incomplete Fragmented Not precise Dynamic External Unmanageable Democratic ???
  5. 5. Data Science Ultimate Goal – Improve Decision making principles Frameworks Data analytic thinking • Extract patterns • Mining for useful knowledge • Models • Process • Stages • Assess how data can improve performance • Understand data science • See data oriented competitive threats • Question
  6. 6. Big Data Advantage – Analytics and decision management Decision making Data deluge Techniques Solutions Rewards
  7. 7. Techniques Statistical Visualization Semantics Automation Predictive Analytics
  8. 8. Plunge Issues Problems
  9. 9. Source:, Cloud Predictive Analytics most used to gain customer insight, 10/24/2013 Data types in a Big Data context
  10. 10. Big Data Use Cases – Financial Sector Source – HP Sponsored white paper, Case for Big Data in the Financial Services sector, IDC Financial Insights opinion 2012
  11. 11. Big Data Use Cases – Government Source – IBM, “Accelerate Analytics and harness Big data within government”
  12. 12. Big Data Capabilities – health care Source: McKinsey Report titled Big Data Revolution in health care, exhibit 9
  13. 13. Big Data Analytics capabilities – travel and transportation Source: IBM, Big Data and Analytics in Travel and Transportation white paper, figure 4 Maintenance and Engineering – asset management data, Spec sheets, product data Capacity and Pricing optimization
  14. 14. Data Analytics for Software Assessment/ Evaluation Source: Context and challenges Meeting milestones Documentation User base growth over time bugs Developer involvement over time
  15. 15. Analytics - Explained Source: Analytics 3.0 by Thomas H Davenport, HBR, Dec 2013 Analytics 1.0 Era of business intelligence, go beyond intuition, fact based comprehension for decision making. Era of enterprise data warehouse. Dominant for about 50 years. Analytics 2.0 From about 2005 onwards, Internet based social network firms – Google, eBay, LinkedIn..Not only internal, externally sourced, sensors, public data initiatives, multi media recordings. Innovative technologies NoSQL, Hadoop, machine learning. Computational and analytical skills Analytics 3.0 Data enriched offerings for every industry. Driven by analytics, rooted in enormous amounts of data. Co-existence of traditional and new.
  16. 16. Information Providers Insight Providers Companies Capitalizing on Analytics Essence of Analytics 3.0: “The resolve by a company’s management to compete on analytics not only in the traditional sense (by improving internal business decisions) but also by creating more valuable products and services” Analytics 3.0 by Thomas H. Davenport, Dec 2013, Harvard Business Review
  17. 17. Ability to handle new varieties of data – voice, text, log files, images, video on a large scale Sensors and operational data gathering devices in motion to optimize Cost savings of storage – data base to database appliance to a Hadoop cluster Big companies always wrestled with the data volume issues. Bigness is not new! Variety is new! What is different from the past? Source: Big Data in Big companies, May 2013:
  18. 18. Big Data Techniques – explained Sources: Data Science for Business, Chapter 2, Business Problems and Data Science solutions Too Big to Ignore by Philip Simon, Chapter 3, elements of persuasion: Big Data Techniques Techniques Statistical – Regression, A/B Testing Data visualization - Heat Maps, Time Series analysis Automation – Machine learning, Sensors, Nano technology, RFID and NFC Semantics – natural language processing, text analytics, sentiment analysis Predictive analytics Collaborative Filtering Business problems to Data Mining tasks
  19. 19. BI reporting Visualization Functional Applications Industry Applications Predictive Analytics Content Analytics Analytics solutions Source: IBM Big Data Application layer
  20. 20. Source: Information week, 16 top big data analytics platforms, 1/30/ 2014 Top 16 Big Data Analytics Platforms
  21. 21. Platform connections Business platforms, Gang of four – Amazon, Apple, Google, Facebook More businesses setting platform trends – Industry wide transformation Netflix, LinkedIn Third Platform – popularized by IDC for social, mobile, cloud, Big data/ analytics and emerging markets
  22. 22. Mainframe, Terminals Level 1 platforms ‘70s Tiered architectures (client server – 2 tier), (’80 - 90s) Multi tiered architectures (2000+ ) Social, mobile, Cloud, Big data/ analytics Convergence (2010 +) Value shifts for the enterprise
  23. 23. Big Data Optimizations – Concept Distributed optimization Parallel optimization Large scale optimizations
  24. 24. Optimizations and Challenges – know how for handling bottle necks Computational challenges
  25. 25. Myths and Overlaps Not just another hype of data related decisions and insights – requires a new mindset
  26. 26. People – roles Data Scientists Statisticians Business Analysis • Find story in a data set • Experimental, exploratory • Data mining • Statistical analysis • Predictive model development • • Multi dimensional analysis • Visual, data discovery References: Davenport, T. H., & Patil, D. J. (2012)] Harvard Business Review, October 2012, pp 70- 76
  27. 27. Big Data and Analytics technologies – supplementing RDBMS’s Scalable MPP Data warehouse Hadoop NewSQLGraph Database NoSQL Reference: WHITE PAPER Discovering the Value of a Data Discovery Platform, Sponsored by: Teradata, Dan Vesset, September 2013
  28. 28. Impact on management New skills and new management style References: McAfee, A., & Brynjolfsson, E. (2012). Big Data: The Management Revolution. (cover story). Harvard Business Review, 90(10), 60-68. Data driven companies, evidence based decisions look for opportunities based on Big data in every business function Leadership, talent, technology, Organizational culture Experimental and exploratory
  29. 29. Introductory EMC Videos – Animated Big Ideas – Simplifying cluster architectures Big Ideas - How big is Big data? Big ideas – Why Big Data matters Big Ideas – Demystifying Hadoop EMC – Big Ideas videos feature=view_all
  30. 30. Conclusions Smart Internet of things Predictions Big data evolution RFID, sensors, NFC Standards ODaF Conclusions