Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big data and Analytics


Published on

Information is the principle driver of competitive advantage. How it is collected, analysed and communicated determines our success. No single resource is more critical to organisational survival.

The amount of data in the world is exponentially increasing, to a point where companies capture significant amounts of information about their customers, suppliers, and operations. Millions of networked sensors are being embedded in everything from mobile phones to cars. Social networks and location data from mobile devices will continue to fuel this exponential data growth. These huge data pools are commonly being referred to as "big data".

This talk examines how analytics and big data are exploiting information to drive competitive advantage.

Published in: Technology, Business

Big data and Analytics

  1. 1. Big Data and Analytics The emerging driver of competitive advantageKevin Magee Contact detailsPartner © Open Window Analytics 2012
  2. 2. What do we do?Our clients span organisations:• looking to introduce Analytics into their business for the first time,• facing challenges leveraging the Analytics investment they have already made,• looking to bring innovative Analytics products and services to market © Open Window Analytics 2012
  3. 3. Today’s Topics• What’s Analytics All About?• The Possibilities with Analytics• A quick look at some emerging technologies• Upcoming challenges & opportunities © Open Window Analytics 2012
  4. 4. The Challenge of Information• Information is the principle driver of • Paradoxically, the greater the volume of competitive advantage. *It’s the new oil+ information we collect, the greater the prospect of uncertainty – How it is collected, analysed and communicated determines our success. – No single resource is more critical to organisational survival. – But like oil, it must first be found, extracted, refined and distributed before its value can be truly appreciated• “a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it” - Herbert Simon (1971) © Open Window Analytics 2012
  5. 5. So, what is Analytics anyway?• It’s about uncovering • Depending on who you patterns, outliers, relati talk to, Analytics onships, and other projects are: insights in data. – Technology projects – Actuarial projects• It’s part: – Machine learning – Data – Applied statistics – Information Technology – Operations research – Human powered – Business Intelligence © Open Window Analytics 2012
  6. 6. Related History Static & Interactive Query, Excel, OLAP, Dashboards, Statistics, data Reports Visual discovery Scorecards mining, Optimisation (mainstream) 1970’s 1980’s 1990’s 2000’s 2010’s• But, Analytics have been used in business since the time of Frederick Winslow Taylor in the late 19th century.• Henry Ford also measured pacing of assembly lines. © Open Window Analytics 2012
  7. 7. What’s Big Data really all about? McKinsey say: “Big Data: The next frontier for innovation, competition, and productivity”Data captured from:CustomersSuppliers Transactional records Unstructured textOperations Internet clicks RFID Geospatial GPS signals Digital multimediaSensors And more…Social networksPublic data Big DataMultimedia © Open Window Analytics 2012
  8. 8. How much data?Digital Universe IDC Digital Universe Study 2011 1.8 Zettabytes 2020 35 Zettabytes Sources: Cisco, comScore, Radicati Group, Twitter, YouTube Number of emails every second 2.9 million Data consumed by households each day 375 megabytes Video uploaded to YouTube per minute 20 hours Data processed by Google per day 24 Petabytes Tweets per day 50 million Minutes on Facebook per month 700 Billion Data sent / received by mobile internet users 1.3 Exabytes Products ordered on Amazon per second 72.9 items 1 Zettabyte = 1,000,000,000 Terabytes © Open Window Analytics 2012
  9. 9. The smartest organizations are already capitalizing on increasedinformation richness and analytics to gain competitive advantage. Top performers use analytics 5 times more than lower performers – MIT Sloan (Autumn 2010)Companies that invest heavily in advancedanalytical capabilities outperform the S&P 500on average by 64% - Accenture research 2011Companies that invest heavily in developing analytical skills andadopting an analytical mindset recover quicker from economicdownturns - Accenture research 2011 © Open Window Analytics 2012
  10. 10. The Value of Analytics © Open Window Analytics 2012
  11. 11. Analytics Landscape Competitive Advantage Optimisation Predictive What’s the best that can happen? Visual Analytics Intelligence Show me Predictive all of this... Modeling What will happen next? Forecasting What if these trends continue? Statistical Analysis Descriptive Why is this happening? Analytics Alerts What actions are needed? Query Drilldown Where exactly is the problem? Ad Hoc Reports How many, how often, where? Standard Reports What happened? Degree of©Intelligence Open Window Analytics 2012© SAS Institute, with some OWA amendments
  12. 12. Common Analytic Applications (in various industries) Retail Promotions, replenishment, shelf management, demand forecasting, inventory replenishment, price & merchandising optimisation Manufacturing Supply chain optimisation, demand forecasting, inventory replenishment, warranty analysis, product customisation, new product development Financial services Credit scoring, fraud detection, pricing, underwriting, claims, customer profitability Transportation Scheduling, routing, yield management Healthcare Drug interaction, preliminary diagnosis, disease management Hospitality Pricing, customer loyalty, yield management Energy Trading, supply, demand forecasting, compliance Government Fraud / waste / error, case management, crime prevention Online Web metrics, site design, online recommendationsAdapted from “Analytics at Work” (Davenport et al 2010) © Open Window Analytics 2012Adapted from “Analytics at Work” – Tom Davenport, Jeanne Harris, Robert Morison (2010)
  13. 13. Analysis Framework (it’s not just about what’s inside the walls…) Feedback Decisions Requirements Intelligence Operations (Estimates) (Recommendations) Primarily focused Primarily focused externally (out of Goals internally (under our control). our control). Purposes What is likely to People happen that is What can we / relevant to our should we do success or failure? about it?Framework courtesy of Kristan J. Wheaton (Mercyhurst College) © Open Window Analytics 2012
  14. 14. IT Needs to Help Businesses Run, Grow, and Transform IT Portfolio Spending Introduce new products Improve existing Sustain existing products and services products and services and services The Business ChallengeSource: Gartner 2011 IT Metrics Spending and Staffing Survey © Open Window Analytics 2012
  15. 15. Big DataPreparing Interpreting Interpreting Big Data will lead to new markets, products, and services© Open Window Analytics 2012
  16. 16. Data Supply Chain © Open Window Analytics 2012 © Open Window Analytics 2012
  17. 17. THE POSSIBILITIES(SOME EXAMPLES) © Open Window Analytics 2012
  18. 18. Reading list• Competing on Analytics (Tom Davenport et al)• Blink (Malcolm Gladwell)• McKinsey Big Data report• The New Know (Thornton May)• Analytics at Work (Davenport et al)• The Long Tail (Chris Anderson)• Visualize This (Nathan Yau)• Information is Beautiful (David McCandless)• How to Lie with Statistics (Darrell Huff) © Open Window Analytics 2012
  19. 19. Chest Pain Diagnosis• Cook County hospital in Chicago• Problem – No budgets; Cardiac care expensive to deliver; Overwhelmed ER; 2-8% of patients across US get sent home when having genuine heart attack; and lots of other problems – Wanted to figure out if there was a better and quicker way of identifying who needs care and level of care• Solution found with Analytics! © Open Window Analytics 2012
  20. 20. Less is More Traditional medical model is to take case history – gather as much info as possible – more info = better diagnosis► Cook County implemented a radical system for predicting chest pain cases that didn‘t bother with history but on 4 pieces of specific information and a decision tree. ► Turns out, less information is better in this case!► Doctors guessed right between 75 and 89% of the time.► The algorithm guessed right >95% of the time!Window Analytics 2012 © Open
  21. 21. Penny Post• Charles Babbage compared the cost of transporting mail with the cost of sorting it• He found the sorting to be inefficient and more costly• Standardising the cost of mail delivery (within a delivery and weight range) to one penny greatly reduced sorting costs.• Sir Rowland Hill introduced Penny Post to Britain based on Babbage recommendations © Open Window Analytics 2012
  22. 22. WWII Aircraft Armour Placement• Common wisdom: – Place heavier armour on parts of plane most shot up after mission• Physicist Patrick Blackett recognised data statistically biased to surviving planes – The shot down planes were the ones of interest• He reasoned: – If a part of a plane could be shot and not bring down plane, that part needed no extra armour.• Solution: – Statistical analysis on the places in common between returning planes not shot down were Make sure you are likely where shot down planes HAD been shot. analysing the right data – Therefore, where extra armour needed! © Open Window Analytics 2012
  23. 23. Another WWII Problem• Are bigger or smaller warship convoys better to protect merchant ships from U-boats?• Small convoys – eluded U-boat detection better than larger ones• Large convoys – better at counterattacking• Analysis revealed – Probability of detection did not vary significantly with convoy size – Therefore making larger convoys the most efficient size © Open Window Analytics 2012
  24. 24. Amazon’s Long Tail … a pioneer in its relentless use of Web site design testing and optimization, constantly evaluating everything, including minutiae such as the color and shape of tabs on the site ….given the volume of traffic on• Selling ‗less of more‘ Amazon.coms Web site, even a slight optimization of its design can mean millions of dollars in additional sales• Amazon is the reverse …By basing its Web site design of the 80/20 rule decisions on usage data and not necessarily on aesthetics or internal designers gut instinct, has managed to keep its user interface closely aligned with its ultimate goal, which is turning visitors into buyers Open Window Analytics 2012 ©
  25. 25. Sports, Medicine, and Cars―The New England Patriots American Footballteam have managed to win the Super Bowl three • The Veteran Administrations use oftimes in four years — using an analytical evidence-based medicine and predictiveapproach. analytics (along with automatedPatriots…renowned for their extensive study of decisions for treatment protocols)game film and statistics…. reads articles by translates into this - only 25-30% ofacademic economists on statistical probabilities of medical decisions are scientifically-football outcomes. based!The team uses data and analytical models • Honda makes good use of text analyticsextensively, both on and off the field. In-depth to flag early problems in cars byanalytics help the team select players and stay analysing warranty claims calls bybelow the NFL salary cap. customers or dealers to HQ - a great example where simple automated analysis and flagging created valueOff the field, the team uses detailed analytics toassess and improve the "total fan experience."At every home game…people have specificassignments to make quantitativemeasurements of the stadium food, parking,personnel, bathroom cleanliness and otherfactors.‖ Be specific Know your questionRead MoneyBall – about Oakland A‘s Focus on the right © Open Window Analytics 2012 information
  26. 26. Supply and Demand • Beer and Poptarts are not natural companions • Walmart, using Analytics, have found that the 2 biggest selling items during a Hurricane warning are Strawberry Poptarts and Beer • They use this information for supply chain management and goods distribution optimising sales of both • ….They sell the essential stuff too © Open Window Analytics 2012
  27. 27. Storm PlanningHome Depot Supply chain managers place orders in November, based on past stormdata, so products like gas cans, generators, and plywood are stocked in three hurricane-specific distribution centers by June. Before Hurricane Gustav in 2008, 500 trucks full ofsupplies went to distribution centers. © Open Window Analytics 2012
  28. 28. Hide n Seek• O2 mobile phone company use personalised menus to maximise value of limited phone interface - and uses predictive analytics to personalise• The decision to display a certain set of options to a mobile phone user is often hidden as companies dont think of each new list as a decision - they think of it as "the list―• Netflix is similar - giving each customer a personalised website experience based on recommendations, ratings, segmentation © Open Window Analytics 2012
  29. 29. Test, Test, Test• 5 keys to Obama Campaign Success 1. Define quantifiable success metrics. 2. Question assumptions. 3. Divide and conquer. 4. Take advantage of circumstances. 5. Turn your customers into evangelists• Based on data from analytics, a passive “Learn More” button with a static and non-Obama centric picture of a family trumped the rest of the variations• No assumptions should be made and that unique scenarios would need unique approaches © Open Window Analytics 2012
  30. 30. THE POWER OF VISUALISATION © Open Window Analytics 2012
  31. 31. Traditional way of presenting information Defence Spending Corporate Revenues Market values2009 data © Open Window Analytics 2012
  32. 32. © Open Window Analytics 2012
  33. 33. © Open Window Analytics 2012
  34. 34. LESSONS LEARNED © Open Window Analytics 2012
  35. 35. Get your thinking right• Its not all about "big data" and complex maths or even advanced software tools.• More essential in transforming your business with analytics is to: • ask the right questions of the data and then • effectively communicate the results of the analysis,• Therefore, critical thinking, and creative communication approaches are just as important (if not more so) than the technology used to enable these Open Window Analytics 2012 © insights.
  36. 36. EMERGING TECHNOLOGY © Open Window Analytics 2012
  37. 37. Big-Data Processing Systems (not a complete list – illustrative purposes only) OLTP Analytic Hadoop NoSQL Real-time Databases Platforms (MapReduce) StreamingOracle, DB2, SQL Netezza, Vertica, Cloudera, EMC, IBM, Cassandra, Storm, Hstreaming,Server, etc Exadata, Teradata HortonWorks, etc MongoDB, S4, StreamBase appliances, SAS, etc DynamoDB, MarkLogic, Attivio, etcTransaction systems EDW to replace MySQL Online data archive for Distributed system for Distributed real-time or SQL Server in fast- all data (but mostly querying unstructured + stream processingEnterprise data growing companies unstructured) datawarehouse hub Continuous Analytic data marts to Staging area to feed the Graph system for computation offload the DW DW understanding relationships Free standing analytical Analytical system when sandboxes (big data, you want to query all Key value pair storage extreme performance, the raw data (Hbase, for rapid data capture etc) Hive, Pig etc) and analysis Analytical system when Key value cache for in- you can’t wait until data memory lookups and is modelled and put in operations DW (Hbase, Hive, Pig) © Open Window Analytics 2012
  38. 38. The New Analytical Eco-systemDiagram courtesy of Wayne Eckerson • These architectures are more analytical • Give power users greater options (access & mix corporate with own data) • Bring unstructured / semi-structured data – Hadoop / nonrelational DB’s © Open Window Analytics 2012
  39. 39. SOME UPCOMING CHALLENGES © Open Window Analytics 2012
  40. 40. Challenges & Opportunities (not a complete list )• Egocentric networks • Collaborative analytics• SoLoMo intelligence • Beyond the Desktop layering • Data Management for – Social Local Mobile Analytics still a problem• Mobile BI / Analytics • The real-time challenge• Analytics in the Cloud • Relevancy & Recency• The Big Data challenge challenges• Ease of use / • Augmented analytics Consumerisation of • Consumable analytics Analytics – The next Billion users © Open Window Analytics 2012
  41. 41. ANY QUESTIONS? © Open Window Analytics 2012