Impact of big data on analytics
 

Impact of big data on analytics

on

  • 2,364 views

What is the impact of Big Data on Analytics from a Data Science perspective.

What is the impact of Big Data on Analytics from a Data Science perspective.

Presented at the Big Data and Analytics Summit 2014, Nasscom by Mamatha Upadhyaya.

Statistics

Views

Total Views
2,364
Views on SlideShare
1,686
Embed Views
678

Actions

Likes
6
Downloads
66
Comments
0

4 Embeds 678

http://www.content-loop.com 668
https://twitter.com 8
http://www.slideee.com 1
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Impact of big data on analytics Impact of big data on analytics Presentation Transcript

  • Impact of Big Data on Analytics Mamatha Upadhyaya
  • 2 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Big Data and Analytics Summit 2014 The terms Big Data and Analytics are used simultaneously
  • 3 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya However, analytics, predictive modeling, advanced analytics, data science is not new!
  • 4 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya So what does this mean for Analytics? Yes, the amount of data that is available to us is exploding And Big Data Platforms and Commodity Hardware and bringing in additional capabilities So what does this mean for Analytics? Media is rife with Big Data and Analytics AND The Data Scientist makes it from Nerd to the most cool person!! …makes it to on top of CIO agenda Big Data and analytics is touted as the panacea for all problems
  • 5 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Data Science perspective – A Data Science perspective Big Data and AnalyticsImpact of
  • 6 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya A brief history of Data Science Pre 1800s 1800-1900 1900-1940 1940-1960 1960 1970 1980 1990 2000 2010  Text/ string search  1974 Peter Naur “Concise Survey of Computer Methods”, Data Science, Datalogy  Knuth – Art of Computer Programming.  1976 – SAS Institute  1977 The International Association for Statistical Computing (IASC). Computer Science Data Technology Visualization Mathematics/ OR Statistics  Probability  Correlation  Bayes Theorem.  Regression, Least Squares  Time Series.  Theoretical Foundations of Modern Stats  Hypothesis, DOE  Mathematical Statistics.  Bayesian Methods  Time Series Methods (Box Cox, Survival, etc.)  Stochastic Methods.  Simulation, Markov  Computational Statistics.  Decision Science  Pattern recognition  Machine learning.  Liebniz – Binary Logic.  Babbage, Lovelace  Boolean Algebra  Punch cards.  Turing machines  Information Theory  Weiner & Cybernetics  Von Neumann Architecture.  Calculus  Logarithms  Newton-Raphson.  1989 First KDD Workshop  Gregory Piatetsky-Shapiro.  Sort & Search Algorithms – Dijkstra, Kruskal, Shell Sort, …  Heuristics – Simulated Annealing, …  Graph Algorithms  Multigrid methods  Tree based methods.  Database Marketing  Data Mining, Knowledge Discovery  “Data science, classification, and related methods.”  William Cleveland: Data Science  Leo Breimann: Statistical Modeling: 2 Cultures.  Optimization Methods  Fourier and other transforms  Matrix & Generalizations  Non-euclidean geometries.  Applications to Military, manufacturing, Communications.  1962 John W. Tukey, Future of Data Analysis  Networks  Assignment Problems  Automation  Scheduling.  First IBM Computers  DBMS.  Removable Disk drives  Relational DBMS.  Desktop, floppy  SQL, OOP  High level languages.  William Playfair  Charles Minard  Florence Nightingale.  Catrography  Astronomical Charts.  John Tukey  Jacques Bertin.  Edward Tufte.  Grammar of Graphics  Word Cloud, Tag Cloud.
  • 7 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Drivers of change Data Availability Technology Ability to Handle Structured and unstructured data Platform Cost Agility Business Expectation Digital Experience Strategic Initiatives New Business Models
  • 8 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya TABLE OF CONTENTS  Data drivers  Technology drivers  So what does all of this increased activity mean to Talent!
  • 9 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya 8 TB data everyday 10 TB data everyday 152 million blogs on the internet Data availability Possible Technically but very expensive Not efficient enough to handle the amount & type of data generated by newer internet-scale technologies Big Data Internal data M&A New Tech adoption Need to access Unstructured data External data 2 billion internet users * Hortonworks CEO Rob Bearden Digital Customer IOT Legacy data management system are not designed to handle heavy demand of data consumption “85% of that data is coming from net-new data sources.” – mobile, social media, and web- and machine-generated data*… and this will increase. RFID
  • 10 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Improving traditional analytics Understand Act Measure Change in Market Share Campaign Effectiveness Revenue/ profit metrics/ reports Customer Churn/ retention reports Customer Next Best Offer Cross Sell/ Up Sell Opportunities CRM (Customer Info, billing, etc.) Subscription and Usage Summary External data (Demography, etc.) Customer Profiling  Demographic Segments  Behavior (usage/ profit/ satisfaction/ etc.) based profiles. Product Association and Product Mix Customer Profitability/ Life Time Value Product Purchase Propensity Score Targeted Retentions Strategies Existing Customer Analytics Insights CDR, IPDR data Customer Service Data Network Data Usage Based Profiling Customer Links/ Network Analysis Drivers of Satisfaction Network Performance and Service Levels New Analytics from Big Data Social Media Data and Analysis Social Media and Web Data Sentiment Analysis Social Media Influence Analysis Drivers of Sentiment Churn Scores
  • 11 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya …but it is all about business value KS Statistic Accuracy Ratio ROC Curve Gini Coefficient Implemented Cut-Off Cut-off neighborhood Shift Baseline Population Current Population NewApplicants PSI: Distributional Shift Scorecard Score Strategic KPIs Reduce Costs Regulations Compliance Increase Revenue
  • 12 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Engineering analytics: Makes it a reality with Big Data + Data Science on top of traditional mathematical models Various Data  Usage Pattern, Health Monitoring  Alarms & Control, Benchmarking  Predictive Rules for optimal recommendation from connected Assets  Failure Prediction & Root Cause Analysis  Resource Scheduling.  Design for Reliability  Benchmark data/content Mining  Reduced Order Modeling for high volume Simulation and Testing  Supplier Risk Modeling  Weight & Cost analytics. Design, Testing & Production Operation Service Manufacturer User Service Provider Analytics Engines Mfg & other Guideline Specification & Performance Benchmark Reports System Topology Financial Sensors/Telemetry – usage, operations setting, events /alarms logs, etc. Failure/ Warranty Claims Field/Technical Inspection Notes Contract/Service History Social Media and Third Party Reliability Testing/Simulation Supplier/OEM transactions Value Chain First Time Right Product Design Connected Assets, Operations Control & Predictive Maintenance Supplier Medium Volume, Low Speed, Domain Specific High Speed, High Volume and Domain Neutral Data Behaviors
  • 13 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya As more data becomes available to the Data Scientist, so does the complexity… Product Affinities Private Label Analysis Customer Purchase Patterns Campaign Response Effectiveness Shrinkage & Productivity Analysis Category Scorecard & Contribution to GM Promotion Decomposition Promotion Mix Optimization Product Price Optimization Pricing with Consumer Perception Analysis Assortments Optimization for Market Basket Analysis Shopper Trip Mission Analysis Shopper Market Basket Shopper Brand Sentiment Analysis Product Behavior Scan Adjacencies Analysis Out of Shelf Analytics Scoring of Stores/ Retail Chains Cross-Channel Order Management Inventory Optimization at DCs (SCM) Demand/ Volume Forecasting Social Impact on Category/ Brand Consumption Promotion Halo/ Cannibalization Pricing Elasticity Analysis Shopper Segmentation Shopper Demographics Shopper Loyalty Base RFM Analysis Product/ Brand Switching Trial & Repeat Category Uniqueness, Popularity Indices Category Leakage Tree Store Clustering Category/ Brand Offer Conversion Cross-Sell Up-Sell Shopper Assortment Price Promotion ProductCompetition Category Tactics DataNeeds + Other Consumer + Survey+ Social Data + Household Panel + Loyalty/ CRM Data + Syndicated + Promotions Data (IRi/ Nielsen) + POS Data + Campaign + Shipments + Public* Data Public* Data includes Weather, Census, Topography, Ordinance etc Maturity Stages
  • 14 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Availability of data is changing the way we address some traditional business problem Pharmaceutical Companies have used physical surveys to identify KOL. Big data and analytics is pioneering the way to use a data driven objective approach to identifying and monitoring KOL  Selection of right KOLs can help in better utilization of these marketing funds  A key success factor for these marketing spends is the correct methodology to identify KOLs  Managing brand perception for the key Opinion Leaders is crucial for Brand Management.
  • 15 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya TABLE OF CONTENTS  Data drivers  Technology drivers  So what does all of this increased activity mean to Talent!
  • 16 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Technology New Technologies are allowing us to manage this at a fraction of the cost & faster than ever before. Traditional DataWarehouse BIG DATA 1/30 of the cost Data does not have to be isolated in repressive silos
  • 17 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Big Data technologies are enabling a new approach Response time Volume Hadoop Data warehouses PB TB GB Hour Min Sec SubSec In-memory databases Event processing tools Real-time Applications
  • 18 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Data and technology implications… Model development  Reduced timelines • Access all data from a „data lake‟ • Data discovery and visualization tools to reduce EDA timelines • In-memory/ in-database/ high performance analytics and parallelized algorithms  Increased analytical capability • Implement techniques like Graph/ Network analysis, ensemble methods, matrix algorithms at scale • Analyze structured and unstructured data on one platform  Improved accuracy • Analyze much larger data sets • Ability to personalize for a segment of one, for e.g. targeting). Model deployment  Seamless deployment (In-database, PMML) • Decreases error in deployment  Big data deployment • Analytics on exabytes, scoring in MB/ sec  Real time deployment • Response (alert/ recommendations) in milliseconds or less  Adaptive, machine learning algorithms • “learn” and respond to recent events  Availability and velocity of data leads to change in analytical approach • for e.g. Can move from „complex algorithms for precision prediction of failure modes‟ to „real time monitoring, alerts and control processes‟.
  • 19 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Decreased response time Customer experience Information is becoming the new battleground Business expectation
  • 20 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Analytics is playing an ever important role Increased Focus on identifying the customer across all channels Segmentation to Micro segmentation to the individual Personalized Messaging and offers – Increased Individual Customer Centricity Gradual evolution of Customer Analytics Past  Customer segments who are most likely to respond to targeted campaigns for new products offers  Can tailor offers to specific to each customer segment  Mostly delivered through mass mail campaigns and in store promotions. Now  Micro segmentation  Analyze customer behavior and buying patterns across channels  Delivery through email, web, mass mail campaigns. Moving toward  Historical individual customer behavior and buying patterns across channels  Individual customer consumption pattern  In-store basket analytics  Additional dimensions Location & time  Targeted Strategies to pre-empt customers from visiting competition  Instantaneous Delivery in store or a proactive delivery via mobile to bring the customer to store. Segment to Individual to Individual @ time, place and behavior You have purchased Cheese, here are the offers on Bagels You are within 2 KMs of a store offering 50% off garden furniture Do you need coffee?
  • 21 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Analytics is only as good as the implementation… Analytics has long excelled in silos …as the amount of data and business expectation increases, this will no longer be feasible IT will move from a facilitator role to an enabler role Decreased response time will mean end to end integrations – enterprise architecture teams will need to be involved… The Data Science team will have to work along with technology teams to effectively serve the end customer
  • 22 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya much of which is outside the organization Increased availability of data Analytics as a Service and Data Monetization New service models Decreasing Time value of data!
  • 23 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya TABLE OF CONTENTS  Data drivers  Technology drivers  So what does all of this increased activity mean to Talent !
  • 24 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya Scalability and industrialization to address skill shortage Technical skills (Coding, Statistics, Math) + Perseverance +Creativity + Intuition +Presentation Skills +Business Savvy = Great Data Scientist! Key to a Great Data Scientist  Identified four Data Scientist clusters based on how data scientists think about themselves and their work, not • Years of experience, • Academic degrees, favorite tools • Titles, pay scales, org charts.  Most successful data scientists are those with substantial, deep expertise in at least one aspect of data science, be it statistics, big data, or business communication  T-Shaped Skills.
  • 25 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya …so can analytics solve all our problems Help us acquire customers Product Recommendation engine Solve World Hunger Crop Sciences Keep us fit! Catch the bad guys Numbers Win FIFA “German national football team uses real time analytics for a competitive edge” Get you married! Dating sites, Matchmaking Analytics in Healthcare
  • 26 Big Data & Analytics Copyright © 2014 Capgemini. All rights reserved. Impact of Big Data on Analytics | Mamatha Upadhyaya So what is it Big Data and Analytics cannot do!!!
  • The information contained in this presentation is proprietary. Copyright © 2014 Capgemini. All rights reserved. Rightshore® is a trademark belonging to Capgemini. www.capgemini.com/bim About Capgemini With more than 130,000 people in over 40 countries, Capgemini is one of the world's foremost providers of consulting, technology and outsourcing services. The Group reported 2013 global revenues of EUR 10.1 billion. Together with its clients, Capgemini creates and delivers business and technology solutions that fit their needs and drive the results they want. A deeply multicultural organization, Capgemini has developed its own way of working, the Collaborative Business Experience™, and draws on Rightshore®, its worldwide delivery model.