Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Driven Insights: Cypher 2017 presentation


Published on

The presentation details how to generate insights from data automatically using AI.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Data Driven Insights: Cypher 2017 presentation

  1. 1. Data-driven Insights Sept 21st 2017 Presentation by: Gurpreet Singh & Gopi Suvanam G-Square Solutions
  2. 2. Data Driven Insights – An Introduction  Need of Insights for Business: Business leaders and analytics teams are looking to derive meaningful actionables from the business data, on the go.  Conventional BI tools wont work: MIS/BI reports or only visualizations will not help the cause – what is needed is an ability to get data-driven insights which can help them get the pulse of the business and suggest meaningful action points.  Key is AI and Machine Learning: Data driven insights are the fuel for every business decision being taken. Rather than relying on an analyst to get insights, business users can get insights directly from Robo-data-scientists using AI and machine learning. A point of view can be a dangerous luxury when substituted for insight and understanding Marshall McLuhan, Canadian Communications Professor
  3. 3. Data-to-Insight-to-Action journey Source: Genpact report on Data-to-insight The critical aspect in the Analytics function which is often neglected is the Data-to- insight and Insight-to-action journey. 1. Provide Visibility – Descriptive analytics 2. Manage effectiveness – Insighting on the data 3. Execute actions – Prescriptive analytics ‘He who searches for pearls must dive below’ – John Dryden
  4. 4. How will we prefer to infer data? ‘Once we know something, we find it hard to imagine what it was like not to know it’ Chip & Dan Heath, Authors of Made to Stick, Switch
  5. 5. Some insighting illustrations Opportunities multiply as they are seized – Sun Tzu Insights
  6. 6. Solving the problem of automated insighting Break insighting into several sub problems Analyse data for each sub- problem Do you care: Are the insights interesting enough?
  7. 7. Breaking down the problem Is there a seasonality of trend? Are KPIs related to any factors strongly? Are factors related to each other? Is there any interaction effect? Are there sub-trends in Factors? What are the causal relationships?
  8. 8. Analysing each part: Deep data-mining ARIMA Model Multivariate analysis Mutual Information Deeper relationships through Recursive Trees Causality test Outliers
  9. 9. A note on Mutual Information › Variance gives dispersion in normal distribution › Entropy gives a measure understand dispersion for any distribution › 𝐻 𝑋 = 𝑋 𝑝 𝑋 𝑙𝑜𝑔 1 𝑝(𝑋) › Mutual information is a measure of the mutual dependence between the two variables. › 𝐼(𝑋; 𝑌) = 𝑥,𝑦 𝑝 𝑥, 𝑦 𝑙𝑜𝑔 𝑝(𝑥,𝑦) 𝑝(𝑥)𝑝(𝑦) › Point-wise mutual information is similar to mutual information, but it refers to a single event whereas mutual information is the average of all possible events.
  10. 10. Consolidating Insights Is the insight statistically significant? Are the underlying variables important for the user? This is where use of machine learning becomes important to identify the right set of insights, curate them and present them in an automated fashion
  11. 11. Narrating Insights: Natural Language Generation Generate several templates Keep the language simple and direct Take care of grammar Make it interesting Use right ajectives
  12. 12. Insights to automated decision making Automated insighting tool should also give out insights in structured format { "insight_type": "Trend_Analysis_Variable", "sub_type": "month" "priority": ""high", "variable": "Region", "level":"Vidarbha & Chattisgarh“ "insight": "Total PARValue has increased over one month for WB & OR by 27.0% not changed muchVidarbha & Chattisgarh by 0.0%", "Total PARValue has overall decreased however comparatively lowest fall over one quarter for Vidarbha & Chattisgarh by -27.5% and significantly decreased for East - UP by 120.4%", "campaign_text": "Focus on Vidarbha & Chattisgarh" }
  13. 13. Automated Insights in Action : Narrator
  14. 14. Tech Stack Columnar DB for structured data: MonetDB NoSQL DB for structured data: MongoDB Python Pandas, Numpy, Scipy, NLTK D3JS Angular
  15. 15. 15 Q & A Let data Insights lead the way forward. +91 22 43470408 Mumbai, India @company/g-square- solutions-pvt-ltd @GSquareSolution @GSquareSolutions1