Data Driven Insights – An Introduction
Need of Insights for Business: Business leaders and analytics teams are looking to derive
meaningful actionables from the business data, on the go.
Conventional BI tools wont work: MIS/BI reports or only visualizations will not help the
cause – what is needed is an ability to get data-driven insights which can help them get the
pulse of the business and suggest meaningful action points.
Key is AI and Machine Learning: Data driven insights are the fuel for every business
decision being taken. Rather than relying on an analyst to get insights, business users can
get insights directly from Robo-data-scientists using AI and machine learning.
A point of view can be a dangerous luxury when substituted for insight and understanding
Marshall McLuhan, Canadian Communications Professor
Source: Genpact report on Data-to-insight
The critical aspect in the Analytics function
which is often neglected is the Data-to-
insight and Insight-to-action journey.
1. Provide Visibility – Descriptive
2. Manage effectiveness – Insighting on
3. Execute actions – Prescriptive
‘He who searches for pearls must dive below’ – John Dryden
How will we prefer to infer data?
‘Once we know something, we find it hard to imagine what it was like not to know it’
Chip & Dan Heath, Authors of Made to Stick, Switch
Some insighting illustrations
Opportunities multiply as they are seized – Sun Tzu
Solving the problem of automated insighting
Break insighting into
several sub problems Analyse data for each sub-
problem Do you care: Are the
Is there a seasonality of trend?
Are KPIs related to any factors strongly?
Are factors related to each other?
Is there any interaction effect?
Are there sub-trends in Factors?
What are the causal relationships?
Analysing each part: Deep data-mining
A note on Mutual Information
› Variance gives dispersion in normal distribution
› Entropy gives a measure understand dispersion for any distribution
› 𝐻 𝑋 = 𝑋 𝑝 𝑋 𝑙𝑜𝑔
› Mutual information is a measure of the mutual dependence between the two variables.
› 𝐼(𝑋; 𝑌) = 𝑥,𝑦 𝑝 𝑥, 𝑦 𝑙𝑜𝑔
› Point-wise mutual information is similar to mutual information, but it refers to a single
event whereas mutual information is the average of all possible events.
Is the insight statistically
Are the underlying variables
important for the user?
This is where use of machine
learning becomes important to
identify the right set of insights,
curate them and present them in
an automated fashion
Narrating Insights: Natural Language Generation
Keep the language
simple and direct
Take care of
Make it interesting
Use right ajectives
Insights to automated decision making
should also give
out insights in
"level":"Vidarbha & Chattisgarh“
"insight": "Total PARValue has increased over one month
for WB & OR by 27.0% not changed muchVidarbha &
Chattisgarh by 0.0%", "Total PARValue has overall
decreased however comparatively lowest fall over one
quarter for Vidarbha & Chattisgarh by -27.5% and
significantly decreased for East - UP by 120.4%",
"campaign_text": "Focus on Vidarbha & Chattisgarh"