Trends in analytics - Feb 2019

Trends in Analytics
Rahul Saxena
February 2019

About me
© Rahul Saxena 2February 2019
Process Re-engineering
Operations & Engineering
ERP
CRM
Analytics Systems
Analytics, Operations, & Strategy
Decision Analytics Systems
Business
Analytics
Tulane University, New Orleans (’92)
SCRA, Jamalpur (IRSME ’85)
IIT, Kanpur (’82)
Business
Architecture

Analytics is a hot topic … hotter than ERP and CRM
https://trends.google.com/trends/explore?date=all&q=%2
Fm%2F02gcn9,%2Fm%2F02jv7,%2Fm%2F02016
Interest over time: Numbers represent search interest relative to the highest point on the chart
for the given region and time. A value of 100 is the peak popularity for the term. A value of 50
means that the term is half as popular. A score of 0 means that there was not enough data for
this term.

Analytics is associated with Artificial Intelligence (AI)
https://trends.google.com/trends/explore?date=all&q=%2
Fm%2F02gcn9,%2Fm%2F0mkz,%2Fm%2F01hyh_,%2F
m%2F0bs2j8q,%2Fm%2F0jt3_q3
Interest over time: Numbers represent search interest relative to the highest point on the chart
for the given region and time. A value of 100 is the peak popularity for the term. A value of 50
means that the term is half as popular. A score of 0 means that there was not enough data for
this term.
Data Scientist: The Sexiest Job of
the 21st Century, HBR Oct 2012
Apache Hadoop 1.0
Nov 2011

How Google uses analytics
Google's mission statement is “to organize the world's
information and make it universally accessible and useful”
… and its users’ online activities are monetized into ad
revenue
Google organizes
data, makes it
accessible,
collects user data
Google analyzes
data and locates
ad opportunities
Google places
ads for its
customers

TRADITIONAL BUSINESSES
• Advertising
• Marketing
• Sales
Google provides decision intelligence for advertisers
COST
Place ads via
Google
Get sales from ads
on Google
Google organizes
data, makes it
accessible,
collects user data
Google analyzes
data and locates
ad opportunities
Google places
ads for its
customers
COST REVENUE
DATA INSIGHT DECISION EXECUTION

Amazon and Netflix go from data to execution, in-house
Google
Facebook
Amazon
Netflix
Traditional
Businesses
INTENTION to INSIGHT INSIGHT to VALUE
Google organizes data,
makes it accessible,
collects user data
Google analyzes data and
locates personalized ad
opportunities
Google places ads for its
customers
Get sales from Google ads
Facebook provides a
social media platform,
collects user data
Facebook analyzes data
and locates personalized
ad opportunities
Facebook places ads for
its customers Get sales from Facebook ads
Amazon provides an
ecommerce platform,
collects user data
Amazon analyzes data
and determines
personalized offerings
Amazon customers buy
more and more from
Amazon
Amazon gets a large and
growing share of retail
business
Netflix gives access to a
lots of movies and TV
shows for most devices
Netflix provides
personalized suggestions
to customers
Netflix customers act on
it’s suggestions for
continued watching
Netflix gets a large and
growing share of the
content business

Google
Facebook
Amazon
Netflix
Traditional
Businesses
Success in analytics gets a reaction
Privacy!
Manipulation!
Disruption!
Death of
traditional retail!

Trends in Analytics for 2019
Data Security
Privacy & Digital
Ethics
Collaborative
Intelligence &
Cobots
Autonomous
Vehicles
Robotics
Dark Data, Data
Prep & Cross-
Linking
Predictive
Analytics
Artificial
Intelligence
Internet of Things
(IoT)
Data Quality,
Traceability &
Trust
Surveillance
Machine Learning

Trends in Analytics for 2019, arranged into categories
Data Security
Privacy & Digital
Ethics
Collaborative
Intelligence & Cobots
Autonomous
Vehicles
Robotics
Internet of Things
(IoT)
Surveillance
Predictive Analytics
Artificial IntelligenceMachine Learning
Data Quality,
Traceability & Trust
Dark Data, Data Prep
& Cross-Linking

The underlying trends in Analytics
Better decision
algorithms
More analytics
success stories
More industry
disruption
More data sources
Better/new analysis
(same/new use)
More privacy and
ethical concerns
More quality
concerns
More data
vulnerability
Either new entrants or industry
competitors will use analytics for
“disruptive” competitive advantage

Decision algorithms already rule the financial markets
Post-disruption … business as usual

Some recent success stories for analytics
Shell built an analytics platform based on
software from several vendors to run predictive
models to anticipate when more than 3,000
different oil drilling machine parts might fail
Airlines Reporting Corp. (ARC), settles $88
billion worth of airfare transactions between
airlines and travel agencies. ARC builds
custom reports for its customers, which it’s
migrating from a Teradata data warehouse to
Snowflake on AWS.
TD Bank’s data analytics team spent several years creating an
enterprise Hadoop data lake, giving business analysts the ability to
pull data from the data lake and make it usable and actionable.
Cargill’s animal nutrition unit developed
iQuatic, an app that helps shrimp farmers
reduce the mortality rate of their yields. The
app predicts biomass in shrimp ponds based
on temperature, pH and nutrition, and works
in concert with Cargill’s iQuatic automated
shrimp feeding system.
Dr. Pepper Snapple Group’s sales staff are armed with
iPads that tell them what stores they need to visit, what
offers to make, and other crucial metrics. Algorithms show
staffers how they are executing against their expected
projections, including whether they are on track to meet
their plan and how to course-correct if they are not.

Shell optimized 3,000 SKUs stock at ~50 locations
◼ Analyze demand patterns
◼ Create better stocking recommendations
◼ Simulation model (Monte Carlo), run on a 50
node Apache Spark™ cluster
Shell built an analytics platform based on
software from several vendors to run predictive
models to anticipate when more than 3,000
different oil drilling machine parts might fail

Cargill’s iQuatic decision support for shrimp farmers
Cargill’s animal nutrition unit developed iQuatic, an app that helps
shrimp farmers reduce the mortality rate of their yields. The app
predicts biomass in shrimp ponds based on temperature, pH and
nutrition, and works in concert with Cargill’s iQuatic automated
shrimp feeding system.
Cargill’s iQuatic
automated shrimp
feeding system can
execute the feed advice
On-farm data:
dissolved oxygen, pH,
animal size, etc.
Weather and market
data
Operations dashboard
for pond status and
shrimp growth
Determine trends
Track actuals vs. plan
Feeding control
Farm management
and production
planning

Types of analytics systems
ARC Snapple
Shell CargillTD Bank
Data Exploration
Dashboards &
Reports
Decision Support
Systems
Decision Cycle
Systems

You go from data to value in four stages
Stage 1
Data Exploration
Stage 2
Dashboards &
Reports
Stage 3
Decision Models
Stage 4
Decision Cycles
Data
Value from
better decisions

The typical situation with analytics systems
The analytics squeeze
Stage 2
Dashboards &
Reports
Stage 3
Decision Models
Data
Value from
better decisions
Data exploration is
just a step in a project,
no owner for rapid
innovation
Data quality, usage &
governance issues
No decision
support process
Hundreds of metrics, reports
& dashboards
(an information flood)
• Cost overruns,
rework, delays
• Operations failures,
latency, firefighting
• Adoption gaps,
errors
• What is the return on our
analytics investment?
• Why does analytics take so
long? We need speed!
• Help us trust the data, make
data quality predictable
Hero model: analytics leaders
work with business counterparts
to help them use analytics
Lack of adoption
in decision making

Add analytics capabilities
Solve the squeeze
Fast & cheap
→ fail-fast
Data Services
on a
Big Data Infrastructure
Stage 1
Data Exploration
Stage 2
Dashboards &
Reports
Stage 3
Decision Models
Stage 4
Decision Cycles
Data
Value from
better decisions
Fast & cheap
→ fail-fast
Unshackle
innovation
Process to drive
analytics to decisions
to outcomes
Enterprise-wide data
with live metadata &
predictable quality
Focus on decisions &
execution instead of a
flood of metrics, drive
adoption, track ROI
Migrate workloads to
Stage 1, Stage 4, and
Data Services
Big Data technologies
reduce data
management overload
Reliable
performance
Visible &
shareable
innovation
Extract value from
earlier investments
by tying them to the
Decision Cycle

Different systems for each stage of analytics
Data Services: usage, storage, quality, and supply
Data Exploration Dashboards & Reports Decision Support Systems Decision Cycle Systems

Analytics hits the knowing/doing gap
❖Knowing what to do ❖Doing it
Good insights hit the knowing/doing gap … when the insights
can be directly connected to guiding a decision, but aren’t
If the data-to-insights capability is built without targeting decisions, analytics
deliverables are generated without any idea of how they can be used … we
hit the “useless analytics” gap

Would you trust Alexa or Siri to
◼ Order dinner
◼ Select a doctor
◼ Pick your life partner

Would you trust an algorithm to
◼ Place ads on Facebook
◼ Prioritize leads
◼ Select candidates to hire
◼ Determine job ratings and bonus payouts
◼ Decide product portfolio priorities and budget allocation

Algorithms beat human experts
Philosophy of Science, 69
pp. S197–S208
September 2002 …
we’re still “working
on it” now
People have to contend
with deep-seated biases
List of cognitive biases
https://en.wikipedia.org/wiki/List_
of_cognitive_biases

Improving decisions with machine intelligence
❖Decision list
❖Decision rights
❖Decision methods
o Machine Intelligence Assesses,
Decides & Executes
o Airline ticket prices
o Facebook ad placements
o Amazon recommendations
etc.
o Machine Intelligence Assesses
o Humans Review, Decide &
Execute
Collaborative decision algorithms
• Provide clarity on decision objectives & constraints
• Help people to trust the decision algorithm
• Set decision criteria & weights
Where people aren’t ready to hand over the entire
decision to an algorithm, we can apply collaborative
decision-making, in which people work with algorithms
to get buy-in, make better choices, and reduce the
effects of cognitive biases
Some decisions have already been automated
(i.e., handed over to algorithms), because
people cannot make them as fast and
accurately
All decisions can be listed, and
we can determine who’s involved
in taking each decision (decision
rights), and how the decision will
be arrived at (the decision
method).
For instance, supply chain planners are provided advice that they
can accept or decline, the algorithm doesn’t enforce the best plan

Machine intelligence can be measured and improved
ADVISE
ADAPT
Machine intelligence starts with assisting humans to take a decision, by taking
into consideration the same decision factors humans should use to generate
assessments and guidance. They convert decisions into specified methods,
providing a systematic institutional brain that generates advice.
The results of the decisions are measured and the decision algorithms
scientifically criticized. The algorithms that do these assessments can be termed
“hindsight algorithms”, that illuminate the effects of the decisions and help correct
course. They institutionalize the ability to learn from experience: to adapt.
A machine intelligence
• Relentlessly checks for gaps and opportunities, helps prioritize
• Provides advice based on best practices
• Adapts based on experience

An analytics system for used car sales by dealers
ADVISE
ADAPT
Machine intelligence assesses
the advice, execution and
results to determine the most
effective strategies for pricing
and promoting used cars and to
tune the decision algorithms as
the market changes
Machine intelligence provides
advice on price and promotion
adjustments for used car
inventory
Sales managers review
the advice, change prices
and enable promotions
DECIDE
Sales managers aren’t ready to hand over the entire
decision for pricing and promotions to an algorithm,
because each used car and each dealership is unique.
So we deploy collaborative decision-making, with
algorithms that generate and communicate advice on
pricing and promotions to sales managers.DATA
• Leads (internal)
• Sales (internal)
• Inventory (internal)
• Market intelligence
Tune the
Decision Advice
Algorithm

A general depiction of the analytics system
ADVISE
ADAPT
• Exceptions
• Grace
• Mercy
• Insights:
Intention to
Advice
• Decision &
Execution
Support
• Introspect:
Intention to
Results
• Adaptation
(Learning &
Course
Correction)
• Objectives
• Constraints
• Process
Hindsight algorithms assess the
decision process (from intention
to results), driving continuous
improvement in the system
Machine Intelligence applied to
decision support and execution,
spanning the spectrum from
automated decisions to
collaborative decisions
Humans provide purpose,
direction, alignment, and
empathy
DECIDE

Understand how analytics technology works
◼ No omniscience: no system is fully instrumented, not everything is measured and
fed into databases for analysis
◼ No prescience: no algorithm can perfectly forecast the future except in trivial cases
◼ No wisdom: no algorithm can always determine the best path in advance (it can
select the best available option based on its current data); introspection and
adaptation (in hindsight) is required to check the results against intentions and
provide course corrections
The design of decision support tools faces a talent bottleneck, as Operations Research and
Decision Analysis experts are hard to find. Then comes the task of getting people to use the
decision algorithms. Even when analytics insights are available, we often find a gap between
knowing and doing. The skills needed to bridge this gap are getting built up and will lead to a
new set of professionals possibly called Decision Coaches (ref. Dr. Steven Barrager),
Analytics Translators (ref McKinsey), or Decision Advisors.

Decisions are expertly enabled and scientifically
improved with analytics technology
Human intelligence assesses information, humans use their minds to
review, decide & execute … decision models are not made external
and explicit, decisions remain implicit and person-dependent
Decision models are made explicit and systematic,
so decisions can be subjected to the scientific
method of evaluation and improvement
Evaluation and improvement of
decision models is also made
systematic
ADVISE
ADAPT
INFORM
Make better decisions
by leveraging expertise
embedded in a system:
a decision algorithm
Ensure that the
decision algorithm is
continually tested and
improved
1
2

The machine intelligence becomes increasingly useful
Machine intelligence assesses, humans review, decide & execute …
humans learn to trust the machine, and teach the machine to make
better assessments
Machine intelligence assesses many decisions …
humans use the machine to help in many decisions
Machine intelligence used for all major decisions
… people rely on it to manage the organization

Summary
Better decision
algorithms
More analytics
success stories
More industry
disruption
More data sources
Better/new analysis
(same/new use)
More privacy and
ethical concerns
More quality
concerns
More data
vulnerability
Data Exploration Dashboards & Reports
Decision Support
Systems
Decision Cycle
Systems
ADVISE ADAPT

Trends in analytics - Feb 2019

More Related Content

Similar to Trends in analytics - Feb 2019

Recently uploaded

Trends in analytics - Feb 2019