This session was recorded in San Francisco on February 5th, 2019 and can be viewed here: https://youtu.be/4a_Y0L7suBc AI is real. Enterprises use it to automate decisions, hyper-personalize customer experiences, streamline operational processes, and much more. However, for most enterprise technology leaders, AI technologies and use cases are still far too mysterious. The field is moving fast. Enterprise leaders must forge a coherent, pragmatic AI strategy that is tied to business outcomes. In this session, guest speaker Forrester Research Vice President & Principal Analyst Mike Gualtieri will demystify enterprise AI, identify use cases most likely to succeed, and, most importantly, provide key advice to enterprise leaders that are charged with moving AI forward in their organization. Bio: Mike's research focuses on software technologies, platforms, and practices that enable technology professionals to deliver digital transformations that lead to prescient digital experiences and breakthrough operational efficiency. His key technology coverage areas are AI, machine learning, deep learning, AI chips and systems, digital decisions, streaming analytics, prescriptive analytics, big data analytical platforms and tools (Hadoop/Spark/Flink; translytical databases), optimization, and emerging technologies that make software faster and smarter. Mike is also a leading expert on the intersection of business strategy, artificial intelligence, and innovation. Mike provides technology vendors with actionable, fine-tuned advisory sessions on strategy, messaging, competitive analysis, buyer-persona analysis, market trends, and product road maps for the areas he directly covers and adjacent areas that wish to launch into new markets or use new technologies. Mike is a recipient of the Forrester Courage Award for making bold calls that inspire leaders and guide great business and technology decisions.

1. Making AI Happen Without
Getting Fired
Mike Gualtieri, VP & Principal Analyst
5 February 2019

3. AI is set to dominate enterprise innovation
for many years to come.

4. 4© 2019 FORRESTER. REPRODUCTION PROHIBITED.
“AI” implementation gains momentum
Q.A13: What are your firm's plans to use the following analytics technologies?
(Artificial intelligence)
Base: 2094, 2106*, 1742 + data and analytics decision-makers
Source: Forrester Data Global Business Technographics® Data And Analytics Survey, 2016, 2017, 2018
No to AI
Yes to AI
6%
29%
25%
40%
7%
23%
20%
51%
8%
19% 20%
53%
Don't know Not interested/no immediate
plans
Planning to implement within
the next 12 months
Implementing, implemented or
expanding2016 2017* 2018+

5. 5
ADVICE
FORRESTER
Set proper expectations for AI
by using the right definition.

6. Forrester recognizes two types of AI:
Pure and Pragmatic.

7. Pure AI strives to imitate comprehensive
human intelligence…

8. Pragmatic AI is narrower in scope, but often
exceeds human intelligence.

9. 9© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Source: Forrester Research, Inc.
Pragmatic AI
Nope!
Huge pre-
requisite
Yep!

10. 10© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Pragmatic AI is not one technology. It is comprised
of one or more building block technologies.
PhysicsDataHumans

11. Machine learning algorithms analyze data to
models that make predictions, take
decisions, or identify context.

12. Focus on pragmatic
AI powered by
machine learning.

13. I shall.

14. 14
ADVICE
FORRESTER
Choose more than one
high-ROI use case.

15. There are as many use cases as there are
business processes and customer experiences.

16. 16© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Enterprise decision makers agree that ML is applicable and
important to all aspects of the business.
Base: 353 global decision makers involved with machine learning and familiar with enterprise applications
Q1-Please rate how much you agree or disagree with the following statements
Hyper-personalization!
Operational Efficiency!
Competitiveness!
Customer experience!

17. Predict supply-chain issues when there is still
time to do something about it.

18. Prevent this dude from cyber attacking.

19. Prevent operations shutdowns by predicting
machine failure before it happens.

20. Automate inspections to optimize deployment of field
personnel.

21. Customize catering to reduce cost by 12%.

22. Find intelligence on potential investment
opportunities.

23. Image source: iStockphoto
Hyper-personalize customer experiences with targeted
offers.

24. Deep learning stoked-up accuracy on images,
voice, and natural language.

25. Diagnose disease more accurately and faster
than doctors.

26. Automatically assess damage and repair
costs.

27. Mitigate freight train slowdowns to save
~$70M per year.

28. Choose multiple AI
projects like a VC
chooses startups.

29. I shall.

30. 30
ADVICE
FORRESTER
Insist on comprehensive
access to enterprise data.

31. Garbage In = Garbage Out

32. Organizations have dozens and hundreds of
applications that generate valuable data.

33. 33© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Enterprise data is super rich - needed for successful,
pervasive machine learning
› Customer transaction data
› Point-of-sale data
› Customer and supplier
contract data
› Inventory data
› Supply chain data
› Product/service data
› ERP and manufacturing data
› Supplier transactions
› R&D data
› Sales and CRM data
› Marketing/advertising data
› Human resources data
› Finance/accounting data

34. Algorithms get all the
press, but it is the
data that leads to
success.

35. Understood.

36. 36
ADVICE
FORRESTER
Go faster with Auto-ML.

37. Data scientists aren’t expensive. They are
inefficient.

38. 38© 2019 FORRESTER. REPRODUCTION PROHIBITED.
The ML model building lifecycle is highly iterative and
continuous.
Model training
using ML
algorithms
ML
models
Big data ingestion,
processing and
preparation
Model
scoring/inferencing
in applications
Retraining on
new data
Production
Development

39. 39© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Machine learning solutions fall under three market
segments
Multimodal Notebook-based
Widest breadth of
workbench tools
Code-first
workbench for R,
Python, etc.
Automation-
focused
Designed specifically
for automation

40. Auto-ML solutions dramatically compress the model
building lifecycle: feature engineering, algorithm
selection, evaluation, and tuning.

41. Data-savvy users can build machine learning models
for many business-worthy use cases.

42. Add Auto-ML to your
repertoire of machine
learning model
development tools.

43. Yessir.

44. 44
ADVICE
FORRESTER
Know when to quit.

45. Machine learning is not guaranteed to work…

46. …that’s why you identify more than one potential
use case.

47. If the data doesn’t fit,
you must quit.

48. Roger that
big guy.

49. 49
ADVICE
FORRESTER
Keep production models
fresh.

50. ML models are probabilistic.

51. 51© 2019 FORRESTER. REPRODUCTION PROHIBITED.
ML model performance can decay over time
Positive
Minimum
Business Value
BusinessValue
Time deployed
Negative
Maximum
Business Value
Model
Model
Model
= probabilistic model

52. 52© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Dev-developed code always runs as written
PositiveBusinessValue
Time deployed
Negative Code Code
= deterministic code
Code

53. 53© 2019 FORRESTER. REPRODUCTION PROHIBITED.
ML models must be monitored, retrained, and often
remodeled.BusinessValue
Time deployed
PositiveNegative
Maximum
Business ValueModel
Model
Model
= probabilistic model
Negative
Business Value

54. Model operations is protection against
undesirable results.

55. Model staging because the real evaluation is on
production data.
A B

56. Monitor monitoring to make sure the model is
still performing as expected and not doing harm.

57. DevOps collaboration to have a repeatable model
deployment process.

58. 58© 2019 FORRESTER. REPRODUCTION PROHIBITED.
The MLOps Engineer role acts to usher models safely
within the entire lifecycle
MLOps
Engineer
DevOpsApp Dev
Data Science
App Design
Data Engineer
Business
ModelOps
ModelOps
Ops

59. Integrate ModelOps
with DevOps.

60. Aye, aye
sir.

61. 61
ADVICE
FORRESTER
Get your business and IT
ducks in a row.

62. A machine learning model is only successful if it
is deployed.

63. ML models impact business process.

64. Ml models impact application design and development.

65. Engage with
business and app
dev early.

66. Easier said
than done,
but…ok.

67. 67
ADVICE
FORRESTER
You don’t have to do what the
model tells you to do.

68. Govern rules using
digital decisioning.

69. I shall.

70. A few models here and there are valuable and
significant, but they are a mere drop in the
bucket compared to what is possible.

71. 11001001101
0100100
010011001
010
Historical
Transactions
Customerdata
Security
Data infrastructure must meet the unique
requirements of machine learning.

73. AI is the fastest growing workload on the
planet earth.

74. Ten percent of firms leveraging AI will use
digital decisioning to enhance and govern
AI models.
1. Set proper expectations for AI.
2. Choose more than one use-case.
3. Insist on comprehensive data
access.
4. Go faster with auto-ML.
5. Know when to quit.
6. Keep production models fresh.
7. Get business and IT engaged early.
8. Ignore the model to protect the

75. FORRESTER.COM
Thank you
© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Mike Gualtieri
mgualtieri@forrester.com