Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

2. @jamet123 #decisionmgt © 2016 Decision Management Solutions 2 James Taylor Decision Management Solutions ▶ We work with clients to improve their business by applying business rules & analytic technology to automate & improve decisions. ▶ Vendor-neutral ▶ Original DMN submitter ▶ Using decision modeling since 2011 ▶ I have spent the last 14 years focused on the development of analytic applications using Decision Management

3. @jamet123 #decisionmgt © 2016 Decision Management Solutions 3 Agenda ▶ Challenges Deploying Analytics ▶ Five Questions To Ensure Deployment ▶ Decision Modeling for Analytics ▶ Decision Models Answer The Five Questions ▶ Wrap Up

7. @jamet123 #decisionmgt © 2016 Decision Management Solutions 7 Challenges Deploying Analytics Analytic results are prioritized, not business resultsIneffective deployment leaves analytics on the shelf Timeliness and sustainability are not considered

8. @jamet123 #decisionmgt © 2016 Decision Management Solutions 8 Operational Systems Analytic Systems Analytics need to drive operations to add value 40%-60% of analytic models are not deployed and used Models On The Shelf Add No Value

9. @jamet123 #decisionmgt © 2016 Decision Management Solutions 9 Analytics Must be Deployed ? ? “A presentation ABOUT a model, however interesting, can’t power the website” “A 40 page report, full of math”

10. @jamet123 #decisionmgt © 2016 Decision Management Solutions 10 Business Results ▶ Too many projects focus on analytic accuracy ▶ Need to focus on business value ▶ Teams must connect their analytic to this value

12. @jamet123 #decisionmgt © 2016 Decision Management Solutions 12 Five Questions To Ensure Deployment 1. Does the model evaluate well against business objectives? 2. Who’s using the model, to decide what, where and how? 3. What will it take to make the model actionable? 4. What’s the plan for capturing and analyzing data about model effectiveness? 5. When do we need to revisit the model to make sure it is still effective?

14. @jamet123 #decisionmgt © 2016 Decision Management Solutions 14 Decision Modeling Frames Analytics ▶ Provides “structure” - who, what, how, when ▶ Standardizes approach to decision making ▶ Provides “transparency” of decision approach ▶ Fosters innovation in the business model ▶ Improves/changes the business model ▶ Promotes “buy in”

15. @jamet123 #decisionmgt © 2016 Decision Management Solutions 15 Decision Models Frame The Problem Decision Input Data Knowledge Sources Sub- decisions Analytic Source: DecisionsFirst Modeler @jamet123 #decisionmgt

16. @jamet123 #decisionmgt © 2016 Decision Management Solutions 16 Decision Models Put Analytics In Context Process Decision Motivation Organization Department Team Role Role Team Department Organization Analytic

18. @jamet123 #decisionmgt © 2016 Decision Management Solutions 18 Five Questions To Ensure Deployment 1. Does the model evaluate well against business objectives? 2. Who’s using the model, to decide what, where and how? 3. What will it take to make the model actionable? 4. What’s the plan for capturing and analyzing data about model effectiveness? 5. When do we need to revisit the model to make sure it is still effective?

19. @jamet123 #decisionmgt © 2016 Decision Management Solutions 19 1. Does the model evaluate well against business objectives? ▶ Objectives of the sponsors ▶ Measures of success ▶ For instance ▶ Reduce claims in litigation ▶ Improve the loss ratio ▶ Objectives of the modelers ▶ Measures of accuracy ▶ For instance ▶ Error rate ▶ Lift Business Measures Not Analytic Ones

21. @jamet123 #decisionmgt © 2016 Decision Management Solutions 21 2. Who’s using the model, to decide what, where and how? ▶ What is the analytic used for? ▶ Who has to use the analytic model? ▶ Where is the analytic used? ▶ How is the analytic used? ▶ The analytic is used in an explicit decision ▶ Those that make the decision ▶ The decision has a business process context ▶ The other elements of the decision are clear

23. @jamet123 #decisionmgt © 2016 Decision Management Solutions 23 Take Action With Business Rules ▶ Adding business rules to analytic models turns predictions into actions ▶ Managing sets of business rules and analytic models in a single decision flow, a single execution, encapsulate decision-making ▶ Business user find business rules approachable, increasing their comfort with analytic decision- making ▶ Business rules give them some control over what to DO about the predictions you give them

24. @jamet123 #decisionmgt © 2016 Decision Management Solutions 24 4. What’s the plan for capturing and analyzing data about model effectiveness? Decision Made Supporting Decisions Scores Business Results ▶ Decision Models show what to store

25. @jamet123 #decisionmgt © 2016 Decision Management Solutions 25 5. When do we need to revisit the model to make sure it is still effective? Data Changes Direct Outcomes Indirect Outcomes Business Results ▶ Decisions models reveal tracking needs

27. @jamet123 #decisionmgt © 2016 Decision Management Solutions 27 Five Questions To Manage Deployment 1. Does the model evaluate well against business objectives? 2. Who’s using the model, to decide what, where and how? 3. What will it take to make the model actionable? 4. What’s the plan for capturing and analyzing data about model effectiveness? 5. When do we need to revisit the model to make sure it is still effective? ▶ Decision models link analytics to decisions to business metrics ▶ Decision models show who owns, makes and cares about decisions ▶ Decision models wrap analytics in rules-based decisions ▶ Decision outcomes can be captured with input data, business results and scores ▶ Decision models clarify business context, pace of change and show data usage clearly

29. @jamet123 #decisionmgt © 2016 Decision Management Solutions 29 Decision Most Analytic Projects Leave Decisions In The Dark Modeling Decisions Brings Transparency 29©2015 Decision Management Solutions

30. @jamet123 #decisionmgt © 2016 Decision Management Solutions 30 Data Business Understanding Data Understanding Modeling Data Preparation Evaluation Deployment CRISP-DM Phases 1. Business Understanding 2. Data Understanding 3. Data Preparation 4. Modeling 5. Evaluation 6. Deployment

31. @jamet123 #decisionmgt © 2016 Decision Management Solutions 31 Specific Analytic Decision Analytic Models Overall decision to be improved Business Understanding As a Decision Model Regulations and Policies Data

32. @jamet123 #decisionmgt © 2016 Decision Management Solutions 32 Decision Models Ensure Analytic Deployment Tie the analytic to decisions and decisions to business results Specify the decision(s) that use the analytic Use the decision context to frame deployment and sustainability needs

33. @jamet123 #decisionmgt © 2016 Decision Management Solutions 33 Decision Modeling For Analytic Projects ▶ Steps 1. Training in decision modeling 2. Establish business value 3. Model effective decision-making 4. Add process, organization, business context 5. Frame and scope the analytic project ▶ Deliverables ▶ Decision Model showing impact of proposed analytic ▶ Analytic, business and IT teams trained & self-sufficient ▶ Best practices and go-forward recommendations ▶ Software, consulting and training

34. @jamet123 #decisionmgt © 2016 Decision Management Solutions 34 Decision Modeling for Analytic Projects ▶ Learn more ▶ Download the five questions brief ▶ bit.ly/5QAnalytics ▶ Ready to start? ▶ Kick off a decision modeling project today ▶ bit.ly/DMNAnalytics

36. @jamet123 #decisionmgt © 2016 Decision Management Solutions 36 Decision Modeling for Analytic Projects ▶ Learn more ▶ Download the five questions brief ▶ bit.ly/5QAnalytics ▶ Ready to start? ▶ Kick off a decision modeling project today ▶ bit.ly/DMNAnalytics

37. @jamet123 #decisionmgt © 2016 Decision Management Solutions 37 DecisionsFirst Modeler Collaborative, cloud-based decision modeling software that puts decisions at the heart of your business architecture, building better requirements and driving successful business rules implementations Key Features Multi-user, Collaborative Decision Model and Notation (DMN) Support Shared Database, Reusable components Automated Consistency and Completeness Checks Living Repository

Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

Recommended

More Related Content

What's hot

What's hot(20)

Viewers also liked

Viewers also liked(9)

Similar to Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model

Similar to Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model(20)

Recently uploaded

Recently uploaded(20)

Analytics Teams: 5 Things You Need to Know Before You Deploy Your Model