Bringing Analytical Models in to The Business Field
Upcoming SlideShare
Loading in...5
×
 

Bringing Analytical Models in to The Business Field

on

  • 385 views

To maximize the business value from analytics, organizations need to develop robust deployment practices that get models rapidly into the hands of staff interacting with customers during that moment ...

To maximize the business value from analytics, organizations need to develop robust deployment practices that get models rapidly into the hands of staff interacting with customers during that moment of truth when behavior can be influenced. In this slide, IIA Faculty Member and President of Corios LLC, Robin Way highlights experience with financial services companies who have successfully brought analytic models into the field, supporting their governance and monitoring their performance.

For more information on how your company can be helped using analytics, follow this link to the DELTA-Powered Analytics Assessment TM.

http://info.iianalytics.com/healthcarebenchmarking

To listen to this webinar, follow this link to register: http://iianalytics.com/resources/archived-webinars/model-deployment-the-moment-of-truth/

Statistics

Views

Total Views
385
Views on SlideShare
385
Embed Views
0

Actions

Likes
1
Downloads
20
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Sarah

Bringing Analytical Models in to The Business Field Bringing Analytical Models in to The Business Field Presentation Transcript

  • MODEL DEPLOYMENT: THE MOMENT OF TRUTH Presenter: Robin Way IIA Lead Faculty and President, Corios
  • MODEL DEPLOYMENT: THE MOMENT OF TRUTH Presenter: Robin Way IIA Lead Faculty President, Corios Inquiries about IIA services should be directed to sales@iianalytics.com 2 | 2013 © IIA All Rights Reserved
  • About IIA IIA is an independent research firm for organizations committed to accelerating their business through the power of analytics. We believe that in the new data economy only those who compete on analytics win. We know analytics inside and out - it’s what we do. IIA works across a breadth of industries to uncover actionable insights gleaned directly from our network of analytics practitioners, industry experts and faculty. The result? Our clients learn how best to leverage the power of analytics for greater success in the new data economy. 3
  • BIG IDEAS • Many companies are not realizing the full economic potential of their analytic model assets due to lack of adoption. • A common language for understanding analytic models and using them in the business needs to be shared with IT and the field. • The financial benefits of proper execution and deployment of analytic models is exceptionally compelling. Inquiries about IIA services should be directed to sales@iianalytics.com 4 | 2013 © IIA All Rights Reserved
  • BACKGROUND • Corios has conducted over 50 model development and deployment projects with North American businesses. • Unlike in model development, few common processes exist in the world of model deployment. • This leads to long project cycles, increased technical and business risk, reduced quality, and reduced adoption of model-based insights by the field. Copyright© 2013 IIA All Rights Reserved
  • WHAT NEEDS TO CHANGE? 1. Establish explicit connections between model scores and business decisions. 2. Data structures and systems used in development and deployment are different, and are changing even faster. 3. Changes in models and their performance over time need to be easily interpretable, assessed and catalogued. 4. To deploy and execute models in production requires practices that are not commonly adopted by model developers. Copyright© 2013 IIA All Rights Reserved
  • THE FIVE D’S OF MODEL DEPLOYMENT Copyright© 2013 IIA All Rights Reserved Develop: Model development and packaging Decisions: Tie operational business decisions to model scores Data: Operationalizing analytic model deployment in a specific data architecture Delta: Monitor the workflow and performance of models in the field Deploy: Implement analytic models via a software development life cycle
  • #1: MODEL DEVELOPMENT Copyright© 2013 IIA All Rights Reserved
  • DEVELOP: LESSONS LEARNED • Understand the capabilities and limitations of the IT production facilities and standards up front. • Educate IT about the analytics process. • The tools you use and the models you build need to be robust to changing field conditions. Copyright© 2013 IIA All Rights Reserved
  • #2: DECISIONS • Scores are not decisions. A decision is the proactive response of the business to the prospective customer behavior, involving the expenditure of resources. • An evidence-based, virtuous cycle needs to be constructed for making the best decisions. • Test and learn practices are the best option for businesses willing to commit to longer-term testing strategies that maximize the value of information. Copyright© 2013 IIA All Rights Reserved
  • CASE STUDY: NEXT BEST OFFER STRATEGY The challenge • Client: The retail marketing group for one of the top 50 banks in the world. • The bank issues hundreds of offers every month via 30+ campaigns. • Marketing leadership knew that ranks and scores weren’t enough to make the final decision about which offer to assign to each customer. They also had to balance product sales goals, cross-product halo effects, over-time contact strategies and offer cost and margin contribution alongside customer likelihood to respond. • In short, which offer should they give each customer in order to grow profitably? The solution • The bank implemented a mathematical campaign optimization routine to allocate the margin- maximizing offer per client for each monthly campaign wave. • After only four months of design and implementation, the bank released their first optimized campaigns out the door. The results • Comparing year-over-year campaign results, the optimized campaigns executed over a two- month period in summer 2012 produced a net increase of $22 in gross sales per customer. In the aggregate, profitable sales growth averaged $3.5 million in monthly incremental campaign-driven income. Copyright© 2013 IIA All Rights Reserved
  • DECISIONS: LESSONS LEARNED • Don’t only implement business rules based on judgment and experience; also build prescriptive rules optimizing for trade-offs between alternatives in pure dollars and cents terms. • The best performing offers and treatments are not the ones you will issue tomorrow; they will be the offers that your organization has refined over several waves of disciplined, rigorous trials, paired with conscientious measurement. Copyright© 2013 IIA All Rights Reserved
  • #3: DATA ARCHITECTURES • Batch scoring in the warehouse worked fine 10 years ago. • Larger pools of models, more rapid refresh, and innovative model scoring methods now dominate attention. In-database analytics In-memory analytics Over-time analytics On-demand analytics Copyright© 2013 IIA All Rights Reserved
  • CASE STUDY: IN-DATABASE ANALYTICS The challenge • Client: The data sciences team for a large investment brokerage. • The brokerage has built response scoring models for their customers, used to advise financial advisors on a periodic basis. • The brokerage needed to dramatically reduce the cycle time needed to develop new models and to refresh their scores, so that financial advisors could have daily updates on the best treatment for their customers, and hence grow their most profitable relationships. • The solution • Corios designed and implemented massively-parallel model training and model scoring routines, and trained the client team to modernize the remainder. • The results • Compute tasks that previously required days to run, now run in a few minutes or less, on billions of transactions. • This also returned lifts on models of 10-20% by taking more transactional predictors into account, not previously possible, due to computational scale limits. Copyright© 2013 IIA All Rights Reserved
  • DATA: LESSONS LEARNED • Batch warehouse updates still work well if your refresh timelines are monthly and customers don’t migrate rapidly. • Otherwise: • Consider in-database analytics when transaction-level trends are important for predictions, and you’ve already made a commit to data appliances or distributed file systems. • Consider in-memory analytics when stakeholders need to make detail-sensitive decisions with intense, interactive visual input. • Consider over-time analytics for advanced trending, such as stress testing, customer survival, and new product adoption. • Consider on-demand scoring for deployment on individual customers, given a high degree of confidence in your models’ reliability, strong believability in the field, and patience for testing. Copyright© 2013 IIA All Rights Reserved
  • #4: DELTA: MONITORING MODEL PERFORMANCE • Document the scope, ownership, quality assurance, approval and signoff, and lifecycle of a model asset. • Understand the model’s insights, including the interpretation, defensibility, and identification of the best business actions to take based on a model score. Copyright© 2013 IIA All Rights Reserved
  • EXAMPLE: CREDIT RISK MODEL CALIBRATION Copyright© 2013 IIA All Rights Reserved
  • DELTA LESSONS LEARNED • Lack of visibility into model inventories is a source of regulatory pressure that needs to be met now, rather than “a few years from now”. • Workflows for tracking the model asset through its stages of development, review, approval and retirement, and commit the appropriate staffing to support. • Models are worth hundreds of times their development cost, deserving prudence comparable to managing assets such as human resources or physical plant. Copyright© 2013 IIA All Rights Reserved
  • #5: DEPLOY • Deploying models in production needs to be a structured, disciplined activity. • The software development life cycle (“SDLC”) is an apt model, with some minor adaptations. Copyright© 2013 IIA All Rights Reserved
  • THE SDLC APPLIED TO MODEL DEPLOYMENT Copyright© 2013 IIA All Rights Reserved
  • DEPLOYMENT: LESSONS LEARNED • Select a single platform for model development and deployment in order to reduce the quality assurance challenges that arise when translating from one platform’s code base to another. • Quality assurance can be improved by using peer review, version and change management practices, and standards for model documentation and packaging. • Model performance should be tracked both on accounts that get scored on-demand during the day, as well as on all accounts on a periodic basis, for tracking model stability. Copyright© 2013 IIA All Rights Reserved
  • THE FIVE D’S OF MODEL DEPLOYMENT Copyright© 2013 IIA All Rights Reserved Develop: Model development and packaging Decisions: Tie operational business decisions to model scores Data: Operationalizing analytic model deployment in a specific data architecture Delta: Monitor the workflow and performance of models in the field Deploy: Implement analytic models via a software development life cycle
  • JOIN US FOR THE NEXT IIA WEBCASTS July 24, 2013 Analytics 3.0: Opportunities for Healthcare Thomas H. Davenport and Jack Phillips Register at iianalytics.com Inquiries about IIA services should be directed to sales@iianalytics.com 23 | 2013 © IIA All Rights Reserved
  • research@iianalytics.com