As companies around the world look to get a jump on AI efforts, there’s one major question: with dozens of potential AI use cases but limited resources, how can organizations prioritize the right projects? Carefully selected and well-designed Enterprise AI projects facilitate faster and more efficient collaboration among AI scientists and engineers and ultimately help organizations set up for AI success. In this presentation we share a comprehensive framework for: 1) Choosing relevant projects: defining needs from business lines and cost/benefit analysis. 2) Advanced project scoping: aligning stakeholders, anticipating key steps, defining success metrics. 3) Operationalization of AI projects: challenges and best practices.

1. January 2020
AI Projects – Lifecycle and Best Practices

2. Today’s focus: Best Practices for AI Projects

3. Analytics Project Lifecycle
Key stages and Key stakeholders
Data Scientists
Data Engineers
Business Analysts IT
Scope
Scoping
Scope
Scope

4. Choosing Relevant Projects
Cost / Benefit Analysis – Use a Quick Proxy
Collect Needs from Business Lines and go through a Cost / Benefit analysis
Business
Value
Complexity
Churn Prediction
Marketing Attribution
Fraud Management
Compliance
Semantic AI
Credit Default Risk
ATM maintenance
Compliance & Regulatory
Cost Reduction
Branding
Revenue Increase
Customer Satisfaction
Overall Value
Availability of data
Complexity of data
Complexity of modelling
Complexity of deployment
Overall Complexity
AML & Financial Crime
Next Best Action / NBP

5. Advanced Project Scoping – ML Project Canvas
Align the different stakeholders, anticipate the key steps, define success metrics

6. Project Build Phase
Collaboration:
• ensure all the stakeholders can participate up to their skills and domain expertise
• minimize loss of information between the steps and stakeholders
• maximize adhesion of the end beneficiaries
Iteration:
• data acquisition should happen by incremental steps
• data preparation, feature engineering and machine learning are an iterative process
• start with simple, robust and explainable models
Functional Validation:
• keep the end goal in mind when building the model and the data preparation steps
• think about the service in production: what data is available, what are the requirements…
• measure current metrics, extrapolate the value and establish quantified objectives for each stage
Collaboration, Iterative Process, Functional Validation
Example – Churn
Management

7. Deployment in Production
Operationalization Challenges (O16N)
ROI: Not all use cases can / should be o16n-ed
● Business Value (ROI of better decisions, time saved...)
● Estimate the replicability of the use case
● Measure before / after
Complexity: There are different levels of o16n-ation
● Level 1 - CSV extract
● Level 2 - dashboard or external viz tool
● Level 3 - automate and push insights into DB
● Level 4 - automate and push insights directly into process or application
Cost: those o16n levels have different costs depending on the organization
● ML readiness, Agility, infrastructure, governance…
L1 L2 L3 L4
o16n complexity
Data Product ROI
Churn Prediction
Deployment Strategy

8. Operationalization of AI projects
Master the key challenges to successful o16n (AVOID RECODING EVERY PROJECT BEFORE PRODUCTION)
Challenges and Best Practices
Link
• IT Environment Consistency
• Packaging and Release, CI/CD process
• Performance, Scalability and Robustness
• Model Lifecycle Management
• Functional Monitoring
• Auditing
Refactor & Robustify Integrate Set Targets Test and Deploy Monitor
Refactor whole data flow
from design one in same
environment
In-built query enrichment
Replicable integration
Define checks and metrics
around model and data
flow performance and
behavior
Dynamic deployment of
endpoints
Automated monitoring
strategy of data flows and
end points

9. Links to ressources
• Defining Successful AI Projects
• https://pages.dataiku.com/hubfs/defining-successful-ai-projects.pdf
• Data Science Use cases in Banking and Financial Services
• https://pages.dataiku.com/hubfs/AI-Banking-White-Paper.pdf
• Data Science Operationalisation Guidebook
• https://pages.dataiku.com/hubfs/operationalization-guidebook.pdf
• Any other questions, feel free to email me:
• vincent.destoecklin@dataiku.com

10. partnerships@dataiku.com