Slides of the project definition workshop at disummit June 26, 2019 . We're all caught in the AI hype - and I daily meet AI enthousiasts who focus heavily on technology, cool algorithms on a modern infrastructure. And business leaders who dream AI will solve our problems automagically. While dreaming and innovating is cool, modern and exciting, such focus typically increases business impact only marginally. The best way to increase impact, is having the project definition right. In this workshop, you'll walk away with a "Project definition checklist " - an exhausitive list of issues you want to understand before you start building your algorithms. Throughout the years this has proved to be a useful asset for junior data scientists, (aspiring) data science managers and (project) management.

1. Copyright DISUMMIT2019
Project Definition
Workshop
Geert Verstraeten
Managing Partner
Python Predictions
1
slideshare.net/pythonpredictions
Project Definition Workshop - disummit 2019 - @pythongeert

2. Copyright DISUMMIT2019 2
is it a car?
or is it a plane?
it’s a workshop!
Project Definition Workshop - disummit 2019 - @pythongeert
(I need your participation)

3. Copyright DISUMMIT2019 3Project Definition Workshop - disummit 2019 - @pythongeert
“Let’s say you want to save millions of dollars – you just push this button here...”
What some managers expect – “automagical” results

4. Copyright DISUMMIT2019 4Project Definition Workshop - disummit 2019 - @pythongeert
How some
(tech) companies
oversell AI today

5. Copyright DISUMMIT2019
We don’t need more buzzwords
We need more impact
5Project Definition Workshop - disummit 2019 - @pythongeert
(and the impact of project definition is huge)
(it’s important to solve a problem correctly,
but it’s even much more important to solve the correct problem)

6. Copyright DISUMMIT2019
(the type of algorithms we talk about)
(predicting who will have a car accident)

7. Copyright DISUMMIT2019
(predicting who will repay their debt)

8. Copyright DISUMMIT2019
(predicting who will buy shoes)

9. Copyright DISUMMIT2019
(predicting who will choose green energy)

10. Copyright DISUMMIT2019
(predicting injuries)

11. Copyright DISUMMIT2019
(predicting burnout)

12. Copyright DISUMMIT2019 12Project Definition Workshop - disummit 2019 - @pythongeert
traditional & streamlined modern & adaptive undefined
“waterfall” “agile” “ad hoc”
Popular project management approaches

13. Copyright DISUMMIT2019
● Traditional approach (since 1956?)
● Originates in manufacturing and construction industries
● A project is a sequence of stages with dependencies
● Progress and dependencies can be represented in Gantt chart:
● Works best when plan does not change
Project Definition Workshop - disummit 2019 - @pythongeert
Waterfall project management

14. Copyright DISUMMIT2019
Agile project management (eg SCRUM)
● More recent and popular for software development
● Sprints (1 to 4 weeks) focus on delivering working increments
● Involve end users as soon as possible
● Performed within a cross-functional team, focus on collaboration,
ownership and fun
● Typical roles:
○ Product Owner – holds product vision, links stakeholders
○ ScrumMaster – coaches SCRUM, removes impediments
○ Development team – builds product, self-organised
● Works best when plans change continuously
Project Definition Workshop - disummit 2019 - @pythongeert

15. Copyright DISUMMIT2019 15Project Definition Workshop - disummit 2019 - @pythongeert
traditional & streamlined modern & adaptive undefined
“waterfall” “agile” “ad hoc”
Popular project management approaches

16. Copyright DISUMMIT2019 16
Some
are very explorative
Data Science problems

17. Copyright DISUMMIT2019 17
Predictive problems

18. Copyright DISUMMIT2019 18
Predictive problems
Modeling
Definition
Preparation
Usage
Validation
Most
predictive problems
are…erm. … predictable
(they can be solved using waterfall project mgmt. approaches)

19. Copyright DISUMMIT2019
Project
Definition
Data
Preparation
Model
Building
Model
Validation
Model
Usage
19Project Definition Workshop - disummit 2019 - @pythongeert
Phases in building predictive models
https://www.linkedin.com/pulse/predictive-analytics-soup-story-geert-verstraeten/

20. Copyright DISUMMIT2019 20
Project definition!
Provided that the basics are available
Project Definition Workshop - disummit 2019 - @pythongeert
Organisation Talent Infrastructure Data
https://www.slideshare.net/pythonpredictions/winning-with-analytics-data-innovation-summit-2015-made-in-belgium

21. Copyright DISUMMIT2019 21
Winning a Michelin star
Project Definition Workshop - disummit 2019 - @pythongeert

22. Copyright DISUMMIT2019 22
Time
Future
Behavior
FuturePast
Current
Profile
What we use to predict
= ‘potential predictors
What we aim to
predict = ‘target’
Historic
Behavior
Historic
Profile
Building the model Using the model
“Training” “Scoring” / “Validation”
Time
Gap
Time
Gap
Project Definition Workshop - disummit 2019 - @pythongeert
The predictive timeline

23. Copyright DISUMMIT2019 23Project Definition Workshop - disummit 2019 - @pythongeert
(the workshop
result)

24. Copyright DISUMMIT2019
Project definition checklist (1 / 3)
Project plan
• Plan project start, project deadline and
milestones in detail:
Stakeholders
Sponsors, client project managers, client
analysts, data / IT responsibles, DPO, etc
Timeline
Definition of the predictive timeline (past,
timegap, future)
24
• signoff project definition
• signoff base table creation
• predictor quality and interpretation
• model quality and interpretation
• model scoring or handover
• proven business results
• monitoring
Project Definition Workshop - disummit 2019 - @pythongeert

25. Copyright DISUMMIT2019
Project definition checklist (2 / 3)
Goal
• The organisational goal - what does the
organisation aim to reach by
constructing a model?
• The analytical goal - what should we
aim to predict?
Granularity
At which level do we want to generate
predictions?
Population
For who (or what) we want to generate
this prediction?
• Are there different sub-groups?
Understand available data
Which data sources are involved? Is there
metadata?...
25Project Definition Workshop - disummit 2019 - @pythongeert

26. Copyright DISUMMIT2019
Project definition checklist (3 / 3)
26
Basetable definition
• ID variable(s) - How do we uniquely
identify a case?
• Target variable(s) - Defining what we
want to predict. This should be specific
and unambiguous.
• Predictors - What type of information
can be computed to help us predict?
• Profiling - What type of information
should be computed to interpret the
result
Plan usage
• How will the organisation use the
scores or the outcome?
• How will the organisation evaluate
decisions or campaigns?
• How will the model help the business
reach its business goal?
• Can we construct a business case?
What is the current benchmark?
• Are there important potential issues or
pitfalls in implementation?
Project Definition Workshop - disummit 2019 - @pythongeert

27. Copyright DISUMMIT2019 27
Get your project
definition right!
Maximise the
impact of your
investments
Project Definition Workshop - disummit 2019 - @pythongeert