The case for agile in business intelligence - presented to Philadelphia Business Intelligence Users Group on July 27, 2017 by Cathy Carleton

1. Using an Agile Framework
in a BI Team
Philly BI Users Group Meetup– July 27, 2017
Cathy Carleton
1

2. 2
• Waterfall, Agile & "Wet Agile"
• Adapting to Lean Requirements
• Tools of the Trade
• Tracking Progress and Communicating
• Sprints and Scrums
• The New Rhythm of Delivery
• Managing Dependencies in an Agile Framework
• Managing Expectations - The Definition of "Done"
WHAT WE’LL COVER

3. 3
• How most projects were run before agile, and many still are:
WATERFALL MODEL

4. 4
• Each phase on the critical path is sequential and dependent
• Deployment doesn’t happen until all steps are complete
WATERFALL MODEL

5. 5
A misconception or error early in the project can be
carried through the project, unidentified until delivery.
BI projects can last months or even years. The original
project concept may be inadequate and/or outdated
by the time it’s delivered.
Opportunity costs accrue until project delivery.
RISKS OF WATERFALL

6. 6
• Yes, it’s a methodology of IT delivery…but one that brings a
seismic change to a business.
• It’s a new way of doing business.
• If it’s going to work, the whole business has to adapt to IT’s
new workflows – when historically, it’s been the reverse.
AGILE – WHAT IS IT?

7. 7
• http://agilemanifesto.org/
THE AGILE MANIFESTO – February 2001
• http://agilemanifesto.org/

8. 8
12 AGILE PRINCIPLES
1) Promote customer satisfaction with early & continuous software delivery
2) Welcome changing requirements, even late in the development cycle
3) Deliver working software frequently
4) Business people and developers work together daily
5) Trust and support motivated people to get the job done
6) Communicate most effectively through face-to-face conversation
7) Measure progress through working software
8) Maintain a constant pace indefinitely
9) Enhance agility via attention to technical excellence & good design
10) Maximize the work not done – simplicity is essential
11) Self-organizing teams generate the best architectures, requirements & designs
12) Reflection on how to be more effective, at regular intervals

9. 9
FINE FOR SOFTWARE DEVELOPMENT – BUT BUSINESS INTELLIGENCE?
Yes.
It’s choosing people-centered architecture over data
or object-centered architecture.
It’s iterative delivery and continuous improvement.

10. 10
1. Prioritizing – Choosing to do the most meaningful work
2. Incremental – Conserving resources by making small, time-
boxed bets, enabling continuous & iterative delivery
3. Socializing – Always-on communication & transparency
4. Exploring – Intellectual curiosity, challenging assumptions
5. Validating – Estimates realistic? Requirements still relevant?
6. Empirical – Getting the facts faster, including user feedback
7. Liberating – Autonomous, self-organizing, shedding what is
no longer needed
SEVEN BEHAVIORS OF AGILE THAT WILL WORK IN BI
Credit: Robert MacGregor – Lead Agile Coach, EPAM

11. 11
Estimation is tricky. Agile is adaptive. Not psychic.
Stakeholders get mad at you when you don’t “welcome
changing requirements” 2 hours before deployment.
Even on long-term projects, progress is no longer
invisible. Deliverables become near-term. Not everyone
will welcome this development.
RISKS OF AGILE

12. 12
STAKEHOLDERS’ VIEW OF WATERFALL DATA WAREHOUSE PROJECTS
Step 1: Requirements
Give requirements.
For hours and hours.
Until you lose your voice.
Or nod off.

13. 13
STAKEHOLDERS’ VIEW OF DATA WAREHOUSE PROJECTS
Step 2: Approval
Approve the requirements.
Do it, man.
Get your life back.

14. 14
STAKEHOLDERS’ VIEW OF DATA WAREHOUSE PROJECTS
Step 3: Wait
Now it’s time to wait.

15. 15
STAKEHOLDERS’ VIEW OF DATA WAREHOUSE PROJECTS
Step 4: Wait More

16. 16
STAKEHOLDERS’ VIEW OF DATA WAREHOUSE PROJECTS
Step 5: Worry
Where is this project?
Um, what was this
project all about again?

17. 17
STAKEHOLDERS’ VIEW OF DATA WAREHOUSE PROJECTS
Step 6: The Big Reveal
Is this what we wanted?
Is it still relevant? Is it still
sufficient?
Will people use it?

18. 18
STAKEHOLDERS’ VIEW OF DATA WAREHOUSE PROJECTS
Step 6: The Big Reveal
And why did it come in
so far over budget?

19. 19
WATERFALL/AGILE HYBRID MODELS

20. 20
WATERFALL/AGILE HYBRID MODELS
• Controversy – Many in agile circles believe anything less than a
full agile transformation is doomed to fail.
• Other claim it is more efficient than waterfall alone, especially in
enterprises that won’t or can’t embrace a full agile transformation
• My View: Holding 15-minute stand-up meetings every morning
doesn’t make you agile, but communicating more often is a start.

21. 21
WATERFALL/AGILE HYBRID MODELS - AGILEWASHING
INSERT ONE OF THE FOLLOWING:
• Daily stand-ups – oh look, we do scrum!
• Search-&-Replace “Release” with “Sprint”
• Purchase whiteboards and sticky notes
DO EVERYTHING ELSE THE SAME

22. 22
“Responding to change over following a plan”
Comprehensive up-front requirements are discarded in agile.
“Working software over comprehensive documentation”
Requirements are gathered iteratively via in-person or Skype
interviews. In some enterprises, requirements can fit on a Post-
It. In many others, the code IS the final documentation.
ADAPTING TO LEAN REQUIREMENTS

23. 23
ADAPTING TO LEAN REQUIREMENTS
Documentation is a safety net. Embracing lean requirements is
an act of courage for the tech side and the business side.

24. 24
User Stories - As a <your role>, I want <desired data> so that
<reason you want it>.
Examples:
“As a supply chain analyst, I want to access sales data from newly opened stores
within 24 hours of transaction so that I can determine inventory demands.”
“As a marketing manager, I want monthly modeled churn propensity scores at the
Customer ID level so that I can make retention offers to those most likely to leave.”
ADAPTING TO LEAN REQUIREMENTS

25. 25
TOOLS OF THE TRADE – AGILE OFFICE ENVIRONMENT
Encourages face-to-face collaboration. Great for daily stand-ups.
Not so great for talking to your client. Or your dermatologist.

26. 26
TOOLS OF THE TRADE – AGILE MANAGEMENT SOFTWARE

27. 27
TOOLS OF THE TRADE – AGILE MANAGEMENT SOFTWARE
Buying a pricy putter won’t
turn me into a great golfer.
Spending IT budget on an agile
tool doesn’t turn your shop
into an agile organization.
AGILE IS BIGGER THAN THE IT
DEPARTMENT

28. 28
TRACKING PROGRESS & COMMUNICATING
Agile Culture – your stakeholders must think about what they
need. And communicate with you.

29. 29
TRACKING PROGRESS & COMMUNICATING
Agile Culture – your stakeholders must think about what they
need. And communicate with you. I know. Freaking nightmare.

30. 30
TRACKING PROGRESS & COMMUNICATING – KANBAN

31. 31
TRACKING PROGRESS & COMMUNICATING – KANBAN ON STEROIDS

32. 32
TRACKING PROGRESS & COMMUNICATING – TEAMS
Agile teams ideally are co-located. But they don’t have to be.

33. 33
TRACKING PROGRESS & COMMUNICATING – ESTIMATING WORK
Planning poker – each team member estimates
levels of effort by showing a card – without being
influenced by each other
Dot voting – a little more group influence, but
still democratic in its process
Affinity mapping – grouping similar tasks
And there are many more ways to estimate

34. 34
SPRINTS & SCRUMS
Sprints are time-boxed periods that usually last 2 to 4 weeks.
They conclude with a working deliverable, which is augmented
with more features/functionality in the next sprint. And the next.
Scrums are 15-minute daily stand-up meetings.
Coffee’s okay. Sitting isn’t.
Each team member answers 3 questions:

35. 35
SPRINTS & SCRUMS
1) What did you complete since we last met?
2) What do you plan to accomplish today?
3) What might get in your way?

36. 36
RHYTHM OF DELIVERY - WATERFALL
Project Architect
Procure All
Requirements,
Map all Project
Deliverables
Data Architect
Full Project
Architecture
Developer
All Coding QA Tester
All Testing
(handoff)
(handoff)
(handoff)

37. 37
THE NEW RHYTHM OF DELIVERY - AGILE
SPRINT Project Architect Data Architect Developer QA Tester
0 MVP Requirements - - -
1 Deliverable A - - -
2 Deliverable B Deliverable A - -
3 Deliverable C Deliverable B Deliverable A -
4 Deliverable D Deliverable C Deliverable B Deliverable A
5 - Deliverable D Deliverable C Deliverable B
6 - - Deliverable D Deliverable C
7 - - - Deliverable D

38. 38
MANAGING DEPENDENCIES IN AN AGILE FRAMEWORK
Project Managers and Product Owners must work together closely.
• Put higher priority on a blocker pre-requisite task (even if the
task has lower business value)
• Fake it till you make it – mock out or implement a facsimile of
the missing data or process to keep making progress
• Reprioritize to move the task with dependency later in the
sprint (Agile tools help manage dependencies)

39. 39
MANAGING EXPECTATIONS – THE DEFINITION OF “DONE”
Done – cross-functional teams need to figure out the definition
and agree on “done” – ideally, everybody has skin in the game:
Good: Finally sending the crusty old AS400 to hardware heaven
Retiring server licenses
Delivering working features
Better: Usage targets
Measurably better business outcomes

40. 40
• Agile in itself is not a business objective.
Agile helps achieve business objectives.
• Agile transforms businesses, not strategies. It
can’t turn a failed business strategy into a
success – only deliver it more efficiently.
• Agile does not mitigate a profound lack of
resources. It will make the resources that do
exist more productive.
Parting thoughts…

41. 41
THANK YOU!
Cathy Carleton
Blog: www.cathycarleton.com
Email: cathy@cathycarleton.com
Twitter: @MkgtMeetsIT