Going Beyond 'What Success Looks Like' - Using Data to Achieve Successful Projects

1. Going Beyond
‘What Success
Looks Like’ – Using
Data to Achieve
Successful Projects
25th November 2015

2. Overview
 Introductions
 Leveraging the most out of the data you already have
 Setting up baselines and real-time KPI dashboards
 Making better decisions from your data
 Presenting Benefits Realisation in a way the business will
understand
 A LateRooms Case Study
 Conclusions
2 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

3. A little bit of LateRooms.com history
LateRooms.com was born in Salford, Greater
Manchester in 1999, starting life as an 'on the
day for the day' booking site for unsold rooms.
By 2007 we had joined the world's leading
leisure travel group, TUI Travel PLC. We're still
loving life here in Manchester today, now
offering over 55,000 properties worldwide
with more UK hotels than anybody else.
Who are LateRooms.com?
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects4
And these days you can still book on the day of
your stay, naturally, but also up to a whole year
in advance - to make things nice and flexible for
you.

4. With a lot of people who like to ‘Standup’…
We work in IT
5 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

5. And focus heavily on collaboration and continuous delivery
We’re structured in product teams
6 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

6. Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
9
Leveraging the
most out of the
data you already
have

7. But it’s siloed within departments or tools
Data, data everywhere…
10 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

8. Who will defend it to the death…
And it has different owners
11 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

9. But nobody thinks their data has quality issues
And nothing lines up
12 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

10. To help alleviate the pain
There are a number of tools and techniques we use as
BA’s to identify data within our organisations, how
close it is to meeting our requirements and how
accessible it is.
Some of the techniques we use include:
• Requirements & value analysis
• Stakeholder identification
• GAP analysis
• Benchmarking analysis
BA Activities that we undertake at this stage
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects13

11. Dust off the business case
At some point in your project someone
committed to some benefits that were
expected.
Where there is a benefit there has to be a
measurement:
• Higher sales
• Lower costs
• Greater customer retention
• Improved efficiencies
Requirements & value analysis
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects14

12. Who’s data is it and how can they help
Data is often collected by organisations to
support a number of requirements. While
this data can be repurposed it may not be
fit for purpose. Consider the following:
• Who in your team, organisation or
external to your organisation may
already have data that you can use?
• What was the data originally captured
for and is there any restriction on its use?
• Are there any data sets that could be
used to support what you’re trying to
measure?
Stakeholder identification
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects15

13. Does the data meet your requirements or does it need enhancing?
Once you have identified your data sources it’s
important to work out if they meet your
requirements or if they need to be enhanced to
support your needs.
If your data source represents a subset of the data
needed you may need to decide between
enhancing your data collection (if that is an
option). Alternatively you may be able to combine
data sets if there is a common unique attribute
across both sets (however, this may be a risky
strategy if the data sets get out of sync).
Consider:
• ROI for enhancing
• How “accurate” you need the data to be
• If a combined data set is reliable enough
GAP Analysis
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects16
Measurementperformance

14. Don’t let the data become stale
Once data is collected you should decide on a
strategy to ensure it remains fresh.
• Set regular reminders to review data sources
• Get added to distribution lists to hear about
changes in data collection projects (Business
Intelligence or data warehouse updates?)
• Periodically validate that the data being collected
is still relevant and that it’s collection mechanism
has not been superseded by an alternative means
Benchmark and improve
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects17

15. Setting up baselines
and real-time KPI
dashboards
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
6

16. And measure it continuously
Once you have identified your data
sources, stakeholders and any gaps
you have in it, the next step is to give
it some meaning.
• Define your targets
• Define your thresholds
• Really good 
• Really bad 
• Know what happens if you breach
a threshold…
• Decide on measurement intervals
and make sure they happen.
Give your data meaning
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects19

17. ..and make sure they’re really big!
Make your dashboards visible
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects20

18. Make it happen
“If Engineering at Etsy has a
religion, it’s the Church of Graphs.
If it moves, we track it. Sometimes
we’ll draw a graph of something
that isn’t moving yet, just in case it
decides to make a run for it.”
Measure Anything, Measure
Everything
Make it part of your culture
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects21

19. Making better
decisions from your
data
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
22

20. And allows us to manage scope
• Data supports our ability to prioritise features in a much more effective way
• It removes ambiguity of business benefits
• It supports continuous delivery of value
Data gives prioritisation context to requirements
23 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

21. So we need to run some tests
By generating some variants of your product you can simulate an experiment to start collecting data.
This can allow us to test hypothesis’ while not risking our core KPIs.
As long as you have measurement in place to identify the success or failure of a test.
Sometimes the data isn’t available
24 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects

22. To chase value at every opportunity
Traditional projects
If we plan and execute our projects in this way, at what point do we know if what we delivered is the right thing?
Data + Learning = Pivoting
25 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
Data driven projects
Continuous measurement (and analysis) allows us to continually chase the most valuable thing next, providing
organisational agility to change.

23. 29
Case Study
New Booking Proccess
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects

24. Our AS IS State
• An existing booking form on a
legacy technology stack that was
being deprecated.
• A change in Personal Card
Information (PCI) regulations
meant work needed to be
undertaken to remain compliant.
• An out-dated UI/UX that was no
longer in keeping with the rest of
the user’s journey.
• Was not fully supported across all
devices and had an adaptive
approach to mobile (2 code
bases).
The problem
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects30

25. Document title31

26. Document title32
9 trust and
confidence
messages
Title
First name
Last name
Repetition
(three times) Links

27. Lean UX
The approach
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects33
Users
Learn
T
Build
Measure

28. Sketching & prototyping
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects34

29. Built the prototype
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects35

30. Available data
Funnel reports….
Reviewing our data
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects36
Conversion rates…
Missing data

31. Where are we and where do we want to be?
Defined our KPIs
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects37
The problem
• We had no mechanism within our
BI tool that gave us a baseline of
how breakfast sales performed.
The solution
• Implement a tracking mechanism
to collect data on our AS IS
breakfast sales performance.
• Work with stakeholders to define
a target for the next 6 months.
• Set thresholds that would alert to
higher or lower than expected
throughput.
Next
• Experiment…

32. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects38

33. Document title39
Orange = 29 Actions
Green = 19 themes
Yellow = Observations

34. Document title40

35. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects41

36. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects42
OLD FORM NEW FORM
12 6Links Links

37. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects43
OLD FORM NEW FORM
20 12Form elements Form elements

38. A better experience for all
 User experience improvements
drove a conversion uplift
 Benefits were delivered earlier
than planned
 The data helped us pivot
• We never actually finished the
‘planned’ project..
• The ROI didn’t justify it.
 We worked towards deprecating
2 legacy codebases
 We now have a baseline of data and
measurements that we can build on
in the future
The benefits
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects44

39. 45
Conclusions
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects

40. Wrapping it up…
1. Measuring success is linked right back to your initial requirements and
stakeholder desires. You’re helping your team prove that they’ve been met with
real data
2. Measuring success is a continuous process, measuring it at the end is too late…
make it part of your culture
3. Measuring success continuously reduces risk and allows for innovation and
agility
4. Evidence of success is the best reward of all, and will really help you as a BA
Conclusions
46 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects