Data Warehouse automation across the entire information delivery value chain gives productivity a double boost – once for the gain in efficiency, and secondly by shrinking cycle times (exponential gains, Little’s Law) Keynote at Data Warehouse Automation conference, 20 September 2012, by Tom Breur

1. Cycle time & Automation:
hidden value & business cases
Tom Breur
Data Warehouse Automation conference
www.dwhautomation.com
Amsterdam, 20 September 2012

2. Little’s Law (1)
L = λ × W
where (in Operations Research):
L = number of customers in a system
λ = average arrival rate
W = average time in the system
www.xlntconsulting.com 2

3. Characteristics of Little’s Law
 Little’s Law pertains to the system as a
whole, and also its constituent parts
 No assumption is made with regards to
variable distribution(s)
 Little’s Law holds for systems in “steady
state”, e.g.: neither starting nor shutting
down
www.xlntconsulting.com 3

4. Queuing theory
www.xlntconsulting.com 4

5. Little’s Law (2)
L = λ × W
where (in Agile BI):
L = number of features/requests WIP
λ = average arrival rate
W = average time “in the system”
www.xlntconsulting.com 5

6. What is “time in system”?
 Sprint length?
 Time from entry in Product BackLog (PBL)
until delivery?
 Time from initial feature request until
delivery?
 Time from information need until delivery?
What cycle are you referring to?
www.xlntconsulting.com 6

7. Development “queue” (=cycle)
requirements
analysis Design Implementation Verification Maintenance
Story
prepping Sprint Maintenance
(design) (code) (unit test) (system test) (acceptance test)
“Zero Sprint work” “Sprint work” “New Sprint work”
www.xlntconsulting.com 7

8. How can you reduce cycle time?
 Concurrent development
 “Swarming”
 Reduce WIP
 Automation
 (and others)
www.xlntconsulting.com 8

9. Concurrent development (1)
 Waterfall: you can avoid mistakes/rework
by getting good requirements upfront
 The most costly mistakes arise from
forgetting important elements early on
 Detailed planning (BDUF) requires:
 early (ill informed) decisions
 uses more time
 leading to less tangible products to resolve
ambiguity
www.xlntconsulting.com 9  vicious cycle

10. Concurrent development (2)
 Agile: decide at “last responsible moment”
 decisions that haven’t been made, don’t ever
need to be reverted
 No “free lunch” – deferring decisions
requires:
 anticipating likely change
 coordination/collaboration within team
 close contact with customers
www.xlntconsulting.com 10

11. “Swarming”
 Of all Stories/tasks in
a Sprint, only one lies
on the “critical path”
 “Impediments” signal completion of a Story
is jeopardized
 Swarming is (should be) default response
www.xlntconsulting.com 11

12. Reduce Work-in-Progress (WIP)
 Central idea Lean/Kanban: set WIP limits
 More WIP leads to longer (and less
predictable) lead times
 Running our of WIP triggers a standstill –
how can this be beneficial??
www.xlntconsulting.com 12

13. Automation
Can take on many different forms:
 Standardized processes, templates, etc.
 ETL/DDL generation
 Staging
 hub (for 3-tiered DWH architectures)
 data marts
 Maintenance
 version control
 documenting “as built” design
www.xlntconsulting.com 13

14. Business case for automation
 Little’s Law: L = λ × W
 Information delivery
as a cycle implies
that throughput gains
accrue exponentially
over time
 Gains anywhere along
the cycle contribute to productivity
www.xlntconsulting.com 14

15. Conclusion
 The information value chain has inherent
variation
 manage “the system” accordingly
 Reduce cycle time by “managing” variation
 working “in parallel”
 automation
 Gains from sustainable shortening of cycle
time are exponential
www.xlntconsulting.com 15