Your SlideShare is downloading. ×
0
Agile BI/DW Daniel Fagerström [email_address]
Experience <ul><li>Used agile methods 10 years
Developed BI/DW 5 years
Led large BI/DW project during last year using Scrum
Database lead at alaTest.com </li></ul>
Why agile? <ul><li>Deliver early
Reduce risk
Make the project steerable and transparent
Make development efficient and fun </li></ul>
How?
Just follow the book? <ul><li>Yes, but …
What are the user stories?
Agile DB practices?
Agile operations? </li></ul>
Overview <ul><li>Before starting
Project cycles
Agile planning
Agile DB practices
Agile operations </li></ul>
Before the project <ul><li>Introduce agile methods while selling
Agile architecture, dimension models
Agile estimation based offering </li></ul>
Sprint 0 <ul><li>Collocation (set up team room)
Appropriate HW, SW, test server
Involve operations
Upcoming SlideShare
Loading in...5
×

Agile BI/DW - Aalborg 2009

1,399

Published on

Using agile methods in data warehouse/business intelligence projects. Presentation at workshop at Aalborg Universtity

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,399
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
70
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Agile BI/DW - Aalborg 2009"

  1. 1. Agile BI/DW Daniel Fagerström [email_address]
  2. 2. Experience <ul><li>Used agile methods 10 years
  3. 3. Developed BI/DW 5 years
  4. 4. Led large BI/DW project during last year using Scrum
  5. 5. Database lead at alaTest.com </li></ul>
  6. 6. Why agile? <ul><li>Deliver early
  7. 7. Reduce risk
  8. 8. Make the project steerable and transparent
  9. 9. Make development efficient and fun </li></ul>
  10. 10. How?
  11. 11. Just follow the book? <ul><li>Yes, but …
  12. 12. What are the user stories?
  13. 13. Agile DB practices?
  14. 14. Agile operations? </li></ul>
  15. 15. Overview <ul><li>Before starting
  16. 16. Project cycles
  17. 17. Agile planning
  18. 18. Agile DB practices
  19. 19. Agile operations </li></ul>
  20. 20. Before the project <ul><li>Introduce agile methods while selling
  21. 21. Agile architecture, dimension models
  22. 22. Agile estimation based offering </li></ul>
  23. 23. Sprint 0 <ul><li>Collocation (set up team room)
  24. 24. Appropriate HW, SW, test server
  25. 25. Involve operations
  26. 26. Contact the data source owners </li></ul>
  27. 27. Cycles <ul><li>Release cycle </li><ul><li>Envision & fund
  28. 28. Plan
  29. 29. Build
  30. 30. Deploy & review </li></ul><li>Sprint </li><ul><li>Plan
  31. 31. Develop
  32. 32. Demo
  33. 33. Review </li></ul><li>Daily Scrum </li></ul>Sprint Daily Scrum Sprint Planning Sprint Demo Sprint Retrospective Release cycle
  34. 34. Planning <ul><li>DW for loyalty card company
  35. 35. Epic </li><ul><li>As marketing manager I want to use the DW to understand customer behavior </li></ul><li>Theme </li><ul><li>As a campaign manager I want to follow sale over time for a certain store </li></ul><li>User story </li><ul><li>I want the customer dimension to be updated daily </li></ul></ul>
  36. 36. Planning (decomposition) <ul><li>Architecture driven story decomposition </li><ul><li>See Agile Data Warehousing, Ralph Huges </li></ul><li>Use course architecture as common language for Product owner and developers
  37. 37. Architectural dimensions </li><ul><li>Subject area, Target layer, Transformation, type, Refresh type, Refresh frequency, Automation, User friendliness </li></ul><li>Each user story must enable feedback </li></ul>
  38. 38. Dimension model Customer name city Store name region Product name class Date year month ReceiptLine amount
  39. 39. Planning (decomposition) <ul><li>Start with the most important </li><ul><li>Star (subject area)
  40. 40. Dimension
  41. 41. Attributes in the dimension </li></ul></ul>
  42. 42. Architecture Customer registry Product registry Org. registry Transactions Raw data Staging Dimension model Marts OLAP Cube Reporting Applications ETL
  43. 43. Planning (decomposition) <ul><li>ETL – Start with simpler/earlier tasks </li><ul><li>Transformation type </li><ul><li>Direct data -> aggregations -> business rules </li></ul><li>Refresh frequency </li><ul><li>One time load -> monthly -> weekly -> daily </li></ul><li>Target layer </li><ul><li>Raw -> staging -> presentation -> end user view </li></ul><li>Refresh type </li><ul><li>Link -> snapshot -> incremental </li></ul></ul></ul>
  44. 44. Planning (decomposition) <ul><li>BI </li><ul><li>Transformation type
  45. 45. Refresh frequency
  46. 46. Automation </li><ul><li>On demand on work station -> on demand posted on server -> scheduled </li></ul><li>User friendliness </li><ul><li>Single table -> linked table -> navigation -> pick list -> dash board
  47. 47. Using expert analysis tools -> reports </li></ul></ul></ul>
  48. 48. Breadth or depth? <ul><li>Breadth </li><ul><li>Simplified load of the most important dimension models
  49. 49. Early feedback, earlier build of dependent systems </li></ul><li>Depth </li><ul><li>Complete load of one dimension at a time
  50. 50. Early deployment of usable sub systems </li></ul></ul>
  51. 51. Agile DB Practices <ul><li>Agile DB refactoring </li><ul><li>Scott Ambler
  52. 52. Change scripts for everything </li></ul><li>Common code ownership </li><ul><li>Everybody can can change the models
  53. 53. DB models in visual modeling tool </li></ul><li>Don't become a bottleneck </li><ul><li>Broad contact area between team members and users, operations and source data specialists </li></ul></ul>
  54. 54. Agile DB Practices <ul><li>Course architecture
  55. 55. Agile modeling
  56. 56. Just in time design
  57. 57. Agile documentation </li></ul>
  58. 58. Agile DB Practices <ul><li>DB Schema in repo
  59. 59. Subset data
  60. 60. Your own DB and development sandbox
  61. 61. Staged deployment: dev -> test -> prod
  62. 62. Continuous deployment
  63. 63. TDD? </li></ul>
  64. 64. Agile Operations <ul><li>Involve operations from day one
  65. 65. Operations part of the project
  66. 66. Deploy at demo/production server often
  67. 67. Start source data deliveries ASAP </li></ul>
  68. 68. Agile Operations <ul>-> Operations, support and maintenance during the project <li>Kanban for that? </li></ul>Project time #Features in prod.
  69. 69. Conclusions <ul><li>Some mental barriers for DB/DW/BI-developers
  70. 70. Transparent
  71. 71. Steerable
  72. 72. Works great!
  73. 73. Happy customers (and developers)! </li></ul>
  74. 74. Literature
  75. 75. www.danielfagerstrom.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×