Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Disciplined Agile Data Management

161 views

Published on

How does data management fit into agile development? How can data professionals take an agile approach to data management? What mindset do data professionals need to succeed in an agile world?

Published in: Software

Disciplined Agile Data Management

  1. 1. Disciplined   Agile Data   Management
  2. 2. Scott  Ambler • Helps  enterprise-­class  organizations  around  the   world  to  improve  their  processes  and  organization   structures • Thought  leader  of: – Agile  Modeling  (AM)  method – Agile  Data  (AD)  method • Co-­creator  of  the  Disciplined  Agile  (DA)  framework • Author  of  20+  books • scott [at]  scottambler.com ©  Disciplined  Agile  Consortium   2 @scottwambler
  3. 3. ©  Disciplined  Agile  Consortium   3
  4. 4. ©  Disciplined  Agile  Consortium   4 Agenda • The  Disciplined  Agile  Framework • The  Agile  Data  Mindset • Disciplined  Agile  Data  Management • Parting  Thoughts
  5. 5. Disciplined Agile  Enterprise Disciplined Agile  IT Disciplined DevOps DAD The  Disciplined  Agile  Framework ©  Disciplined  Agile  Consortium   http://www.disciplinedagiledelivery.com/the-­agile-­tractor-­engine-­analogy/ 5
  6. 6. The  Disciplined  Agile  Framework ©  Disciplined  Agile  Consortium   6
  7. 7. An  Agile  Mindset for  Data  Management ©  Disciplined  Agile  Consortium   7
  8. 8. Evolution  over   Definition ©  Disciplined  Agile  Consortium   8
  9. 9. Holistic  Organization  Over  Data  Management ©  Disciplined  Agile  Consortium   9
  10. 10. Collaboration  over  Documentation ©  Disciplined  Agile  Consortium   10
  11. 11. Sufficiency  over  Perfection ©  Disciplined  Agile  Consortium   11 What  we  want Where Disciplined Agilists initially  aim Where traditionalists typically  aim  
  12. 12. Cross  Functional  Over  Specialized  Staff ©  Disciplined  Agile  Consortium   12 Specialized  Staff Cross-­Functional  Staff
  13. 13. Automated  Over  Manual  Processes ©  Disciplined  Agile  Consortium   13 Data  Automation  opportunities: – Automated  regression  testing – Data  schema  analysis – Continuous  database  integration – Continuous  database  deployment – Operational  database  monitoring  with   real-­time  dashboards
  14. 14. Disciplined  Agile  Data  Management ©  Disciplined  Agile  Consortium   14
  15. 15. The  Data   Management   Process  Blade ©  Disciplined  Agile  Consortium   15
  16. 16. Comparing  Data  Quality  Strategies ©  Disciplined  Agile  Consortium   16 Long-­term  payback, difficult  to  keep  updated Transform  logic  little  more  than a  band-­aid over  source  problems Ensures  that  future  changes  don’t “break”  previous  implementations Detailed  executable  specification and  validation Reduces  feedback  cycle,  thereby minimizing  cost  of  fixing  problems Safely  address  production  problems
  17. 17. Evolve  Data  Quality  Assets • Keep  this  as  light-­weight  and  collaborative  as  possible • Test  data  management  can  be  challenging: – Privacy  issues  surrounding  production  data – Some  test  data  will  need  to  be  engineered – Configuration  management  of  test  data – Continuous  database  integration  requires  automated  restoration  of  test   data ©  Disciplined  Agile  Consortium   17
  18. 18. Refactoring  Legacy  Data  Sources ©  Disciplined  Agile  Consortium   18 Database  refactoring   immediately  addresses  legacy   quality  problems  BUT  requires   long-­term  administration  to   remove  implementation   scaffolding  
  19. 19. Data  Security • Security  addresses  a  range  of  issues: – Data  security – Facilities  security – Staff  vetting – Staff  protection – And  more • Implication:  Data  management  should  collaborate  closely  with   Security  professionals ©  Disciplined  Agile  Consortium   19
  20. 20. Specifying  Data  Structures • Modeling  is  effective  at  thinking  through  and  communicating  high-­ level  ideas – Fundamental  lean  concept  is  that  models  of  complex  things  should  be   very  simple  in  nature,  not  detailed • Tests,  when  automated  and  better  yet  written  before  the   implementation  that  they  validate,  are  very  effective  as  detailed   specifications ©  Disciplined  Agile  Consortium   20
  21. 21. Agile  Data  Governance • Effective  governance  focuses  on  motivation  and  enablement,  not  on   policing  the  creation  of  artifacts • Data  governance  is  one  of  many  aspects  of  IT  Governance • Agile  strategies: – Collaborative  support  of  teams  and  stakeholders – Light-­weight  guidance – Automated  monitoring(tests,  schema  analysis) ©  Disciplined  Agile  Consortium   21
  22. 22. Agile  Data  Quality  Strategies ©  Disciplined  Agile  Consortium   22
  23. 23. ©  Disciplined  Agile  Consortium   23
  24. 24. Shifting  Your  Data  Management  Mindset ©  Disciplined  Agile  Consortium   24
  25. 25. How  to  Adopt  Agile  Data  Management  Strategies 1. Expect  (a  lot)  better 2. Surface  your  challenges 3. Invest  in  your  staff 4. Hire  agile  coaches  with  deep  experience  in  both  agile  and  data   management 5. Invest  in  automation  infrastructure – Question  your  existing  “data  quality  tools” – Data  is  a  critical  component  of  your  CI/CD  pipeline – DB  testing  and  refactoring  tools  are  proven  in  practice ©  Disciplined  Agile  Consortium   25
  26. 26. scott [at]  scottambler.com Twitter:  @scottwambler DisciplinedAgileConsortium.org DisciplinedAgileDelivery.com Thank  You! ©  Disciplined  Agile  Consortium   26
  27. 27. Would  You  Like  This  Presented   to  Your  Organization? Contact  us  at  ScottAmbler.com ©  Disciplined  Agile  Consortium   27
  28. 28. Scott  Ambler  +  Associates  is  the  thought  leader  behind  the  Disciplined   Agile  Delivery  (DAD)  framework  and  its  application.  We  are  a  boutique   IT  management  consulting  firm  that  advises  organizations  to  be  more   effective  applying  disciplined  agile  and  lean  processes  within  the   context  of  your  business. Our  website  is  ScottAmbler.com We  can  help ©  Disciplined  Agile  Consortium   28
  29. 29. Additional Slides ©  Disciplined  Agile  Consortium   29
  30. 30. Values  for  Agile  Data  Management • Evolution  over  definition • Holistic  organization  over  data  management • Sufficiency  over  perfection • Collaboration  over  documentation • Cross-­functional  staff  over  specialized  staff • Automation  over  manual  processes ©  Disciplined  Agile  Consortium   30

×