Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Leveraging Cloud data to optimize your product decisions and Agile processes - Ronen Aseo & Efrat Egozi Levi

2,234 views

Published on

In this session we will share innovative directions in which the Hewlett Packard Agile Manager development team, leverages big data analytics to optimize its agile processes and align with customer feedback.

We will focus on
- What are challenges of developing a an Enterprise product in continuous delivery
- Personas involved in the process and their related challenges
- Importance of Customer feedback and incorporating it in the development process

We will discuss how production data is utilized to support decision making, prioritization and continuous improvement in development ,quality and product usability.

  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Leveraging Cloud data to optimize your product decisions and Agile processes - Ronen Aseo & Efrat Egozi Levi

  1. 1. © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Leveraging cloud data to optimize your Agile processes Ronen Aseo, Product Manager Efrat Egozi-Levi, Senior Data Scientist, CTO office
  2. 2. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
  3. 3. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3 HP Agile Manager defined Agile project management Release & User Story mgmt. Defect management ALM/PPM synchronization ALI integrations Key Capabilities A SaaS solution for Agile Instant-on service Regular, incremental releases driven by user feedback Web-based, intuitive UI Complements HP ALM/QC Key Characteristics
  4. 4. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4 Ø  3x number of users within the last year Ø  Monthly drops to production Ø  Vast internal adoption throughout HP Our Story http://www.tonycooke.org/
  5. 5. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5 HP Agile Manager Development Teams Agile Manager Product characteristics •  3 applications •  Teams are co-located •  On premise & SaaS product Products ALM Synchronizer Application Lifecycle Intelligence Agile Manager Sunnyvale, CAProduct Management Prague, CZDevelopment Yehud, ILDevelopment Ho-chi minh, Vi Development
  6. 6. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6 AgM Delivery Model SaaS Production Farms (External) Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Sprint 6 SaaS Production Farm (Internal – early adapters) Test Farm (trunk) Nightly deployment trunk Deploy after two weeks on internal farm, including required bug fixes RELEASE/PSI Internal branch Bug fixes Test Farm – internal branch Bug fixes Deploy after two weeks on internal farm, including required bug fixes Drop 1 Drop 2 Drop 1 Drop 2 Sprint length: 2 weeks
  7. 7. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7 Source: http://www.redeemthecommute.com/
  8. 8. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8 Challenges Roadmap Features Coverag e Resource s Customer Requests
  9. 9. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. BIG DATA - TO THE RESCUE Source: http://coffeebreakwithgamechangers.tumblr.com/
  10. 10. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10 Data Science Office @ HP Software Add data- science and analytic capabilities in HP products machine learning graph analysi s static / dynamic code analysis other technique s
  11. 11. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11 •  We want to make informed product/dev decision based on real users behavior •  We have real user monitoring today e.g. Google Analytics •  Information is event based not flow based •  Flow based is hard to understand •  There is code coverage but this doesn’t reflect real user usage •  What we need is real user coverage and insights Problem scope Event-based analysis in Google analytics
  12. 12. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12 Problem scope External Flows Ope n Chat Write Msg Attac h Send SendLogi n Test Flows Feature: Use Chat Login Open Chat Write Msg Send Ope n Chat Write Msg Attac h Send SendLogi n Ope n Chat Write Msg Attac h Send SendLogi n
  13. 13. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13 Real user test coverage algorithm •  Difference between two flows based on their similarity score definition •  The similarity score combines 2 criteria (Law & Order): –  the identical steps, which exist in both flows –  the similarity in the step order in each flow •  Match flows based on the difference function with an adaptive sensitivity threshold •  Compute the weighted similarity for a set of matched production flows per test Production Flows Internal / Test Flows 1. Login > 2. Open Chat > 3. Write Msg > 4. Send 1. Login > 2. Open Chat > 3. Attach > 4. Write Msg > 5. Send 1. Login > 2. Open Chat > 3. Send 1. Login > 2. Open Chat > 3. Attach > 4. Write Msg > 5. Write Msg > 6. Write Msg > 7. Attach > 8. Write Msg > 9. Send 1. Login > 2. Open Chat > 3. Write Msg > 4. Send > 5. Attach Usage : 50% Coverag e: 78% Feature X 84 78 70 75
  14. 14. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14 Key Findings Ø  The automated tests cover 35% of production flows Ø  50% of production events are not covered by any automatic test Ø  Interesting insights from comparing internal vs. external user flows From the motion picture “Moneyball”,2011
  15. 15. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15 Significant increase in real user test coverage after gathering insights 0 10 20 30 40 50 60 70 80 100 150 200 250 300 350 400 450 Drop 1 stabilization Drop 2 New Content #tests Realusercoverageintests
  16. 16. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16 Example: Dev Summary External Flows Internal Flows Release mngmnt > Taskboard > Zoom > ali summary > all > builds Release mngmnt > Taskboard > Zoom > ali summary > all > retrospective Release mngmnt > Taskboard > Zoom > Kanban > Kanban: Configure Team Storyboard> project configuration teams > Taskboard > Zoom > ali summary > all > shared items •  Release mngmnt > Taskboard > Zoom > ali summary > all > retrospective > Taskboard > Zoom > Kanban > Kanban: Configure Team Storyboard> project configuration teams > tab: Storyboard •  Release mngmnt > Taskboard > Zoom > Kanban > Kanban: Configure Team Storyboard> project configuration teams > Taskboard > Zoom > ali summary > all > shared items •  Release mngmnt > Taskboard > Zoom > ali summary > all > builds •  Release mngmnt > Taskboard > Zoom > ali summary > all > retrospective •  Compared: external customers to internal customers •  Surprise: page is used similarly, regardless of dev configuration •  => Cancel “prompt to config” plans Feature: Dev Summary Same usage in external and Internal
  17. 17. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17 Example: Planning board •  Planning board event is rarely used internally – Should we invest? •  Surprise: Planning board is used as the 1st step in many flows Planning board Kanban Release backlog [?] Sprint Backlog …
  18. 18. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.18 •  Surprise: “Sprint Closure” flows have high usage (>20%) but low coverage (3%) •  Sprint closure should have high priority and additional test automation Example: Sprint closure
  19. 19. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19 Benefits - Recap Data driven decision making Better understanding & evaluation of customer needs Improved real user test coverage Optimize resource allocations
  20. 20. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Thank You Contact: ronen.aseo@hp.com eegozi@hp.com
  21. 21. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21 Release Planning •  Feature Ranking •  Feature Scoping - MMF •  Initial Feature Sizing •  Define Feature Lead •  Define Release Scope
  22. 22. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22 Definition Of Done - DoD ü  Unit tests are written and green ü  Acceptance tests are defined by QA and passed on top of a build ü  Main functionality of the user story is covered by automated acceptance tests ü  Sanity and Regression tests are passed green after the user story is checked in 0 C/H open defects ü  Majority of Medium defects are fixed ü  All fixed defects are validated ü  The user story is pushed to production, monitored and green
  23. 23. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23 Delivery Model to Production Release Criteria Progress Status Target Current Status Test Coverage On Track 100% •  100% Quality On Track No Critical defects No C/H New features defects No C/H customer encountered defects No Functional Regression defects <10 High defects 70% M defects fixed •  0 Critical defects •  0 C/H New features defects •  1 C/H customer encountered defects •  3 Functional regression •  10 open High defects •  61% of the M defects were fixed Performance On Track No C/H issues No C/H issues Security On Track No C/H issues No C/H issues
  24. 24. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24 Product Owner Challenges •  How to prioritize features and enhancement requests? •  Can I cut a story from a feature and it will still have value for the user? (MMF) •  How will a new feature impact current usage? •  Does my acceptance criteria cover the end user’s flow? •  Are features being used the way they are “supposed” to? •  Is there a difference between how features are used internally vs. externally?
  25. 25. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25 Team Challenges •  How much to invest in automation? •  What areas to automate and how to automate? •  What to include in our Regression testing?

×