Four Keys to Efficient DevOps
•Download as PPTX, PDF•
3 likes•568 views
Every DevOps implementation is unique, but certain challenges are the same. Establishing fast feedback loops, formulating a better definition of done, and anticipating technical gaps to delivery are all milestones to transformation. In this session, our panel of experts will share their experiences in DevOps transformation. Learn how to: -Improve your team’s definition of done to cut down rework -Identify automation gaps that lead to later holding patterns -Inject fast feedback to catch bugs as early as possible -Establish autonomy and ownership over winning customer-focused experiences
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Four Keys to Efficient DevOps
Similar to Four Keys to Efficient DevOps (20)
More from Perfecto by Perforce
More from Perfecto by Perforce (20)
Recently uploaded
Recently uploaded (20)
Four Keys to Efficient DevOps
- 1. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile 4 Keys to Efficient DevOps Patterns to Enable Transformation in Enterprise Development Web: perfectomobile.com Twitter: @perfectomobile
- 2. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile ● Session is being recorded ● Presentation will be shared ● Questions and conversation ○ Twitter: #DevOpsMilestones ○ WebEx chat panel ○ Q&A at the end of the webinar Housekeeping
- 3. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Today’s Panel Carlo Cadet Director, Product Marketing & Evangelism Perfecto Tzvika Shahaf Director, Product Management Perfecto Sean Williams Principal Software Engineer Prism Tech Studios James Brown Director, Enterprise Solutions VisionIT Raj Subramanian Senior UI Automation Engineer CSG International
- 4. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile What We’ll Cover 1. Improving your team’s Definition of Done 2. Addressing automation gaps early 3. Fast feedback in your pipelines 4. Autonomy for Customer-focused Experiences
- 5. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Poll: Where are you on your DevOps Journey? ● We fully embrace DevOps across our organization ● Many teams have re-organized and built efficient tool chains ● There are a few pilots underway to implement DevOps ● We talk about DevOps a lot, but haven’t quite started ● Not going there
- 6. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Improving Your Team’s Definition of Done 4 Milestones on the Path to Efficient DevOps Patterns to Enable Transformation in Enterprise Development
- 7. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Predictable flow ● Clock speed mismatch ○ Business needs to innovate ○ Development needs to deliver ● Delivering predictably ○ Establish cadence that business can rely on ● The “real” flow of our teams ○ Leftover work impacts next sprint’s bandwidth ○ Confidence in go / no-go Waterfall (months) Agile (weeks) DevOps (days)
- 8. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile What goes into a Definition of Done? Constructing: ● Common criteria ● Previous RCA on re-work ● Go/no-go blockers Improving: ● Retros: rejected work ● Minimize tech debt ● Better DoD fit per story https://www.scrumalliance.org/community/articles/2008/september/definition-of-done-a-reference
- 9. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Examples of tailoring Definition of Done ● Code produced (all ‘to do’ items in code completed) ● Built without errors ● Unit tests written and passed ● Peer reviewed and corrected ● Deployed to system test environment and passed system tests ● Passed UAT (User Acceptance Testing) ● Any build/deployment/configuration changes documented and communicated Perfecto: ‘Done is DONE: (partial list) ● Builds without warnings/lint? ● Automated tests ● Feature flags ● Peer code review ● Acceptance / UI review ● Feature documentation ● Code samples DoD ~ AC =Story Design & code Code pushed / JIRA updated
- 10. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
- 11. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Addressing Automation Gaps Early 4 Milestones on the Path to Efficient DevOps Patterns to Enable Transformation in Enterprise Development
- 12. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Areas of Automation “gaps” ● CI ● Device/Simulators ● Maintenance ● Test Coverage ● Performance From the Digital Quality Handbook (pg. 70): ● Android 6: Fingerprint ● Android 7: Split screen ● iOS 9: 3D Touch ● iOS 10: Voice commands bit.ly/digqual
- 13. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Why to find them early? The Key is... ● Quick Feedback ● Better Visibility ● Building “Trust” for the automated tests
- 14. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile How to find them early? ● Plan automation as a team ○ What areas need to be automated? ○ What areas are more stable? ○ Cost vs Value ● Treat automation as separate stories/requirements ● Do estimation with “buffer time” in mind
- 15. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Fast Feedback in Your Pipelines 4 Milestones on the Path to Efficient DevOps Patterns to Enable Transformation in Enterprise Development
- 16. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Enhanced Quality Visibility Throughout Your Pipeline
- 17. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Meet Velocity Release Goals Increase efficiency & Optimize UX Focus more on feature development and validation, less on analysis Empower continuous quality while eliminating flaky tests for faster cycles
- 18. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Tools & processes must deliver value
- 19. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Shortening the Feedback Loop cycle in CI
- 20. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile My Build failed – but why?
- 21. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile My Build failed – but why?
- 22. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Autonomy for Customer-focused Experiences 4 Milestones on the Path to Efficient DevOps Patterns to Enable Transformation in Enterprise Development
- 23. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Autonomy Process Autonomy Cultural Autonomy
- 24. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Process Autonomy Automated Builds and Deployments Dev,QA and Staging Environments Documentation … and more
- 25. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Cultural Autonomy Empower your team to do what’s best for the end user ● Share end user data ● Share business goals and priorities ● Nurture an owner mindset
- 26. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Takeaway: The definition of done is a fundamental communication of success, both within the DevOps team and out to the larger enterprise. Build “Trust” and “Respect” the automated tests Necessity of quality visibility across the entire pipeline to support timely decision making Teams are more motivated and make better decisions when development teams stay close to the real customer experience
- 27. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile What We’ve Covered 1. Improving your team’s Definition of Done 2. Addressing automation gaps early 3. Fast feedback in your pipelines 4. Autonomy for Customer-focused Experiences
- 28. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Q&A (open ended question about “what have you seen works?”)
- 29. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Additional Resources ● List of links from the group (1 per) ● The Path to Releasing Confidently in DevOps: http://info.perfectomobile.com/release-confidently-in-devops.html ● Peopleware: Productive Projects and Teams: https://www.amazon.com/Peopleware-Productive-Projects-Teams- 3rd/dp/0321934113/
- 30. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Appendix
- 31. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Abstract Title: 4 Milestones on the Path to Efficient DevOps How many new initiatives does it take to transform an organization? Answer: as many as it takes to drive change. Every DevOps implementation is unique but certain challenges are the same. In our experience with enterprises, establishing fast feedback loops over your code commits, formulating a better definition of done on features, anticipating technical gaps to delivery, and fostering a culture of autonomy are milestones to transformation. In this session, Perfecto Developer Advocate Paul Bruce will host a dialog with a panel of development experts, sharing their experiences in DevOps transformation. We will discuss: · Improving your team’s definition of done to cut down rework · Identify automation gaps in planning that lead to later holding patterns · Inject fast feedback throughout your pipelines to catch bugs as early as possible · Establish autonomy and ownership over winning customer-focused experiences
- 32. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile What you don’t know...hurts. Examples of automation “gaps”... ● Existing functionality ○ Appium on iOS 10 ○ Low platform coverage in CI ● New features ○ Hardware dependencies (camera, location, BLE) ○ Test coverage in DoD [Panelist attests, offer example] From the Digital Quality Handbook (pg. 70): ● Android 6: Fingerprint ● Android 7: Split screen ● iOS 9: 3D Touch ● iOS 10: Voice commands bit.ly/digqual
- 33. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile Who’s in the planning room? ● Idea in backlog (PM / PO) ● Prioritization / grooming (PO) ● Defining (PO / dev) ● Scoping (dev / test / ops) ● Estimation (dev / test / ops) ● Signoff (all) [panelist example of who’s in planning sessions]
- 34. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile A tale of two features [panelist] ● Feature 1: didn’t have the right people in the room ○ Subsequently, later automation was delayed due to technical gap ○ Impact on business was x ● Feature 2: all members in planning (DevOps) ○ Planning included domain knowledge of all delivery aspects ○ Potential gaps in automation were identified early ○ Estimation included this work ○ Delivery dates were met
Editor's Notes
- Sean: other “steak”-holders unrelated to the end user, DoD -> tailoring to teams ⇐ no longer a fan of his own question :-) James: what’s the most challenging aspects of improving development Raj: what does “automation” mean to you? Gaps come from diff. mindsets
- Why is this a thing? Perspective Developers focus on the code QA focuses on the testing Product Manager focuses on product Communication is Fundamental to Success History of software development Requirements System Requirement Specification Use Cases User Stories DevOps Complexity grows exponentially with team size Mutual Expectations Change is Expected Teams Stakeholders Vision Product Roadmap Politics Points Raj can contribute to this slide -Often Testers are often put in a situation to give go/no-go which is not right. Testers provide relevant information about the product and then the business needs to make that decision based on the information. Testers are “Information brokers” not “Gate Keepers”
- Definition of Definition of Done A clear, consistent yardstick yielding a binary answer Simple checklist Between 3 and 10 items Lightweight and Focused How Context Specific Team Driven Tailor based on context Dimensions of Done SDLC Development Testing Unit Functional Enterprise Project Management Quality Gates Architectural Review System Integration Deployment Product Shippable Customer/Client Dependencies Raj’s points -Have a checklist for Definition of Done and make it visible to the team -The story is complete only when all the points in the checklist are completed
- Sean’s example of DoD: At one company I was at, we had a very light product department relative to developers so getting full requirements was always a challenge. We ended up settling on Acceptance Criteria as the main definition of done. This was a blend of user stories and testing behavior. Here’s an example for adding a Forgot Password link to login: Go to the Log In page. There should be a “Forgot Password” link. Click on the “Forgot Password” link. You should go to a “Forgot Password” page, and it should match the attached mockup. Enter an invalid email. The form should show an error message (specified elsewhere in the requirements) Enter an email of a user that doesn’t exist in the system. The form should show an error message. And so on...
- Challenges Agreement on the definition Who Decides Buy in
- CI · Frequency - Want the automated tests to run each time a check in happens · Visibility - No one knows what is going on and when test fails/pass · Reporting · Identifying/eliminating flaky tests Device/Simulator/Emulator · Where do you want to run this on device or simulators? · On Simulators difficult to tests o Wi-Fi/4g/Airplane mode switching (Simulator and emulators cannot tap into Airplane mode) Maintenance · Who maintains the automated tests? · Who maintains the server machines running the automated tests? What if you run out of space or want to run multiple instances of your tests o People are going to cloud testing for this exact same reason Test coverage · Have 2 sets of tests o Smoke tests – Most Stable and only high level functionalities and runs quickly o Regression tests/suites – Run during regression testing phase or periodically based on requirement Performance · How fast does your tests run? Does it take 5 mins, 30 mins, 1 day? o People want quick feedback on this
- Raj talks about Need quick feedback if something fails/broken. Find defects “Early” Need to keep everyone involved in automation effort as it is a “Whole Team” approach not just the automation team . Make it visible to everyone through e-mails, TV boards, stand ups etc Make people respect the automated tests. If it fails, whoever broke if fixes it. It is not only the automation team members who keep fixing it. Have stable tests and remove flaky tests. Increase your and the automated tests credibility Sean can comment on: As a developer, I can attest to trust for automated tests being key. If you have a test that is failing sporadically for a reason other than broken functionality, disable the test until it can be fixed. You don’t want get developers used to seeing tests fail because if they see too many failing and they are not responsible for the failures, developers get the habit of being skeptical towards test failures - or even flat-out ignoring them. James: Automation empowers the team to find more complicated or subtle defects and perform more robust edge case testing The early identification of defects significantly reduces cost to remediate
- Raj talks about -How automation gaps are addressed during planning meetings -Who is there in the planning meetings - Whole Team includes PO/Solution architect/Tech Doc writer/Dev’s/QA/s -Examples on this slide --Through discussion how we identified not to automate “Guide Story” as there was another story which included complete redesign of the guide - Devs pointed it out and thus, it helps to plan automation as with the whole team. Saved a lot of time for automation testers --Automation needs considerable work/coding. Treat it a separate stories. Have “kick off” meetings on expectations/acceptance criteria, then have development, Code review, Testing, Demo and then Complete just like a normal development story --”Buffer Time” - Under-stimated different data setup needed for different types of products - Movies with different captions, Movies with different audio lang, Single Episode, mulit episode, rental products for one of the stories. Learnt a valuable lesson of under estimating.
- We use a lot of industry standard tools and cloud services here at Perfecto. Everyone has their tools,
- The goal of autonomy is to let people and processes fly with as little interference as possible Autonomy = end-to-end ownership of how to accomplish the goals that leadership sets Ever worked as the sole developer on a project or at a brand new company? Development goes very fast compared to larger companies. Part of that is because there are no existing systems, but part of it is that a lot of other stuff becomes involved outside of code. Two types of autonomy: Process autonomy (automation): construction of systems that minimize the time and error associated with repeated delivery; enables faster work Cultural autonomy: The business identifies goals, the team figures out how to accomplish them. Aligns team effort to delivering value through a great customer/end-user experience.
- Construction of systems that minimize the error associated with repeatable work. Minimize error, enable faster work. There are many things that fall into this, and you can fall anywhere on the range: Automated Builds and Deployments - Maybe you don’t have anything. Maybe you have automated builds but they need to be manually deployed, etc. Environments: maybe there’s no instructions for developers on how to create a local development environment. Maybe you have a staging environment that simulates production. Documentation: if you were new, how much information can you find out without having to ask someone who already knows it? And is it easy enough to get to that information? There are other examples: Automated UI tests, Unit tests What you need is very custom to your team and project: If you have a big team, just a small change in some of these can save significant time If you’re a small team, you may not need these Prioritize based on your team and tackle one thing at a time. To give an example, I have a project right now that is a small application that only needs to be up for three hours for a live event. Everything is manual - I upload files to the server when I change them, I manually SSH into the server and restart it. Doing automated builds/deployments there would be a waste of my time.
- Developers probably don’t use the product, or if they do, they don’t with the same mindset as an end user. It’s a culture thing, so must be slowly implemented over time Share end-user data with developers: email campaign results, A/B testing results, interesting GA data At one startup I was at in the entertainment industry, the analytics team sent out a bi-weekly report with the stats relevant to each dev team and an explanation of any changes. A/B testing results were available on the internal wiki. Let developers sit in on end user interaction: feedback sessions, focus groups, demos Product can share their notes from these sessions Share business objectives and priorities with developers: High-level business priorities (ex, quarterly goals) KPIs of features after rollout (ex, is conversion up after changing our contact form design?) Be aware of non-end user stakeholders: Investors External people w/purchasing power for your product At another startup in the energy sector, I’d sit in on demos to hear what people commented on. At one SaaS company, we had a feature that was never used but was in the product because people with purchasing power liked to see that it was there on demo.