• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Testing in a continuous delivery environment

Testing in a continuous delivery environment



Talk at Iqnite conference for software quality and testing about how we have do continuous delivery at here.com ...

Talk at Iqnite conference for software quality and testing about how we have do continuous delivery at here.com

Together with Siddharth Somasundaram




Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Testing in a continuous delivery environment Testing in a continuous delivery environment Presentation Transcript

    • Testing in a Continuous Delivery Environment Siddharth Somasundaram Stefan Verhoeff
    • HERE ● Formerly known as Nokia / NAVTEQ Maps ● Powers Bing, Facebook and Yahoo Maps ● 4 out of every 5 cars with in-dash navigation are powered by HERE
    • HERE ● Maps on the Web offering here.com
    • Testing Before Agile ● Testers are only involved in the later stage ● Long regression testing cycle before release ● Developers and Testers have separate work streams ○ Leads to conflicts between them ● Business pressure to release the software ○ Compromise on quality?
    • Importance of Agile ● Requirements change over time ● Testers are part of the planning process ● The entire team is responsible for Quality ● Faster release cycle ○ From once every 3 months to daily releases
    • Continuous Delivery ● Release early and often to production ● Automate testing and deployment ● Faster feedback, reduce risks and overhead for releasing ● Hot trend in the industry, especially web companies
    • But what about Quality Assurance? ● Lots of automation for regression testing ● Build enough trust in the test coverage ● Focus shifts from prevention of bugs to rapid discovery and resolution ● Exploratory testing can be decoupled from releasing
    • How do we do it? ● Team Culture and Technical practices we found to be useful
    • Team Culture Practices ● One agile Team, One Product and One Goal ○ Product Owner ○ Scrum Master ○ Developers / Testers ○ Offshore teams ● Communication solves most of the problems
    • Collaborate using Work Agreements The team formulates rules and hold each other accountable. For example: ● Running tests before committing ● No commits on red build ● First priority is to keep the build green
    • Build Monitor visible in every room
    • Pair Programming
    • Share knowledge with Tech Talks
    • Combine developer and operation teams
    • Technical Practices
    • Continuous Integration
    • Test Driven Development ● Code and tests are developed at the same time
    • Build Pipeline ●
    • Test Coverage at all levels © Alister Scott http://watirmelon.com/2011/06/10/yet-another-software-testing-pyramid/
    • Eliminate Flaky Tests ● Essential to keep the team trusting the test results ● Mocking dependencies
    • Feature Flags ● No branching ○ Prevent integration problems ○ Quality built in ● Turn features on in production when they are ready ● Quality boundaries for different release stages
    • Logging & Monitoring ● What happens after we go live? ● Production monitoring ensures edge cases are covered ● Real-time metrics for: system, application, business
    • Logging & Monitoring ● Quickly responding to alerts ● Production is the ultimate proof the code works Bug introduced We fixed it :-)
    • In Conclusion...
    • Is it Successful? The Metrics 2010 Now Release quarterly Release daily Mostly manual testing 7000+ unit tests, 400+ GUI tests Preparing release takes weeks Release package ready in 45 minutes Deployment takes days Deploy 3 sites in 10 minutes Management signoff required Team empowered to make changes
    • Deployments 2012 vs 2013
    • Business Benefits ● Reduce risk involved in big changes ● Reduce time to market ● Smaller more frequent feature rollouts ● Quickly learn from feedback on product changes
    • Business Benefits ● Predictable releases ● Improved team morale ○ Less time spend in stressful firefighting ○ Less bug fixing, more feature delivery ○ Satisfaction of delivering to the customer
    • Challenges ● Everybody agreeing on the process ● Discipline comes over time only ● Feedback needs to be given quickly ● High investment in test automation and keeping it running smoothly ● Need to react to incidents quickly
    • Future Plans ● Deploy every build to production ● Keep on investing in Automation and Monitoring ● Canary deployments to validate new releases before global rollout ● Leverage quick cycle time for A/B testing driven feature development
    • HERE Checklist Team Culture ● One Team ● Work Agreements ● Build Monitor in the room ● Pair Programming ● Tech Talks ● DevOps Technical ● Continuous Integration ● Test Driven Development ● Build Pipeline ● Test Coverage on all levels ● Feature Flags ● Logging & Monitoring
    • Merci Beaucoup Thank You Danke Schön Grazie