0
Laurent PY
CEO, Smartesting
Laurent.py@smartesting.com
@py_laurent

www.smartesting.com

Acceptance testing
in DevOps
Our long and painful
journey
towards DevOps
Overview of our development process 2004/06


Product: CertifyIt, eclipse plug-in in JAVA (Model-Based-Testing)
– Waterfa...
Tasting agility end of 2006


Product: CertifyIt, eclipse plug-in in JAVA (Model-Based-Testing)
– First XP and then Scrum...
Now we are experiencing DevOps - 2012


Product: Zest, testing platform in the cloud
– Still Scrum process
– We do TDD an...
What we’ve learnt


Shorter iterations (1 to 4 weeks) lead to
massive test automation completed by
exploratory testing

...
What we’ve learnt


To achieve this level of speed (DevOps context),
acceptance tests should be:
– Understandable to both...
ATDD & Refactoring

My view as product owner & tester:
Acceptance tests need to be continually
reviewed and refactored jus...
ATDD & Refactoring

My view as product owner & tester:
Acceptance tests need to be continually
reviewed and refactored jus...
Acceptance testing
driven development
Shift left
Daily meeting

Product Backlog
Sprint Backlog

Product at the end of the
iteration

Sprint
1 to 4 weeks

Accept...
Acceptance Testing Driven Development (ATDD)


Acceptance test is a powerful artifact to improve
communication



Test a...
Acceptance Testing Driven Development (ATDD)


Benefits:
– Improve communication and collaboration between project
stakeh...
Continuous acceptance testing
Zest: create acceptance tests with DSL


Key features:
–

–

Suggestion to promote the reuse of Action Word and avoid dup...
Collaboration through acceptance tests & DSL
Tester
Create acceptance
Tests

Product Owner
Validate acceptance
tests

Deve...
Define new business entities…

Define progressively your business terminology with
Action Words. Enable collaboration base...
…or define business entities right from the tests

Promote test steps into Action Words (what developers
call extract func...
Evils of duplication
A fundamental principle
Reuse, reuse and reuse Action Words!

Suggestions

Create and maintain consistent scenarios for
your project
Analyse and optimize continuously your tests

When duplications are identified, refactoring
options are suggested!
Add, remove, modify Action Words and Scenarios

Add parameters to Action Word

Propagate automatically to
the scenarios

T...
Generate scripts

Use of Action Words significantly decrease the
cost of automation and accelerate the overall testing
cyc...
So your acceptance tests are…
Understandable
Definition of business domain tests enable
better alignment of the team

• DS...
And remember

Acceptance tests need to be
continually reviewed and
refactored just like code!!!
Give a try
Zest-testing.com
Laurent PY
CEO, Smartesting
Laurent.py@smartesting.com
@py_laurent

www.smartesting.com
Upcoming SlideShare
Loading in...5
×

Acceptance Testing in DevOps

1,046

Published on

To support efficiently ATDD or BDD at the speed and scale required in Agile and DevOps projects, your acceptance criteria need to:
- Be understandable by business experts, Product owners, developers (defining the business terminology)
- Facilitates test authoring as well as maintenance (based on a semantic and not just text)
- Can be transformed automatically into code for test execution

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,046
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Acceptance Testing in DevOps"

  1. 1. Laurent PY CEO, Smartesting Laurent.py@smartesting.com @py_laurent www.smartesting.com Acceptance testing in DevOps
  2. 2. Our long and painful journey towards DevOps
  3. 3. Overview of our development process 2004/06  Product: CertifyIt, eclipse plug-in in JAVA (Model-Based-Testing) – Waterfall process – 1 release every 6 months – Few tests and no TDD – 1 month (x5 people) to do release testing before deployment – Poor quality impacting customers’ feedback and adoption 3
  4. 4. Tasting agility end of 2006  Product: CertifyIt, eclipse plug-in in JAVA (Model-Based-Testing) – First XP and then Scrum process – We do TDD – Continuous integration (code & unit test) – 1 customer release every 3 months (1 operation release every 3 weeks) – 1 man/month to do release testing before deployment – Good feedback from customers! 4
  5. 5. Now we are experiencing DevOps - 2012  Product: Zest, testing platform in the cloud – Still Scrum process – We do TDD and… – …Acceptance Testing Driven Development (ATDD), 100% automated – Acceptance tests are part of the CI process – We do several deployments a day (≈10) 5
  6. 6. What we’ve learnt  Shorter iterations (1 to 4 weeks) lead to massive test automation completed by exploratory testing  Acceptance tests become the specification  Testing starts earlier in the development process: Shift left! Req Management & Definition Test Planning Execution Defect management
  7. 7. What we’ve learnt  To achieve this level of speed (DevOps context), acceptance tests should be: – Understandable to both developers and business experts to strengthen alignment and enable the ‘shift left’ – Maintainable to manage efficiently changes in requirements and keep pace with continuous deployment – Automated to enable rapid execution and feed-back  ATDD with Business Domain Language and refactoring capabilities 7
  8. 8. ATDD & Refactoring My view as product owner & tester: Acceptance tests need to be continually reviewed and refactored just like code!!!
  9. 9. ATDD & Refactoring My view as product owner & tester: Acceptance tests need to be continually reviewed and refactored just like code!!! Martin Fowler
  10. 10. Acceptance testing driven development
  11. 11. Shift left Daily meeting Product Backlog Sprint Backlog Product at the end of the iteration Sprint 1 to 4 weeks Acceptance testing Shift left Acceptance testing Scrum team PO developers testers Scrum master
  12. 12. Acceptance Testing Driven Development (ATDD)  Acceptance test is a powerful artifact to improve communication  Test as the definition of ‘STOP’  Written prior to development by tester  Based on a business DSL (domain specific Language)  Confirmed with stakeholders  Mostly automated Test in natural language Test fixture Code
  13. 13. Acceptance Testing Driven Development (ATDD)  Benefits: – Improve communication and collaboration between project stakeholders – Shared understanding of what a successful implementation means – Better coverage of business expectations – Faster feed back  Challenges: – New methodology that requires rigor and discipline – Find the right balance between people/process/tool
  14. 14. Continuous acceptance testing
  15. 15. Zest: create acceptance tests with DSL  Key features: – – Suggestion to promote the reuse of Action Word and avoid duplication – Refactoring: when an Action Word is modified, impacts are performed automatically across the tests – Optimization: inspection features enable to continuously optimize the acceptance tests –  Define progressively your Action Words and your business domain language for test authoring Create scripts and accelerate test automation (Ruby, Java, XML…) Integrations with: Watir, Appium… 15
  16. 16. Collaboration through acceptance tests & DSL Tester Create acceptance Tests Product Owner Validate acceptance tests Developer Automate acceptance tests • DSL • Refactoring capabilities 16
  17. 17. Define new business entities… Define progressively your business terminology with Action Words. Enable collaboration based on acceptance tests.
  18. 18. …or define business entities right from the tests Promote test steps into Action Words (what developers call extract function). This is refactoring!
  19. 19. Evils of duplication
  20. 20. A fundamental principle
  21. 21. Reuse, reuse and reuse Action Words! Suggestions Create and maintain consistent scenarios for your project
  22. 22. Analyse and optimize continuously your tests When duplications are identified, refactoring options are suggested!
  23. 23. Add, remove, modify Action Words and Scenarios Add parameters to Action Word Propagate automatically to the scenarios Test refactoring enables to perform automatically impact analysis and test maintenance tasks
  24. 24. Generate scripts Use of Action Words significantly decrease the cost of automation and accelerate the overall testing cycle
  25. 25. So your acceptance tests are… Understandable Definition of business domain tests enable better alignment of the team • DSL • Refactoring capabilities Can be automated Good design based on Action Words streamlines the test automation phase Maintainable Refactoring and optimization features dramatically accelerate maintenance
  26. 26. And remember Acceptance tests need to be continually reviewed and refactored just like code!!!
  27. 27. Give a try Zest-testing.com Laurent PY CEO, Smartesting Laurent.py@smartesting.com @py_laurent www.smartesting.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×