Agile Testing

Agile testing tools and approaches Anand Ramdeo Nathan Bain

What we were building?

Adopting Agile.

Scrum Product Owner, Technical Owner, Scrum Master & Team including tester.

Sprint Product Backlog Sprint Backlog 3 Week Sprint Daily Burn New Priority Product Owner Next Sprint Please. Technical Owner

Process Highlights

PO, TO and tester to meet before sprint planning to complete stories and acceptance criteria.

Four column task board to ensure every task is tested.

Unit testing, code review, show and tell – They were part of every story.

End of sprint demo, bug bash, retrospective and release (when applicable) were part of every sprint.

Approach to Testing

Test automation using open source tools like FitNesse, Selenium, Twill etc.

Manual exploratory testing.

Very little emphasis on documented test cases as such.

Feature capabilities metrics during releases.

Defect management using Jira.

Feature Capability Metric Can be reported Can be moderated Comments Can be scheduled Can be published Article Comments Opera Safari FF3 FF2 IE8 IE7 IE6 Feature

Environment & Automation

Python, Django, PyUnit, Selenium RC (with Python), Twill, FitNesse (With WebTest), .NET etc.

Continuous Integration for all the projects.

Continuous and auto deployment for 'demo' environment.

Manual deployment for 'snapshot' environment.

A bit of Twill

Open source library in Python.

Can be used for testing web application – below browser.

Best suited for validating at page source, http request / response level.

Can be used from command line in interactive mode or from Python scripts.

Extremely powerful if used with other Python libraries.

How we used it? 1. Get Page 2. Get All the links 3. Get first link and if link is not external and crawler has not visited it, open link. 4. Get Page Source 5. Validate all the rules you want to validate on this page 6. Repeat 1 to 5 for all the pages. * Title and meta tags like keywords and description are present for all the pages and is not generic. * Instrumentation code is present on all the pages. * Every image has an alternative text associated with it. * Ad code is coming from the right server and has all the relevant information we need. * Every page contains at least two ads and no more than four ads. * And many more… Pseudo Algorithm Validation Rules Links pointing to content server in stage environment. Editorials not using SEO tools to populate meta tags. Links used by editors are dead. SA Sample Defects

A bit of Selenium

Selenium RC was used with Python.

Domain Specific Language was created for abstraction and robustness.

Started exploring ui-elements to abstract element locators.

Automation was driven from feature – capabilities and releases where possible.

Abstraction With UI-Element

selenium.type("q", “TestingGeek");

selenium.click("btnG");

LOGIN_BUTTON = "css=input[value='Log in']" DELETE_CONFIRMATION = "css=input[value='"Yes, I'm sure"']" myMap.addPageset({ name: 'allPages', description: 'All WLL Pages', pathRegexp: '.*' }); myMap.addElement ('allPages', { name: 'register' , description: 'Register link on all the pages' , locator: "xpath=//*[@id='user']/dd/a[1]" }) sel.click("ui=allPages::register()")

DSL & Data Driven Testing

stations =

{ 'sgrfm' : { 'splitter' : '/east/ipswich/sgr-fm.html', 'link' : 'Go to heart ipswich', 'cookie' : 'heart_ipswich' },

More stations…

gusto_dsl.login_to_admin(…) gusto_dsl.write.an_article(….) gusto_dsl.goto_sitehome_from_gusto(…) gusto_dsl.open_page(…) gusto_dsl.asserts_for_articles(..)

Use power of language for Data Driven Testing

Invest in creating Domain Specific Language to make your tests readable, maintainable.

# Go to home page # Go to splitter page for a specific station # Click on the link to go to the new website # Go back. # Ensure that cookies is set up properly. # Go to the home page. # Ensure that home page is redirected automatically based on the information in cookies.

Would have been better if..

Selenium test suite, Twill etc. were also included in CI.

Automation for stories were included as a task in the story itself.

Session Based Test Management was used to highlight manual exploratory testing.

Controlled releases, technical debt management and more test automation.

Key Learning

If you have few testers in a team, you can afford to automate. If you do not have, you can not afford to not automate.

Thank You.

Anand Ramdeo

www.TestingGeek.com

@testinggeek

www.AtlantisSw.com

[email_address]

Nathan Bain

Things I wish I had known as a new Agile Tester

You are the testing expert!

Agile is a new practice – the rulebook is still being written

Software Development Methodologies are not written with testing in mind

We are just beginning to understand how Agile testing fits in

Huge amounts of information being published every day on blogs, user groups etc.

What should I do?

Listen and Learn

Read, Research & Follow

Discuss, Suggest, Offer an Opinion and come to an Agreement

As a non-tester, how can I help?

Share your opinion on how testing should be approached on your project

If you disagree, then discuss this with the test team

Try to be Constructive and Supportive

If you don’t feel involved, then involve yourself!

Testing is no longer a phase in a project – it’s a continuous activity – testers should always be involved.

I found that I wasn’t invited to meetings discussing the fundamental requirements of projects.

If you don’t capture the requirements early, then how can you test them later?

What should I do?

Invite yourself to meetings

Explain to team members the importance of capturing requirements early

Suggest how you can add value

Keep the tester involved in all phases of product development.

Invite testers to products meetings.

Don’t assume that they are not going to be interested.

Don’t assume that testers have nothing to contribute to certain meetings.

No more quality police!

Testing on waterfall was a phase – we were paid to find as many bugs a possible in a short space of time

Finding a bug was a triumph – but was that how the developers saw things?

Fostered an Us V’s Them relationship

Now we are all on the same team

What should I do?

Keep finding bugs – you will be thanked for it!

Communicate first, raise a bug report later.

Get into the team spirit - contribute to the teams efforts.

Treat your testers as you would any other team member.

Discuss bugs with the testers – it may not require a bug report.

Recruit your own deputies

There are never enough testers

There will be times when testing becomes the bottleneck on the project due to the lack of resources

Other times, testing something becomes the priority – set release dates!!

What should I do?

Agile teams are multi-disciplined – if one area is suffering, then we should all help put

Recruit your Developers, Project Managers, Product Managers and anyone else to help out with testing

Be open to the idea of testing when needed

Keep an eye on the project board – if testing is a bottleneck, then offer to help out

ABC – Always Be Communicating

No more huge documentation documents.

Requirements are communicated via emails, wiki’s, presentations, IM conversations, - verbally!

If you don’t communicate – then the steam train will keep on rolling, and you have missed an opportunity

What should I do?

At first, you may miss your requirements documents – learn to communicate with team members and stakeholders

Get involved in meetings

Don’t be afraid to ask questions – that’s what Product Manager’s are there for

Communication is a 2-way process.

Copy your testers in on all your project emails – don’t assume it’s not relevant to them.

Invite testers to meetings – formal and informal.

Brett Pettichord “ The reason Agile teams can do with less planning is because feedback allows you to make sure that you are on course. If you don’t have meaningful feedback then you’re not agile. You’re just in a new form of chaos. ”

http://www.io.com/~wazmo/blog/archives/2008_08.html#000285

Be Organised, yet Lightweight

As the name suggests, Agile team members should be quick to react and not bogged down by paperwork, processes etc.

However, you cannot work without some kind of structure and planning

You also need to be able to track your progress and report back to the team

Be Organised, yet Lightweight

What should I do?

You don’t need huge test scripts which detail every button click - lists of features, areas to be tested, browser combinations etc. will help planning/tracking.