1. The document describes an iterative software testing process used for Scrum and Waterfall projects at the Nokia Technology Institute.
2. The process involves test planning, specification, execution and reporting using open source tools like TestLink, Mantis, Selenium, and Marathon. Automated tests were created to test web and desktop applications.
3. The process was applied to two projects - a customer survey system using Scrum and a factory production monitoring system using Waterfall. Test activities were broken into iterations to prioritize testing of key features. Automation helped improve test coverage, efficiency and catch defects earlier.
SAP Testing with SolMan and SAP Quality CenterLaxmi2121
This Presentation introduces you to the latest testing techniques using tools and capabilities of SAP Solution Manager along with HP’s Quality Center and gives you strategies and best practices to evaluate and improve your integrated testing procedures and environments.
At SQA Solution the objectives of SAP System Testing are to verify that the installed system, which includes the SAP software, custom code and manual procedures, perform per business requirements:
Executes as specified and without error.
Validates with the users and management that the delivered system performs in accordance with the stated system requirements.
Ensures that the system works with other existing systems, including but not limited to interfaces, conversions, and reports.
Do The Right Thing - Empowering Your Test TeamsSmartBear
Margo Visitacion of Forrester Research discusses how empowered testing teams can work effectively with their business counterparts to plan and execute highly focused test cycles that, through a combination of manual and automated testing will assure that the highest quality software is being released.
SAP Testing with SolMan and SAP Quality CenterLaxmi2121
This Presentation introduces you to the latest testing techniques using tools and capabilities of SAP Solution Manager along with HP’s Quality Center and gives you strategies and best practices to evaluate and improve your integrated testing procedures and environments.
At SQA Solution the objectives of SAP System Testing are to verify that the installed system, which includes the SAP software, custom code and manual procedures, perform per business requirements:
Executes as specified and without error.
Validates with the users and management that the delivered system performs in accordance with the stated system requirements.
Ensures that the system works with other existing systems, including but not limited to interfaces, conversions, and reports.
Do The Right Thing - Empowering Your Test TeamsSmartBear
Margo Visitacion of Forrester Research discusses how empowered testing teams can work effectively with their business counterparts to plan and execute highly focused test cycles that, through a combination of manual and automated testing will assure that the highest quality software is being released.
Running Realistic Load Tests: Answers to the Most Important Questions - Yours!SmartBear
Hear two software industry veterans as they discuss the answers to the load testing you asked. Get a better understanding of how to make your load tests more realistic. In this webinar, Lino Tadros, CEO of Falafel Software and Sergei Sokolov, Director of Testing and Performance Products at SmartBear Software team up to lead a dynamic discussion about load testing.
StarWest 2012 - Agile Defect Management: Focus On PreventionDavid Jellison
Efficient agile organizations focus on defect prevention rather than downstream defect discovery, because discovering defects during or after testing adds to development costs. Delaying discovery and repair of defects can make an agile team feel like they are operating in a mini-waterfall. Sharing his experience with Scrum/ Kanban teams, Leveraging the lean concept of limiting work in progress, David Jellison describes how grouping defects into two major categories—work-in-progress defects and escaping defects—reduces development costs and improves reliability in the field. Dave illustrates how to manage problem discovery early and minimize the existence of escaping defects. Treating escaping defects as the exception rather than the norm results in a much smaller defect backlog and increased customer satisfaction. This approach encourages increased collaboration between quality engineers and developers, and shifts the focus of team measures from defect counts to product delivery velocity and cycle time, with increased confidence in quality as the work is completed.
Running Realistic Load Tests: Answers to the Most Important Questions - Yours!SmartBear
Hear two software industry veterans as they discuss the answers to the load testing you asked. Get a better understanding of how to make your load tests more realistic. In this webinar, Lino Tadros, CEO of Falafel Software and Sergei Sokolov, Director of Testing and Performance Products at SmartBear Software team up to lead a dynamic discussion about load testing.
StarWest 2012 - Agile Defect Management: Focus On PreventionDavid Jellison
Efficient agile organizations focus on defect prevention rather than downstream defect discovery, because discovering defects during or after testing adds to development costs. Delaying discovery and repair of defects can make an agile team feel like they are operating in a mini-waterfall. Sharing his experience with Scrum/ Kanban teams, Leveraging the lean concept of limiting work in progress, David Jellison describes how grouping defects into two major categories—work-in-progress defects and escaping defects—reduces development costs and improves reliability in the field. Dave illustrates how to manage problem discovery early and minimize the existence of escaping defects. Treating escaping defects as the exception rather than the norm results in a much smaller defect backlog and increased customer satisfaction. This approach encourages increased collaboration between quality engineers and developers, and shifts the focus of team measures from defect counts to product delivery velocity and cycle time, with increased confidence in quality as the work is completed.
Agile teams deliver working, fully tested software every 1 to 4 weeks.
New teams wonder how testing can fit into that timeframe.
Join us as we walk through the typical test activities within an Agile iteration.
In software testing, there are many paths between the entry and exit of a software program. So it’s difficult to fully test all paths of even a simple unit. This is a challenge when we design test cases.
This is a draft of a presentation for a course on Visual Studio 2010 Unit Testing, I've uploaded mainly because I tried to create a Metro Style presentation, so if everyone like it, he can use as base for own presentation.
Releasing fast code - The DevOps approachMichael Kopp
Agile makes you Develop faster, DevOps also makes you Deploy faster but how do you make your Application faster?
Many currently used Performance Management practices don’t work anymore as they are too time consuming. It takes a new approach to track performance in Continuous Integration, get more value out of Load Testing and leverage production data for performance optimization.
We will show you real world examples on how the new DevOps approach can work.
This ppt is done by my dear classmate Sap, almost each ppt I have uploaded is copied from net and other sources.I hope this will b a little useful for students..
Agile Open Source Performance Testing Workshop for Business ManagersClever Moe
Use Selenium, soapUI, Sahi, TestMaker Performance Testing In Your Organization. These are slides from a free Webinar titled "The Agile Open Source Way To Load Test, Scalability Test, and Stress Test In Your Organization". Your organization may have adopted Agile Software Development Methodology and forgot about load and performance testing! Learn the Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
Where Neev can help? A few Use Cases
Why Neev as an Independent Testing Partner?
Capabilities
Functional Testing (Web / Mobile / Cloud)
Non-Functional Testing
Testing Tools (Web / Mobile)
Agile Testing Expertise
Case Studies
Agile Open Source Performance Test Workshop for Developers, Testers, IT OpsClever Moe
Training For Selenium, soapUI, Sahi, TestMaker Performance Testing. Slide deck from the free Webinar titled "Technical Training On The Agile Open Source Way To Load Test, Scalability Test, and Stress Test." Learn the Agile Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.
Implementing Test Automation in Agile ProjectsDominik Dary
All new features at eBay Europe are developed using SCRUM. One key success factor for those projects is to have a reliable end-to-end test automation safety net. This presentation illustrates how in addition to a robust automation toolset it is essential to have an integrated approach to test automation design:
Test Aspects - Test Aspects are used to do the functional design of the end-to-end automation test cases. Since this is done upfront, the tester is able to focus on the what rather than the how.
Modeling of the Biz Domain Layer - The Biz Domain Layer is an abstraction layer above the user interface that is implemented in the test code. This layer is divided into pages and flows which are then used in the tests.
Test Implementation - Tests are written in Java, stored in SVN and are executed using the WebDriver Grid (Selenium2). Tests execution can be triggered by all team members via a continuous integration server (Hudson).
Lean Test Automation – it is important to retain and maintain the quality of the tests. Key success factors for this are: Code Reviews, Software Craftsmanship, Test Aspect Reviews and the “Definition of Done”.
Following an integrated approach to test automation ensures high efficiency, low overhead and easier maintenance.
Ashwini Malthankar
Organization: ThoughtWorks
Topic: Effective use of CI by QA
Presented in vodQA - THE TESTING SPIRIT! on Oct 7, 2010 in ThoughtWorks, Pune
Selenium is a software testing framework for automating tests of Web applications. Join Frank Cohen for a Workshop to learn the basics of Selenium. A screencast for this presentation is on the PushToTest.com Web site.
Functional testing addresses particular aspect of the user experience, accessibility, and performance, so when your users receive the app's final version, they can be sure that it works seamlessly. Functional Testing is an essential aspect of application development and helps ensure the app that you are building is stable and works well
Sbqs 2010 Processo de Teste de Software para Scrum
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
1. 22nd IFIP International Conference on Testing Software and Systems – ICTSS 2010
QUALITY
Federal University of Amazonas - UFAM VALIDATION
Nokia Technology Institute - INdT
Bug
MANTIS
INdT
Iterative Software Testing Process for Scrum and
Waterfall Projects with Open Source Testing Tools
AGILITY
Experience
SELENIUM
Verification TestLink
Eliane Collins & Vicente Lucena
SCRUM Natal – RN , Nov 2010
2. Summary
1. Introduction
2. Environments Context
3. Software Testing
Process
4. Execution Results
5. Conclusion
Company Confidential
4. INTRODUCTION
Different projects with different development process
request Test activities.
The Software Testing process must be agile for all projects.
Project 1 used Scrum. It is an agile development process
focusing on teamwork .
The test activities must follow the scrum and they may be
executed efficiently and fast .
Company Confidential
4
5. INTRODUCTION
The Project 2 used Waterfall software development process.
The system test activities must be done for the whole system
in the test phase.
Software Testing automation is one of the main approaches
that has been applied to decrease testing costs and time.
Test Case
Test Case Test Case
Test Case
Test Planning
Test Planning Defect Report
Defect Report
Specification
Specification Execution
Execution
Sequential software testing process
Company Confidential
5
7. Environments Context
Nokia Technology Institute – INdT
It is an independent and nonprofit institute committed to conducting
research and development of technological solutions through
application development, new technologies and concepts.
The INdT area where the experiment was designed is called Product
Manufacturing Technologies (PMT).
Company Confidential
7
8. Environments Context
Project 1
A system to provide an online customer satisfaction survey result to
the development team by e-mail.
The system registers the projects, leaders, customers, team members
and the customers’ answers.
The system sends the reminders to customers online to answer the
questions on the survey and forward the results to the development
team.
The project was developed under the Scrum process.
Web Platform: MVC Architecture, Ruby and Java languages, Rails
framework and MySQL database.
Company Confidential
8
9. Environments Context
Project 1 - Scenario
In the schedule there wasn’t any time planned for test activities.
No formal Software Testing process was done.
Just one test analyst.
Short time for test execution.
No financial resources available for this activity.
Company Confidential
9
10. Environments Context
Project 2
A software project to provide a graphic analysis of factory production
tests in real time in order to advise professionals to support
manufacturing on the quality trends for production line failures.
The project development used the Waterfall process approach.
Software platform : Client-Server, Java Swing, IDE Eclipse environment
and Oracle database.
The System has twelve screens, where six of them were used to show
and analyze graphic charts and the other six had the data inputs,
reports and user configuration.
Company Confidential
10
11. Environments Context
Project 2 - Scenario
In the schedule there wasn’t any time planned for test activities.
No formal Software Testing process was done.
Outdated documentation of the system requirements.
Functionalities implemented but not tested
No financial resources available for this activity.
4 weeks for a tester and a test analyst to do all test activities for the
whole system
Company Confidential
11
13. Software Testing Process
Iterative Software Test Process
Test Inputs: Project Plan, Requirements or Backlog, Development
Environment.
Test Deliveries: Test Plan, Test Cases Specification, Test Case Execution
Report and Summary Defect Report.
Functional Test Automation for basic test cases.
Exploratory Testing.
Create Test Cases derived from Backlog stories or Requirements.
The tester was a scrum team member.
Interaction with the Development Team
Company Confidential
13
14. Software Testing Process
Iterative Software Test Process
Start
Environment
Environment Criation/ Update
Criation/ Update
Planning
Config. Test Cases
Test Cases
Config.
NO
Summary
Summary End of
End of Automated
Automated
Defect Report
Defect Report Iterations?
Iterations?
YES Test Scripts
Test Scripts
End
Regression Test
Test
Tests Execution
Execution
New System
New System Defect Report
Defect Report
Version
Version
Report
Report
Generation
Generation
Company Confidential
14
15. Software Testing Process
Open Source Automated Test Tools
TestLink to automate the managing, planning, test cases specification
and test execution reports.
Mantis for Defect Tracking tool, friendly user interface to interact with
developers and can be integrated with TestLink.
Company Confidential
15
17. Software Testing Process
Web Functional Open Source Automated Test Tools
Selenium IDE and Selenium Core.
Aproach rec-and-play for Mozila Firefox and Internet Explorer
browsers.
Test scripts and Test Suite created in HTML code.
Company Confidential
17
18. Software Testing Process
Automated Testing Tools:
For functional test automation a new open source tool was chosen to
test the java swing desktop application: MARATHON.
Approach rec-and-play recording the user actions creating test scripts
in Jython language and run them later in the tool .
Company Confidential
18
20. Execution– Project 1
Sprint Test Activities in First week:
Test Environment Configuration
Automated test scripts creation. The test scripts were implemented in
the browser Mozilla Firefox.
These scripts were edited and gathered in a test suite, and reused in
other test cases;
the test cases were done according to the acceptance criteria
emphasizing test cases creation for exceptions.
The system analyst should review the test cases to ensure coverage of
all functionalities;
Company Confidential
20
21. Execution– Project 1
Sprint Test Activities in Second week:
The test scripts were executed for Firefox and Internet Explorer
browsers. Manual Exploratory tests were also executed by the tester.
At Mantis and TestLink, the defects were registered, described and
assigned to the developer who was notified by e-mail.
The Reports were automatically generated by TestLink.
When the defects had been fixed, the entire test suite was executed by
Selenium doing the regression test.
It ensures that another part of the system was not affected by code
changes.
If the acceptance criteria were met, the same activities of the process
will begin again for the next Sprint.
Company Confidential
21
22. Execution– Project 2
Goal: To ensure in a short time the main features stressed.
Same test process for the scrum, but with some adaptations:
A meeting with the development team to understand the project and
verify the requirements documentation updates.
The testers asked for the prioritized list of the project features where
the most important features had higher priority (As a backlog list).
The test iterations were planned having the most priority features
being tested first.
Test Activities executed in 4 iterations (4 weeks to test all
applications).
Company Confidential
22
23. Execution– Project 2
Test Activities for iteration:
Plan the test cases for the priority features in the TestLink Tool.
Automated test scripts creation for test cases in Marathon.
Execute the automated tests and the exploratory manual tests too.
The defects found are registered in Testlink and Mantis.
When the developer fixes the defect, the code changes, so all
automated tests are executed to do the regression tests.
The Automated reports are generated and sent to the Project leader.
The development team releases another system version and the
same activities of the process will begin again for the next test
iteration.
Company Confidential
23
24. Results
General Results:
100% of the functionalities from backlog were covered in Project 1.
65% of the system defects were from browser incompatibilities in
Project 1.
77 defects were registered in Mantis, where 30% were interface
defects and 70% were functional defects in Project 2.
The regression tests showed defects in the functionalities that were
considered done.
Good evaluation of the projects.
Company Confidential
24
25. Results
Waterfall Results Comparison
Test
Test Process Coverage Regression cicles Defects
Cases
Manual and
65% 40 0 27
Sequential
Iterative and
100% 75 3 77
Automated Process
Company Confidential
25
27. Conclusion
Lessons learned:
Plan Test Activities and engage the test team from the beginning of
the project.
Automated test scripts had to be reworked and updated with every
backlog change for each sprint.
The scrum developer team had to estimate more time to defects
fixing because of the large amount of problems found during the
regression tests.
Exploratory tests found the usability and performance problems.
The test script creation has become faster because of the reuse of the
script code.
Company Confidential
27
28. Conclusion
The test activities broken into iterations made it possible to
stress the most important features, making the system attend
to what was requested by the client.
Search for new automation tools for other development
platforms for further projects in the company.
Search for new techniques to improve agile test process as
Test based on Models to reduce the rework.
Company Confidential
28