SlideShare a Scribd company logo

Taming the Testing Beast

•Download as PPTX, PDF•
•
Marc Balcer
Marc Balcer

How do you know what to test? There never seems to be enough time (or money or people) to adequately test systems. Rapid cycle times brought on by the adoption of agile approaches only seem to make things worse. Even with automated systems for running tests, you're still left with the time-consuming problem of designing many test cases. This webinar will introduce you to a simple yet powerful technique for modeling functional requirements and a tool that uses those models to generate effective test cases. This approach can be used for unit testing, user testing, and more. Tests can be developed from early specifications (use cases and user stories) and in response to bug reports. You'll see how to easily produce a good set of test cases – exhaustive enough to give your software a good workout, but intelligent enough not to waste time and effort.

Software
1 of 60
Taming the Testing Beast Marc J. Balcer Chief Architect Model Compilers LLC
How do you know what to test?
Test Case TAME – Test Authoring Made Easy Test Worksheet TAME Test Case Test Case Test Case Test Case Excel Open Office
Test Case Employee Add an expenseto an expensereport
Employee Add an expenseto an expensereport Test Case Expense Type Expense Date Amount Notes Inputs Expense Added No Date Error Bad Date Error Amount Missing Error Amount Too Large Warning Expected Results “Stateless”
Test Case Expense Type Expense Date Amount Notes Inputs Expense Added No Date Error Bad Date Error Amount Missing Error Amount Too Large Warning No Project Error Employee Error Expected Results Project Exists Employee Current Other Items Exist Preconditions “Stateful”
Test Case TAME – Test Authoring Made Easy Test Worksheet TAME Test Case Test Case Test Case Test Case Inputs Preconditions Expected Results Test Suite
How do you get a good set of test cases?
Example: A Login Page Inputs
Generic Login Command Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Inputs Characteristic Values (“choices”)
Kinds of Input Values Numeric Symbolic Enumerated Composite A number Specified as • Range • Precision (number of decimal places) May include a unit of measure A character string Specified as • Size • Pattern (regular expression or equivalent) A list of values Specified as the set of values May be ordered or unordered May have a corresponding numeric or symbolic value A single value composed of other values Examples: • Mailing Address • Grid Coordinate
Numeric Type Examples Type Name Description Examples Year A four-digit integer between 1900 and one plus the current year 2012 is good 1832 is not good 342.34 is not good “huh” is not good Money A US Dollar value between 0 and 1000.00 to a precision of 2 decimal places. 345.23 is good -60 is not good 34.219 is not good Count A nonnegative number used to indicate a count of items between 0 and 100. 46 is good 245 is not good -10 is not good Date A calendar date to the precision of one day 15-Jan-2010 is good 4/3/2010 is good* June 6 is not good * How this is interpreted depends upon whether you are in the US or Europe…
Symbolic Type Examples Type Name Description Examples ISBN Book Number 10 or 13 decimal digits The last character may be an X instead of a zero. 0-201-74804-5 is good 4-87197-308-5 is good 9781492217947 is good 9321G4E3 is not good ISO Country Code 2 capital letters US is good GB is good EUR is not good Order Number 4 digits plus a dash plus 6 digits: [0-9]{4}-[0-9]{6} (or d{4}-d{6}) 2014-002432 is good 1986-231321 is good 503-3324-324 is not good Title String 1 to 100 characters, including non-ASCII (foreign language) characters Pretty much anything as long as it’s not more that 100 characters long.
Common Input Values • Errors • Missing (empty, null, nothing, unchecked, …) • Too short • Too long • Wrong characters (bad syntax) • Out of range • Acceptable Values • Good syntax • Special characters (e.g. quotes, < > characters) • Non-ASCII and non-Latin characters
Generate All Combinations
Results Generic Login Command Errormessage "EmptyUserID"is displayed UserIDTooLong UserIDhasinvalid characters EmptyPassword PasswordTooLong UserIDorpassword incorrect UserIDorpassword incorrect Userloggedin Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x x Input Password Field Empty x Too long x Valid Syntax x x x
Preconditions Generic Login Command Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Facts about the environment (usually set up in advance)
Forming Combinations Generic Login Command Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID 4 choices 3 choices 2 choices 2 choices 4 × 3 × 2 × 2 = 48 test cases
All Combinations = Many Useless Test Cases Redundant (duplicate) User ID empty Password too long User: none with the user ID Password: does not match the user ID  Result: Empty User ID error User ID: empty Password: good syntax User: none with the user ID Password: does not match the user ID  Result: Empty User ID error Make No Sense User ID empty Password too long User: user with the user ID Password: does not match the user ID User ID bad syntax Password too long User: user with the user ID Password: does not match the user ID
Define Results Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Expected Results
Simple Result Marking Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax Input Password Field Empty x Too long x Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID
Combination Results Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Input Password Field Empty Too long Valid Syntax x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x
Multiple Paths to the Same Result Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x x Input Password Field Empty Too long Valid Syntax x x Condition User no user with the user ID x user with the user ID x Condition User Password does not match the user ID x matches the user ID same
Fully Marked Worksheet Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x x Input Password Field Empty x Too long x Valid Syntax x x x Condition User no user with the user ID x user with the user ID x x Condition User Password does not match the user ID x matches the user ID x
Forming Test Cases Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax Input Password Field Empty x Too long x Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Test Case 1 Inputs: User ID Field : Empty Expected Results: Empty User ID error ------------------------------------ Test Case 2 Inputs: User ID Field : Too long Expected Results: User ID Too Long error ------------------------------------ Test Case 3 Inputs: User ID Field : Has invalid characters Expected Results: User ID has invalid characters error
No Mark – Don’t Care Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax Input Password Field Empty x Too long x Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Test Case 1 Inputs: User ID Field : Empty Expected Results: Empty User ID error Any choice for password will result in this same error message.
Default Choice Select a default if you want to put that “don’t care” category into the test case.
Combine All Choices Select all choices if you want to combine all choices into test cases. Alternatively check the category name.
Multiple Paths to the Same Result Generic Login Command UserIDorpassword incorrecterror UserIDorpassword incorrecterror Input User ID Field Empty Too long Has invalid characters Valid Syntax x x Input Password Field Empty Too long Valid Syntax x x Condition User no user with the user ID x user with the user ID x Condition User Password does not match the user ID x matches the user ID Test Case 6 Preconditions: User : no user with the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User ID or password incorrect -------------------------------------------- --- Test Case 7 Preconditions: User : user with the user ID User Password : does not match the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User ID or password incorrect
Combination Results Generic Login Command Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Input Password Field Empty Too long Valid Syntax x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x Test Case 8 Preconditions: User : user with the user ID User Password : matches the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User logged in
Multiple Combinations Generic Login Command (With Special Inputs) Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Valid with non-ASCII chars x Input Password Field Empty Too long Valid Syntax x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x Form all combinations of these inputs. 2 × 4 = 8 test cases
Limits Generic Login Command (With Special Inputs) LIMITS Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Valid with non-ASCII chars 1 x Input Password Field Empty Too long Valid Syntax x Valid with quotes 1 x Valid with HTML chars 1 x Valid with non-ASCII chars 1 x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x Each choice will only be used in one test case.
Generating Test Cases Test CaseTest Worksheet TAME Test Case Test Case Test Case Test Case Excel Open Office Test Case List Test Summary Table Test Protocol Document
Test Case List Test Cases for Generic Login Command (With Special Inputs) ---------------------------------------------------- Test Case 1 Preconditions: Inputs: User ID Field : Empty Expected Results: Empty User ID error ---------------------------------------------------- Test Case 2 Preconditions: Inputs: User ID Field : Too long Expected Results: User ID Too Long error ---------------------------------------------------- Test Case 3 Preconditions: Inputs: User ID Field : Has invalid characters Expected Results: User ID has invalid characters error
Test Summary Table User ID Field Password Field User User Password Results... 1 Empty Empty User ID error 2 Too long User ID Too Long error 3 Has invalid characters User ID has invalid characters error 4 Empty Empty Password error 5 Too long Password Too Long error 6 Valid Syntax Valid Syntax no user with the user ID User ID or password incorrect error 7Valid Syntax Valid with quotes no user with the user ID User ID or password incorrect error 8 Valid Syntax Valid with HTML chars no user with the user ID User ID or password incorrect error 9 Valid Syntax Valid Syntax user with the user ID does not match the user ID User ID or password incorrect error 10 Valid Syntax Valid with quotes user with the user ID does not match the user ID User ID or password incorrect error 11 Valid Syntax Valid with HTML chars user with the user ID does not match the user ID User ID or password incorrect error 12 Valid Syntax Valid Syntax user with the user ID matches the user ID User logged in 13 Valid Syntax Valid with quotes user with the user ID matches the user ID User logged in 14 Valid Syntax Valid with HTML chars user with the user ID matches the user ID User logged in
Default Protocol Document
Creating a Custom Protocol How to log in: • Click a link to go to the login page • Enter the user ID • Go to the password field • Enter the password • Click the login button • Observe whether • an error message is displayed, or • the application indicates that the user is logged in
Creating a Custom Protocol How to log in: • Click a link to go to the login page • Enter the user ID • Go to the password field • Enter the password • Click the login button • Observe whether • an error message is displayed, or • the application indicates that the user is logged in We want real values for each of these parameters.
Defining Values Generic Login Command (With Special Inputs) LIMITS EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x x Too long x x Has invalid characters x x Valid Syntax x x x x x Valid with non-ASCII chars 1 x x Input Password Field Empty x x Too long x x Valid Syntax x x x x x Valid with quotes 1 x x x x Valid with HTML chars 1 x x x x Valid with non-ASCII chars 1 x x Condition User no user with the user ID x x user with the user ID x x x x x x x Condition User Password does not match the user ID x x matches the user ID x x x
Defining Values UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Parameter Name : value
Custom Protocol Template Test Case «TestCaseNumber» Preconditions «TableStart:Precondition»«Cate goryName» «ChoiceName»«TableEnd:Precondition» Inputs «TableStart:Input»«CategoryNa me» «ChoiceName»«TableEnd:Input» Results «TableStart:Result»«ResultName»«TableEnd:Result» Procedure: (Warning: If you customize this section do not use numbered lists. Word MailMerge doesn’t have the sense to reset the numbering.)  Open the website  Click the Login button  Enter «User ID» into the User ID field.  Hit tab  Enter «Password» into the password field  Click the Login button Now verify that: «TableStart:Result»  «ResultName» «TableEnd:Result» Word Mailmerge Fields User ID and Password values filled in here
Simple Values Generic Login Command (With Special Inputs) UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x
Complex Values Generic Login Command (With Special Inputs) UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x
Complex Values Generic Login Command (With Special Inputs) UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x
Default Values Generic Login Command (With Special Inputs) LIMITS EmptyUserIDerror UserIDTooLong error UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars 1 x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes 1 x Valid with HTML chars 1 x Valid with non-ASCII chars 1 x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x * indicates the default value
Values Applied to the Protocol Test Case 7 Preconditions User no user with the user ID Inputs User ID Field Valid Syntax Password Field Valid with quotes Results User ID or password incorrect error Procedure:  Open the website  Click the Login button  Enter charlie into the User ID field.  Hit tab  Enter G"oo"d into the password field  Click the Login button Now verify that:  User ID or password incorrect error
Going Beyond the Basics • Create variability in procedures • Action Choices • Using TAME when bugs are found • Root Cause Analysis • Modify the Tests • Scaling Up • Higher Level Tests • Results become Preconditions
Additional variability Test Case 7 Preconditions User no user with the user ID Inputs User ID Field Valid Syntax Password Field Valid with quotes Results User ID or password incorrect error Procedure:  Open the website  Click the Login button  Enter charlie into the User ID field.  Hit tab  Enter G"oo"d into the password field  Click the Login button Now verify that:  User ID or password incorrect error - Enter a URL directly - Click a link on the company homepage - Click a link on an unsecured page - Tab to the next field - Click into the password field - Click the Login button - Tab to the Login button and press Enter
Action Choices Test Case 7 Preconditions User no user with the user ID Inputs User ID Field Valid Syntax Password Field Valid with quotes Results User ID or password incorrect error Procedure:  Open the website  Click the Login button  Enter charlie into the User ID field.  Hit tab  Enter G"oo"d into the password field  Click the Login button Now verify that:  User ID or password incorrect error • Enter a URL directly • Click a link on the company homepage • Click a link on an unsecured page • Tab to the next field • Click into the password field • Click the Login button • Tab to the Login button and press Enter For each of these • Create “action” choices • Use the action choices in the protocol
Unforseen Circumstances A user ID that includes spaces causes a crash Three failed logins lock the user’s account • Add “contains spaces” as a choice • Add a value definition that includes spaces • Determine how to handle a user ID that includes spaces: is it allowed or is it an error? • Add “Preceding failed logins” as a new preconditions with several choices • Determine whether a locked account presents a different error message • Add result selectors
Scaling Up Employeehas expensesto report Log In CreateNew ExpenseReport Add Expense Item Submit to Manager Expense Report Submitted Open ExpenseApp already logged in not logged in Login is just one step in a larger process
Scaling Up Employeehas expensesto report Log In CreateNew ExpenseReport Add Expense Item Submit to Manager Expense Report Submitted Open ExpenseApp already logged in not logged in Create separate test specifications for each step in the process
Scaling Up – Business Processes EmployeeManagerController Employeehas expensesto report Record Expenses Approve Expense Report Pay Reimbursement Reimbursement Reject Expense Report Resubmit Expense Report Submitted Expense Report Rejected Expense Report Resubmitted Expense Report Approved Expense Report Review New Expense Report Review Resubmitted Expense Report Cancel Expense Report ExpenseReport hasbeen canceled Consider whether to resubmit Employeehas expensesto report Log In CreateNew ExpenseReport Add Expense Item Submit to Manager Expense Report Submitted Open ExpenseApp already logged in not logged in
Generating Test Cases Test CaseTest Worksheet TAME Test Case Test Case Test Case Test Case Excel Open Office Test Case List Test Summary Table Test Protocol Document
Comparative Testing Tools • Run Tests • White-box, code-centric NUnit, JUnit • UI-based, capture-playback Ranorex, Telerik • Design by Contract • Write a specification, construct a single test case Cucumber / Gherkin
Where to use TAME • Unit testing • specify and generate test cases that can run against software modules (classes, functions). • GUI testing • whether a traditional installed application, a website, or a mobile app, use TAME to specify the contents of UI pages, then generate test cases to validate user-facing behavior. • Regression testing • update the test specification and the test cases whenever new features are deployed or bugs are fixed.
Where to use TAME • Project Planning • create initial test specifications to estimate the number of test cases to be run and project the cost of the test effort. • Retrospectives • use the test specifications to probe features to better understand the root causes of failures.
TAME for All • TAME helps the analyst to think through what needs to be tested. • TAME gives the tester a platform for creating many test cases quickly. • TAME gives the project manager a means for sizing and scoping the testing effort.
TAME software Training Courses “Writing Testable Requirements and Effective Test Cases” . “Modeling Techniques for Business Analysts”
Thank you Marc J. Balcer Chief Architect Model Compilers LLC marc@modelcompilers.com http://www.modelcompilers.com

Recommended

1.getting started with c
1.getting started with c1.getting started with c
1.getting started with c
Hardik gupta
 
Transitory Electronic Communication: Recordless Messaging in the Context of a...
Transitory Electronic Communication: Recordless Messaging in the Context of a...Transitory Electronic Communication: Recordless Messaging in the Context of a...
Transitory Electronic Communication: Recordless Messaging in the Context of a...
jonneiditz
 
Project Management Skills for QA
Project Management Skills for QAProject Management Skills for QA
Project Management Skills for QA
QASource
 
B2 B Commerce
B2 B CommerceB2 B Commerce
B2 B Commerce
Ritesh Verma
 
QA Fes 2016. Claudiu Draghia. Testing Challenges
QA Fes 2016. Claudiu Draghia. Testing ChallengesQA Fes 2016. Claudiu Draghia. Testing Challenges
QA Fes 2016. Claudiu Draghia. Testing Challenges
QAFest
 
ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011
Professional Testing
 
Agile Testing - Challenges
Agile Testing - ChallengesAgile Testing - Challenges
Agile Testing - Challenges
Mohan Krishna Kona
 
ISTQB REX BLACK book
ISTQB REX BLACK bookISTQB REX BLACK book
ISTQB REX BLACK book
Naga Chiranjeevi Kota
 

Recommended

1.getting started with c
1.getting started with c1.getting started with c
1.getting started with c
Hardik gupta
 
Transitory Electronic Communication: Recordless Messaging in the Context of a...
Transitory Electronic Communication: Recordless Messaging in the Context of a...Transitory Electronic Communication: Recordless Messaging in the Context of a...
Transitory Electronic Communication: Recordless Messaging in the Context of a...
jonneiditz
 
Project Management Skills for QA
Project Management Skills for QAProject Management Skills for QA
Project Management Skills for QA
QASource
 
B2 B Commerce
B2 B CommerceB2 B Commerce
B2 B Commerce
Ritesh Verma
 
QA Fes 2016. Claudiu Draghia. Testing Challenges
QA Fes 2016. Claudiu Draghia. Testing ChallengesQA Fes 2016. Claudiu Draghia. Testing Challenges
QA Fes 2016. Claudiu Draghia. Testing Challenges
QAFest
 
ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011ISTQB - Foundation Level Syllabus 2011
ISTQB - Foundation Level Syllabus 2011
Professional Testing
 
Agile Testing - Challenges
Agile Testing - ChallengesAgile Testing - Challenges
Agile Testing - Challenges
Mohan Krishna Kona
 
ISTQB REX BLACK book
ISTQB REX BLACK bookISTQB REX BLACK book
ISTQB REX BLACK book
Naga Chiranjeevi Kota
 
Software Testing-Dynamic testing technique-Mazenet solution
Software Testing-Dynamic testing technique-Mazenet solutionSoftware Testing-Dynamic testing technique-Mazenet solution
Software Testing-Dynamic testing technique-Mazenet solution
Mazenetsolution
 
AcceptCriteria_TestCases_TestScripts
AcceptCriteria_TestCases_TestScriptsAcceptCriteria_TestCases_TestScripts
AcceptCriteria_TestCases_TestScripts
Russell Pannone
 
ISTQB, ISEB Lecture Notes- 4
ISTQB, ISEB Lecture Notes- 4ISTQB, ISEB Lecture Notes- 4
ISTQB, ISEB Lecture Notes- 4
onsoftwaretest
 
tPlan
tPlantPlan
tPlan
guest4c6fd6
 
Testing and Testable Code
Testing and Testable CodeTesting and Testable Code
Testing and Testable Code
Pawel Szulc
 
Designing Great Forms
Designing Great FormsDesigning Great Forms
Designing Great Forms
Josh Fraser
 
Chapter 15.pptx
Chapter 15.pptxChapter 15.pptx
Chapter 15.pptx
Fatima130137
 
Basic test cases one should take care while implementing web application
Basic test cases one should take care while implementing web applicationBasic test cases one should take care while implementing web application
Basic test cases one should take care while implementing web application
khushbu vaghasia
 
Design Test Case Technique (Equivalence partitioning And Boundary value analy...
Design Test Case Technique (Equivalence partitioning And Boundary value analy...Design Test Case Technique (Equivalence partitioning And Boundary value analy...
Design Test Case Technique (Equivalence partitioning And Boundary value analy...
Ryan Tran
 
Spss series - data entry and coding
Spss series - data entry and codingSpss series - data entry and coding
Spss series - data entry and coding
Dr. Majdi Al Jasim
 
PHP Form Validation Technique
PHP Form Validation TechniquePHP Form Validation Technique
PHP Form Validation Technique
Morshedul Arefin
 
Adding gift questions in swayam 2.0
Adding gift questions in swayam 2.0Adding gift questions in swayam 2.0
Adding gift questions in swayam 2.0
aschrdc
 
Vb.Net 01 To 03 Summary Upload
Vb.Net 01 To 03 Summary UploadVb.Net 01 To 03 Summary Upload
Vb.Net 01 To 03 Summary Upload
Hock Leng PUAH
 
Unit testing
Unit testingUnit testing
Unit testing
st.mary's womens engineering college,budampadu
 
Example acrolinx report-cfpb
Example acrolinx report-cfpbExample acrolinx report-cfpb
Example acrolinx report-cfpb
Center for Plain Language
 
Testers Desk Presentation
Testers Desk PresentationTesters Desk Presentation
Testers Desk Presentation
Quality Testing
 
3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf
3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf
3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf
sangeeta borde
 
Lecture7 pattern
Lecture7 patternLecture7 pattern
Lecture7 pattern
Châu Thanh Chương
 
Online satsang examination system
Online satsang examination systemOnline satsang examination system
Online satsang examination system
Jignesh Bhadani
 
Unit testing (workshop)
Unit testing (workshop)Unit testing (workshop)
Unit testing (workshop)
Foyzul Karim
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AIBusinessGPT - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
AGATSoftware
 
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAMWSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2
 

More Related Content

Similar to Taming the Testing Beast

AcceptCriteria_TestCases_TestScripts
AcceptCriteria_TestCases_TestScriptsAcceptCriteria_TestCases_TestScripts
AcceptCriteria_TestCases_TestScripts
Russell Pannone
 
Unit testing (workshop)
Unit testing (workshop)Unit testing (workshop)
Unit testing (workshop)
Foyzul Karim
 

Similar to Taming the Testing Beast (20)

Software Testing-Dynamic testing technique-Mazenet solution
Software Testing-Dynamic testing technique-Mazenet solutionSoftware Testing-Dynamic testing technique-Mazenet solution
Software Testing-Dynamic testing technique-Mazenet solution
 
AcceptCriteria_TestCases_TestScripts
AcceptCriteria_TestCases_TestScriptsAcceptCriteria_TestCases_TestScripts
AcceptCriteria_TestCases_TestScripts
 
ISTQB, ISEB Lecture Notes- 4
ISTQB, ISEB Lecture Notes- 4ISTQB, ISEB Lecture Notes- 4
ISTQB, ISEB Lecture Notes- 4
 
tPlan
tPlantPlan
tPlan
 
Testing and Testable Code
Testing and Testable CodeTesting and Testable Code
Testing and Testable Code
 
Designing Great Forms
Designing Great FormsDesigning Great Forms
Designing Great Forms
 
Chapter 15.pptx
Chapter 15.pptxChapter 15.pptx
Chapter 15.pptx
 
Basic test cases one should take care while implementing web application
Basic test cases one should take care while implementing web applicationBasic test cases one should take care while implementing web application
Basic test cases one should take care while implementing web application
 
Design Test Case Technique (Equivalence partitioning And Boundary value analy...
Design Test Case Technique (Equivalence partitioning And Boundary value analy...Design Test Case Technique (Equivalence partitioning And Boundary value analy...
Design Test Case Technique (Equivalence partitioning And Boundary value analy...
 
Spss series - data entry and coding
Spss series - data entry and codingSpss series - data entry and coding
Spss series - data entry and coding
 
PHP Form Validation Technique
PHP Form Validation TechniquePHP Form Validation Technique
PHP Form Validation Technique
 
Adding gift questions in swayam 2.0
Adding gift questions in swayam 2.0Adding gift questions in swayam 2.0
Adding gift questions in swayam 2.0
 
Vb.Net 01 To 03 Summary Upload
Vb.Net 01 To 03 Summary UploadVb.Net 01 To 03 Summary Upload
Vb.Net 01 To 03 Summary Upload
 
Unit testing
Unit testingUnit testing
Unit testing
 
Example acrolinx report-cfpb
Example acrolinx report-cfpbExample acrolinx report-cfpb
Example acrolinx report-cfpb
 
Testers Desk Presentation
Testers Desk PresentationTesters Desk Presentation
Testers Desk Presentation
 
3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf
3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf
3. Test Scenarios & Test Cases with Excel Sheet Format (1).pdf
 
Lecture7 pattern
Lecture7 patternLecture7 pattern
Lecture7 pattern
 
Online satsang examination system
Online satsang examination systemOnline satsang examination system
Online satsang examination system
 
Unit testing (workshop)
Unit testing (workshop)Unit testing (workshop)
Unit testing (workshop)
 

Recently uploaded

Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...
Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...
Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 

Recently uploaded (20)

BusinessGPT - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AIBusinessGPT - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
 
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAMWSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
 
WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...
WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...
WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...
 
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
 
Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...
Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...
Abortion Pill Prices Boksburg [(+27832195400*)] 🏥 Women's Abortion Clinic in ...
 
WSO2Con2024 - Hello Choreo Presentation - Kanchana
WSO2Con2024 - Hello Choreo Presentation - KanchanaWSO2Con2024 - Hello Choreo Presentation - Kanchana
WSO2Con2024 - Hello Choreo Presentation - Kanchana
 
WSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaSWSO2CON 2024 Slides - Open Source to SaaS
WSO2CON 2024 Slides - Open Source to SaaS
 
WSO2Con2024 - Software Delivery in Hybrid Environments
WSO2Con2024 - Software Delivery in Hybrid EnvironmentsWSO2Con2024 - Software Delivery in Hybrid Environments
WSO2Con2024 - Software Delivery in Hybrid Environments
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public AdministrationWSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
 
WSO2Con2024 - Navigating the Digital Landscape: Transforming Healthcare with ...
WSO2Con2024 - Navigating the Digital Landscape: Transforming Healthcare with ...WSO2Con2024 - Navigating the Digital Landscape: Transforming Healthcare with ...
WSO2Con2024 - Navigating the Digital Landscape: Transforming Healthcare with ...
 
WSO2Con204 - Hard Rock Presentation - Keynote
WSO2Con204 - Hard Rock Presentation - KeynoteWSO2Con204 - Hard Rock Presentation - Keynote
WSO2Con204 - Hard Rock Presentation - Keynote
 
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdfAzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
 
WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...
WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...
WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...
 
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptxArtyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public AdministrationWSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
 
WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
WSO2CON2024 - Why Should You Consider Ballerina for Your Next IntegrationWSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
 
WSO2Con2024 - Unleashing the Financial Potential of 13 Million People
WSO2Con2024 - Unleashing the Financial Potential of 13 Million PeopleWSO2Con2024 - Unleashing the Financial Potential of 13 Million People
WSO2Con2024 - Unleashing the Financial Potential of 13 Million People
 

Taming the Testing Beast

  • 1. Taming the Testing Beast Marc J. Balcer Chief Architect Model Compilers LLC
  • 2. How do you know what to test?
  • 3. Test Case TAME – Test Authoring Made Easy Test Worksheet TAME Test Case Test Case Test Case Test Case Excel Open Office
  • 4. Test Case Employee Add an expenseto an expensereport
  • 5. Employee Add an expenseto an expensereport Test Case Expense Type Expense Date Amount Notes Inputs Expense Added No Date Error Bad Date Error Amount Missing Error Amount Too Large Warning Expected Results “Stateless”
  • 6. Test Case Expense Type Expense Date Amount Notes Inputs Expense Added No Date Error Bad Date Error Amount Missing Error Amount Too Large Warning No Project Error Employee Error Expected Results Project Exists Employee Current Other Items Exist Preconditions “Stateful”
  • 7. Test Case TAME – Test Authoring Made Easy Test Worksheet TAME Test Case Test Case Test Case Test Case Inputs Preconditions Expected Results Test Suite
  • 8. How do you get a good set of test cases?
  • 9. Example: A Login Page Inputs
  • 10. Generic Login Command Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Inputs Characteristic Values (“choices”)
  • 11. Kinds of Input Values Numeric Symbolic Enumerated Composite A number Specified as • Range • Precision (number of decimal places) May include a unit of measure A character string Specified as • Size • Pattern (regular expression or equivalent) A list of values Specified as the set of values May be ordered or unordered May have a corresponding numeric or symbolic value A single value composed of other values Examples: • Mailing Address • Grid Coordinate
  • 12. Numeric Type Examples Type Name Description Examples Year A four-digit integer between 1900 and one plus the current year 2012 is good 1832 is not good 342.34 is not good “huh” is not good Money A US Dollar value between 0 and 1000.00 to a precision of 2 decimal places. 345.23 is good -60 is not good 34.219 is not good Count A nonnegative number used to indicate a count of items between 0 and 100. 46 is good 245 is not good -10 is not good Date A calendar date to the precision of one day 15-Jan-2010 is good 4/3/2010 is good* June 6 is not good * How this is interpreted depends upon whether you are in the US or Europe…
  • 13. Symbolic Type Examples Type Name Description Examples ISBN Book Number 10 or 13 decimal digits The last character may be an X instead of a zero. 0-201-74804-5 is good 4-87197-308-5 is good 9781492217947 is good 9321G4E3 is not good ISO Country Code 2 capital letters US is good GB is good EUR is not good Order Number 4 digits plus a dash plus 6 digits: [0-9]{4}-[0-9]{6} (or d{4}-d{6}) 2014-002432 is good 1986-231321 is good 503-3324-324 is not good Title String 1 to 100 characters, including non-ASCII (foreign language) characters Pretty much anything as long as it’s not more that 100 characters long.
  • 14. Common Input Values • Errors • Missing (empty, null, nothing, unchecked, …) • Too short • Too long • Wrong characters (bad syntax) • Out of range • Acceptable Values • Good syntax • Special characters (e.g. quotes, < > characters) • Non-ASCII and non-Latin characters
  • 17. Preconditions Generic Login Command Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Facts about the environment (usually set up in advance)
  • 18. Forming Combinations Generic Login Command Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID 4 choices 3 choices 2 choices 2 choices 4 × 3 × 2 × 2 = 48 test cases
  • 19. All Combinations = Many Useless Test Cases Redundant (duplicate) User ID empty Password too long User: none with the user ID Password: does not match the user ID  Result: Empty User ID error User ID: empty Password: good syntax User: none with the user ID Password: does not match the user ID  Result: Empty User ID error Make No Sense User ID empty Password too long User: user with the user ID Password: does not match the user ID User ID bad syntax Password too long User: user with the user ID Password: does not match the user ID
  • 20. Define Results Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax Input Password Field Empty Too long Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Expected Results
  • 21. Simple Result Marking Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax Input Password Field Empty x Too long x Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID
  • 22. Combination Results Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Input Password Field Empty Too long Valid Syntax x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x
  • 23. Multiple Paths to the Same Result Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x x Input Password Field Empty Too long Valid Syntax x x Condition User no user with the user ID x user with the user ID x Condition User Password does not match the user ID x matches the user ID same
  • 24. Fully Marked Worksheet Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x x Input Password Field Empty x Too long x Valid Syntax x x x Condition User no user with the user ID x user with the user ID x x Condition User Password does not match the user ID x matches the user ID x
  • 25. Forming Test Cases Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax Input Password Field Empty x Too long x Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Test Case 1 Inputs: User ID Field : Empty Expected Results: Empty User ID error ------------------------------------ Test Case 2 Inputs: User ID Field : Too long Expected Results: User ID Too Long error ------------------------------------ Test Case 3 Inputs: User ID Field : Has invalid characters Expected Results: User ID has invalid characters error
  • 26. No Mark – Don’t Care Generic Login Command EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax Input Password Field Empty x Too long x Valid Syntax Condition User no user with the user ID user with the user ID Condition User Password does not match the user ID matches the user ID Test Case 1 Inputs: User ID Field : Empty Expected Results: Empty User ID error Any choice for password will result in this same error message.
  • 27. Default Choice Select a default if you want to put that “don’t care” category into the test case.
  • 28. Combine All Choices Select all choices if you want to combine all choices into test cases. Alternatively check the category name.
  • 29. Multiple Paths to the Same Result Generic Login Command UserIDorpassword incorrecterror UserIDorpassword incorrecterror Input User ID Field Empty Too long Has invalid characters Valid Syntax x x Input Password Field Empty Too long Valid Syntax x x Condition User no user with the user ID x user with the user ID x Condition User Password does not match the user ID x matches the user ID Test Case 6 Preconditions: User : no user with the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User ID or password incorrect -------------------------------------------- --- Test Case 7 Preconditions: User : user with the user ID User Password : does not match the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User ID or password incorrect
  • 30. Combination Results Generic Login Command Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Input Password Field Empty Too long Valid Syntax x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x Test Case 8 Preconditions: User : user with the user ID User Password : matches the user ID Inputs: User ID Field : Valid Syntax Password Field : Valid Syntax Expected Results: User logged in
  • 31. Multiple Combinations Generic Login Command (With Special Inputs) Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Valid with non-ASCII chars x Input Password Field Empty Too long Valid Syntax x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x Form all combinations of these inputs. 2 × 4 = 8 test cases
  • 32. Limits Generic Login Command (With Special Inputs) LIMITS Userloggedin Input User ID Field Empty Too long Has invalid characters Valid Syntax x Valid with non-ASCII chars 1 x Input Password Field Empty Too long Valid Syntax x Valid with quotes 1 x Valid with HTML chars 1 x Valid with non-ASCII chars 1 x Condition User no user with the user ID user with the user ID x Condition User Password does not match the user ID matches the user ID x Each choice will only be used in one test case.
  • 33. Generating Test Cases Test CaseTest Worksheet TAME Test Case Test Case Test Case Test Case Excel Open Office Test Case List Test Summary Table Test Protocol Document
  • 34. Test Case List Test Cases for Generic Login Command (With Special Inputs) ---------------------------------------------------- Test Case 1 Preconditions: Inputs: User ID Field : Empty Expected Results: Empty User ID error ---------------------------------------------------- Test Case 2 Preconditions: Inputs: User ID Field : Too long Expected Results: User ID Too Long error ---------------------------------------------------- Test Case 3 Preconditions: Inputs: User ID Field : Has invalid characters Expected Results: User ID has invalid characters error
  • 35. Test Summary Table User ID Field Password Field User User Password Results... 1 Empty Empty User ID error 2 Too long User ID Too Long error 3 Has invalid characters User ID has invalid characters error 4 Empty Empty Password error 5 Too long Password Too Long error 6 Valid Syntax Valid Syntax no user with the user ID User ID or password incorrect error 7Valid Syntax Valid with quotes no user with the user ID User ID or password incorrect error 8 Valid Syntax Valid with HTML chars no user with the user ID User ID or password incorrect error 9 Valid Syntax Valid Syntax user with the user ID does not match the user ID User ID or password incorrect error 10 Valid Syntax Valid with quotes user with the user ID does not match the user ID User ID or password incorrect error 11 Valid Syntax Valid with HTML chars user with the user ID does not match the user ID User ID or password incorrect error 12 Valid Syntax Valid Syntax user with the user ID matches the user ID User logged in 13 Valid Syntax Valid with quotes user with the user ID matches the user ID User logged in 14 Valid Syntax Valid with HTML chars user with the user ID matches the user ID User logged in
  • 37. Creating a Custom Protocol How to log in: • Click a link to go to the login page • Enter the user ID • Go to the password field • Enter the password • Click the login button • Observe whether • an error message is displayed, or • the application indicates that the user is logged in
  • 38. Creating a Custom Protocol How to log in: • Click a link to go to the login page • Enter the user ID • Go to the password field • Enter the password • Click the login button • Observe whether • an error message is displayed, or • the application indicates that the user is logged in We want real values for each of these parameters.
  • 39. Defining Values Generic Login Command (With Special Inputs) LIMITS EmptyUserIDerror UserIDTooLong error UserIDhasinvalid characterserror EmptyPassword error PasswordTooLong error UserIDorpassword incorrecterror UserIDorpassword incorrecterror Userloggedin UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x x Too long x x Has invalid characters x x Valid Syntax x x x x x Valid with non-ASCII chars 1 x x Input Password Field Empty x x Too long x x Valid Syntax x x x x x Valid with quotes 1 x x x x Valid with HTML chars 1 x x x x Valid with non-ASCII chars 1 x x Condition User no user with the user ID x x user with the user ID x x x x x x x Condition User Password does not match the user ID x x matches the user ID x x x
  • 41. Custom Protocol Template Test Case «TestCaseNumber» Preconditions «TableStart:Precondition»«Cate goryName» «ChoiceName»«TableEnd:Precondition» Inputs «TableStart:Input»«CategoryNa me» «ChoiceName»«TableEnd:Input» Results «TableStart:Result»«ResultName»«TableEnd:Result» Procedure: (Warning: If you customize this section do not use numbered lists. Word MailMerge doesn’t have the sense to reset the numbering.)  Open the website  Click the Login button  Enter «User ID» into the User ID field.  Hit tab  Enter «Password» into the password field  Click the Login button Now verify that: «TableStart:Result»  «ResultName» «TableEnd:Result» Word Mailmerge Fields User ID and Password values filled in here
  • 42. Simple Values Generic Login Command (With Special Inputs) UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x
  • 43. Complex Values Generic Login Command (With Special Inputs) UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x
  • 44. Complex Values Generic Login Command (With Special Inputs) UserID: UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes x Valid with HTML chars x Valid with non-ASCII chars x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x
  • 45. Default Values Generic Login Command (With Special Inputs) LIMITS EmptyUserIDerror UserIDTooLong error UserID: abcdefghijklm UserID:56%#@!*(a UserID:gary* UserID:charlie UserID:garcía Password: Password: 56%#@!*(a Password:Good* Password:Bad Password:G"oo"d Password:Go<o>d& Password:¿sueÑo Input User ID Field Empty x Too long x x Has invalid characters x Valid Syntax x x Valid with non-ASCII chars 1 x Input Password Field Empty x Too long x Valid Syntax x x Valid with quotes 1 x Valid with HTML chars 1 x Valid with non-ASCII chars 1 x Condition User no user with the user ID x user with the user ID x x x x x Condition User Password does not match the user ID x matches the user ID x x * indicates the default value
  • 46. Values Applied to the Protocol Test Case 7 Preconditions User no user with the user ID Inputs User ID Field Valid Syntax Password Field Valid with quotes Results User ID or password incorrect error Procedure:  Open the website  Click the Login button  Enter charlie into the User ID field.  Hit tab  Enter G"oo"d into the password field  Click the Login button Now verify that:  User ID or password incorrect error
  • 47. Going Beyond the Basics • Create variability in procedures • Action Choices • Using TAME when bugs are found • Root Cause Analysis • Modify the Tests • Scaling Up • Higher Level Tests • Results become Preconditions
  • 48. Additional variability Test Case 7 Preconditions User no user with the user ID Inputs User ID Field Valid Syntax Password Field Valid with quotes Results User ID or password incorrect error Procedure:  Open the website  Click the Login button  Enter charlie into the User ID field.  Hit tab  Enter G"oo"d into the password field  Click the Login button Now verify that:  User ID or password incorrect error - Enter a URL directly - Click a link on the company homepage - Click a link on an unsecured page - Tab to the next field - Click into the password field - Click the Login button - Tab to the Login button and press Enter
  • 49. Action Choices Test Case 7 Preconditions User no user with the user ID Inputs User ID Field Valid Syntax Password Field Valid with quotes Results User ID or password incorrect error Procedure:  Open the website  Click the Login button  Enter charlie into the User ID field.  Hit tab  Enter G"oo"d into the password field  Click the Login button Now verify that:  User ID or password incorrect error • Enter a URL directly • Click a link on the company homepage • Click a link on an unsecured page • Tab to the next field • Click into the password field • Click the Login button • Tab to the Login button and press Enter For each of these • Create “action” choices • Use the action choices in the protocol
  • 50. Unforseen Circumstances A user ID that includes spaces causes a crash Three failed logins lock the user’s account • Add “contains spaces” as a choice • Add a value definition that includes spaces • Determine how to handle a user ID that includes spaces: is it allowed or is it an error? • Add “Preceding failed logins” as a new preconditions with several choices • Determine whether a locked account presents a different error message • Add result selectors
  • 51. Scaling Up Employeehas expensesto report Log In CreateNew ExpenseReport Add Expense Item Submit to Manager Expense Report Submitted Open ExpenseApp already logged in not logged in Login is just one step in a larger process
  • 52. Scaling Up Employeehas expensesto report Log In CreateNew ExpenseReport Add Expense Item Submit to Manager Expense Report Submitted Open ExpenseApp already logged in not logged in Create separate test specifications for each step in the process
  • 53. Scaling Up – Business Processes EmployeeManagerController Employeehas expensesto report Record Expenses Approve Expense Report Pay Reimbursement Reimbursement Reject Expense Report Resubmit Expense Report Submitted Expense Report Rejected Expense Report Resubmitted Expense Report Approved Expense Report Review New Expense Report Review Resubmitted Expense Report Cancel Expense Report ExpenseReport hasbeen canceled Consider whether to resubmit Employeehas expensesto report Log In CreateNew ExpenseReport Add Expense Item Submit to Manager Expense Report Submitted Open ExpenseApp already logged in not logged in
  • 54. Generating Test Cases Test CaseTest Worksheet TAME Test Case Test Case Test Case Test Case Excel Open Office Test Case List Test Summary Table Test Protocol Document
  • 55. Comparative Testing Tools • Run Tests • White-box, code-centric NUnit, JUnit • UI-based, capture-playback Ranorex, Telerik • Design by Contract • Write a specification, construct a single test case Cucumber / Gherkin
  • 56. Where to use TAME • Unit testing • specify and generate test cases that can run against software modules (classes, functions). • GUI testing • whether a traditional installed application, a website, or a mobile app, use TAME to specify the contents of UI pages, then generate test cases to validate user-facing behavior. • Regression testing • update the test specification and the test cases whenever new features are deployed or bugs are fixed.
  • 57. Where to use TAME • Project Planning • create initial test specifications to estimate the number of test cases to be run and project the cost of the test effort. • Retrospectives • use the test specifications to probe features to better understand the root causes of failures.
  • 58. TAME for All • TAME helps the analyst to think through what needs to be tested. • TAME gives the tester a platform for creating many test cases quickly. • TAME gives the project manager a means for sizing and scoping the testing effort.
  • 59. TAME software Training Courses “Writing Testable Requirements and Effective Test Cases” . “Modeling Techniques for Business Analysts”
  • 60. Thank you Marc J. Balcer Chief Architect Model Compilers LLC marc@modelcompilers.com http://www.modelcompilers.com