SlideShare a Scribd company logo
Codifying Game & Test Knowledge in Tests? 
How much information should your test cases (or test missions, charters, or 
other types or similar test artifacts) include? What are the pros and cons of 
adding lots of detailed information in your test cases? These are questions I will 
discuss in this article, based on my experience with testing. 
So imagine you have a test case, which just states that you should test the 
feature, and confirm that you experience the desired outcome. If you have no 
previous experience with the game and it’s features, performing such a test will 
be very difficult, and most likely counter-productive. If you are an experienced 
game tester who has worked with the game and it’s features extensively, it may 
be enough information for you. 
So the key point here is: 
 Context-specific domain knowledge (knowing the game and the features) 
affects the game tester’s requirements on the test cases (or similar test 
artifacts) 
What if you are very experienced in general but do not have any game specific 
knowledge? Most likely you would run some exploratory testing on the feature 
before the actual test to understand how it works, and then discuss your findings 
with a developer or designer. This way you get the context-specific domain 
knowledge you need to perform the tests using the very vague test cases. 
Key point: 
 General game testing experience can allow you to mitigate the lack of 
context-specific domain knowledge, if you have the means to bridge this 
gap through exploration of the game/feature and having the proper 
communication channels with designers/developers 
But there are obviously exceptions. If you have to enter certain settings into test 
tools or environments to be able to test the feature, and this information is not 
readily available, it will be difficult to perform the tests. This could be part of the 
test case, or of some other test documentation, depending on the context. 
Key point: 
 Specific test tool and test environment information needs to be available 
to the tester regardless of experience 
So why not just write very detailed test cases, with intricate step-by-step 
instructions for how to do every single thing, so that even someone without any 
experience could do it? That would make all of the above irrelevant. 
There are a few reasons.
Firstly it is very costly to write these kinds of test cases. The game testers have to 
spend a lot of time creating these test cases. Time they could have spent doing 
more valuable things, like actually testing the game, or other preparations for 
testing. 
Secondly these test cases will require much more maintenance. Every time a 
small detail in the feature changes, the tests need to be updated. This is an 
additional cost apart from the upfront creation cost. 
Thirdly these types of detailed, scripted tests restrict the tester and reduce the 
variation and innovation in the tests performed. You will test everything exactly 
the same every time, and there is a greater risk of missing bugs at the periphery 
of the test. The tester is executing a test case, rather than focusing on testing the 
feature. Running the exact same test over and over again is often not an optimal 
use of resources. 
Fourthly it gives false confidence that inexperienced game testers can just 
perform these tests, and that the confidence in this results will be the same as if 
an experienced tester performed the tests. Even though an inexperienced tester 
can run through a detailed, scripted test, this does not mean that an experienced 
game tester wouldn’t have done it better. And I don’t mean faster – I mean 
finding tricky bugs in the area, which the detailed test artifact intends to cover. 
You cannot separate test design from test execution and think that test execution 
is something mindless which can be outsourced to random people. Good test 
execution requires experience and opportunities to utilize that experience when 
you perform your tests. 
Key points: 
 High up-front cost 
 High maintenance cost 
 Reduces variation and innovation in tests 
 False confidence - detailed test cases cannot replace experience 
 Test execution cannot be seen as a mindless follow-up to test design 
But what about if you are an inexperienced tester – isn’t detailed test cases a 
good way to learn the feature/game, and get the initial experience you need? 
This is a tempting proposition. An experienced tester transfers his knowledge to 
an inexperienced tester through a test case. Personally I am unconvinced this is a 
good way to transfer knowledge. I think the best way is either through pair-testing, 
mentorships, or through good communication channels. I just don’t think 
detailed test cases are a cost-effective way of transferring knowledge. 
With this in mind I would like to edit the first key-point I presented: 
 Context-specific domain knowledge (knowing the game and the features) 
affects the game tester’s requirements on the test cases (or similar test
artifacts), or other means to acquire knowledge of how to test the 
game/features 
So how do you reach the optimal level of detail in a test artifact? 
1. Make sure that mandatory things like information about test tools and 
test environment is available for the tester when running the test case – 
either in the test case or other test documentation 
2. Let experienced testers write tests and let other experienced testers try to 
run those tests and review the results 
3. Make sure you have the framework in place to train inexperienced testers 
on the job – for example pair-testing, mentors, or good communication 
channels 
But even experienced game testers have to understand what needs to be tested, 
so that they do not forget to test specific parts of the game or feature. So a test 
case that states: “Test the game”, would be problematic for anyone, regardless of 
experience. 
The goal with a test case (or similar test artifact) should be: 
 To help the game tester understand what needs to be tested, not how it 
should be tested in detail 
If an experienced game tester knows what needs to be tested, then the game 
tester can use experience and knowledge to deduct how to test that in the best 
way, with regards to the current context, using minimal documentation. 
These are my views right now in any event. 
/Johan Hoberg

More Related Content

What's hot

Maintaining test fairness
Maintaining test fairnessMaintaining test fairness
Maintaining test fairness
Test Generator
 
Styliff hekovnik homework
Styliff hekovnik homeworkStyliff hekovnik homework
Styliff hekovnik homework
Bor Cerar Lampič
 
PHP unit testing - good and bad practices
PHP unit testing - good and bad practicesPHP unit testing - good and bad practices
PHP unit testing - good and bad practices
Konstantin Stefanov
 
The Nature Of Patterns
The Nature Of PatternsThe Nature Of Patterns
The Nature Of Patterns
Nick Harrison
 
x3ntaur Conceptos Básicos de Preactor
x3ntaur Conceptos Básicos de Preactorx3ntaur Conceptos Básicos de Preactor
x3ntaur Conceptos Básicos de Preactor
Javier Diaz
 
Advanced unit testing – real life examples and mistakes
Advanced unit testing – real life examples and mistakesAdvanced unit testing – real life examples and mistakes
Advanced unit testing – real life examples and mistakes
Milan Vukoje
 
Real Life Unit Testing
Real Life Unit TestingReal Life Unit Testing
Real Life Unit Testing
Dror Helper
 
Testing
TestingTesting
Testing
Bas Bossink
 
Software engineering 22 error detection and debugging
Software engineering 22 error detection and debuggingSoftware engineering 22 error detection and debugging
Software engineering 22 error detection and debugging
Vaibhav Khanna
 
How to write defect
How to write defectHow to write defect
How to write defect
Afsarah Jahin
 
Boundary value analysis and equivalence partitioning
Boundary value analysis and equivalence partitioningBoundary value analysis and equivalence partitioning
Boundary value analysis and equivalence partitioning
Sneha Singh
 
Test case design_the_basicsv0.4
Test case design_the_basicsv0.4Test case design_the_basicsv0.4
Test case design_the_basicsv0.4
guest31fced
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
David Rogers
 
Debugging by induction
Debugging by inductionDebugging by induction
Debugging by induction
Waqar Ali
 
Assignment 1 applications of the scientific method
Assignment 1 applications of the scientific methodAssignment 1 applications of the scientific method
Assignment 1 applications of the scientific method
coursesexams1
 
Debugging
DebuggingDebugging
Dakiry_qastandup_Olia Didyk_testdesign
Dakiry_qastandup_Olia Didyk_testdesignDakiry_qastandup_Olia Didyk_testdesign
Dakiry_qastandup_Olia Didyk_testdesign
Dakiry
 
Unit testing - An introduction
Unit testing - An introductionUnit testing - An introduction
Unit testing - An introduction
Alejandro Claro Mosqueda
 
Introductory Programming With Python
Introductory Programming With PythonIntroductory Programming With Python
Introductory Programming With Python
Brendan McCane
 
Effective unit testing
Effective unit testingEffective unit testing
Effective unit testing
Roberto Casadei
 

What's hot (20)

Maintaining test fairness
Maintaining test fairnessMaintaining test fairness
Maintaining test fairness
 
Styliff hekovnik homework
Styliff hekovnik homeworkStyliff hekovnik homework
Styliff hekovnik homework
 
PHP unit testing - good and bad practices
PHP unit testing - good and bad practicesPHP unit testing - good and bad practices
PHP unit testing - good and bad practices
 
The Nature Of Patterns
The Nature Of PatternsThe Nature Of Patterns
The Nature Of Patterns
 
x3ntaur Conceptos Básicos de Preactor
x3ntaur Conceptos Básicos de Preactorx3ntaur Conceptos Básicos de Preactor
x3ntaur Conceptos Básicos de Preactor
 
Advanced unit testing – real life examples and mistakes
Advanced unit testing – real life examples and mistakesAdvanced unit testing – real life examples and mistakes
Advanced unit testing – real life examples and mistakes
 
Real Life Unit Testing
Real Life Unit TestingReal Life Unit Testing
Real Life Unit Testing
 
Testing
TestingTesting
Testing
 
Software engineering 22 error detection and debugging
Software engineering 22 error detection and debuggingSoftware engineering 22 error detection and debugging
Software engineering 22 error detection and debugging
 
How to write defect
How to write defectHow to write defect
How to write defect
 
Boundary value analysis and equivalence partitioning
Boundary value analysis and equivalence partitioningBoundary value analysis and equivalence partitioning
Boundary value analysis and equivalence partitioning
 
Test case design_the_basicsv0.4
Test case design_the_basicsv0.4Test case design_the_basicsv0.4
Test case design_the_basicsv0.4
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
 
Debugging by induction
Debugging by inductionDebugging by induction
Debugging by induction
 
Assignment 1 applications of the scientific method
Assignment 1 applications of the scientific methodAssignment 1 applications of the scientific method
Assignment 1 applications of the scientific method
 
Debugging
DebuggingDebugging
Debugging
 
Dakiry_qastandup_Olia Didyk_testdesign
Dakiry_qastandup_Olia Didyk_testdesignDakiry_qastandup_Olia Didyk_testdesign
Dakiry_qastandup_Olia Didyk_testdesign
 
Unit testing - An introduction
Unit testing - An introductionUnit testing - An introduction
Unit testing - An introduction
 
Introductory Programming With Python
Introductory Programming With PythonIntroductory Programming With Python
Introductory Programming With Python
 
Effective unit testing
Effective unit testingEffective unit testing
Effective unit testing
 

Viewers also liked

Quality in Games
Quality in GamesQuality in Games
Quality in Games
Johan Hoberg
 
Testing in a scrum team
Testing in a scrum teamTesting in a scrum team
Testing in a scrum team
Johan Hoberg
 
How a Game Tester Adds Value
How a Game Tester Adds ValueHow a Game Tester Adds Value
How a Game Tester Adds Value
Johan Hoberg
 
Information artifact simplicity
Information artifact simplicityInformation artifact simplicity
Information artifact simplicity
Johan Hoberg
 
Systematic inventive thinking and game testing
Systematic inventive thinking and game testingSystematic inventive thinking and game testing
Systematic inventive thinking and game testing
Johan Hoberg
 
Initial thoughts on live user tests for games
Initial thoughts on live user tests for gamesInitial thoughts on live user tests for games
Initial thoughts on live user tests for games
Johan Hoberg
 
Acceptance Criteria as Requirements and Tests
Acceptance Criteria as Requirements and TestsAcceptance Criteria as Requirements and Tests
Acceptance Criteria as Requirements and Tests
Johan Hoberg
 
Testing & Scrum
Testing & ScrumTesting & Scrum
Testing & Scrum
Johan Hoberg
 
The Value-Adding Test Strategist
The Value-Adding Test StrategistThe Value-Adding Test Strategist
The Value-Adding Test Strategist
Johan Hoberg
 
Software testing vs. Game testing
Software testing vs. Game testingSoftware testing vs. Game testing
Software testing vs. Game testing
Johan Hoberg
 
QI, not QA
QI, not QAQI, not QA
QI, not QA
Johan Hoberg
 
Hardware/Software Integration Testing
Hardware/Software Integration TestingHardware/Software Integration Testing
Hardware/Software Integration Testing
Johan Hoberg
 
How to structure testing within the Scrum Framework
How to structure testing within the Scrum FrameworkHow to structure testing within the Scrum Framework
How to structure testing within the Scrum Framework
Johan Hoberg
 
Defining Test Competence
Defining Test CompetenceDefining Test Competence
Defining Test Competence
Johan Hoberg
 
Giving feedback & Scrum
Giving feedback & ScrumGiving feedback & Scrum
Giving feedback & Scrum
Johan Hoberg
 
Exploratory Testing for Developers
Exploratory Testing for DevelopersExploratory Testing for Developers
Exploratory Testing for Developers
Johan Hoberg
 
Why all deadlines are bad for quality
Why all deadlines are bad for qualityWhy all deadlines are bad for quality
Why all deadlines are bad for quality
Johan Hoberg
 
Communicated deadlines = bad quality
Communicated deadlines = bad qualityCommunicated deadlines = bad quality
Communicated deadlines = bad quality
Johan Hoberg
 
Software testing and game testing
Software testing and game testingSoftware testing and game testing
Software testing and game testing
Johan Hoberg
 
Quality, Testing & Agile Methodologies
Quality, Testing & Agile MethodologiesQuality, Testing & Agile Methodologies
Quality, Testing & Agile Methodologies
Johan Hoberg
 

Viewers also liked (20)

Quality in Games
Quality in GamesQuality in Games
Quality in Games
 
Testing in a scrum team
Testing in a scrum teamTesting in a scrum team
Testing in a scrum team
 
How a Game Tester Adds Value
How a Game Tester Adds ValueHow a Game Tester Adds Value
How a Game Tester Adds Value
 
Information artifact simplicity
Information artifact simplicityInformation artifact simplicity
Information artifact simplicity
 
Systematic inventive thinking and game testing
Systematic inventive thinking and game testingSystematic inventive thinking and game testing
Systematic inventive thinking and game testing
 
Initial thoughts on live user tests for games
Initial thoughts on live user tests for gamesInitial thoughts on live user tests for games
Initial thoughts on live user tests for games
 
Acceptance Criteria as Requirements and Tests
Acceptance Criteria as Requirements and TestsAcceptance Criteria as Requirements and Tests
Acceptance Criteria as Requirements and Tests
 
Testing & Scrum
Testing & ScrumTesting & Scrum
Testing & Scrum
 
The Value-Adding Test Strategist
The Value-Adding Test StrategistThe Value-Adding Test Strategist
The Value-Adding Test Strategist
 
Software testing vs. Game testing
Software testing vs. Game testingSoftware testing vs. Game testing
Software testing vs. Game testing
 
QI, not QA
QI, not QAQI, not QA
QI, not QA
 
Hardware/Software Integration Testing
Hardware/Software Integration TestingHardware/Software Integration Testing
Hardware/Software Integration Testing
 
How to structure testing within the Scrum Framework
How to structure testing within the Scrum FrameworkHow to structure testing within the Scrum Framework
How to structure testing within the Scrum Framework
 
Defining Test Competence
Defining Test CompetenceDefining Test Competence
Defining Test Competence
 
Giving feedback & Scrum
Giving feedback & ScrumGiving feedback & Scrum
Giving feedback & Scrum
 
Exploratory Testing for Developers
Exploratory Testing for DevelopersExploratory Testing for Developers
Exploratory Testing for Developers
 
Why all deadlines are bad for quality
Why all deadlines are bad for qualityWhy all deadlines are bad for quality
Why all deadlines are bad for quality
 
Communicated deadlines = bad quality
Communicated deadlines = bad qualityCommunicated deadlines = bad quality
Communicated deadlines = bad quality
 
Software testing and game testing
Software testing and game testingSoftware testing and game testing
Software testing and game testing
 
Quality, Testing & Agile Methodologies
Quality, Testing & Agile MethodologiesQuality, Testing & Agile Methodologies
Quality, Testing & Agile Methodologies
 

Similar to Codifying Knowledge in Tests

Exploratory Testing: Make It Part of Your Test Strategy
Exploratory Testing: Make It Part of Your Test StrategyExploratory Testing: Make It Part of Your Test Strategy
Exploratory Testing: Make It Part of Your Test Strategy
TechWell
 
Software testing _mod_9
Software testing _mod_9Software testing _mod_9
Software testing _mod_9
hellosashi
 
Exploratory Testing, A Guide Towards Better Test Coverage.pdf
Exploratory Testing, A Guide Towards Better Test Coverage.pdfExploratory Testing, A Guide Towards Better Test Coverage.pdf
Exploratory Testing, A Guide Towards Better Test Coverage.pdf
pCloudy
 
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
Mary Clemons
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
TechWell
 
Experiences with Semi-Scripted Exploratory Testing
Experiences with Semi-Scripted Exploratory TestingExperiences with Semi-Scripted Exploratory Testing
Experiences with Semi-Scripted Exploratory Testing
Simon Morley
 
Caveon webinar series - smart items- using innovative item design to make you...
Caveon webinar series - smart items- using innovative item design to make you...Caveon webinar series - smart items- using innovative item design to make you...
Caveon webinar series - smart items- using innovative item design to make you...
Caveon Test Security
 
Defining Test Competence
Defining Test CompetenceDefining Test Competence
Defining Test Competence
Johan Hoberg
 
SAM
SAMSAM
Pairing: The Secret Sauce of Agile Testing
Pairing: The Secret Sauce of Agile TestingPairing: The Secret Sauce of Agile Testing
Pairing: The Secret Sauce of Agile Testing
TechWell
 
70 270 q & a
70 270 q & a70 270 q & a
70 270 q & a
Samaja
 
Software Testing for International Students
Software Testing for International StudentsSoftware Testing for International Students
Software Testing for International Students
cristianoribeirosilva
 
Exploratory Testing - A Whitepaper by RapidValue
Exploratory Testing -  A Whitepaper by RapidValueExploratory Testing -  A Whitepaper by RapidValue
Exploratory Testing - A Whitepaper by RapidValue
RapidValue
 
Effective Testing fo Startups
Effective Testing fo StartupsEffective Testing fo Startups
Effective Testing fo Startups
Testnetic
 
Test Cases Vs Test Scenarios
Test Cases Vs Test ScenariosTest Cases Vs Test Scenarios
Test Cases Vs Test Scenarios
Sneha Singh
 
Test Cases Maintaining & Documenting
Test Cases Maintaining & DocumentingTest Cases Maintaining & Documenting
Test Cases Maintaining & Documenting
Seyed Ali Marjaie
 
Business awareness of testers and the quality of testing
Business awareness of testers and the quality of testing Business awareness of testers and the quality of testing
Business awareness of testers and the quality of testing
KAROLINA ZMITROWICZ
 
Test case writing
Test case writingTest case writing
Test case writing
Afsarah Jahin
 
Develop your inner tester
Develop your inner tester Develop your inner tester
Develop your inner tester
Anne-Marie Charrett
 
How to do usability testing and eye tracking
How to do usability testing and eye trackingHow to do usability testing and eye tracking
How to do usability testing and eye tracking
Objective Experience
 

Similar to Codifying Knowledge in Tests (20)

Exploratory Testing: Make It Part of Your Test Strategy
Exploratory Testing: Make It Part of Your Test StrategyExploratory Testing: Make It Part of Your Test Strategy
Exploratory Testing: Make It Part of Your Test Strategy
 
Software testing _mod_9
Software testing _mod_9Software testing _mod_9
Software testing _mod_9
 
Exploratory Testing, A Guide Towards Better Test Coverage.pdf
Exploratory Testing, A Guide Towards Better Test Coverage.pdfExploratory Testing, A Guide Towards Better Test Coverage.pdf
Exploratory Testing, A Guide Towards Better Test Coverage.pdf
 
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
 
Experiences with Semi-Scripted Exploratory Testing
Experiences with Semi-Scripted Exploratory TestingExperiences with Semi-Scripted Exploratory Testing
Experiences with Semi-Scripted Exploratory Testing
 
Caveon webinar series - smart items- using innovative item design to make you...
Caveon webinar series - smart items- using innovative item design to make you...Caveon webinar series - smart items- using innovative item design to make you...
Caveon webinar series - smart items- using innovative item design to make you...
 
Defining Test Competence
Defining Test CompetenceDefining Test Competence
Defining Test Competence
 
SAM
SAMSAM
SAM
 
Pairing: The Secret Sauce of Agile Testing
Pairing: The Secret Sauce of Agile TestingPairing: The Secret Sauce of Agile Testing
Pairing: The Secret Sauce of Agile Testing
 
70 270 q & a
70 270 q & a70 270 q & a
70 270 q & a
 
Software Testing for International Students
Software Testing for International StudentsSoftware Testing for International Students
Software Testing for International Students
 
Exploratory Testing - A Whitepaper by RapidValue
Exploratory Testing -  A Whitepaper by RapidValueExploratory Testing -  A Whitepaper by RapidValue
Exploratory Testing - A Whitepaper by RapidValue
 
Effective Testing fo Startups
Effective Testing fo StartupsEffective Testing fo Startups
Effective Testing fo Startups
 
Test Cases Vs Test Scenarios
Test Cases Vs Test ScenariosTest Cases Vs Test Scenarios
Test Cases Vs Test Scenarios
 
Test Cases Maintaining & Documenting
Test Cases Maintaining & DocumentingTest Cases Maintaining & Documenting
Test Cases Maintaining & Documenting
 
Business awareness of testers and the quality of testing
Business awareness of testers and the quality of testing Business awareness of testers and the quality of testing
Business awareness of testers and the quality of testing
 
Test case writing
Test case writingTest case writing
Test case writing
 
Develop your inner tester
Develop your inner tester Develop your inner tester
Develop your inner tester
 
How to do usability testing and eye tracking
How to do usability testing and eye trackingHow to do usability testing and eye tracking
How to do usability testing and eye tracking
 

More from Johan Hoberg

Approaches to unraveling a complex test problem
Approaches to unraveling a complex test problemApproaches to unraveling a complex test problem
Approaches to unraveling a complex test problem
Johan Hoberg
 
A business case for a modern QA organization
A business case for a modern QA organizationA business case for a modern QA organization
A business case for a modern QA organization
Johan Hoberg
 
Signing off on Quality
Signing off on QualitySigning off on Quality
Signing off on Quality
Johan Hoberg
 
Quality Information Coverage - A QI Concept
Quality Information Coverage - A QI ConceptQuality Information Coverage - A QI Concept
Quality Information Coverage - A QI Concept
Johan Hoberg
 
The Bug Backlog - An Evergrowing Mountain
The Bug Backlog - An Evergrowing MountainThe Bug Backlog - An Evergrowing Mountain
The Bug Backlog - An Evergrowing Mountain
Johan Hoberg
 
Quality Intelligence: Transparency & Visibility
Quality Intelligence: Transparency & VisibilityQuality Intelligence: Transparency & Visibility
Quality Intelligence: Transparency & Visibility
Johan Hoberg
 
Building a QA Mindset
Building a QA Mindset Building a QA Mindset
Building a QA Mindset
Johan Hoberg
 
What is QI?
What is QI?What is QI?
What is QI?
Johan Hoberg
 
Building High Quality Software
Building High Quality Software Building High Quality Software
Building High Quality Software
Johan Hoberg
 
Testit 2017 - Exploratory Testing for Everyone
Testit 2017 - Exploratory Testing for EveryoneTestit 2017 - Exploratory Testing for Everyone
Testit 2017 - Exploratory Testing for Everyone
Johan Hoberg
 
Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...
Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...
Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...
Johan Hoberg
 
Moving from scripted regression testing to exploratory testing
Moving from scripted regression testing to exploratory testingMoving from scripted regression testing to exploratory testing
Moving from scripted regression testing to exploratory testing
Johan Hoberg
 
Building High Quality Software
Building High Quality SoftwareBuilding High Quality Software
Building High Quality Software
Johan Hoberg
 
QI, not QA
QI, not QAQI, not QA
QI, not QA
Johan Hoberg
 
The Tester Role & Scrum
The Tester Role & ScrumThe Tester Role & Scrum
The Tester Role & Scrum
Johan Hoberg
 

More from Johan Hoberg (15)

Approaches to unraveling a complex test problem
Approaches to unraveling a complex test problemApproaches to unraveling a complex test problem
Approaches to unraveling a complex test problem
 
A business case for a modern QA organization
A business case for a modern QA organizationA business case for a modern QA organization
A business case for a modern QA organization
 
Signing off on Quality
Signing off on QualitySigning off on Quality
Signing off on Quality
 
Quality Information Coverage - A QI Concept
Quality Information Coverage - A QI ConceptQuality Information Coverage - A QI Concept
Quality Information Coverage - A QI Concept
 
The Bug Backlog - An Evergrowing Mountain
The Bug Backlog - An Evergrowing MountainThe Bug Backlog - An Evergrowing Mountain
The Bug Backlog - An Evergrowing Mountain
 
Quality Intelligence: Transparency & Visibility
Quality Intelligence: Transparency & VisibilityQuality Intelligence: Transparency & Visibility
Quality Intelligence: Transparency & Visibility
 
Building a QA Mindset
Building a QA Mindset Building a QA Mindset
Building a QA Mindset
 
What is QI?
What is QI?What is QI?
What is QI?
 
Building High Quality Software
Building High Quality Software Building High Quality Software
Building High Quality Software
 
Testit 2017 - Exploratory Testing for Everyone
Testit 2017 - Exploratory Testing for EveryoneTestit 2017 - Exploratory Testing for Everyone
Testit 2017 - Exploratory Testing for Everyone
 
Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...
Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...
Don’t celebrate failure. Don’t celebrate success. Celebrate commitment, owner...
 
Moving from scripted regression testing to exploratory testing
Moving from scripted regression testing to exploratory testingMoving from scripted regression testing to exploratory testing
Moving from scripted regression testing to exploratory testing
 
Building High Quality Software
Building High Quality SoftwareBuilding High Quality Software
Building High Quality Software
 
QI, not QA
QI, not QAQI, not QA
QI, not QA
 
The Tester Role & Scrum
The Tester Role & ScrumThe Tester Role & Scrum
The Tester Role & Scrum
 

Recently uploaded

SELENIUM CONF -PALLAVI SHARMA - 2024.pdf
SELENIUM CONF -PALLAVI SHARMA - 2024.pdfSELENIUM CONF -PALLAVI SHARMA - 2024.pdf
SELENIUM CONF -PALLAVI SHARMA - 2024.pdf
Pallavi Sharma
 
Digital Twins Computer Networking Paper Presentation.pptx
Digital Twins Computer Networking Paper Presentation.pptxDigital Twins Computer Networking Paper Presentation.pptx
Digital Twins Computer Networking Paper Presentation.pptx
aryanpankaj78
 
309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf
309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf
309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf
Sou Tibon
 
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Transcat
 
Call For Paper -3rd International Conference on Artificial Intelligence Advan...
Call For Paper -3rd International Conference on Artificial Intelligence Advan...Call For Paper -3rd International Conference on Artificial Intelligence Advan...
Call For Paper -3rd International Conference on Artificial Intelligence Advan...
ijseajournal
 
comptia-security-sy0-701-exam-objectives-(5-0).pdf
comptia-security-sy0-701-exam-objectives-(5-0).pdfcomptia-security-sy0-701-exam-objectives-(5-0).pdf
comptia-security-sy0-701-exam-objectives-(5-0).pdf
foxlyon
 
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...
DharmaBanothu
 
ITSM Integration with MuleSoft.pptx
ITSM  Integration with MuleSoft.pptxITSM  Integration with MuleSoft.pptx
ITSM Integration with MuleSoft.pptx
VANDANAMOHANGOUDA
 
Beckhoff Programmable Logic Control Overview Presentation
Beckhoff Programmable Logic Control Overview PresentationBeckhoff Programmable Logic Control Overview Presentation
Beckhoff Programmable Logic Control Overview Presentation
VanTuDuong1
 
FULL STACK PROGRAMMING - Both Front End and Back End
FULL STACK PROGRAMMING - Both Front End and Back EndFULL STACK PROGRAMMING - Both Front End and Back End
FULL STACK PROGRAMMING - Both Front End and Back End
PreethaV16
 
INTRODUCTION TO ARTIFICIAL INTELLIGENCE BASIC
INTRODUCTION TO ARTIFICIAL INTELLIGENCE BASICINTRODUCTION TO ARTIFICIAL INTELLIGENCE BASIC
INTRODUCTION TO ARTIFICIAL INTELLIGENCE BASIC
GOKULKANNANMMECLECTC
 
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
nedcocy
 
ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...
ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...
ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...
Kuvempu University
 
This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...
DharmaBanothu
 
Applications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdfApplications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdf
Atif Razi
 
Open Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surfaceOpen Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surface
Indrajeet sahu
 
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICSUNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
vmspraneeth
 
Accident detection system project report.pdf
Accident detection system project report.pdfAccident detection system project report.pdf
Accident detection system project report.pdf
Kamal Acharya
 
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
upoux
 
DELTA V MES EMERSON EDUARDO RODRIGUES ENGINEER
DELTA V MES EMERSON EDUARDO RODRIGUES ENGINEERDELTA V MES EMERSON EDUARDO RODRIGUES ENGINEER
DELTA V MES EMERSON EDUARDO RODRIGUES ENGINEER
EMERSON EDUARDO RODRIGUES
 

Recently uploaded (20)

SELENIUM CONF -PALLAVI SHARMA - 2024.pdf
SELENIUM CONF -PALLAVI SHARMA - 2024.pdfSELENIUM CONF -PALLAVI SHARMA - 2024.pdf
SELENIUM CONF -PALLAVI SHARMA - 2024.pdf
 
Digital Twins Computer Networking Paper Presentation.pptx
Digital Twins Computer Networking Paper Presentation.pptxDigital Twins Computer Networking Paper Presentation.pptx
Digital Twins Computer Networking Paper Presentation.pptx
 
309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf
309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf
309475979-Creativity-Innovation-notes-IV-Sem-2016-pdf.pdf
 
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
 
Call For Paper -3rd International Conference on Artificial Intelligence Advan...
Call For Paper -3rd International Conference on Artificial Intelligence Advan...Call For Paper -3rd International Conference on Artificial Intelligence Advan...
Call For Paper -3rd International Conference on Artificial Intelligence Advan...
 
comptia-security-sy0-701-exam-objectives-(5-0).pdf
comptia-security-sy0-701-exam-objectives-(5-0).pdfcomptia-security-sy0-701-exam-objectives-(5-0).pdf
comptia-security-sy0-701-exam-objectives-(5-0).pdf
 
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...
 
ITSM Integration with MuleSoft.pptx
ITSM  Integration with MuleSoft.pptxITSM  Integration with MuleSoft.pptx
ITSM Integration with MuleSoft.pptx
 
Beckhoff Programmable Logic Control Overview Presentation
Beckhoff Programmable Logic Control Overview PresentationBeckhoff Programmable Logic Control Overview Presentation
Beckhoff Programmable Logic Control Overview Presentation
 
FULL STACK PROGRAMMING - Both Front End and Back End
FULL STACK PROGRAMMING - Both Front End and Back EndFULL STACK PROGRAMMING - Both Front End and Back End
FULL STACK PROGRAMMING - Both Front End and Back End
 
INTRODUCTION TO ARTIFICIAL INTELLIGENCE BASIC
INTRODUCTION TO ARTIFICIAL INTELLIGENCE BASICINTRODUCTION TO ARTIFICIAL INTELLIGENCE BASIC
INTRODUCTION TO ARTIFICIAL INTELLIGENCE BASIC
 
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
 
ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...
ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...
ELS: 2.4.1 POWER ELECTRONICS Course objectives: This course will enable stude...
 
This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...
 
Applications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdfApplications of artificial Intelligence in Mechanical Engineering.pdf
Applications of artificial Intelligence in Mechanical Engineering.pdf
 
Open Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surfaceOpen Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surface
 
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICSUNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
 
Accident detection system project report.pdf
Accident detection system project report.pdfAccident detection system project report.pdf
Accident detection system project report.pdf
 
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
 
DELTA V MES EMERSON EDUARDO RODRIGUES ENGINEER
DELTA V MES EMERSON EDUARDO RODRIGUES ENGINEERDELTA V MES EMERSON EDUARDO RODRIGUES ENGINEER
DELTA V MES EMERSON EDUARDO RODRIGUES ENGINEER
 

Codifying Knowledge in Tests

  • 1. Codifying Game & Test Knowledge in Tests? How much information should your test cases (or test missions, charters, or other types or similar test artifacts) include? What are the pros and cons of adding lots of detailed information in your test cases? These are questions I will discuss in this article, based on my experience with testing. So imagine you have a test case, which just states that you should test the feature, and confirm that you experience the desired outcome. If you have no previous experience with the game and it’s features, performing such a test will be very difficult, and most likely counter-productive. If you are an experienced game tester who has worked with the game and it’s features extensively, it may be enough information for you. So the key point here is:  Context-specific domain knowledge (knowing the game and the features) affects the game tester’s requirements on the test cases (or similar test artifacts) What if you are very experienced in general but do not have any game specific knowledge? Most likely you would run some exploratory testing on the feature before the actual test to understand how it works, and then discuss your findings with a developer or designer. This way you get the context-specific domain knowledge you need to perform the tests using the very vague test cases. Key point:  General game testing experience can allow you to mitigate the lack of context-specific domain knowledge, if you have the means to bridge this gap through exploration of the game/feature and having the proper communication channels with designers/developers But there are obviously exceptions. If you have to enter certain settings into test tools or environments to be able to test the feature, and this information is not readily available, it will be difficult to perform the tests. This could be part of the test case, or of some other test documentation, depending on the context. Key point:  Specific test tool and test environment information needs to be available to the tester regardless of experience So why not just write very detailed test cases, with intricate step-by-step instructions for how to do every single thing, so that even someone without any experience could do it? That would make all of the above irrelevant. There are a few reasons.
  • 2. Firstly it is very costly to write these kinds of test cases. The game testers have to spend a lot of time creating these test cases. Time they could have spent doing more valuable things, like actually testing the game, or other preparations for testing. Secondly these test cases will require much more maintenance. Every time a small detail in the feature changes, the tests need to be updated. This is an additional cost apart from the upfront creation cost. Thirdly these types of detailed, scripted tests restrict the tester and reduce the variation and innovation in the tests performed. You will test everything exactly the same every time, and there is a greater risk of missing bugs at the periphery of the test. The tester is executing a test case, rather than focusing on testing the feature. Running the exact same test over and over again is often not an optimal use of resources. Fourthly it gives false confidence that inexperienced game testers can just perform these tests, and that the confidence in this results will be the same as if an experienced tester performed the tests. Even though an inexperienced tester can run through a detailed, scripted test, this does not mean that an experienced game tester wouldn’t have done it better. And I don’t mean faster – I mean finding tricky bugs in the area, which the detailed test artifact intends to cover. You cannot separate test design from test execution and think that test execution is something mindless which can be outsourced to random people. Good test execution requires experience and opportunities to utilize that experience when you perform your tests. Key points:  High up-front cost  High maintenance cost  Reduces variation and innovation in tests  False confidence - detailed test cases cannot replace experience  Test execution cannot be seen as a mindless follow-up to test design But what about if you are an inexperienced tester – isn’t detailed test cases a good way to learn the feature/game, and get the initial experience you need? This is a tempting proposition. An experienced tester transfers his knowledge to an inexperienced tester through a test case. Personally I am unconvinced this is a good way to transfer knowledge. I think the best way is either through pair-testing, mentorships, or through good communication channels. I just don’t think detailed test cases are a cost-effective way of transferring knowledge. With this in mind I would like to edit the first key-point I presented:  Context-specific domain knowledge (knowing the game and the features) affects the game tester’s requirements on the test cases (or similar test
  • 3. artifacts), or other means to acquire knowledge of how to test the game/features So how do you reach the optimal level of detail in a test artifact? 1. Make sure that mandatory things like information about test tools and test environment is available for the tester when running the test case – either in the test case or other test documentation 2. Let experienced testers write tests and let other experienced testers try to run those tests and review the results 3. Make sure you have the framework in place to train inexperienced testers on the job – for example pair-testing, mentors, or good communication channels But even experienced game testers have to understand what needs to be tested, so that they do not forget to test specific parts of the game or feature. So a test case that states: “Test the game”, would be problematic for anyone, regardless of experience. The goal with a test case (or similar test artifact) should be:  To help the game tester understand what needs to be tested, not how it should be tested in detail If an experienced game tester knows what needs to be tested, then the game tester can use experience and knowledge to deduct how to test that in the best way, with regards to the current context, using minimal documentation. These are my views right now in any event. /Johan Hoberg