Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Defining Test Competence
In this article I will explore the concept of Test Competence and try to define
what I actually m...
would fall within the range of what we could expect a developer to test, since
they are most likely to have the competence...
 They start with a software system under test and some information about
this system (could be requirements, risks, histo...
Upcoming SlideShare
Loading in …5
×

Defining Test Competence

639 views

Published on

In this article I explore the concept of test competence, and how it is connected to complex problems.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Defining Test Competence

  1. 1. Defining Test Competence In this article I will explore the concept of Test Competence and try to define what I actually mean with it. To understand Test Competence, it is my belief that we must look in the Cynefin Framework [1]. There are many different types of problems, and according to this framework they can be divided into four categories; Obvious, Complicated, Complex and Chaotic. Using this framework we can categorize different test problems: Obvious Test Problems: Tests in which the relationship between cause and effect is obvious to all Complicated Test Problems: Tests in which the relationship between cause and effect requires analysis or some other form of investigation and/or the application of expert knowledge Complex Test Problems: Tests in which the relationship between cause and effect can only be perceived in retrospect, but not in advance An obvious test problem would most likely not require specific test competence to solve. It could be something as simple as pressing an application icon on your mobile phone and expecting the application to start. This is something anyone could test, regardless of specific competence. A complicated test problem would be a little bit harder. To solve this would require an understanding of the system and its architecture, as well as an understanding of how to read and write code. But once the relationship between cause and effect has been established, it is just a matter of verifying it. To me this Sidebar: How does complex behavior arise? “The behavior of a complex system is often said to be due to emergence.” [2] “Emergence is a process whereby larger entities, patterns, and regularities arise through interactions among smaller or simpler entities that themselves do not exhibit such properties.” [3]
  2. 2. would fall within the range of what we could expect a developer to test, since they are most likely to have the competence required to do so, but I would not classify this problem as one that required a high level of test competence. It requires a lot of expert knowledge, but not specifically about testing. Before we continue, let’s take some time to define what I believe lies outside of the Test Competence concept:  Writing reports – Definitely important to a tester, but not really testing  Writing automated tests – A difficult task that requires a good coder, but not necessarily high test competence (choosing what to write automated tests for is another story)  Executing manual regression tests – Might require intricate knowledge of the system and different tools, but if you are just following a script and you have a set scope of tests to run, then you should not need much else  Critical thinking, adaptability and curiosity – Obviously important to a tester, but that goes for many different roles  Risk assessment and prioritization – Some key competencies for a tester, but many other roles also require you to master these  Agile and Scrum – Definitely important for everyone who works in a Scrum team, but not part of the Test Competence concept  Collaboration and communication skills – Again key competencies for a tester, but also for many others There are of course more examples, but hopefully I have framed the concept of Test Competence a little more with these. So if nothing of the above is the elusive Test Competence … then what is it that a good tester does better than anyone else? I believe the answer is: “Solving complex test problems.” Ok, so how does a tester solve a complex test problem using their test competence? What makes their skillset unique? I believe Test Competence consists of (at least) the following components:  Primary: Modeling unpredictability in a complex system  Primary: Having a toolbox of triggers that could cause a system to behave unpredictably  Secondary: Provoking a system to reach an unpredictable outcome  Secondary: Exploring a complex system through tests So what does this really mean, and how do I solve a complex test problem with these components? If an experienced tester is confronted with a complex test problem, then this is how I expect them to approach this problem:
  3. 3.  They start with a software system under test and some information about this system (could be requirements, risks, historical data, etc.)  Based on previous experience (either domain specific or general) and/or based on a toolbox of triggers, such as heuristics, test methods and test techniques, together with information about the system, I expect them to come up with a number of test ideas  Based on these ideas, they then model a faulty system (in their mind or documented in the form of high-level test cases or test missions) which could show an unpredictable behavior  They then provoke the system to try to reach this faulty state (which may or may not require specific tools, and/or understanding of the system) o Documenting how to provoke the system to reach the faulty state would probably result in a step-by-step detailed test case  If they succeed they may have found potentially valuable information, and if the system does not show unpredictable behavior then the system works according to specification  They then continue to explore the system using more test ideas, but also equipped with the new information the previous test provided So what does it mean to model a faulty system? Because this is the key part. Example: Let’s say I have previous experience with performance problems in earlier versions of a software, and I am now tasked with testing the new release of that software. A test idea related to performance is not that far fetched. With this information I must now build a model in my mind of the system where performance problems could occur. Maybe in my system, when performing action A, while I am in state B, during an ongoing action C, the performance of the system breaks down completely. Setting relevant A, B and C requires understanding of the system and/or the users of the system. Once I have built this faulty model in my mind I continue with trying to provoke the system to reach this faulty state, and see if I find some valuable information or not. To model a faulty system is to imagine a system, with a certain set of variables and states, which might behave unpredictably under specific circumstances. This is how I define Test Competence right now. Subject to change. References [1] Cynefin Framework https://en.wikipedia.org/wiki/Cynefin_Framework [2] Complexity https://en.wikipedia.org/wiki/Complexity [3] Emergence https://en.wikipedia.org/wiki/Emergence

×