Form and structure of test case MATTERS!

Form and Structure of Test Case MATTERS.T Ashokash@stagsoftware.com in.linkedin.com/in/AshokSTAG ash_thiruWebinar: Dec 7, 2011, 1430-1530 IST

Form is about the external shape, the way it is presented. Courtesy http://carwallpapers.in Structure is about the composition of the elements of a system. Courtesy http://www.pcars.us© 2011. STAG Software Private Limited. All rights reserved. 2

The Roller Coaster Form & Structure is the key to: Creating the twists & turns From htt p://gadget s.gunaxin .com/ﬁfty -roller-co asters/65403 to Experience the thrill of high G forces yet Ensuring safety and enable it to be built from desired materials© 2011. STAG Software Private Limited. All rights reserved. 3

Test cases or Stimuli is the “combination of inputs”. We know that these are combinatorial in nature. The key is to generate the smallest yet sufﬁcient set and be optimal. i.e Efficient© 2011. STAG Software Private Limited. All rights reserved. 5

The underlying architecture i.e. Form & Structure plays a vital role in ensuring a “scaffolding” that is light yet strong. Effective & Efficient© 2011. STAG Software Private Limited. All rights reserved. 6

Form & Structure of Test Cases must ENABLE: Clarity of the entity that I am testing? The types of defects am I looking for - Clear objective The stage when should I execute these Rapid generation of test cases The importance of these- Clear prioritisation A logical & methodical review Frequency of execution - How frequent? Foster just-enough documentation Objective of assessment - Conformance or Robustness? The ease of conversion into automated scripts?© 2011. STAG Software Private Limited. All rights reserved. 7

A good “Form & Structure” fosters simplicity, robustness and most importantly enables clear thinking. Clarity of thinking is key to producing good artifacts that can be objectively assessed to ensure conﬁdence. ... or shall we say Guarantee?© 2011. STAG Software Private Limited. All rights reserved. 8

Properties to be satisfied by a Good Form & Structure Enable setting a clear baseline for test Design Enable good design Enable objective evaluation of adequacy Review Aid in checking if it is designed well Enable quick yet ﬂexible and robust automation Automation Foster rapid design & test Execution Allow for choosing test cases to execute logically Enable clear & objective assessment of “cleanliness” Assessment Allow for continual improvement and “de-weeding” Improvement© 2011. STAG Software Private Limited. All rights reserved. 9

Hypothesis Based Testing - HBT 2.0 A Quick Introduction Personal, scientiﬁc test methodology. SIX stage methodology powered by EIGHT disciplines of thinking (STEMTM). Setup Hypothesize Cleanliness Criteria Potential Defect Types SUT Nine Stage Cleanliness Assessment Defect Removal Filter Click here to know more about HBT. http://slidesha.re/qBMNiy© 2011. STAG Software Private Limited. All rights reserved. 10

Cleanliness Levels in HBT 2.0 Nine-Stage Filter to Detect Defects Objective Issues That user expectations are met L9 End user value User flows, experience That it deploys well in the real environment L8 Clean Deployment Compatibility, migration That the stated attributes are met L7 Attributes met Performance, security, volume, load... That it does not mess up the environment L6 Environment cleanliness Resource leaks, Compatibility... That end-to-end flows work correctly L5 Flow correctness Business flow conditions, Linkages That the functional behaviour is correct L4 Behaviour correctness Functionality L3 Structural integrity That the internal structure is robust Internal structural issues That the user interface is clean L2 Input interface cleanliness UI issues That inputs are handled well L1 Input cleanliness Input data handling© 2011. STAG Software Private Limited. All rights reserved. 11

HBT Test Case Architecture Nine-dimensional FORM Organised by Cleanliness levels sub-ordered by items (features/modules..), Level segregated by type, ranked by importance/priority, Item sub-divided into conformance(+) and robustness(-), classiﬁed by early (smoke)/late-stage evaluation, Type tagged by evaluation frequency, linked by optimal execution order, classiﬁed by execution mode (manual/automated) Priority Focus Stage A well architected set of test cases is like a effective bait that can ‘attract‘ defects in the Frequency system. Order It is equally important to ensure that they Mode are well organised to enable execution optimisation and have the right set of information to ensure easy automation.© 2011. STAG Software Private Limited. All rights reserved. 12

Focused Scenarios enable Form Rapid and Easy automation L9 End user value Level based test scenarios foster L8 Clean Deployment “single-path” script. L7 Attributes met L6 Environment cleanliness L5 Robustness L4 Behaviour correctness L3 Structural integrity L2 Input interface cleanliness L1 Input cleanliness ... yields shorter scripts ... enables rapid scripting ... makes it more ﬂexible ... eases maintenance© 2011. STAG Software Private Limited. All rights reserved. 13

Clear Assessment of Cleanliness Form L4 PDT10 TT8 PDT9 TT7 Quality report CC1 CC2 CC3 CC4 L3 PDT9 TT6 R1 Met R2 PDT8 TT5 Not met R3 Cleanliness Partially met PDT7 R4 TT4 L2 PDT6 R5 PDT5 TT3 PDT4 L1 PDT3 TT2 PDT2 PDT1 TT1 Stage© 2011. STAG Software Private Limited. All rights reserved. 14

Better Assessment of Adequacy Structure should satisfy impeded by Fault Traceability Cleanliness Criteria Potential Defect Entity under test Types Fault Requirements Test Cases traceability traceability “test for what” “what to test” It is necessary for test cases to be purposeful/goal-focused. Tracing test cases to requirement is a necessary condition for this, but not sufﬁcient! Two way tracing i.e. Requirements + FAULT makes this sufﬁcient.© 2011. STAG Software Private Limited. All rights reserved. 15

Better Assessment of Adequacy Structure Fault Traceability Fault Fault traceability traceability Test for what? PD1 R1 TC1 PD1 PD2 R2 TC2 PD2 PD3 R3 TC3 PD3 ... ... ... ... PDn Rm TCi PDn Requirements traceability What to test? That test cases for a given requirement shall have the ability to detect speciﬁc types of defects FAULT COVERAGE© 2011. STAG Software Private Limited. All rights reserved. 16

Entity Behavioural Stimuli Structure under test scenarios Inputs Conditions Outputs } 1 C4 i1 2 C2 3 i2 C1 4 C3 i3 5 • What an input(stimuli) is an therefore its values depends on the cleanliness level. • Conditions govern behaviour, what a condition is also depends on the cleanliness level© 2011. STAG Software Private Limited. All rights reserved. 17

Structure of a Test Scenario in HBT [EUT-ID + TS-ID] [Behaviour evaluation goal] R1.TS1 Ensure that the system does/does-not or Check that.../Validate that... A clear and concise one-liner. A collection of these shall cover all the behaviours (intended or un-intended behaviours) The structure of test scenario “forces” clarity of thinking and is sharply goal-focused i.e. what do we want to check in the behaviour. Create a behavioural model and use this to generate test scenarios. Techniques landscape© 2011. STAG Software Private Limited. All rights reserved. 18

Structure of a Test Scenario (Detailed) [EUT-ID + TS-ID] [Behaviour evaluation goal] { [Evaluation pre-conditions] [Evaluation steps] [Evaluation post-conditions/oracle] 1. Ensuring consistency in execution Useful for 2. Improving execution efﬁciency automation Clarity of purpose of a scenario is critical. Having a set of steps without clarity of purpose is like listing the directions without being clear about the destination.© 2011. STAG Software Private Limited. All rights reserved. 19

Structure of Test Cases [Combination of values of [EUT-ID + TS-ID + TC-ID] [Expected value] pertinent inputs] {I11 , I21, , I31} Techniques {I12 , I22, , I32} landscape A set of test cases for a scenario is the optimal cartesian product. Document in a tabular form. Input granularity principle What an “input” is is key here. The values for each input depend on “what an input is”. Remember Basic Types & Abstract Data Types in programming languages?© 2011. STAG Software Private Limited. All rights reserved. 20

Results of using this Form & Structure by Applying HBT Ability to question requirements better 1. Find defects in requirements 2. Ensuring adequacy in scenarios/cases. Improved existing scenarios/cases by 2x-3x increasing defect yield Reduced in-process defect escapes by 30% due to better dev test criteria Lower maintenance on automation and better ROI on functional automation Lessen the dependency on the experience as key factor to good test design Able to convince customer logically about completeness© 2011. STAG Software Private Limited. All rights reserved. 21

Summarising... FORM = External Shape STRUCTURE = The way it is composed Level Clarity of test Beneﬁts Form and Structure of Item Test Case MATTERS. NINE Dimensional Form Test adequacy Type Logical review-ability Priority Shorter scripts Stage Structure EUT-->TS-->TC Rapid design Frequency Fault traceability Better criteria Order Cleanliness index Prioritisation Mode© 2011. STAG Software Private Limited. All rights reserved. 22

Thank you! Connect with us... @stagsoft blog.stagsoftware.comHBT is the intellectual property of STAG Software Private Limited.STEMTM is the trademark of STAG Software Private Limited.© 2011. STAG Software Private Limited. All rights reserved. www.stagsoftware.com

Form and structure of test case MATTERS!

by STAG Software Pvt Ltd

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 438

Statistics