Test Strategy “ Layer” the project Identify the components in each “layer” Identify the test basis Draw up a Dependency matrix Formulate a testing “Big Picture” Test Packs Summarising and going forward ….

“ Layer” the project

Identify the components in each “layer”

Identify the test basis

Draw up a Dependency matrix

Formulate a testing “Big Picture”

Test Packs

Summarising and going forward ….

“ Layer” the project Integration services runs on (currently) 3 Biztalk platforms: IS2000 (PoS), IS2002 (ISGM) and IS2004 Each platform contains various abstract layers: Service layer Application layer Database layer Biztalk layer Queue ReceiveFunctions Adapters Orchestrations MessageBoxes ServerHosts Test Environment

Integration services runs on (currently) 3 Biztalk platforms: IS2000 (PoS), IS2002 (ISGM) and IS2004

Each platform contains various abstract layers:

Service layer

Application layer

Database layer

Biztalk layer

Queue

ReceiveFunctions

Adapters

Orchestrations

MessageBoxes

ServerHosts

Test Environment

Components in each layer Service layer IS01, IS02, IS03, IS04, BP01, BP02 etc. Application layer RuleExec, Janus Web, HOPPA etc. Database layer NDTRefData, Alerter, HAT etc. Biztalk layer Queue -  Workqueue, Retry, Scheduled etc. ReceiveFunctions -  Failed Messages, Receiver Adapters -  FTP, HTTP, SOAP etc. Orchestrations – BP01, BP02 etc. MessageBoxes – MsgBox DB, BAM archive etc. ServerHosts – Orchestration host, Tracking host, SendHost adapters etc. Test Environment DEV, Systest 1, Systest 2, CATE 1, CATE 2, little CATE etc.

Service layer

IS01, IS02, IS03, IS04, BP01, BP02 etc.

Application layer

RuleExec, Janus Web, HOPPA etc.

Database layer

NDTRefData, Alerter, HAT etc.

Biztalk layer

Queue -  Workqueue, Retry, Scheduled etc.

ReceiveFunctions -  Failed Messages, Receiver

Adapters -  FTP, HTTP, SOAP etc.

Orchestrations – BP01, BP02 etc.

MessageBoxes – MsgBox DB, BAM archive etc.

ServerHosts – Orchestration host, Tracking host, SendHost adapters etc.

Test Environment

DEV, Systest 1, Systest 2, CATE 1, CATE 2, little CATE etc.

Test basis N This is the documentation that supplies the Expectations , Requirements and Expected Results for ANY given piece of testing effort Prime source of these here is the SUMMIT methodology deliverables – i.e. Project Charter, System Prospectus, System Delivery Spec (SDS), Tech System Design (TSD), Tech Procedure Development (TPD) etc. Test basis should (ideally) consist of SIGNED-OFF DOCUMENTATION ONLY. However – real-world here and it may include draft docs but increased risk of inaccurate Requirements and/or Expected Results All pertinent test basis documents collated as “ KeyDocs ” and are secured into Quality Center

This is the documentation that supplies the Expectations , Requirements and Expected Results for ANY given piece of testing effort

Prime source of these here is the SUMMIT methodology deliverables – i.e. Project Charter, System Prospectus, System Delivery Spec (SDS), Tech System Design (TSD), Tech Procedure Development (TPD) etc.

Test basis should (ideally) consist of SIGNED-OFF DOCUMENTATION ONLY. However – real-world here and it may include draft docs but increased risk of inaccurate Requirements and/or Expected Results

All pertinent test basis documents collated as “ KeyDocs ” and are secured into Quality Center

Dependency matrix N This matrix will highlight the implicit and explicit relationship between IS components on each abstract layer It is used for impact analysis and, hence, to identify likely candidates for regression testing here in IS Where change is known to have occurred to one component in the abstract layer model, the reader can quickly follow the implicit and explicit relationships to other components that are identified in this matrix. As the reader navigates through the matrix, further likely regression test candidates (and hence Test Packs) are identified Change to one particular component may involve regression testing “most” of IS

This matrix will highlight the implicit and explicit relationship between IS components on each abstract layer

It is used for impact analysis and, hence, to identify likely candidates for regression testing here in IS

Where change is known to have occurred to one component in the abstract layer model, the reader can quickly follow the implicit and explicit relationships to other components that are identified in this matrix. As the reader navigates through the matrix, further likely regression test candidates (and hence Test Packs) are identified

Change to one particular component may involve regression testing “most” of IS

Testing Big Picture N This is, in essence, a amalgamation of all of the useful diagrams identified as part of the test basis This model offers a tabular view of the testing and will involve the decomposition of those diagrams in the KeyDocs repository Data from the diagrams is dropped into a two-dimensional table and the information from the diagram (Names in boxes, names against lines, where the lines come from, where the lines go to etc) is then accessible AND, more importantly, reusable from a testing perspective Impact analysis is greatly facilitated by using this Big Picture…You can not only prove the Dependency matrix – which means you can accurately describe what related, regression tests need to be run but you also have a pictorial view of everything testing. INVALUABLE

This is, in essence, a amalgamation of all of the useful diagrams identified as part of the test basis

This model offers a tabular view of the testing and will involve the decomposition of those diagrams in the KeyDocs repository

Data from the diagrams is dropped into a two-dimensional table and the information from the diagram (Names in boxes, names against lines, where the lines come from, where the lines go to etc) is then accessible AND, more importantly, reusable from a testing perspective

Impact analysis is greatly facilitated by using this Big Picture…You can not only prove the Dependency matrix – which means you can accurately describe what related, regression tests need to be run but you also have a pictorial view of everything testing. INVALUABLE

Test Packs N The management will usually have a view at what level of granularity a Test Packs should be designed. Integration Services (IS) have chosen “layers” within platform. i.e ISGM Biztalk, or IS2004 Database, or IS01 service etc. The FEWER Test Packs you choose to have, the MORE has to go into each Test Pack. This increases complexity, which increases the chance of failure in a Test Pack, BUT the MORE Test Packs you have, the more resources required to create, modify, prepare and execute them! If you “spread the load” by having more Test Packs (higher level of granularity), then you have a more agile way of testing change BECAUSE you wont need to regression test ALL Test Packs.

The management will usually have a view at what level of granularity a Test Packs should be designed. Integration Services (IS) have chosen “layers” within platform. i.e ISGM Biztalk, or IS2004 Database, or IS01 service etc.

The FEWER Test Packs you choose to have, the MORE has to go into each Test Pack. This increases complexity, which increases the chance of failure in a Test Pack, BUT the MORE Test Packs you have, the more resources required to create, modify, prepare and execute them!

If you “spread the load” by having more Test Packs (higher level of granularity), then you have a more agile way of testing change BECAUSE you wont need to regression test ALL Test Packs.

Summarising N Test Packs are used to regression test KeyDocs repository contains the (signed-off and master) test basis documentation which will facilitate proof of test coverage (i.e. HAVE we covered everything in the documentation?) Dependency matrix helps you to decide (accurately when used in conjunction with the Big Picture) exactly what else is candidate for test Big Picture gives pictorial representation of ALL things testing on a project/programme. The data/information contained therein is ultimately reusable because it also exists in two-dimensional form! If all this seems VERY COMPLEX, then wait till something CHANGES . i.e. documentation or process. THEN it gets complex because you don’t know what impact that change will have!! The beauty of the above is that I have tools that will automatically decompose documents/diagrams so “change” merely means re-extraction of the data from the changed documents or diagrams (text from documents or text/context information from diagrams) into KeyDocs or Big Picture ……. Next slide elaborates more … .. But this is all talking the talk, now lets walk the walk

Test Packs are used to regression test

KeyDocs repository contains the (signed-off and master) test basis documentation which will facilitate proof of test coverage (i.e. HAVE we covered everything in the documentation?)

Dependency matrix helps you to decide (accurately when used in conjunction with the Big Picture) exactly what else is candidate for test

Big Picture gives pictorial representation of ALL things testing on a project/programme. The data/information contained therein is ultimately reusable because it also exists in two-dimensional form!

If all this seems VERY COMPLEX, then wait till something CHANGES . i.e. documentation or process. THEN it gets complex because you don’t know what impact that change will have!!

The beauty of the above is that I have tools that will automatically decompose documents/diagrams so “change” merely means re-extraction of the data from the changed documents or diagrams (text from documents or text/context information from diagrams) into KeyDocs or Big Picture ……. Next slide elaborates more

… .. But this is all talking the talk, now lets walk the walk

Walk the walk N DocIndex provides the engine that drives the creation of the KeyDocs (test coverage and Expected Results). InternetMiner provides the engine that assists the creation of KeyDocs and operates on Intranet/internet pages and hyperlinks to extract similar information to DocIndex. This is more an ancillary tool that complements DocIndex and is useful for gaining insight into specific areas of expertise. VisioDecompositer provides the engine that drives diagram information (process flow and data flow) into the Big Picture. Here’s some screen shots to give you a better flavour….

DocIndex provides the engine that drives the creation of the KeyDocs (test coverage and Expected Results).

InternetMiner provides the engine that assists the creation of KeyDocs and operates on Intranet/internet pages and hyperlinks to extract similar information to DocIndex. This is more an ancillary tool that complements DocIndex and is useful for gaining insight into specific areas of expertise.

VisioDecompositer provides the engine that drives diagram information (process flow and data flow) into the Big Picture.

Here’s some screen shots to give you a better flavour….

DocIndex 1 of 10 N DocIndex (viewer) main menu

DocIndex (viewer) main menu

DocIndex 2 of 10 KeyDocs main menu

KeyDocs main menu

DocIndex 3 of 10 Display a selectable list of KeyDocs

Display a selectable list of KeyDocs

DocIndex 4 of 10 Now display a summarised list of the sections in the selected KeyDoc

Now display a summarised list of the sections in the selected KeyDoc

DocIndex 5 of 10 Select one of the sections in the document and the text is displayed

Select one of the sections in the document and the text is displayed

DocIndex 6 of 10 The power of this tool lies in the fact that ALL relevant project/programme-level documentation is captured into the tool

The power of this tool lies in the fact that ALL relevant project/programme-level documentation is captured into the tool

DocIndex 7 of 10 The contents of all of the KeyDocs are stored in two tables: tblSectionHeaders – which is a summarised “table of contents” for a KeyDoc

The contents of all of the KeyDocs are stored in two tables: tblSectionHeaders – which is a summarised “table of contents” for a KeyDoc

DocIndex 8 of 10 and File_Decomposition – which is textual body for all KeyDocs

and File_Decomposition – which is textual body for all KeyDocs

DocIndex 9 of 10 There is a keyword/key phrase Search facility for the group of KeyDocs, which allows the user to quickly become acquainted with “project specific” terminology

There is a keyword/key phrase Search facility for the group of KeyDocs, which allows the user to quickly become acquainted with “project specific” terminology

DocIndex 10 of 10 and the reader can, very quickly, become acquainted with the context in which the specific search terms are used.

and the reader can, very quickly, become acquainted with the context in which the specific search terms are used.

InternetMiner 1 of 10 N InternetMiner main menu

InternetMiner main menu

InternetMiner 2 of 10 Enter a suitably relevant URL - in this case, found by doing a “biztalk performance test” search string in Google UK

Enter a suitably relevant URL - in this case, found by doing a “biztalk performance test” search string in Google UK

InternetMiner 3 of 10 And the page(s) are decomposed into InternetMiner , in much the same way as DocIndex

And the page(s) are decomposed into InternetMiner , in much the same way as DocIndex

InternetMiner 4 of 10 The list of decomposed URLs can be viewed (and refreshed either manually or automatically) so that the information contained therein is always up to date

The list of decomposed URLs can be viewed (and refreshed either manually or automatically) so that the information contained therein is always up to date

InternetMiner 5 of 10 And the content from ALL of the decomposed URLs can be viewed at a glance

And the content from ALL of the decomposed URLs can be viewed at a glance

InternetMiner 6 of 10 And relevant search keywords or key phrases can be entered

And relevant search keywords or key phrases can be entered

InternetMiner 7 of 10 And the result be displayed (in various forms) - so that the user can see, contextually, where the search term is used

And the result be displayed (in various forms) - so that the user can see, contextually, where the search term is used

InternetMiner 8 of 10 Alternative views of the results (and search strings) can be made

Alternative views of the results (and search strings) can be made

InternetMiner 9 of 10 And you can quickly build up a picture of the various (in this case technical Biztalk) phrases

And you can quickly build up a picture of the various (in this case technical Biztalk) phrases

InternetMiner 10 of 10 And how they relate to the overall topic (Biztalk Performance testing)

And how they relate to the overall topic (Biztalk Performance testing)

VisioDecompositer 1 of 2 N VisioDecompositer takes a straightforward Visio diagram…..

VisioDecompositer takes a straightforward Visio diagram…..

VisioDecompositer 2 of 2 N And converts it into two-dimensional information that can be “mapped” into test cases or test scripts quite easily

And converts it into two-dimensional information that can be “mapped” into test cases or test scripts quite easily