20091203 Presentatie Eurostar V02

751 views

Published on

I held this presentation during the Eurostar 2009 conference in Stockholm. It gives a good impression in how a simulation of a SIT helps you in determining defects during the design stage of your SOA landscape.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
751
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Every domain is a component with a particular business knowledge. In the central part you see the 5 main domains of our business
  • Boehm: Cost of change
  • Connectivity: Defects in the connectivity during test costs sometimes > 3 months delay; Different release calendars; Or, solved by a work around with results in problems in the future A Quality Management System gives no extra certainty in a SOA environment, because the quality gates are focused on one business domain.
  • Verbeterpunten: Testposten halverwege inschieten
  • 20091203 Presentatie Eurostar V02

    1. 1. Bring requirements to life with a paper SIT - Simulated System Integration Test - 3 december 2009 | Henri Haarmans / Arjan Steltenpool
    2. 2. Goal and program <ul><li>Goal </li></ul><ul><li>After the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape. </li></ul><ul><li>Program </li></ul><ul><li>Company and IT-environment; </li></ul><ul><li>Problems of a SOA-environment; </li></ul><ul><li>Simulated System Integration Test; </li></ul><ul><li>Back at the office ... </li></ul>
    3. 3. Our roles within Nationale-Nederlanden <ul><li>Henri Haarmans </li></ul><ul><li>>25 years experience in IT with Nationale-Nederlanden; </li></ul><ul><li>Several roles within IT; </li></ul><ul><li>For Group Pension, e-Business, Mortgages and Life Retail; </li></ul><ul><li>Currently: </li></ul><ul><ul><li>Lead Analyst for Life Retail; </li></ul></ul><ul><ul><li>Facilitator. </li></ul></ul><ul><li>Arjan Steltenpool </li></ul><ul><li>>10 years experience in IT with Logica (CMG); </li></ul><ul><li>Test Analyst, Test Manager, Trainer; </li></ul><ul><li>Logica is the preferred supplier for testing at Nationale-Nederlanden. </li></ul>
    4. 4. Nationale-Nederlanden is part of the ING Group <ul><li>IT Department </li></ul><ul><li>Employees: </li></ul><ul><ul><li>510 Internal; </li></ul></ul><ul><ul><li>220 External from preferred suppliers; </li></ul></ul><ul><li>IT change budget for software about € 85 M; </li></ul><ul><li>CMM-i level 3. </li></ul><ul><li>Nationale-Nederlanden </li></ul><ul><li>Employees: about 4250 internal; </li></ul><ul><li>One of the biggest insurance companies in the Netherlands; </li></ul><ul><li>Two locations: The Hague and Rotterdam; </li></ul><ul><li>Turnover in 2008 about € 5 Bn; </li></ul><ul><li>Product lines: </li></ul><ul><ul><li>Life Retail; </li></ul></ul><ul><ul><li>Pension; </li></ul></ul><ul><ul><li>Property& Casualty; </li></ul></ul><ul><ul><li>Income; </li></ul></ul><ul><ul><li>Mortgages. </li></ul></ul>We use the Business Domain Model (BDM) for the representation of our organization.
    5. 5. The BDM contains all our business processes and each domain is a component with a particular business knowledge 1. Relationship Management 3. Performance Management & Corporate Support 14 Provider Management 141 Provider Selection 11 Customer (Intermediary and End-Client) Management 111 Customer Insight & Segmentation 114 Prospecting & Campaign Management 112 Customer Interaction Policy 142 Sourcing & Contracting 34 Corporate Support 32 Performance Management 321 Customer & Channel Performance 322 Financial Performance 342 Knowledge Mgt Prod/Claims 343 Communic. Intern / Ext. 344 Human Resources 345 Finance & Accounting 347 Architecture Mgt Intermediary Sales Force End-client Industry Software (e.g. VA package, Rolls, RDW) Internet Intermediary Sales Force End-client Printing at POS Internet 346 ORM & Fraude 113 Customer Interaction Execution 143 Provider Performance 323 Operational Management 2. Service Delivery Channels 341 Legal & Compliance 2. Output Delivery Channels 348 Quality Management 13 Sales & Distribution Management 131 Sales Force Support 133 Intermediary Agreement 132 Intermediary Support 31 Process Support 311 Workflow & Navigation 33 Accounts Receivable Payable 331 Collection & Disbursement 21 Input messages 211 Paper Data Capture 213 Voice 214 Web 212 Electronic 215 Email 22 Output messages 221 Publishing (Paper) 223 Voice 224 Web 222 Electronic 225 Email 8. Banking 4. Life Retail 6. Property& Casualty 7. Income 5. Pensions Conditional: accommodation, infrastructure and security 12 Product Cockpit 121 Product Development management 122 Product & Rules Building 312 Archiving Product lifecycle Life Retail: management and processing Product lifecycle Property & Casualty: management, processing and claim handling Product lifecycle Income: management and processing Product lifecycle Pensions: management and processing Product lifecycle Mortgage: management and processing
    6. 6. For example: a new product for Life Retail results in changes in all the 15 green domains 1. Relationship Management 3. Performance Management & Corporate Support 14 Provider Management 141 Provider Selection 11 Customer (Intermediary and End-Client) Management 111 Customer Insight & Segmentation 114 Prospecting & Campaign Management 112 Customer Interaction Policy 142 Sourcing & Contracting 34 Corporate Support 32 Performance Management 321 Customer & Channel Performance 322 Financial Performance 342 Knowledge Mgt Prod/Claims 343 Communic. Intern / Ext. 344 Human Resources 345 Finance & Accounting 347 Architecture Mgt Intermediary Sales Force End-client Industry Software (e.g. VA package, Rolls, RDW) Internet Intermediary Sales Force End-client Printing at POS Internet 346 ORM & Fraude 113 Customer Interaction Execution 143 Provider Performance 323 Operational Management 2. Service Delivery Channels 341 Legal & Compliance 2. Output Delivery Channels 348 Quality Management 13 Sales & Distribution Management 131 Sales Force Support 133 Intermediary Agreement 132 Intermediary Support 31 Process Support 311 Workflow & Navigation 33 Accounts Receivable Payable 331 Collection & Disbursement 21 Input messages 211 Paper Data Capture 213 Voice 214 Web 212 Electronic 215 Email 22 Output messages 221 Publishing (Paper) 223 Voice 224 Web 222 Electronic 225 Email 8. Banking 4. Life Retail 6. Property& Casualty 7. Income 5. Pensions Conditional: accommodation, infrastructure and security 12 Product Cockpit 121 Product Development management 122 Product & Rules Building 312 Archiving Product lifecycle Life Retail: management and processing Product lifecycle Property & Casualty: management, processing and claim handling Product lifecycle Income: management and processing Product lifecycle Pensions: management and processing Product lifecycle Mortgage: management and processing
    7. 7. The BDM is implemented in a real Service Oriented Architecture (SOA) <ul><li>Characterization: </li></ul><ul><li>Distributed requirement and software development of the applications; </li></ul><ul><li>Communications between domains goes via an Enterprise Service Bus (ESB). </li></ul><ul><li>Complexity: </li></ul><ul><li>We have various hardware platforms; </li></ul><ul><li>We use various programming languages; </li></ul><ul><li>We use in-house built software and vendor applications; </li></ul><ul><li>Transition from the middle-ware in the ESB to Tibco. </li></ul>Consequence of a SOA environment: Defects in the connectivity between applications arise during the System Integration Test (SIT) for the first time.
    8. 8. Goal and program <ul><li>Goal </li></ul><ul><li>After the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape. </li></ul><ul><li>Program </li></ul><ul><li>Company and IT-environment; </li></ul><ul><li>Problems of a SOA-environment; </li></ul><ul><li>Simulated System Integration Test; </li></ul><ul><li>Back at the office ... </li></ul>
    9. 9. In a traditional environment the V-model is a good model for the development and verification of software Paul E. Brook 1986 A Quality Management System with quality gates ( ) and baseline audits provide extra certainty. Expectation Wish Customer Requirements System Requirements Functional and technical Design Unit Built Unit Test System Test (Functional Acceptance Test) User Acceptance Test System Integration Test Regression Test Production AcceptanceTest Test design One team is developing all the software! Unit Design
    10. 10. A Quality Management System gives no extra certainty in a SOA environment The big challenge is: How can we secure that all applications work together as one process to the customer? Expectation Wish Regression Test Production Acceptance Test System Requirements per domain User Acceptance Test per domain In a SOA environment one team develops the Customer Requirements and for each domain different teams develop the System Requirements and the software! Our solution: Test the process after developing the system and interface requirements. System Integration Test ? Customer Requirements
    11. 11. Different stakeholders, different concerns Our solution the Simulated SIT (SSIT) provides also answers to the questions from stakeholders. Test Manager Understanding of the application landscape; Reduction in time to test and fix. Business representative Do the software developers understand my requirements? Software developer Do I understand the Customer Requirements correctly? Lead Analyst Does my solution Work? Project and Business Management Certainty of due date and quality of the system.
    12. 12. Goal and program <ul><li>Goal </li></ul><ul><li>After the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape. </li></ul><ul><li>Program </li></ul><ul><li>Company and IT-environment: </li></ul><ul><li>Problems of a SOA-environment; </li></ul><ul><li>Simulated System Integration Test: </li></ul><ul><ul><li>Goal; </li></ul></ul><ul><ul><li>Preparation; </li></ul></ul><ul><ul><li>The test itself; </li></ul></ul><ul><ul><li>Roles; </li></ul></ul><ul><ul><li>Follow-up; </li></ul></ul><ul><ul><li>Benefits; </li></ul></ul><ul><li>Back at the office ... </li></ul>
    13. 13. Main goal of the SSIT: Find defects in the process flow as soon as possible <ul><li>A SSIT is: </li></ul><ul><li>A simulation of the process between the main applications: </li></ul><ul><ul><li>One analyst and one business representative per domain simulate the application; </li></ul></ul><ul><ul><li>Manually filling in Service Request Forms on paper; </li></ul></ul><ul><ul><li>Manually delivering these forms between domains to represent the IT-interfacing; </li></ul></ul><ul><li>There is no face-to-face contact between the domains; </li></ul><ul><li>Databases and screens are on paper; </li></ul><ul><li>Defects and questions are filled in by the people who are performing the test. </li></ul><ul><li>We don’t test: </li></ul><ul><li>The functions of an application. </li></ul>Good preparation and facilitation is essential for a successful SSIT.
    14. 14. A good preparation is the basis for success <ul><li>When should a SSIT be executed? </li></ul><ul><li>After definition of the System and Interface Requirements, the Technical Application Architecture and Technical Interface Architecture and the Business Use Cases; </li></ul><ul><li>Before starting to build the applications. </li></ul><ul><li>Precondition of a SSIT: </li></ul><ul><li>More than one Business Use Case or one complex Business Use Case; </li></ul><ul><li>More than one application; </li></ul><ul><li>More than one Business Domain. </li></ul><ul><li>Test cases made by the Lead Analyst and Test Analyst: </li></ul><ul><li>Usually 15 - 20 test scenario’s are sufficient; </li></ul><ul><li>A test scenario maps 1-1 with the Business Use Cases; </li></ul><ul><li>For every Business Use Case more than one test case; </li></ul><ul><li>Database information for each application. </li></ul>The preparation for a SSIT depends on complexity, but shouldn’t be more than 40 hrs. The Facilitator of the SSIT plays the role of the devil’s advocate.
    15. 15. The testing day in a nutshell (1 of 3) <ul><li>Kick-off with all the participants in a central meeting: </li></ul><ul><ul><li>Project urgency presented by the Project Manager; </li></ul></ul><ul><ul><li>Rules of the road presented by the Facilitator; </li></ul></ul><ul><ul><li>Explanation of the working of the SSIT presented by the Facilitator. </li></ul></ul>
    16. 16. The testing day in a nutshell (2 of 3) <ul><li>4 test sessions during one day: </li></ul><ul><ul><li>Testing is done by filling in Service Request Forms, the templates , logbook etc.; </li></ul></ul><ul><ul><li>After one hour, a session ends with an evaluation in every ‘break-out’. </li></ul></ul>
    17. 17. The testing day in a nutshell (3 of 3) <ul><li>A central meeting after every test session for discussing the defects, discussing the questions and optimizing the test process; </li></ul><ul><li>Between the test sessions: exercises for getting back the energy ; </li></ul><ul><li>At the end of the day: </li></ul><ul><ul><li>Conclusions of the test cases and actions for solving the defects; </li></ul></ul><ul><ul><li>Action for outstanding questions; </li></ul></ul><ul><ul><li>Evaluation of the day; </li></ul></ul><ul><li>Afterwards: The facilitator makes a SSIT test report including defects and questions. </li></ul>
    18. 18. Visualization of the simulation On every table… Table 6 Table 1 Table 7 Table 2 Table 5 Table 3 Business Rules Back Office Calculation Box Collection & Disbursement Front Office Output Facilitator Test Manager Information Analyst Lead Analyst ‘ Knowledge bank’
    19. 19. The shopping list for the SSIT (1 of 2) <ul><li>A room which can be divided in separate break-outs; </li></ul><ul><li>Magnetic white boards (not essential but very convenient); </li></ul><ul><li>A facilitator for: </li></ul><ul><ul><li>The process of the SSIT; </li></ul></ul><ul><ul><li>Exercises for bringing back the energy; </li></ul></ul><ul><ul><li>Interim evaluations for: </li></ul></ul><ul><ul><ul><li>Solving problems; </li></ul></ul></ul><ul><ul><ul><li>Evaluation of the test scenario’s during the day. </li></ul></ul></ul>
    20. 20. The shopping list for the SSIT (2 of 2) <ul><li>For every business domain: </li></ul><ul><ul><li>A break-out; </li></ul></ul><ul><ul><li>An in-box and an out-box; </li></ul></ul><ul><ul><li>Instructions for the SSIT; </li></ul></ul><ul><ul><li>Templates for: </li></ul></ul><ul><ul><ul><li>Reporting defects; </li></ul></ul></ul><ul><ul><ul><li>Questions; </li></ul></ul></ul><ul><ul><ul><li>The evaluation; </li></ul></ul></ul><ul><ul><li>Screen prints for the online applications; </li></ul></ul><ul><ul><li>A log book; </li></ul></ul><ul><ul><li>Database content; </li></ul></ul><ul><ul><li>Service request forms to other applications. </li></ul></ul>
    21. 21. All participants play their own role during the SSIT Lead Analyst Walking encyclopedia Business People Monitors that the right business rules are used. Information Analyst Processor of the Service Request Forms. Facilitator Controls the process, initiates a test case, facilitates the evaluation etc. Test Manager Postman between the domains
    22. 22. Beware: Follow up after a SSIT is crucial <ul><li>Detailed review on field-level of the services between two applications; </li></ul><ul><li>Defects have to be solved; </li></ul><ul><li>Questions have to be answered; </li></ul><ul><li>Test scenarios can be input for the real SIT; </li></ul><ul><li>Baseline for working procedure. </li></ul>
    23. 23. Some results from SSIT executed in the last year <ul><li>A new product for Life Retail: </li></ul><ul><ul><li>20 Defects during the SSIT: </li></ul></ul><ul><ul><ul><li>6 Missing and wrong services in the TIA; </li></ul></ul></ul><ul><ul><ul><li>3 Defects in data model; </li></ul></ul></ul><ul><ul><ul><li>3 Defects in the manual processing; </li></ul></ul></ul><ul><ul><ul><li>2 misinterpretations of the CR; </li></ul></ul></ul><ul><ul><ul><li>6 Defects in the services; </li></ul></ul></ul><ul><ul><li>Expected delay when not found: > 3 months; </li></ul></ul><ul><li>Online Transaction Module: </li></ul><ul><ul><li>17 Defects during the SSIT: </li></ul></ul><ul><ul><ul><li>3 Missing services in TIA; </li></ul></ul></ul><ul><ul><ul><li>7 Defects in the control function; </li></ul></ul></ul><ul><ul><ul><li>1 Defect in the data of a service; </li></ul></ul></ul><ul><ul><ul><li>4 Defects in business rules; </li></ul></ul></ul><ul><ul><ul><li>2 Defects in the manual processing; </li></ul></ul></ul><ul><ul><li>Expected delay when not found: > 6 weeks. </li></ul></ul>
    24. 24. Benefits: One day hard work during a SSIT results in answers to the concerns of the stakeholders, and also in: <ul><li>Detecting logical errors in the process: </li></ul><ul><ul><li>Interfaces; </li></ul></ul><ul><ul><li>Services; </li></ul></ul><ul><ul><li>Order of services. </li></ul></ul><ul><li>Detecting differences between the data models in different domains; </li></ul><ul><li>Participants: </li></ul><ul><ul><li>Get the same view of the objective of the project; </li></ul></ul><ul><ul><li>Get understanding of the importance of the project; </li></ul></ul><ul><ul><li>Knowing each other promotes efficient communication during testing. </li></ul></ul><ul><li>Business: </li></ul><ul><ul><li>Check on the right process; </li></ul></ul><ul><ul><li>Information about and understanding of the complexity of the test; </li></ul></ul><ul><ul><li>Input for the Knowledge Bank and guidelines. </li></ul></ul><ul><li>SIT: </li></ul><ul><ul><li>Test cases of the SSIT are the base for test cases in the SIT. </li></ul></ul><ul><ul><li>Quality of the SIT becomes higher. </li></ul></ul><ul><li>Test Manager: </li></ul><ul><ul><li>Gets a better understanding of the process to test. </li></ul></ul>Less or no delay in the delivery of the software. Defects can be discussed immediately
    25. 25. Goal and program <ul><li>Goal </li></ul><ul><li>After the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape. </li></ul><ul><li>Program </li></ul><ul><li>Company and IT-environment; </li></ul><ul><li>Problems of a SOA-environment; </li></ul><ul><li>Simulated System Integration Test; </li></ul><ul><li>Back at the office ... </li></ul>
    26. 26. Back at the office … <ul><li>Do you recognize the problem? Painful SITs due to lack of communication between business domains in previous project phases? </li></ul><ul><li>Convince project/program management for the necessity of a SSIT; </li></ul><ul><li>Keep in mind: The Information Analyst is responsible for all requirements, review activities including the SSIT, but a Test Manager can help facilitate the SSIT; </li></ul><ul><li>So … </li></ul>
    27. 27. Thank you for your attention. [email_address] [email_address]

    ×