deeper IBM Global Business Services – Application Services
Upcoming SlideShare
Loading in...5

deeper IBM Global Business Services – Application Services






Total Views
Views on SlideShare
Embed Views



1 Embed 1 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • James Bach
  • Users: Operators Database administrators Performance analysts Capacity planners Storage administrators Security Administrators Users themselves

deeper IBM Global Business Services – Application Services deeper IBM Global Business Services – Application Services Presentation Transcript

  • April 19, 2010 Version 1.0 Application Testing Services University of Stellenbosch – System Testing Disclaimer This document does not constitute a formal offer, but serves to outline the discussion elements for contractual consideration based on IBM’s standard terms and conditions.
  • The current situation
  • Ad Hoc Statements John Roodt : Senior Architect “… you guys are brutal. Brutally honest…” Jolene Saayman : Lead Java Developer “… if it weren't for you I would have left this company a long time ago…” Adam Granger : Lead Java Developer “… you are our safety net…” Michael Kiergard : Customer Project Manager “… you must be involved. I trust you guys…. I don’t trust the developers…” Karen Greeves : Project Delivery Executive “… I can rest assured that after it has been through your hands, then the application is pretty solid…” Meike Voight : SAS Developer “… but what are they going to test if I have already tested it?...” Cherryl Lundt : Project Manager “… I now want to involve you on all my projects...” “ As a software tester, the fate of the world rests on your shoulders. This statement is not an exaggeration if you accept the dual premises that computer software runs the modern world and that all software has bugs – then you will quickly reach the inescapable conclusion that unless the most disruptive bugs are removed, the world as we know it will grind to a halt.” Scott Loveland, Geoffrey Miller, Richard Prewitt, Jr., and Michael Shannon have collectively spent more than 70 years in the software testing trenches. They currently hunt bugs in the IBM mainframe development laboratory in Poughkeepsie, New York. Collectively they have spent more than 70 years in the software testing trenches.
  • Agenda
    • Being a tester
    • Test Strategy : Example case study of mature client
    • Test Methodology : Verification and Validation
    • ________________________________________________________________
    • What every developer needs to know about testing
    • Characteristics of a good tester
  • Being a Tester
    • What people think we are : Testers are negative thinkers, they complain, they like to break things, they take a special thrill in delivering bad news.
    • Who we really are : Tester’s don’t complain, we offer evidence. Tester’s don’t like to break things, we dispel the illusion that things work. Testers don’t enjoy giving bad news, we enjoy freeing our clients from the thrall of false belief.
  • Why Test?
    • To define the role of a testing group is not easy. The following metaphor might best explain this role:
    • "A project is like a road trip. Some projects are simple and routine, like driving to the store in broad daylight. But most projects are more like driving a truck off-road, in the mountains at night. Those projects need headlights. The tester is the headlight . The tester illuminates the road ahead so the developers and managers, however they struggle with the map, can at least see where they are, what they're about to run over, and how close they are to the cliff."
    • The detailed mission of testing groups might differ from company to company, but behind those details is a common factor. Testing is done to find information. Critical decisions about the project or the product are made on the basis of the information found by the test group.
  • IBM Testing Services - Overview Provide integrated testing services that leverage test best practices and highly skilled test professionals from across IBM
      • Deliver flexible testing services
    Exploit technology to deliver innovative solutions Leverage leading practices and Global Reach
      • Lead industry through proven processes, methods, and tools
      • Maintain high customer satisfaction through delivery excellence and measurable business value
      • Provide testing infrastructure On-Demand
    Test Team IBM Provision of experienced test resources to IBM &/or Customer Projects Testing Project and/or Programme Take responsibility for all aspects of testing for a project or programme. Build estimates, provide the team, provide management support Consultancy Services Managed Testing Services “ Testing on Demand” for a subset of testing scope or full Outsourced Testing Test Process Automation Test Environment Management Test Support (Systems, Integration, OAT, UAT) Technical Testing Performance, Load and Stress Education & Skills Transfer Component Solutions Web Testing Test Leadership Legacy Transformation Test Automation Test Process Assessment Capacity Planning Complex Architecture Performance Scalability & Interoperability Software Product Testing Full Lifecycle Test Services tailored
  • Test Strategy – Example Case study Business Case Feasibility Analysis Specification Design Architectural Design Module Design Construction Module Test Acceptance Test System Test Implementation Integration Test Business Realization Business Process Owners Business IT Vendor or Int IT Business Governance
  • Test Strategy – Example Case Study Business Case Feasibility Analysis Specification Design Architectural Design Module Design Construction Module Test Acceptance Test System Test Implementation Integration Test Business Realization Business Process Owners Business IT Vendor or Int IT
    • Business case
    • Requirement Catalogue
    • Change / Decision register
    • Non Functional Requirement Specification
    • Interface Specification
    • Software System Requirement Specification
    • Application Architecture
    • Technical Architecture Definition
    • Test strategy
    • Implementation Strategy
    • Deliverables Catalogue
    • Project Contract/Schedule
    • AsIs and ToBe model
    • Logical Data Model
    • System Context Diagram
    • Project approach
    • Project Management Plan
    • System Design Document
    • Physical Data Model
    • QA Plan
    • Detailed Test Plan
    • Implementation Plan
    • Application Impact
    • Implementation Requirement Specification
    • User Manual
    • Test Data
    • Operational Acceptance Test plan
    • Unit Test report
    • Integration Test report
    • Data Migration Test report
    • Factory Acceptance report
    • Application support documentation
    • Global HelpDesk documentation
    • Operational Management support
    • Operations Handbook
    • User Acceptance Test report
    • Stress and Performance report
    • Application Impact Assessment
    • Operations Acceptance test report
    • Maintenance and Support Contracts
    • Handover Report to Business Owner
    • Close Down report
  • Test Strategy – Example Case Study Test plans AUT, AIT, FAT Test cases AUT, AIT, FAT Execute AUT Execute AIT Execute FAT Test plans SIT, UAT, SPT Test cases SIT, UAT, SPT Execute SIT Execute SPT Execute UAT Promote PROD Warranty Period Design Analysis Construct and Vendor test Cust Test Impl Prod Execute OAT Execute AIA
  • Testing Methodology - Simplified “V”-Model of Testing Capture Requirements Design System Implement System Component Test Integration/System Test Acceptance Tests
    • Test team involved on day one of project
    • Requirements, high-level design are primary drivers for behavioral test phases
    • Low-level design, implementation are primary drivers for structural testing
    • Features dropped during development (left side of V) rather than slipping test phase entry dates or violating test entry criteria
    Application Concept Develop Tests Develop Tests Develop Tests System Tests Integration Tests Unit Tests Verification Validation
  • Testing Methodology - Validation ARE WE BUILDING THE RIGHT SYSTEM? Ideal Reviewers : Review Granularity Business Owners Business Representative Business Analyst Conceptual Application Business Owner Architects Test Engineers Prototypes Simulations High Level Designs Developers Deployers Architects Detailed Design Execution Activities For review phases Requirements/Specification reviews Architectural Design reviews Detailed Design reviews Test Planning
  • Testing Methodology - Verification ARE WE BUILDING THE SYSTEM RIGHT? Ideal Reviewers : Review Granularity Developers DB / Net / System Administrators Deployers Structural (“White Box”) Test Engineers Test Technicians (Some) Developers Behavioral (“Black Box”) Tech Support / Help Desk Sales / Marketing Business Analyst / User Customer Live (“Alpha/Beta”) Execution Activities For review phases Unit Component Integration System Acceptance Pilot
  • Test Process – Main process
  • Test Process – Define Mission
  • Test Process - Verify Test Approach
  • Test Process – Validate Build Stability
  • Test Process – Test and Evaluate
  • Test Process – Achieve Acceptable Mission
  • Test Process – Improve Test Assests
  • Simplified Test Process
  • What every developer needs to know : 1. The product is more than Software
    • Application
    • Code
    • Framework
    • Documentation
    • Manuals
    • Operator instructions
    Kit OS Infrastructure
  • What every developer needs to know : 2. Quality is more than the lack of bugs (FURPS)
    • Functionality
    • Suitability
    • Correctness
    • Interoperability
    • Compatibility
    • Security
    • Installability
    • Usability
    • Understandability
    • Learnability
    • Operability
    • Performance
    • Reliability
    • Maturity
    • Fault tolerance
    • Integrity
    • Recoverability
    • Safety
    • Maintainability
    • Analyzability
    • Changeability
    • Stability
    • Testability
    • Portability
    Does it solve the problem when it works Is it practical to use Does it keep on working Is it economical to build and maintain
  • What every developer needs to know : 3. Quality Assurance is more than testing Quality Assurance is everything you do to minimize the risk of failure and to promote excellence
    • Risk Management
    • Customer Involvement
    • Skillfull developers
    • Process definition and improvements
    • Inspections and active testing
    • Experience based improvements
  • What every developer needs to know : 4. Testing is hard to do
    • Anticipate your users:
    • Data
      • Skills
        • Actions
          • Expectations
            • Environment
    • … to examine a product that is..
    • Invisible
      • Volatile
        • Sensitive
          • Complex
            • Unfamiliar
    • ...using a process that is often…
    • Endless
      • Ambiguous
        • Negative
          • Boring
            • Laborious
    • Think of the permutations!!
    • Functions vs Input data vs state of data
    • Products vs platforms
    • External factors
    • Expectations of customers
    • Timeframes
    • Versions of the product
  • What every developer needs to know : 5. You can make testing easier
    • Document your design
    • User internal error checking
    • Test units before Integration
    • Tell the tester what is new, concerns you or is shaky
    • Test the build yourself…….First
    • Evolve product in functional layers.. Ie Iterative development with testability in mind
    • Build in testability
  • Characteristics of a good tester
    • Know at least some programming . There’s a popular myth that testing can be staffed with people who have little or no programming knowledge. Since they’re testing software, without know some programming, they can’t have any real insights into the kinds of bugs that come into software and the likeliest place to find them.
    • Know the application . The ideal tester has deep insights into how the users will exploit the program’s features and the kinds of cockpit errors that users are likely to make.
    • Intelligence . The single most important quality for testers (just as for programmers) is raw intelligence, good testers, just as programmers, are smart people.
    • Hyper-sensitivity to little things . Good testers notice little things that others miss or ignore. Testers see symptoms, not bugs.
    • Tolerance for chaos . People react to chaos and uncertainty in different ways. If the tester waits for all issues to be fully resolved before starting test design or testing, he won't get started until after the software has been shipped. Testers have to be flexible and be able to drop things when blocked and move on to another thing that is not blocked. Testers always have many irons in the fire.
    • People skills . You can be an effective programmer even if you are hostile and anti-social; that won’t work for a tester. Testers can take a lot of abuse from outraged programmers. A sense of humor and a thick skin will help the tester survive. Testers may have to be diplomatic when confronting a programmer with a fundamental goof. Diplomacy, tact, a ready smile- all work to the independent tester’s advantage.
    • Tenacity (Merriam-Webster’s dictionary explains it as: persistent in maintaining or adhering to something valued or habitual). An ability to reach compromises and consensus can be at the expense of tenacity. The best testers are both socially adept and tenacious where it matters. The best testers are so skilful at it that the programmer never realizes they’ve been had.
    • Organized . There’s just too much to keep track of to trust to memory. Good testers use files, Databases and all the other accoutrements of an organized mind.
    • Sceptical . That doesn’t mean hostile. I mean skepticism in the sense that nothing is taken for granted and that all is fit to be questioned. Only tangible evidence in documents, specifications, code and test results matter. While they may patiently listen to the reassuring words from the programmers ("Trust me. I know where the bugs are.") - and do it with a smile - they ignore such in-substantive assurances.
    • Self-sufficient and tough . If they need love, they don’t expect to get it on the job. They can’t be looking for interaction between them and programmers as a source of ego-gratification and/ or nurturing. Their ego is gratified by finding bugs, with few misgivings about the pain (in the programmers) that such finding might engender.
    • Cunning . Systematic test techniques such as syntax testing and automatic test generators have reduced the need for much cunning, but the need is still with us and undoubtedly always will be because it will never be possible to systematize all aspects of testing. There will always be room for that off-beat thinking that will lead to a test case that exposes a really bad bug.
    • Technology hungry . They hate dull, repetitive work; they’ll do it for a while if they have to, but not for long. The silliest thing for a human to do, in their mind, is to pound on a keyboard when they’re surrounded by computers.
    • Honest . Testers are fundamentally honest and incorruptible. They’ll compromise if they have to, but they’ll righteously agonize over it. This fundamental honesty extends to a brutally realistic understanding of their own limitations as a human being.
  • Thank you