The document outlines a webinar discussing best practices in quality assurance (QA) led by experts from Liveware Inc. and Computer Aid Inc. (CAI), focusing on the importance of QA roles, feedback mechanisms, and the impact of effective QA processes on productivity and product quality. It highlights case studies demonstrating the challenges and successes in implementing QA in various organizations, emphasizing proactive support and experienced personnel to improve processes and achieve compliance.

1. Webinar: QA Three Best Practices March 11, 2009 3:30 PM – 5:00 PM

2. Jorge Luis Boria Sr. VP International Process Improvement Liveware Inc. [email_address] Michael Milutis Director of Marketing Computer Aid, Inc. (CAI) [email_address]

3. About Presenter’s Firm LIVEWARE is a business consultant company in the field of Software Engineering. Since its creation in 1983 it has specialized in the dissemination, application and development of Software Engineering, particularly the concepts of Total Quality and the strategic value of Information Systems. In the past five years it developed a tight relationship with the Software Engineering Institute and is currently the sponsor of five lead appraisers, two certified high maturity lead appraisers, and two CMMI instructors.

4. CAI is a global IT outsourcing firm currently managing active engagements with over 100 Fortune 1,000 companies and government agencies around the world. CAI is a leader in IT Best Practices for legacy support and new development application management. CAI’s focus is directed toward practical implementations that track and measure the right activities in software activity management CAI consistently promises and delivers double digit productivity in its outsourcing and consulting engagements. CAI makes all of this possible through the use of: Standard processes Management by metrics SLA compliance management Detailed cost, resource, and time tracking Capacity management Standard estimation A unique, metrics based methodology along with a proprietary, real time data repository and management system (TRACER®). About Computer Aid, Inc. (CAI)

5. The Project Management Institute’s ISSIG group has accredited this webinar with PDUs Stay tuned! Your PDU code will be displayed at the conclusion of this webinar. PDU CREDITS FOR THIS WEBINAR

6. NOW AVAILABLE! ONLINE WEBINAR RECORDINGS ANYTIME ACCESS! WWW. ITMPI.ORG / LIBRARY

7. Quality Assurance Role Provide teams and management with visibility into the process being used and into the products being built through: reviewing and auditing work products and team activities to verify compliance with procedures and to standards continues in next slide

8. Quality Assurance Role supplying teams and management with feedback on reviews and audits assisting in the definition of plans, standards, and procedures addressing non-compliance issues through vigilance and escalation from previous slide

9. QA vs Testing Structural Verification Integration and System Functional Verification Validation (User Acceptance) Component Functional Verification Non-functional Verification TESTING VALIDATE VERIFY BUILD Coding Requirements High Level Design Component Design Specification QA

10. QA Competency At a minimum: Ensure product and process meets quality system requirements as defined by customer specifications.

11. Typical Approach to QA Process Police Academic approach Very literal interpretation of CMMI Improvement infrastructure independent of application areas Focus on punishment Compliance, not commitment Push

12. Better Approach to QA Process Mentor Proactive Approach Business-oriented interpretation of CMMI CMMI as good practices input Focus on progress Achievements, not compliance Pull

13. Case Study 1 Fast growing federal & state e-commerce company Ten-fold growth in two years Went from one product in the first five years to multiple ones based on initial product Y2K Success Story Product showed no defects over the millennium Defects identified were 30% date related, 70% non-Y2K Developers found defects that had been dormant for years

14. Obstacles to Quality - 1 Original Sins Developer-owned code Success of a non-documented process Buddy-network of the twelve original Peter’s Principle Explosive Growth New products, new people Small attrition (+) Explosive demand (+/-) Difficulties in recruiting (-) Overtime the norm (-)

15. Obstacles to Quality - 2 Matrix Organization Project management vs. Project Administration Flexible yet chaotic team composition Project leadership power struggles Readiness for change highly challenged by volatility of all project components Requires strong discipline to succeed

16. Context - 3 UAT Execution (SDS) Test Report Sys Test Execution (SDS) Test Report Acceptance UAT Test Planning and Preparation System Test Planning and Preparation Unit Test Planning and Preparation Acceptance Requirements (SDS) Acceptance Specifications (TSD) Coding (SDS) Unit Test Execution (SDS) Hand Off Developed Components (SDS)`

17. First Process Change Shared test cases across testing, analysis and development teams Immediate payback with early detection Shorter integration phase New process required new skills SQA to help developers through the processes Requirements revealed as problem Pushback!

18. Context - 3 UAT Execution (SDS) Test Report Sys Test Execution (SDS) Test Report Acceptance UAT Test Planning and Preparation System Test Planning and Preparation Unit Test Planning and Preparation Acceptance Requirements (SDS) Acceptance Specifications (TSD) Coding (SDS) Unit Test Execution (SDS) Hand Off Developed Components (SDS)` Phase End Review Phase End Review Post Mortem Project Review Peer Review Peer Review

19. Progress Attained Requirements problems fixed Peer reviewed and agreed upon by testers and developers alike Baselined, accessible to all and put under change control and as root of the traceability matrix (under testers control) QA now seen as supporting projects Waiving unnecessary procedures No longer “process police”

20. And Compliance? Testing (SQC) as the enforcers (at each transition) Process a consequence of their need to do good work Validated by past success in Y2K and need to use process deliverables Handovers became smooth

21. The Good News Quality counts went up Number of builds between freeze and release reduced by 50% Test preparation time vs. execution time changed from 20%-80% to 70%-30% Moving 50% of the total effort out of the project’s critical path Defects caught by developers increased an order of magnitude Developers pushing involvement of testers early in review of requirements and design specifications

22. Success Factors – 1 QA engineers (proactive support) had process ownership could talk the talk and walk the walk perceived as rescuing the project from itself Testing engineers (compliance control) technical authority from Y2K good reasons to demand process commitment responsibility over the traceability matrix perceived as process-focused, not witch-hunters

23. Case 2: Texas 3 Step Well established financial institution with links to military market Very large market share in their segment Stove pipe structure across divisions Community of overachievers Growth had reached a plateau Very low attrition over the years Opportunities for promotion based on retirement of elders Small market of similar employers starting to pick the managers

24. Obstacles to Quality - 1 Original Sins Lackadaisical approach to quality New QA role filled in with rookies Transition Point Mandate to achieve maturity levels in short periods Correctly identified as a quality, not process problem (to quality through processes)

25. Obstacles to Quality - 2 Lots of generals, few soldiers Readiness for change highly challenged by bureaucratic approach to process Command and control, no matrix organization No funding for a separate SQA organization No objectivity by new hires “ Run of the mill” checklist approach No authority within organization Direct reports to development manager (budget)

26. Process Needs Expectation was improved quality will lower cycle time Releasing important individual efforts Investment of those efforts in quality possible Expert managers could become quality mentors objectivity achieved through independence role seen as cross-pollination and introducing experience to the team

27. New Obstacle Expert manager too expensive to perform menial tasks Filling in checklist Supporting template use Doing the paperwork

28. Solution Three roles and two actors Expert manager from afar as QA expert Interns as QA specialist Role 1: QA-experts help the team define their process and sets up the specialist’s goals and activities Role 2: specialist does the leg work and QA-expert remains on call Role 3: at selected milestones, the QA-expert works with the QA-specialist looking into the outcomes and audits the work done by the specialist for the project

29. Texas 3 Step overall project execution selected milestones when to perform QA audits and reviews prep (all involved) repetitive (specialist) audit (all QA)

30. Progress Attained Experienced managers shared management with mentoring and auditing responsibilities large social reward within the company delegated internal support to the projects keeping objectivity provided by relatively inexperienced personnel.

31. The Good News Steep learning curve 100% process compliance in six months Significantly shortened delivery dates (no hard data)

32. Success Factors – 2 Experienced Managers authority from experience and reputation perceived as process focused, not witch-hunters knowledge on how to make things happen faster and better Internal QA resources artifact responsibility talked the talk and walked the walk perceived as helping the project doing productive work

33. Case 3: Another Division Well established financial institution with links to military market Very large market share in their segment Stove pipe structure across divisions Community of overachievers Growth had reached a plateau Very low attrition over the years Opportunities for promotion based on retirement of elders Small market of similar employers starting to pick the managers Same Old Story

34. Obstacles to Quality - 1 Original Sins Lackadaisical approach to quality New QA role filled in with rookies Transition Point Mandate to achieve maturity levels in short periods Correctly identified as a quality, not process problem (to quality through processes) Same Old Story

35. Obstacles to Quality - 2 Lots of generals, few soldiers Readiness for change highly challenged by bureaucratic approach to process Command and control, no matrix organization No funding for a separate SQA organization No objectivity by new hires “ Run of the mill” checklist approach No authority within organization Direct reports to development manager (budget) Same Old Story

36. Process Change Expectation was improved quality will lower cycle time Releasing important individual efforts Investment of those efforts in quality possible Expert managers could become quality mentors objectivity achieved through independence role seen as cross-pollination and introducing experience to the team Same Old Story

37. New Approach Expert managers seen as stopped in their tracks by lack of open positions neighboring companies picking their up and coming managers opportunity for a new parallel track money not the problem lack of open positions IS the problem

38. Solution New QA function Using experienced senior managers with responsibility over mentoring auditing the work of up-and coming first line managers making “ it” happen Line managers promoted to QA Up and coming getting the jobs

39. Progress Attained Experienced managers shared management with mentoring and auditing responsibilities large social reward within the company delegated internal support to the projects keeping objectivity provided by relatively inexperienced personnel.

40. The Good News New role well positioned by CEO Perceived as promotion in a clogged career ladder New QA group found it very easy to coordinate work Their advice was very sound the organization quickly closed ranks behind them organization was first one in the company to reach Maturity Level 2 achieved in only six months.

41. Success Factors – 3 Experienced Senior Managers authority from experience and reputation perceived as process focused, not witch-hunters knowledge to make things happen faster and better “been there, done that” process responsibility at Level 2 could talk the talk and walk the walk perceived as helping the project do productive work

42. Common CSF – 1 Use of Experienced Personnel had authority from experience and reputation were perceived as process focused, not witch-hunters had the knowledge to make things happen faster and better

43. Common CSF – 2 Focus on progress help the projects to help the process improvement program avoid the “Gotcha!” attitude make thinks happen, treat project members as adults

44. Questions? Your PDU CODE: S010-ITMPI09015

45. CAI Sponsors The IT Metrics & Productivity Institute: Clearinghouse repository of best practices: WWW.ITMPI.ORG Weekly educational newsletter: WWW.ITMPI.ORG / SUBSCRIBE Weekly webinars hosted by industry leaders: WWW.ITMPI.ORG / WEBINARS ACCESS WEBINAR RECORDINGS ANYTIME AT WWW.ITMPI.ORG / LIBRARY

46. Software Best Practices Conferences Around the World WWW.ITMPI.ORG / EVENTS Mar. 10 Orlando, FL Mar. 31 Albany, NY Apr. 9 Baton Rouge, LA Apr. 28 Tallahassee, FL Apr. 30 Detroit, MI May 14 Tampa, FL May 19 Philadelphia, PA June 2 Hartford, CT 2009 Dates and Locations June 9 Houston, TX June 11 New York, NY Sep. 15 Toronto, ON Sep. 22 Chicago, IL Sep. 29 New York, NY Oct. 6 Baltimore, MD Oct. 20 Philadelphia, PA Nov. 10 Princeton, NJ Nov. 17 Ft. Lauderdale, FL

47. Jorge Luis Boria Sr. VP International Process Improvement Liveware Inc. [email_address] Michael Milutis Director of Marketing Computer Aid, Inc. (CAI) [email_address]