2 earthsoft-quality control in software organidation


Published on

"Earthsoft Foundation of Guidance (EFG) is working as an NGO/NPO for students - Education & Career
guidance and for Professionals for soft skills enhancements. I am working on speading , sharing
knowledge; experience globally.It has uploaded important presentations at http://myefg.in/downloads.aspx.
Also https://dl.dropbox.com/u/83265908/Links-events.xls has links for all ppt files.

Read http://tl.gd/jm1gh5

Be mentor using your education, knowledge & experience to contribute for a social cause & do conduct
free training/ workshop seeking help of existing platforms like rotary,etc
Kindly spread to your friends.Thank you!
- Earthsoft Foundation of Guidance
Let us make earth little softer..

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Above processes remove the defects
  • Reviews at each phase during SDLC removes the defects and quality improves
  • Global Standards Compliance Checker - GSCC
    250+ detailed code checks
    Built into the development process
    Impossible to release code that violates these checks
    Started with 11i6, previously, these errors were only discovered by code reviews or testing
    Many errors only occurred on specific datasets
    Automatic Generation of Patch Database (DB) Drivers
    DB driver is portion of patch that updates schema, seed data and pl/sql
    DB driver errors accounted for ~20% of patch install errors
    Started with 11i6, previously these were manually coded, which was extremely complex and error prone
  • Automated Performance Analyzer
    Every SQL statement (~280,000) is run against multiple (5) real world customer databases
    Worst case Explain plan is analyzed
    Bugs automatically logged for worst performing statements
    This activity performed every month
    Fixes consolidated into next release
    Critical issues fixed as one-off fixes
    ~1,500 issues fixed monthly
    Started with 11i5, previously these were only caught by performance testing
    Database group uses this on new versions of the database
    100+ bugs fixed prior to release of 9iR2
  • Issues not easily resolved are escalated for resolution.
    Issues are typically identified throughout the project and logged and tracked through resolution.
    In this section of the plan the following processes are depicted:
    Where issues will be maintained and tracked
    The process for updating issues regularly
    The escalation process
    The vehicle by which team members can access documented issues
    Issue… already impacting the cost, time or quality
    Risk… POTENTIAL negative impact to project
  • Following factors make the quality software
  • 400% increase in automated regression tests,
    Tests created for all key business flows
    Created by dedicated product QA teams
    All products now have RTs
    Used for
    Ongoing maintenance/bug fixes
    Platform certification (e.g. Linux)
    Technology certifications (e.g. 9iR2)
    eBusiness Flows certified prior to release
    With 11i7, 60+ Oracle consultants worked for 2 months to pre certify Oracle implementation flows
    Progress measured daily, and reported to executive management
  • (By changing system dates)
    After server side validations such as "Duplicate ID Check", retry entering valid data and save
    Read only fields Test (press enter key or click to verify that no unexpected events are getting fired)
  • Business Transactions – HKK
    Topacs Screen Flow Diagrams
    Hakkai ERP Application Testing
    Hakkai ERP Application Testing
  • “There is always one more bug.”
    The amount of testing needed to ensure quality is hard to quantify. Very few project teams are able – or are willing – to predict how many bugs there will be in a release. Testing and fixing until all bugs have been found would be a very expensive task, if it were possible. And is it reasonable? Most organizations prefer to deliver a product on time, with a few unimportant defects, rather than to deliver it six months late. Needless to say, each industry has a different level of acceptable quality, for instance, word processors versus aircraft systems.
    In order to provide correct budget and schedule estimates, QA managers need the means to predict fault levels.
  • For translation projects
  • 2 earthsoft-quality control in software organidation

    1. 1. Making earth little softer Earthsoft Foundation of Guidance Edge-Aggressive-Reliable-Trust-Honesty-Soft-Obedient-Fun-Transparent
    2. 2. Making earth little softer Earthsoft Foundation of Guidance (EFG) has uploaded following presentations at http://myefg.in/downloads.aspx Be mentor using your education, knowledge & experience to contribute for a social cause & do conduct free training seeking help of existing platforms. Kindly share with your friends •Motivation for higher study, Planning for study, Education guidance, Career guidance, Career available after SSC & HSC •Personality development – 3 files •How to prepare resume, Tips to attend interview successfully •Religion related –To understand basic religion, Do & Don’t tips •Health related - Be vegetarian, Be healthy, Manage health •Corporate - Project management, Assertiveness, Ownership, Effective communication, Leadership, Be entrepreneur •Finance - To avoid speculation in stock market •Social - Women empowerment, Choosing life partner, conflict resolution, stop ragging, stop alcohol, snakebite treatment About us
    3. 3. Making earth little softer • Organisation • Quality • Quality management • Quality Assurance • Quality Control • Processes • Case studies experience • Strategic Quality Planning Index
    4. 4. Making earth little softer Index
    5. 5. Making earth little softer Quality Control
    6. 6. Making earth little softer Project RAG Status NOTE: Pl update the RAG status from spectrum for last 4 weeks. If any of the parameter is Red / Amber, pl update the action required and status for the current week. Parameter Last Week -1 Last Week Current Week Action Required Status Target Date Budget Scope Effort Schedule Resource Quality
    7. 7. Making earth little softer • Functionality related gap between expectations of client and actual delivery • Defects are injected during Requirement analysis, High & Low level Design and coding phase. • Continuous reviews help to prevent/ remove defect injection • Rigorous testing during Unit level, Integration and application level testing detect the defects. RA R Desig n R Coding R UT IT AT Achieving Quality
    8. 8. Making earth little softer Reviews • Review helps • To confirm client’s requirements • To detect defects in early stages • To reduce rework at later phases
    9. 9. Making earth little softer RA Design Coding UT IT AT Defectsinsystem Various Phases of SDLC To be at better quality after every phase • Removing defects by reviewing & testing • From each deliverable • At each phase Reviews & Testing
    10. 10. Making earth little softer Team structure & reviews PMO Reviews PMO Reviews Reviews of deliverables of each member at every level by supervisor help improving quality Creating proposal Project Plan, Reports Following Process & best practices Review of DD, Coding conventions Reviews of Proposal; Reports, Escalations Compliances of the best practices, client management & business growth Creating DD, Design Framework, Coding convention, Reviews of deliverables of Developers & testers Creating Specification, Code, Unit & SI test cases, Testing, etc Completion checklist, Self review
    11. 11. Making earth little softer Reviews • Assuring:- • Quality of design document impacts quality of the code & the test cases. • To identify the right resource for the reviews depending on entity to be reviewed • Assuring incorporating review comments • Reviews are discontinued when progress is delayed. To assure to have reviews as planned • Tracking review efforts & duration for each entity (like code, unit test cases, etc) (e.g. review time per page)
    12. 12. Making earth little softer Reviews • Deciding on review density before starting the activity • Review comments expected to reduce on repeat cycle • Cause for concerns • No. of review comments & No. of major comments are not decreasing on progress as expected is indication of unstable quality • Volume change of entity (e.g. no of pages for design doc, no. of test cases, etc) pre & post review
    13. 13. Making earth little softer Code reviews • Build detailed code checks into the development process • Make it impossible to release code that violates these checks • Few errors would only be discovered by code reviews or testing • Many errors only occurred on specific datasets
    14. 14. Making earth little softer Performance testing • Execute to check performance of each query • Create database as appropriate • Compile & Check manually all the queries individually for performance time, keep record • Every SQL statement is run against multiple real world customer databases • Improve/ optimise the queries • Log the bugs for worst performing statements
    15. 15. Making earth little softer Issue Management • Issues are restraints to accomplishing the deliverables of the project. • Typically identified throughout the project and logged and tracked through resolution. Issues impact cost, time, quality, most important is reputation and branding! Types of issue – Minor, Normal, Major, Critical, Blocker Minor issue is dangerous!
    16. 16. Making earth little softer Issues/Risks (non-financial) Highlight highest risks for management attention
    17. 17. Making earth little softer Metrics-Development & Enhancement NOTE: Pl highlight the summary from the metrics report Work Packets Effort Variance On-time Delivery Defect Density Field Defect Density Productivity COQ Review Effective ness Goal Actual G A G A G A G A G A G A
    18. 18. Making earth little softer Metrics-Maintenance & Support Provide the SLA compliance summary If you use any tool provide the snap shot of the same. The number of tickets includes the new received in the reporting month as well as the backlog from previous months Priority Number of Tickets/Faults Response SLA Compliance% Resolution SLA Compliance% Remarks Goal Actual Goal Actual P1 P2 P3 P4
    19. 19. Making earth little softer • The perceptions of the user determine the purpose. Quality therefore is a user perception. • Correctness : Does it as per users agreed on? (Unit Level) • Integrity : Does it maintain accurate data? (Integration Level) • Reliability: Does it behave right all the time? (Application Level) • Efficiency : Does it run as well as it could? (Application Level) • Robustness: Does it handle invalid data? (Application Level) Quality & User perceptions
    20. 20. Making earth little softer Testing Improvements • PM decides on type of testing - automated or manual or both • 400% increase in automated regression tests in the same efforts • Tests created for all key business flows • Created by dedicated product QA teams • Used for • Ongoing maintenance/bug fixes • Platform certification (e.g. Linux) • Technology certifications • eBusiness Flows certified prior to release • Progress measured daily, and reported
    21. 21. Making earth little softer Prerequisites of Testing Test Cases • Latest & Completed test cases documents from CVS (or any Configuration management tool) Test Environments • Document that gives the details of H/W and S/W required to start testing • Identical environment which client would have while going live • Development environment • Testing environment
    22. 22. Making earth little softer Flow of Testing • Test cases • Test Data • Unit Testing • Screen Layout, look & feel • Screen Validations • Functionality • Integration Testing • Application Testing • Users Acceptance Testing
    23. 23. Making earth little softer Testing Guidelines Why the testing guidelines? • “Test cases may not cover 100% aspects thus extend your thoughts beyond the written statement of the test case.”
    24. 24. Making earth little softer Who should test There are 2 possible actors • Developers to execute 100% unit testing • Reasons being they take the entire ownership of quality of the delivery • Second possibility is dedicated testing team executing test cases • Testing being skill of expert!
    25. 25. Making earth little softer Unit Testing - UI To use a check list & test for following • Consistency- Control sizes • Basic functionalities • Tables, navigation to next page • Search listing result • Validation error message • Layout (as per Prototype) • Spelling Mistakes • Font (Type & Size) • Icons display • Color Scheme • Screen resolution
    26. 26. Making earth little softer Unit Testing - UI • Tab sequence & Text selection • Links (Check each link to ensure it works) • Current selection indication • Alignments (Text, Numbers, Row, Column) • Compatibility with various browsers • Duplicate display of information • Mention of Copyright
    27. 27. Making earth little softer Unit Testing - Validations • Field type & Size (Numeric, Alphanumerical, Database field width) • Formatted fields (e.g. Date, E-mail, Phone Numbers, Security ID) • Mandatory Fields • Messaging (Alert, Warning, field message, message post submissions, sequence, etc) • Database Records verification • Verify the values stored in database for Radio buttons & Checkboxes • Test spaces trimming (leading & trailing spaces) • Update By & Update on fields
    28. 28. Making earth little softer Unit Testing - Functionality To use matrix of screens & functionalities e.g. File Upload/Download •Try all types of files – image files, office files, animated GIF •Try downloading files – verify they don’t contain garbage •Search using special characters like (%,_,’ etc.) Sorting functionality - Check the number of records before and after the sorting are same • In case of mailing functionality, check for non- existing e-mail address. (The mail should be returned back to some preset mail address)
    29. 29. Making earth little softer Unit Testing - Functionality To use matrix of screens & functionalities •Test User login Case Sensitivity •Test the dependant behaviors of the controls •Enabling / Disabling based on some actions •Data shown in Popup Screens based on some actions (Refreshing) •Language Support Test (Wherever applicable Japanese / German/ French, etc Input) •Test readability by changing encoding type in the browser
    30. 30. Making earth little softer Integration Testing • Refer Integration Matrix while testing • Enter boundary values (Min , Max, 0 , negative etc) • Errors occur when values passed by once screen are used in calculations on other screen
    31. 31. Making earth little softer • Testing Business Transactions • Refer Screen Flow Diagrams while identifying Business Transactions • Identify the User Roles to perform testing (Sales Person, Purchase Manager, Dispatch Person OR User , Administrator) • Ensure Test Data Variety & Test data Volume • Roles & Rights testing (Test for access rights) • Logging Mechanism testing (Errors and User Actions) • Deployment Testing (Perform Deployment as per the Deployment Document) Application Testing
    32. 32. Making earth little softer • Application Menu should be in sequence of the functionality • User Management->Master Data Mgt-> Order Entry->Purchase Mgt->Dispatch • Navigation • Using Browser ‘Back’ & ‘Forward’ buttons • Using ‘Back Button’ & ‘Links’ in the application • Browser dependency • Test for all the browsers as per client request (Min should be IE & Netscape) • Disable Cookies and try to login or perform Transactions Application Testing
    33. 33. Making earth little softer • Perform transaction using multiple instances of application, Open other applications like (Word, Excel, Access, Others) • Quick Actions in the Application • Click Submit button twice • Switch between the screens quickly • Enter data in one instance & try to see it immediately in another instance • Random operations • Enter any number of characters and click anywhere • Clicks on read only area Application Testing
    34. 34. Making earth little softer • Copy Paste of the URLs • Copy URL after Logging in and paste it in another instance of browser • Same operation while in the middle of a business Transactions • Session Timeout verification (test for < / = / > defined timeout value) • Disconnection Test (to verify there is no inconsistent data generated) • Disconnect Net connection in the middle of a transaction • Stop Database service while transaction is in process on the client Application Testing
    35. 35. Making earth little softer • Reports printing • Print Multiple Pages • Try to print without installing any printer driver • Verify Report Margins for the printed copy • Status related tests (Display only ‘Active’ / ‘Live’ records) • Multi-user Testing • Concurrent users (Different users / users with different roles) • Concurrent Transactions (Add / Update / Delete / File Upload) • Locking Issues Test • Common Tests…ref file Application Testing
    36. 36. Making earth little softer How Much Testing Is Enough? • There is always one more bug… • So how much testing is needed to find “all” ? • Cost • Time to market • Feasibility • Rate of detection of defects • Industry-specific standards • To predict how many bugs will be in a release? • In order to provide correct budget and schedule estimates, QA managers need the means to predict fault levels.
    37. 37. Making earth little softer Types of defects • Blocker • Severe/ Critical • Major • Normal • Minor • Trivial • To publish phase wise defect data for each module for above types • Responsibility wise Sample statistics
    38. 38. Making earth little softer Update defect passbook at end of day and check status and trend Defect passbook
    39. 39. Making earth little softerConfidential 41 QAG measures Quality QAG tracks and measures quality using following template Organisation
    40. 40. Making earth little softer Quality measurement No. of defects 1 effort in manmonths 3 Loc per hr 10 Lines of code 4800 no. of defects 1 % of defects to LoC 1*100/4800=2.1% Achieved quality 97.9167% • 1 defect detected by customer for 3 man months of efforts is said to be fairly good quality • Count each type of defect (minor, normal, major, critical or blocker) ; Minor defect is worst!
    41. 41. Making earth little softer Measure where are you • To set a quality baseline & remain determined • To seek feedback • To measure and conclude current levels from desired level • To draw action plan to be at set goal 0 1 2 3 4 5 6 7 8 Vertex NTTD Desired level Current Level Current level Desired level
    42. 42. Making earth little softer Aim for Defect Metrics (sample) Phase Metrics Unit Lower Limit Avera ge Upper Limit Requirement definition Requirement documents Review comments Per 100 pages 9.8 15 24 Design Detail design (DD) Review comments Per 100 pages 7.6 13 21 Program design (PD) Review comments Per 100 pages 4.9 12 30 Coding Unit Testing Test Case Density KLOC 65 110 170 Defect density KLOC 3 6.4 9.7 Integration Testing (SI) Test Case Density KLOC 16 39 92 Defect density KLOC 0.7 2.2 3.6 Testing System Testing (PT) Testing density KLOC 2.3 9.5 39 Bug detection density KLOC 0 0.7 1.4 Entity which is measured, can be managed well
    43. 43. Making earth little softer Bug content analysis Phenomenon Action example Module having major change Suspend test work Review of design document, review of source code with the designer Enhanced test according to test items, bug contents Test density < allowable range Enhanced test according to test items, bug contents Bug detection density is less than allowable Enhanced test according to test item, bug contents Bug detection density exceeds allowable range As per 1 above
    44. 44. Making earth little softer Bug content analysis Phenomenon Action example Bug cause Problem of defective specification Re-check specifications for client, business Design document review with an expert Indetermination of specifications of design doc. ambiguous part Bug cause Problem of development from specification Confirmation of specification to designer Review of design document & source code with the designer Clarification of implementation, implementation method Bug - Other than specification Thoroughness of walk through and source code review Occurrence type Environment Specify reason according to occurrence period, contents Re-examination of test environment, system
    45. 45. Making earth little softer Bug content analysis Occurrence type Action example Operational error, without reproduction, identification mistake) Review of test item, process Re-adjustment of design document, test environment Educational guidance to designer Coding/ modification mistake Thoroughness of source code review Occurrence of important bug that impacts test progress Suspend test work Review of design document & review of source code Enhanced test according to the test item, bug contents occurrence of similar bugs Check the latency of a similar bug in another module
    46. 46. Making earth little softer • No. of pages/ characters of design document & LOC of the code • Check planned No. of test cases (a statistical index page for test cases shows the progress) • Test cases planned & actually execution • To capture No. of test cases executed on daily basis and cumulative basis • No. of defects detected on daily and cumulative basis – check if volume is apt • Calculate defect density–check if volume is apt • Check if any delay in execution, understand & remove the delay Quantitative analysis
    47. 47. Making earth little softer •Check the pending test cases on daily & cumulative basis – check if volume is apt •Bug fixing on daily and cumulative basis - check if volume is less or more •Check if any delay in fixing, understand & remove the delay Quantitative analysis Matrix Analysis Test density Low Target value Upper limit Bug Density -Low Less Bugs detection Retest adding few more test cases Member skill level? Target Value Write additional test cases Judge verification of the result qualitatively Judge verification of the result qualitatively Upper Limit Multiple bugs occurred Multiple bugs occurred Suspend testing, get into other measure like code review, etc
    48. 48. Making earth little softer Quantitative analysis # Category Details Type 1 Specification Inadequacy Description Inadequacy Interface Error Logical Error Data definition error Table definition error Format Error Others Wrong description Ambiguous Standard Violation Document not corrected Document not consistent 2 Issue while deploying Slipping over the specification Lack of understanding the Lack of confirming the specification Insufficient investigation of specs 3 Issues except the matters related to specification Less knowledge for language usage Halfway publicize / communication Standard violation Not checked while re-using Not checked while modifying Unforced mistake
    49. 49. Making earth little softer • Root cause analysis – assign root cause to each bug & conduct analysis (fishbone diagram) • Severity wise • Developer wise • Tester wise • Defect injection phase v/s defect detection • Defect injection phase v/s root cause • Daily status (like a bank passbook) Quantitative analysis Defect detected RA-Review HLD, IT -Review LLD, UT-Review Code review UT IT AT Defect injected RA HLD LLD Coding
    50. 50. Making earth little softer Process • This PCB is released as Organization PCB • This PCB should be referred to when providing Organizational Data to the customers • Each project depending on the size, complexity and customer requirement shall define the metrics for the project. • This would be done in Process Plan as part of the project initiation. • In case the project targets are outside the UCL or LCL limits an approval would be required from the respective VP.
    51. 51. Making earth little softer PCB Contents • Productivity Data • Phase wise effort distribution • Effort – Actual Vs. Estimated • Schedule – Actual Vs. Estimated • Defect Metrics
    52. 52. Making earth little softer Productivity Data LOC/ 1 MM Productivity LCL Target UCL Java 700 1000 1300 VB.Net 800 1500 2200 C# .net 700 950 1200 C 700 1000 1300 This data applies to all the efforts spent on all SDLC Phases – from RA to Acceptance testing To finalise parameters for data to be captured To create PCB data for each project Thus organisational data will be created Analyse & use the data
    53. 53. Making earth little softer Plan to Collect Data • PCB shall be updated every three months, End of every quarter - Jun, Sep, Dec, and Mar • For development, testing & maintenance project • Finalise project metrics during kickoff of project • The project metrics collection plan shall be finalized as part of the project initiation process (Process Plan) • The projects shall contribute Phase End data or Milestone data every six months • To share PCB data during project closure
    54. 54. Making earth little softer Project Closure To conduct a meeting with all stake holders To share following information estimated & actual • Efforts • Schedule • Defect Matrices • Profitability • Client happiness Index Detailing • Key learning • Best performance • What next? Remember
    55. 55. Making earth little softer Thank You Author has worked at as Sr management professional Vertex Software, now NTTData Tech Mahindra SoftBridge Solutions, India & extremely thankful to these organisation for offering exciting opportunity to work in related areas, gain experience & contribute & a gentleman having deep passion about quality..Mukand Bhagwat