QUALITY ASSURANCE & POST-AWARD PERFORMANCE MEASUREMENT <ul><li>Marc Berson, Director, Grant Thornton LLP </li></ul><ul><li...
QUALITY ASSURANCE AND POST-AWARD PERFORMANCE TOPICS Policy & Guidance Quality & Performance throughout an Acquisition Less...
IN BRIEF &quot;It's much more difficult to measure nonperformance than performance.“ - Harold S. Geneen, CEO  ITT &quot;Wh...
POLICY & GUIDANCE <ul><li>Performance Based Service Acquisitions  defined </li></ul><ul><ul><li>Public Law 106-398, sectio...
SAMPLE GOVERNMENT ACCOUNTABILITY OFFICE REPORT &quot;Could use  performance incentives  more effectively&quot; &quot;Has r...
WHY IS QUALITY ASSURANCE AND PERFORMANCE MONITORING IMPORTANT? <ul><li>Continuity and  consistency  in the delivery of pro...
ADDRESSING QUALITY AND PERFORMANCE THROUGHOUT THE ACQUISITION LIFE-CYCLE <ul><li>Critical to a performance-based acquisiti...
PLANNING
THREE CONCEPTS TO CONSIDER IN DESIGNING PERFORMANCE MEASURES Alignment Measure Types Framework Measures should directly su...
BALANCED SCORECARD FRAMEWORK What measures within each perspective can help indicate whether an objective was achieved? Pr...
MEASURE SELECTION CRITERIA <ul><li>MEANINGFUL  – Related significantly and directly to organizations mission and goal </li...
PROCUREMENT ACTIVITIES
INTEGRATING PERFORMANCE METRICS Statement of Objectives QASP Monitoring Procedures PRS Performance Metrics Performance mea...
IMPORTANT QUALITY AND PERFORMANCE SOLICITATION REQUIREMENTS
WHAT IS A QUALITY MANAGEMENT PLAN? <ul><li>Documents   how   an organization structures its quality system that describes ...
EXECUTING AN INCENTIVE PROGRAM   <ul><li>Clearly articulate  incentive clauses  in both the solicitation and resulting con...
MANAGE & MEASURE
WHAT IS THE PERFORMANCE REQUIREMENTS SUMMARY (PRS) <ul><li>Typically part of both the solicitation and the QASP that ident...
SAMPLE PRS Definition Unit of Measure APL Sources of Data Measure Detailed description of measure Unit in which to express...
WHAT IS THE QUALITY ASSURANCE SURVEILLANCE PLAN (QASP) <ul><li>Defines process for  performance monitoring  & evaluation b...
QASP IN ACTION <ul><li>The QASP provides the bridge between the work requirements in the SOO and actual performance by the...
CORE QASP COMPONENTS <ul><li>Ensure data is: </li></ul><ul><ul><li>Obtained </li></ul></ul><ul><ul><li>Recorded </li></ul>...
REPORTING PERFORMANCE CONSIDER ENABLING TECHNOLOGIES <ul><ul><li>Enabling technologies provide  accountable, intuitive, au...
REPORT PERFORMANCE COST AND PERFORMANCE MANAGEMENT TOOLS
LESSONS LEARNED <ul><li>Planning </li></ul><ul><ul><li>Incorporate quality and performance  throughout   the acquisition l...
QUESTIONS AND ANSWERS
GRANT THORNTON CONTACTS n Marc Berson Director   T 703.637.2716 C 202.236.1182 E  [email_address]   Eric Heffernan Senior ...
Upcoming SlideShare
Loading in...5
×

Quality Assurance &amp; Post Award Performance Measurement

955

Published on

QUALITY ASSURANCE &amp; POST-AWARD PERFORMANCE MEASUREMENT

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
955
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Not associated with evaluating the details of how the SP accomplishes the work
  • Quality Assurance &amp; Post Award Performance Measurement

    1. 2. QUALITY ASSURANCE & POST-AWARD PERFORMANCE MEASUREMENT <ul><li>Marc Berson, Director, Grant Thornton LLP </li></ul><ul><li>Eric Heffernan, Senior Manager, Grant Thornton LLP </li></ul>Thursday, November 20 2:10 – 3:10 pm
    2. 3. QUALITY ASSURANCE AND POST-AWARD PERFORMANCE TOPICS Policy & Guidance Quality & Performance throughout an Acquisition Lessons Learned Questions & Answers
    3. 4. IN BRIEF &quot;It's much more difficult to measure nonperformance than performance.“ - Harold S. Geneen, CEO ITT &quot;What gets measured gets done, what gets measured and fed back gets done well, what gets rewarded” gets repeated - John E. Jones, Federal Judge &quot;You get what you measure. Measure the wrong thing and you get the wrong behaviors.&quot; - John H. Lingle , Business Author
    4. 5. POLICY & GUIDANCE <ul><li>Performance Based Service Acquisitions defined </li></ul><ul><ul><li>Public Law 106-398, section 821 </li></ul></ul><ul><ul><li>FAR Section Subpart 37.6 </li></ul></ul><ul><ul><li>FAR Section Subpart 46.4 </li></ul></ul><ul><li>OFPP pamphlet No. 4: &quot;A Guide to Writing and Administering Performance Statements of Work for Service Contracts (1980) </li></ul><ul><li>SARA Acquisition Advisory Panel Report Chapter 2: Improving Implementation of PBSA in the Federal Government </li></ul><ul><li>2006 Survey of Acquisition Executives finds few know how to do it right! </li></ul>
    5. 6. SAMPLE GOVERNMENT ACCOUNTABILITY OFFICE REPORT &quot;Could use performance incentives more effectively&quot; &quot;Has relied on invalidated contractor data to monitor and assess contractors performance&quot;
    6. 7. WHY IS QUALITY ASSURANCE AND PERFORMANCE MONITORING IMPORTANT? <ul><li>Continuity and consistency in the delivery of products or services </li></ul><ul><li>Means to hold vendor accountable and provide a basis for incentives/disincentives </li></ul><ul><li>Provides a structured process for evaluation </li></ul><ul><li>Means to hold the vendor to an expected level of service </li></ul>
    7. 8. ADDRESSING QUALITY AND PERFORMANCE THROUGHOUT THE ACQUISITION LIFE-CYCLE <ul><li>Critical to a performance-based acquisition, yet often times the most overlooked </li></ul><ul><li>Guided far less by law, regulation, and policy </li></ul><ul><li>Is in the best interest of all parties for a successful contract </li></ul><ul><li>Leads to improved performance over time </li></ul><ul><li>Transparency of post award monitoring with other federal agencies </li></ul><ul><ul><li>Agency IGs </li></ul></ul><ul><ul><li>GAO </li></ul></ul><ul><ul><li>OMB </li></ul></ul>
    8. 9. PLANNING
    9. 10. THREE CONCEPTS TO CONSIDER IN DESIGNING PERFORMANCE MEASURES Alignment Measure Types Framework Measures should directly support or be in alignment with objectives and requirements Measures should be defined in the context of a performance management framework Various types of measures should be considered
    10. 11. BALANCED SCORECARD FRAMEWORK What measures within each perspective can help indicate whether an objective was achieved? Program Success Financial Customer Learning & Growth Process EVM Customer Complaints Certified Employees Error Rate Cause & Effect Relationship Integration of External & Internal Measures
    11. 12. MEASURE SELECTION CRITERIA <ul><li>MEANINGFUL – Related significantly and directly to organizations mission and goal </li></ul><ul><li>VALUABLE – Designed to measure the most important activities of the organization </li></ul><ul><li>BALANCED – Inclusive of several types of measures </li></ul><ul><li>PRACTICAL – Affordable price to retrieve and/or capture data </li></ul><ul><li>COMPARABLE – Used to make comparisons with other data over time </li></ul><ul><li>CREDIBLE – Based on accurate and reliable data </li></ul><ul><li>TIMELY – Designed to use and report data in a usable timeframe </li></ul><ul><li>SIMPLE – Easy to calculate and understand </li></ul>
    12. 13. PROCUREMENT ACTIVITIES
    13. 14. INTEGRATING PERFORMANCE METRICS Statement of Objectives QASP Monitoring Procedures PRS Performance Metrics Performance measures must be identified for requirements defined in the SOO Monitoring tools and techniques are outlined in detail
    14. 15. IMPORTANT QUALITY AND PERFORMANCE SOLICITATION REQUIREMENTS
    15. 16. WHAT IS A QUALITY MANAGEMENT PLAN? <ul><li>Documents how an organization structures its quality system that describes their: </li></ul><ul><ul><li>Policies and procedures </li></ul></ul><ul><ul><li>Criteria for application </li></ul></ul><ul><ul><li>Roles and responsibilities </li></ul></ul><ul><ul><li>Tools used for measuring quality </li></ul></ul><ul><ul><li>Processes and corrective action </li></ul></ul><ul><li>Provides a &quot;blueprint&quot; on how vendors will implement specific quality assurance and quality control activities for a specific activity </li></ul><ul><li>QMP level of detail should mirror the scope of the program </li></ul>
    16. 17. EXECUTING AN INCENTIVE PROGRAM <ul><li>Clearly articulate incentive clauses in both the solicitation and resulting contact, which may include non-monetary alternatives </li></ul><ul><ul><li>Financial impact </li></ul></ul><ul><ul><li>Frequency of assessment </li></ul></ul><ul><li>Develop auditable and verifiable data collection trail </li></ul><ul><li>Determine clear payout (or reduction) timelines and prepare finances accordingly </li></ul><ul><li>Be diligent! </li></ul>
    17. 18. MANAGE & MEASURE
    18. 19. WHAT IS THE PERFORMANCE REQUIREMENTS SUMMARY (PRS) <ul><li>Typically part of both the solicitation and the QASP that identifies the performance outcomes expected by the Government </li></ul><ul><li>Can be transformed into a &quot;scorecard&quot; for overall vendor performance </li></ul><ul><li>Determines the how well a vendor is performing </li></ul>
    19. 20. SAMPLE PRS Definition Unit of Measure APL Sources of Data Measure Detailed description of measure Unit in which to express measure Maximum deviation from standard Sources to seek for data collection Wait time Error Rate Reorders as a percent of total orders fulfilled Percentage <ul><li>90% order accuracy </li></ul><ul><li>Order log in workload management system </li></ul>Average time customers wait in line before cashier greets them – weekly average Minutes <ul><li>80% of customers greeted in 5 minutes </li></ul><ul><li>Wait time log </li></ul>
    20. 21. WHAT IS THE QUALITY ASSURANCE SURVEILLANCE PLAN (QASP) <ul><li>Defines process for performance monitoring & evaluation by the government </li></ul><ul><li>Designed to implement and enforce the PRS </li></ul><ul><li>Identifies the surveillance methods that will be used to apprise the Government of the vendor performance </li></ul><ul><li>The Government retains the right to modify the QASP </li></ul><ul><ul><li>Reasons may include: surveillance method, increase/decrease degree of effort, and change procedures </li></ul></ul><ul><li>Identifies procedures to correct deficiencies observed in vendor performance </li></ul>
    21. 22. QASP IN ACTION <ul><li>The QASP provides the bridge between the work requirements in the SOO and actual performance by the vendor. </li></ul>SOO QASP
    22. 23. CORE QASP COMPONENTS <ul><li>Ensure data is: </li></ul><ul><ul><li>Obtained </li></ul></ul><ul><ul><li>Recorded </li></ul></ul><ul><ul><li>Analyzed </li></ul></ul><ul><ul><li>Reported </li></ul></ul>
    23. 24. REPORTING PERFORMANCE CONSIDER ENABLING TECHNOLOGIES <ul><ul><li>Enabling technologies provide accountable, intuitive, automated processes to collect data, measure it, and evaluate vendor performance while: </li></ul></ul><ul><ul><ul><li>Reducing levels of effort </li></ul></ul></ul><ul><ul><ul><li>Increasing standardization </li></ul></ul></ul><ul><ul><ul><li>Enhancing the quality of work products </li></ul></ul></ul><ul><ul><ul><li>Providing and storing data that can be tracked and audited </li></ul></ul></ul>
    24. 25. REPORT PERFORMANCE COST AND PERFORMANCE MANAGEMENT TOOLS
    25. 26. LESSONS LEARNED <ul><li>Planning </li></ul><ul><ul><li>Incorporate quality and performance throughout the acquisition life-cycle </li></ul></ul><ul><ul><li>Assess the cost impacts of any quality and performance initiatives </li></ul></ul><ul><ul><li>Contract type may help determine the type and degree of required oversight </li></ul></ul><ul><li>Procurement </li></ul><ul><ul><li>Include quality and performance management frameworks as part of your solicitation documents (B, C, L, M) </li></ul></ul><ul><ul><li>Government should determine performance measures to support program goals </li></ul></ul><ul><ul><li>Consider the use of a performance oversight board </li></ul></ul><ul><li>Manage and Measure </li></ul><ul><ul><li>The QASP should be developed on how the program will organize itself to monitor the selected vendor </li></ul></ul><ul><ul><li>Modify the procedures as necessary, should be a living document </li></ul></ul><ul><ul><li>Provide quality and performance monitoring training to all stakeholders </li></ul></ul><ul><ul><li>Prioritize limited resources to accurately document vendor performance </li></ul></ul><ul><ul><li>Create mechanism to recommend changes to the PRS, may require a contract modification </li></ul></ul><ul><ul><li>Automate Quality & Performance Management System(s) </li></ul></ul>
    26. 27. QUESTIONS AND ANSWERS
    27. 28. GRANT THORNTON CONTACTS n Marc Berson Director T 703.637.2716 C 202.236.1182 E [email_address] Eric Heffernan Senior Manager T 703. 637.2769 E Eric.Heffernan @gt.com Jennifer Palazzolo Manager T 703. 637.2846 E Jen.Palazzolo @gt.com Global Public Sector 333 John Carlyle Street Suite 400 Alexandria, Virginia 22314 703.837.4400
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×