1. Vendor Performance Evaluation RICHARD L. GAY, CPPO, RSBO PURCHASING MANAGER OFFICE OF PURCHASING October 2009

2. VENDOR PERFORMANCE EVALUATIONS Why?

3. Compliance with Board Policy & Rule 3231-Vendor Performance Evaluation, approved September 2007. To effectively manage vendors doing business with the Baltimore County Public Schools. Vendors have the right to receive performance feedback, whether it is positive or negative. If feedback is negative, the vendor is entitled to be informed of why their performance is unsatisfactory and what corrective action is required. This procedure applies to all contracts with a value greater than $500,000, and is voluntary for other contracts.

4. Why Monitoring is Fair…. Should be 360° Each participating party needs to be able to rate the other allowing for open and constructive feedback. Constructive Comments Should pinpoint exactly what needs to be improved and how to do it. Consensus Isolated low scores look insignificant when the other team members are rating everyone as well. No Surprises Because the process is ongoing through the life of the project, there are no surprises and no frustrations over “after the fact” evaluations, when it is too late to change. Many Opportunities Multiple intervals offer many opportunities to improve based on suggestions received.

5. Why Monitoring is Fair…. Hard and Easy Raters Administrators can determine whether certain raters are consistently demanding and disgruntled, or alternatively indiscriminant in passing out 10’s to everyone. 20 Criteria, 4 Categories Breaking the interval criteria into categories provides many opportunities to score well. Positive Emphasis Project managers can use the ratings process to emphasize teamwork, positive and constructive feedback, rewards, and recognition. Averaged Final Scores Final scores are averaged on multiple intervals and ratings by multiple raters. Recognition Many opportunities to recognize good or excellent performance.

6. Why Monitoring Can Be Done Performance-based contracting typically requires procurement, technical, legal, and program staff to do more up-front work than more traditional acquisition processes. Further, it requires considerable collaboration among staff. However, this investment of time yields a high payoff for all, allowing public sector agencies to get the best value for the dollars spent. When performance evaluations are used there were demonstrated savings of at least 15%, better contractor performance, and higher satisfaction with the results of the contracts.

7. Develop a performance work statement (PWS) or statement of objectives (SOO). This is a description of how the work to be done under the contract is framed as a desired outcome or set of results, rather than a set of tasks. To generate the statement, an interdisciplinary team typically conducts one or more brainstorming sessions to consider what the ultimate "customer" really needs.

8. Decide how to measure and manage performance. Measurable performance standards Incentives/remedies Performance assessment or quality assurance plan

9. "Projects that would be considered as most appropriate for the use of the PerformanceEvaluation include: Large Projects: For example, a project requiring a large investment of resources and staff, with an equal opportunity to gain large benefits. New Technology Projects: For example, a project requiring new technology where the agency does not have an in-house staff with the needed skill or expertise. High-Risk Projects: A project that contains enough cost, new technology, or impact on the business that the agency should look outward for solutions and expertise.

10. Among the kinds of projects that are not good candidates for performance evaluation are those; with easy to define deliverables, such as hardware replacement, projects where it is not possible to assemble an interdisciplinary team, and projects that affect governmental functions with no performance measurements.

11. MONITORING BCPS’ procurement personnel or project staff shall verify that the certified MBE’s listed in the MBE participation schedule are actually performing the work. BCPS’ procurement personnel shall ensure that MBE subcontractors are receiving compensation as set forth in the MBE participation schedule by ensuring that the contractor submits monthly reports, listing any unpaid invoices over 30 days old received from any certified MBE subcontractor, the amount of each invoice, and the reason payment has not been made. The MBE liaison and/or the Public School Construction Program will conduct reviews as deemed necessary to confirm compliance with the minority business enterprise participation requirements. The MBE liaison will maintain appropriate records, and shall assist the Public School Construction Program in on-site or post-audit reviews upon request.

12. How?

14. The VPR is found on the Purchasing Intranet webpage, under Forms & Publications.https://intranet.bcps.org/offices/purchasing/xls/Form_VendorPerform_NonConst.xls

16. A VPRmaybe completed for contracts with a value less than $500,000, and for any open ended long-term or blanket order contracts.

17. A VPR may be completed at the discretion of BCPS personnel, whether performance is positive or negative.

18. Purchasing cannot take action on the negative performance of any vendor without adequate documentation!

20. NON-CONSTRUCTION CONTRACTSReport Disposition Based on the documented information provided, the Office of Purchasing will contact the vendor and investigate the matter. The Office of Purchasing will report to the vendor the findings, their right to respond, and their right to appeal the disposition. The Office of Purchasing will inform the report initiator of the results. The Office of Purchasing maintains a file on all vendor performance evaluation documentation.

21. How? CONSTRUCTION CONTRACTS

22. CONSTRUCTION CONTRACTS The Office of Purchasing uses Rating Source, an internet evaluation application, to evaluate performance while projects are under way. Rating Source is a 360° evaluation application wherein each party rates the other. The Office of Purchasing works with Facilities to set up project intervals and determines project raters (team members). Project intervals and team members are chosen based on the scope of the project. Team members will be provided instructions entitled Vendor Performance Evaluation Procedures. Team members are asked to submit evaluations at predetermined intervals or milestones.

23. Project Intervals Design Development Construction Documents 50% Complete 100% Construction Document Submission Bid Phase Initial Construction End of First Phase End of Second Phase Final Completion

24. CONSTRUCTION CONTRACTS Project Rating Request: A notification email will be generated to every project team member when it is time to rate a project. The email provides each team member with a link to access the project to be rated, along with login and password information.

25. CONSTRUCTION CONTRACTS After clicking on the project link: Sign on with the user information provided to you by Rating Source.

26. CONSTRUCTION CONTRACTS Click on Complete An Open Survey.

27. CONSTRUCTION CONTRACTS Click on the survey that you have been asked to complete. *Detailed instructions entitled Vendor Performance Evaluation Procedures, will be provided to you by the Office of Purchasing.*

28. CONSTRUCTION CONTRACTSRatings Results Results Upon completion of interval ratings by all team members, Rating Source compiles the results. The results summary is shared with the Project Manager for feedback. Team members can request access to Rating Source to view their results by contacting the Office of Purchasing. An email to grant access with login and password information be will be sent to the requestor. Click on the link to login to Rating Source.

29. CONSTRUCTION CONTRACTSRatings Results Results-Beta Version Click on Reports.

30. CONSTRUCTION CONTRACTSRatings Results Results Click on the survey that you would like to review for an overall score.

31. CONSTRUCTION CONTRACTSRatings Results Results For more detailed results, click on the monitor name.

32. CONSTRUCTION CONTRACTSRatings Results Results-Beta Version Review each survey by clicking on the View button.

33. CONSTRUCTION CONTRACTSRatings Results

34. CONSTRUCTION CONTRACTSRatings Results

35. CONSTRUCTION CONTRACTSRatings-Legacy System BCPS first began rating construction projects using the older Legacy System version of Rating Source. Enter your User ID and Password; click on the Legacy System button.

36. CONSTRUCTION CONTRACTSRatings-Legacy System Click on the Team Monitor link.

37. CONSTRUCTION CONTRACTSRatings-Legacy System Select your Monitor Name in the drop-down list.

38. CONSTRUCTION CONTRACTSRatings-Legacy System Click View Reports button along bottom of page.

39. CONSTRUCTION CONTRACTSRatings-Legacy System Below is a view of Overall Reports that can also be printed by clicking on the Print button.

40. CONSTRUCTION CONTRACTSRatings-Legacy System More detail is obtained by clicking on the Rated Company link.

41. BCPS GENERAL CONTRACTORSVendor Performance Evaluations A Rating Source-based evaluation process was performed on BCPS BCDPW pre-qualified general contractors in April, 2007. Each general contractor provided client references on jobs over past three years.

42. BCPS GENERAL CONTRACTORSVendor Performance Evaluations These references were contacted and asked to provide vendor performance information via a survey. Vendor Performance Evaluation Sets

44. Requests for completed evaluation surveys should be made to the Office of Purchasing. Surveys will be received via email.