A strategic solution for purchasing organizations

  • 815 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
815
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
15
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A group decision support system for RFP evaluation A strategic solution for purchasing organizations Complete eRFP Tutorial Available at: http://erfp.sysoft.com 100 Wood Ave South Ph: 732-549-9002 Suite 111 erfp@sysoft.com Iselin, NJ 08830 http://erfp.sysoft.com
  • 2. Bringing RFP Evaluations Into The 21st Century Paul J. Brennan, CPPB, C.P.M. For years, those in the procurement field, both public and private, have heard the expectations of how our profession would dramatically change from the implementation of e-procurement technology. While most of those in the procurement profession will certainly agree that our jobs today have changed dramatically over the past five years; it remains a fact that many of the processes involved in source selection have seen little change or have remained basically the same. While most of the Enterprise Resource Planning Software and Procurement Software that is available today has embraced “e-procurement” by offering E-Bidding, Bid Tabulation, Automatic Supplier Notification, and other new features, most if not all of the software providers have not provided an automated solution to the evaluation of Request for Proposals. As more and more government organizations move away from the “low bid or low cost approach” to contracting, the Request for Proposal process is becoming the preferred method of source selection to identify the “best qualified” or “best value” proposal. Rockland County, New York, like most other government agencies had seen their use of the Request for Proposal increase significantly over the past five years. Already issuing over 225 Competitive Bids on an annual basis, Rockland began to see the number of Requests for Proposals issued each year reach an average of 50 – 60. Along with the increased use of the RFP came many operational issues including, how to manage multiple evaluations being conducted at the same time, how to ensure thorough, independent and objective evaluations by multi-disciplinary teams, how to take evaluation data from paper rating forms and roll that information up to meaningful reports that can assist the evaluation teams in making informed decisions, and how to document the decision making process so that the source selection decision can be justified to administrators and elected officials. Rockland County determined that it needed an automated group decision support system to help manage the RFP evaluation process. With no knowledge of such software currently available, the County issued a Request for Information, which was advertised nationally to attempt to locate such a software product. The County received many responses with offers to build such a system from scratch; and only two responses from software providers that had similar software available or in development. One of the responses came from a software provider named Sysoft for a product called eRFP. eRFP was designed as a group decision support system for RFP evaluations. At the time of Sysoft’s response to the County’s Request for Information, the design of the eRFP software was ninety-five percent complete but had yet to be beta tested. The timing of our RFI could not have been better as the County identified a new software product as well as a software provider that was looking for a government agency to partner with to help beta test the software and identify any design changes needed based on using the system to evaluate actual government projects. The result was a contract that benefited both the County as well as Sysoft. Since implementing the eRFP evaluation system the County has come to rely on the use of this on-line evaluation tool to make better sourcing decisions. The system is database driven and uses the most current web-based technologies. The software is deployable through an organization’s Intranet or through the Internet. Internet access allows evaluators in many different locations to easily evaluate the same proposals. Since the software is web-based, it also allows evaluators access from their home computers. Allowing evaluators to work from home while evaluating proposals can lead to more in-depth evaluations, usually in a shorter timeframe.
  • 3. Rockland County’s experience has been very positive from the beginning of this project. There have been the normal bumps in the road or “bugs” that are normally experienced when one is participating in a software development project. The use of a web-based software product has enabled Sysoft to respond quickly to any software issues that arose during this project. From a procurement management standpoint, the eRFP software has permitted us to track the progress of each evaluator, capture evaluator’s comments for use at caucus meetings, identify strengths and weaknesses of each proposal, score proposals in a more objective and less subjective manner, and produce detailed reports that assist in the decision making process. Using the eRFP software has given the County the ability to easily identify the “best qualified” firm or proposal. By ensuring the selection the most qualified firms, the County anticipates that it will realize cost savings by having projects completed on time and within budget. The eRFP software has highlighted the various effects of using “Cost” as an evaluation category and how the use of cost as an evaluation category may skew evaluation results; resulting in lesser qualified firms or proposals being selected. This software has also enabled the County to begin including “Risk Assessments” as part of the evaluation process. Proposals are now evaluated in relation to Risks associated with the proposed approach or Statement of Work; Risks associated with the proposed schedule, Risks associated with the proposed personnel; and Risks associated with the financial capability of the firm to complete the project. Using the software has also helped the County assess the strengths and weaknesses of evaluators, and has helped identify evaluators who require additional training as well as those who may need to be removed from future consideration as an evaluator. From the evaluator’s point of view the eRFP software gives them the ability to enter their evaluation on-line. The software guides each evaluator through an organized set of on-line forms with pre-defined evaluation criteria and standards. Evaluators have access to the relevant sections of the RFP that was issued for the project without having to revert to a paper copy of the RFP. Evaluators can record strengths, weaknesses, deficiencies, and statements regarding their rating of each evaluation criteria. The use of this software has helped provide a more objective evaluation process by having the evaluators complete the evaluation with no knowledge of the scoring formulas, and by asking for comments in relation to each rating they assign. When all of the evaluators assigned to a project have completed their evaluations, rating scores and comments are automatically consolidated into a detailed summary report that is used during a caucus meeting to develop a short list of the best proposals for further consideration. The short listed firms are normally invited to make formal oral presentations. After each oral presentation is completed, evaluators log back onto the eRFP system to score and provide comments on the oral presentation. At this time the evaluators also complete the Risk Assessment of each of the short-listed proposals. The final evaluation report combines the scores of the written proposal evaluation, the oral presentation evaluation, and the risk assessment evaluation. The software documents the entire evaluation process, from project set-up and the selection of evaluation criteria, to the final award recommendation. The software captures all notes generated during the evaluation process. This documentation helps justify the selection of the most qualified firm or proposal, especially when the cost is higher than other proposals. In summary, the use of software to facilitate the RFP evaluation process has enabled Rockland County to reduce the time and cost of proposal evaluations, reduce the subjectiveness of evaluations, provide a complete audit trail, eliminate paper intensive processes, and increase evaluator efficiency.
  • 4. Sysoft eRFP proposal evaluation system automates Request for Proposal evaluations. Although a seemingly straightforward management responsibility, the task of managing the evaluation process for request for proposals requires careful administration and the tracking of a substantial amount of information. If the evaluation for request for proposals is mainly a paper driven process, it makes the collection of evaluation data and the development of reports a very time consuming task. Sysoft eRFP automates the source selection process for various types of contracts including but not limited to: Service Contracts, Rental and Lease Contracts, Professional Service Contracts and Software and Technology Contracts. The software assists in providing quality evaluations, complete and traceable documents and timely contract award decisions. Features • Database driven and Web based • Deployable within Internet, Intranet and Extranet • The ability for evaluators to enter their evaluation online • The ability to view progress and change task assignments of evaluators • The ability to assign risk assessments • Provides on-line access to relevant acquisition documents • Provides electronic evaluation forms with pre-loaded evaluation criteria and standards • Guides the evaluator through an organized set of on-line forms to create clarifications, deficiencies, ratings, rationale and impact statements • Ability to store proposal strengths and weaknesses in a notepad area • On-line access to all evaluations allows source selection officials to monitor progress and provide feedback • Ratings are automatically consolidated • During caucus sessions, individual ratings and evaluations can be viewed on a projector screen • Consensus ratings are generated on-line • Standard and customized reports can be printed throughout the process • Easy evaluator assignment • Email notification for evaluation deadlines and automated email/Fax capability Benefits • Reduce the time and cost of proposal evaluations • Helps reduce subjectiveness of evaluations • Availability of complete audit trail • Eliminates paper intensive processes • Increases evaluator efficiency • Instant on-line monitoring of the evaluation process • Facilitates more precise evaluations • Ensures complete traceability of evaluations • Availability of real-time interim and final reports • Availability of various cost and other analytical models
  • 5. Experience of Rockland County - New York Management The most significant benefit of using eRFP is that it provides the ability to easily identify the “best qualified” firm or proposal. By selecting the most qualified firms or proposals, agencies should realize cost savings by having projects completed on time and within budget. eRFP helps show the various effects of using “Cost” as an evaluation category and helps an agency to determine if cost should be a weighted evaluation category. The system shows how the use of cost as an evaluation category can skew evaluation results; resulting in lesser qualified firms or proposals being selected. Management gains the ability to track the progress of the evaluators; which helps meet target dates to complete the evaluations. In general the use of the system assists an organization in keeping the evaluation process on schedule. The system can also help management assess the strengths and weaknesses of the evaluators, and helps identify evaluators who require additional training, and those who should be removed from evaluation teams. Better Evaluations The eRFP system provides a more objective evaluation process by having the evaluators complete the evaluation with no knowledge of the scoring formulas, by asking for comments in relation to each evaluation criteria, and by providing detailed summary reports for use at evaluation team meetings. The system allows each evaluator to easily score and enter comments for each proposal. Since the system is web-based, it allows access from multiple locations, including giving evaluators access from their home computers. Allowing evaluators to work from home while evaluating proposals can lead to more in-depth evaluations; usually in a shorter timeframe. Documentation The eRFP system documents the entire evaluation process, from project set-up and the selection of evaluation criteria, to the final award recommendation. It gives the evaluation team the ability to enter notes regarding a specific proposal, or the evaluation process in general. This documentation helps justify the selection of the most qualified firm or proposal, especially when the cost is higher than other proposals. This documentation also easily identifies the area in which a proposal is weak, and helps eliminate proposals that may be very weak in one key area. Process Customization The eRFP system allows customization of the RFP evaluation processes. A library of templates (managed and maintained by the eRFP configurator) makes the eRFP system flexible to handle any RFP process. It also allows evaluation of any type of RFP.
  • 6. eRFP Process eRFP Process Administrator sets up Cost Models: the RFP evaluation project based on Step 1 evaluation criteria, customized 1. Competitive Range Analysis weight assignments, upset levels, eRFP Process analytical models available in the 2. Least Cost Evaluation Administrator system, selects evaluators and 3. Greatest Benefit for Dollar Cost assigns tasks to evaluators. 4. Cost as an Evaluation Category eRFP Process Administrator adds Evaluation Criteria: Step 2 proposals to the RFP evaluation project. This will allow the evaluators - Initial written proposals eRFP Process assign values to proposals and add - Technical Administrator detailed comments as they evaluate - Personnel proposals and criteria assigned to - Relevant Past Performance them. - DBE/MBE criteria - Cost - Oral evaluations - Personnel eRFP Process Administrator adds -Team issues Step 3 multiple references. This will allow - Approach evaluators or the administrator to - Technical response assign points based on reference - Past Performance eRFP Process Administrator feedbacks - Risk criteria Evaluators login and find the proposals waiting for their evaluation. Step 4 They evaluate, assign ratings to Reports: criteria and add unlimited detailed comments for clarification. System Evaluators monitors and facilitates their  Written Proposal Evaluation progress. Evaluators can generate Report reports on their own evaluations  Print version of Written Proposal Evaluation Report  Oral and Risk Summary Report  Print version of Oral and Risk Summary Report The process administrator can  Overall Combined Report monitor the overall progress and  Print version of Overall Combined Step 5 change task assignments if needed. Report On completion or at any point of time eRFP Process the process administrator can Administrator generate evaluation reports, caucus facilitation reports, add caucus notes, shortlist proposals, award the selected proposal and archive the project for future use.
  • 7. eRFP Architecture eRFP Process Administrator Intranet/Extranet Database Administration, Data Input, Query and Std. PCs with Windows and Reports Browser Access, Oracle or SQL Server Intranet/ Extranet Cloud Evaluator Authentication Evaluator Firewall Std. PCs with Windows and Browser Web Server (If connection to Internet is required) Internet Cloud Std. PCs with Windows and Browser Evaluator
  • 8. eRFP – Evaluator Task Assignment You can assign tasks to evaluators and change the task assignment, if required at anytime. eRFP – View Progress You can view progress of the RFP evaluation process at anytime to ensure completion before the caucus meeting.
  • 9. County of USA Department of General Services - Purchasing Division eRFP Evaluation System Written Proposal Evaluation Report Date: 1/8/2004 Time: 3:26:17 PM Project Name: RFP-CR-003 Technical Assistance Project Project Number: 0000000144 Upset Level Imposed: Yes Selected Cost Model: Competitive Range Analysis Proposal: EAK Inc. Number of evaluators who completed initial evaluations: 5 out of 5 assigned Proposal: ATI Number of evaluators who completed initial evaluations: 5 out of 5 assigned Proposal: KRL and Associates Number of evaluators who completed initial evaluations: 5 out of 5 assigned Proposal: MultipleSystems Inc. Number of evaluators who completed initial evaluations: 5 out of 5 assigned Proposal: Reliable Systems Inc. Number of evaluators who completed initial evaluations: 5 out of 5 assigned Initial Written Proposal Evaluation Ranked by Overall Score Proposal: MultipleSystems Inc. Overall Score=3.35 Proposal: ATI Overall Score=3.32 Proposal: Reliable Systems Inc. Overall Score=2.72 Proposal: EAK Inc. Overall Score=2.24 Proposal: KRL and Associates Overall Score=2.21
  • 10. Overall Caucus Comments: Proposal: MultipleSystems Inc. Shortlisted Proposal Initial Evaluation of Written Proposal Technical Personnel Past DBE/MBE Cost Criteria Yes MultipleSystems Criteria Criteria Performance Criteria Inc. Competitive Weight=40% Weight=30% Weight=30% Weight=0% Range Analysis Weighted Weighted Weighted Weighted Score: Score: Score: Score: 3.30 2.96 3.80 0.00 Good Good Excellent N/A Caucus Comments: Technical Criteria • Criteria: The Proposal showed a full and complete understanding of the need(s) or problem the proposed solution will address Evaluator: Sam Adams Comment: Solution anticipates turning over to County at some point for maintenance. County has little resident expertise in this area. Evaluator: Kara Blasco Comment: MultipleSystems Inc. and their partner companies are apparently very aware of the needs of this type of project. However, there is one area in which their approach differs from what we envision. Repeatedly, there is reference to a single or "the" technology vendor. We see the very strong likelihood that the best final product will be realized by acquiring technology from a variety of vendors, picking and choosing the items that are best for us and the vendor that seems best suited for our needs. I would see this as one of the most important functions and strangely out of place in an otherwise amazingly detailed and comprehensive proposal. • Criteria: The proposal showed knowledge of potential technical, regulatory, economic, environmental, or other issues that will be addressed by the technical approach Evaluator: Kara Blasco Comment: Obviously the Multiplesystems Inc. team, especially the Principle in Charge John Poloski, is on the leading edge of the industry edge of the industry's effort to establish technical and regulatory guidelines that this rapidly developing application of ITS technology to transit requires. Their proposal provides a extensive reference to the process of regulatory development, the base from which it is developing and the progress to date. John Poloski has been and still is a prominent figure in this effort. This report consists of 30 more pages. For the purpose of this brochure the report is truncated here.
  • 11. County of USA Department of General Services - Purchasing Division eRFP Evaluation System Oral and Risk Summary Report Project Name: RFP-CR-003 Technical Assistance Project Project Number: 0000000144 Proposal: ATI Number of evaluators who completed Oral Evaluations: 4 out of 5 and number of evaluators who completed Risk Evaluations: 5 out of 5 Proposal: MultipleSystems Inc. Number of evaluators who completed Oral Evaluations: 4 out of 5 and number of evaluators who completed Risk Evaluations: 5 out of 5 Ranking Based on Oral Evaluation Score Proposer's Name: ATI Oral Evaluation Score: 12.50 Risk Evaluation Score: 8.25 Less Risk Score (MAX Risk - Risk EVAL Score): 27.75 ORAL Less RISK Proposer's Name: MultipleSystems Inc. Oral Evaluation Score: 10.00 Risk Evaluation Score: 12.25 Less Risk Score (MAX Risk - Risk EVAL Score): 23.75 ORAL Less RISK Overall Caucus Comments: Proposer's Name: ATI Oral Evaluation Score: 12.50 Risk Evaluation Score: 8.25 Personnel Past Performance Team Issues Approach Technical Response Score: 6.50 Score: 1.00 Score: 2.00 Score: 2.00 Score: 1.00 Caucus Comments:
  • 12. Evaluator Comments - ORAL Personnel • Criteria: Has the key management been identified? Evaluator: Samuel Johnson Comments: Appears to be very qualified and experienced. I believe they have the capabilities to make this project a success. Evaluator: Kara Blasco Comments: Assigned project manager is a recognized expert and leader in the integrated technology field. Evaluator: John Tish Comments: Key management has been identified. Project manager is a leader in this field. • Criteria: Have the key consultants and other team members been identified? Evaluator: Samuel Johnson Comments: I don't believe this question is applicable. Evaluator: Kara Blasco Comments: Good presentation on approach to the work, chain of command, use of staffing resources. Evaluator: John Tish Comments: They did not guarantee that the three individuals in each of the specialty areas would be assigned for the entire project. Not all of the team members were present. • Criteria: Has the presentation been clear as to the role and responsibilities of key management and consultants and team members? Evaluator: Samuel Johnson Comments: All of the members of the Avail team have specific responsibilities. all are centered around the project manager. • Criteria: Will the project manager and other presenter(s) be able to communicate with elected officials, staff, and the public Evaluator: Samuel Johnson Comments: Pete Bennett speaks very well. Evaluator: Kara Blasco Comments: Project Manager is frequent presenter and recognized authority at industry conferences and is an excellent public speaker and one to one communicator. This report continues for another fifteen pages. It includes evaluator comments for oral and risk evaluation. In order to keep this brochure short and simple the report is truncated at this point.
  • 13. County of USA Department of General Services - Purchasing Division eRFP Evaluation System Overall Combined Report Date: 1/8/2004 Time: 8:44:10 PM Project Name: RFP-CR-003 Technical Assistance Project Project Number: 0000000144 Ranking based on Initial Written + Oral Score Proposal: ATI Overall Score=15.82 Risk Evaluation Score: 8.25 Proposal: MultipleSystems Inc. Overall Score=13.35 Risk Evaluation Score: 12.25 Ranking based on Initial Written + Oral + Risk Score (75% of Overall Initial Written & Oral + 25% of Less Risk) Proposal: ATI Overall Score with Risk =18.80 Overall Score without Risk =15.82 Risk Evaluation Score: 8.25 Proposal: MultipleSystems Inc. Overall Score with Risk =15.95 Overall Score without Risk =13.35 Risk Evaluation Score: 12.25 Proposal Awarded: ATI Award Notes:
  • 14. Proposer's Name: ATI Combined Overall Score: 15.82 Risk Evaluation Score: 8.25 Initial Written Evaluation Score: 3.32 Oral Evaluation Score: 12.50 Risk Evaluation Score: 8.25 Oral Score Personnel Past Performance Team Issues Approach Technical Response Score: 6.50 Score: 1.00 Score: 2.00 Score: 2.00 Score: 1.00 Initial Written Score Initial Technical Initial Personnel Initial Past Performance Score: 3.71 Score: 3.25 Score: 2.85 Proposer's Name: MultipleSystems Inc. Combined Overall Score: 13.35 Risk Evaluation Score: 12.25 Initial Written Evaluation Score: 3.35 Oral Evaluation Score: 10.00 Risk Evaluation Score: 12.25 Oral Score Personnel Past Performance Team Issues Approach Technical Response Score: 5.25 Score: 0.75 Score: 1.75 Score: 1.50 Score: 0.75 Initial Written Score Initial Technical Initial Personnel Initial Past Performance Score: 3.30 Score: 2.96 Score: 3.80
  • 15. eRFP Configurator You can customize eRFP evaluation process with the configurator. You can register and select detailed criteria for your evaluation project template
  • 16. You can specify the evaluation process in your project template You can also customize the formula and rating system used in your project template
  • 17. eRFP Builder You can easily create the RFP through the template based RFP builder