Your SlideShare is downloading. ×
Performance Measures
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Performance Measures

367
views

Published on

Published in: Business, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
367
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Enhanced Forest Management Pilot Project Performance Measures April 1997 (Revised September, 1997) (Updated September, 1998) (Updated November, 1999) EFMPP
  • 2. Ministry of Forests Enhanced Forest Management Project Performance Measures Table of Contents Page 1. Introduction.........................................................................................................................................1 2. Methodology.......................................................................................................................................1 3. Performance Measures Framework...................................................................................................1 3.1 Characteristics of Performance Measures....................................................................................1 3.2 The Steps to Developing Performance Measures.........................................................................2 4. EFM Project Objectives and Performance Measures.........................................................................3 4.1 Overall Project Measures ............................................................................................................3 4.2 Linkage of Objectives and Measures............................................................................................5 4.3 Committee Objectives and Measures...........................................................................................5 Appendices A Pilot Project Evaluation Framework B Interviews - Participants and Interview Guide
  • 3. Enhanced Forest Management Pilot Project Performance Measures1 Page 1 1. Introduction The objective of this project is to clearly define the EFM project objectives and establish a preliminary set of performance measures and an evaluation framework by which to set targets and track overall project progress and outcomes. 2. Methodology Minutes from meetings and documentation on the EFM project were reviewed. A draft summary of objectives with some examples of performance measures was compiled. A set of interview questions relating to project objectives and expectations was faxed to several individuals associated with the EFM project prior to a half hour telephone interview with each (see Appendix B). The purpose of the interviews was to: • clarify the EFM project objectives; • develop a few key performance measures for the various aspects of the EFM project; and • obtain comments on the EFM project committees and process Comments from the interviewees are incorporated into the objectives and performance measures presented in Section 4.0 3. Performance Measures Framework 3.1Characteristics of Performance Measures Effective performance measures will often have the following characteristics: • they support the project strategy • they show trends • they include targets that promote continuous improvement • they contain both leading and lagging measures (i.e. performance drivers vs. outcomes) • they have impacts that are ‘forecastable’ and/or measurable • they are balanced (not all economic, or environmental etc.) • they are easily measured with current systems Each measure should have its own specification document which contains the following information: • Measurement name: an easy to understand name that describes the activity or result being measured and can be recognized as supporting a critical strategy or objective • Definition: the meaning of the measurement • Computation: the precise equation including the specific data components and data sources
  • 4. Enhanced Forest Management Pilot Project Performance Measures2 Page 2 • Champion: the person who is knowledgeable about the measurement area and is responsible for measurements, results and implementation • Frequency of Collection/Reporting: how often the measurement will be collected and reported • Purpose: the reason for using the measurement, including its link to strategies • Reporting levels: who is going to get reports on the measurement and in what format 3.2The Steps to Developing Performance Measures 1. Define project objectives and desired outcomes. 2. Once the objectives are defined, develop the performance measures. Ask questions such as : • how will you know when objective x is achieved? Measures can focus on: • inputs (what is consumed by the program) These are also referred to as “leading” performance measures or “performance drivers” • outputs (what is produced by the program) • outcomes (results or impacts of the program activities) Both outputs and outcomes can be referred to as “lagging” performance measures that result from the performance drivers (see Section 4.2). Minimize the number of performance measures listed. Ask questions like: • is this measure relevant to the strategies or objectives of the program? • who will use it and how? • who will be accountable for the measure? • how easily can the measure be understood and communicated? • how easy will it be to gather the data for the measure? 3. Develop the measurement strategy. For each measure, outline how the information will be collected, by whom and how often. 4. Record all of the above in a table to demonstrate the linkage between strategy or desired outcome and the measures. This ensures that the measures are linked to the strategy and are not proposed simply because they would be nice to know.
  • 5. Enhanced Forest Management Pilot Project Performance Measures3 Page 3 4. EFM Project Objectives and Performance Measures 4.1 Overall Project Measures There are two overall objectives for the EFM project which bridge all of the elements of sustainability (economic, environmental and social). These are: • Maximize forest productivity; and, • Operationalize (implement) local forest management strategies within existing land use plans and zoning initiatives Underlying these overall project objectives are specific project objectives which fall under one of the elements of sustainable forestry: Economic • Optimize economic efficiency • Empower local forest industry managers Environmental • Optimize biological response • Accomplish objectives within the parameters of the Forest Practices Code and higher level plans Social • Optimize a positive impact on the community • Involve the public in the development of a local forest management strategy This set of objectives will need to be balanced in order for the project outcomes to be sustainable. A balanced set of objectives and appropriate performance indicators will then form the decision framework against which FRBC funding priorities can be developed and benefits assessed. Figure 1 illustrates the interrelationship. Figure 1 : Enhanced Forest Management Project Objectives : Sustainability Analysis SOCIAL ENVIRONMENT (community) (ecosystem) • Optimize • Optimize the • Positive Biological • Impact on the Response to •Positive Community Management •Positive Public Involve Input • • • Maximize Forest Productivity • • Successfully Implement Management Strategy within existing Land Use Plan Ojectives • Empower Local Managers • Optimize Economic Efficiency ECONOMIC (industry)
  • 6. Fi g u r e 2 EF M P r o j e c t Obj e c t i v e s an d Perfor m a n c e Me a s u r e s Fr a m e w o r k Overall Optimize Forest Productivity Successfully Implement Enhanced Optimize Positive Impact Optimize the Biological Response to Forest Optimize Economic Empower Local Managers Involve Public • Increase pilot site harvestable MAI Forest Management Strategy on Community Management Input Efficiency • EFM strategy incorporated • Adequate stakeholder Project • Increase productive land base • Develop and implement local forest • Increase economic activity • Resource inventories demonstrate that the non-timber • Decrease investment needed into forest development plans representation on advisory Objectives • Decrease area lost to unsalvageable losses management strategies in each pilot generated per m3 resource land use objectives are not compromised by to increase the harvestable • Local forestry manager has groups area within existing Land Use Plan harvested enhanced forest management activities • Revenue/ha clear accountability for forest • Stakeholders either support Objectives • Promote education on • Optimize biological effectiveness of forest management • Cost/Benefit per stand, forest management and results on EFM Strategy or recognize • EFM Strategy focusing FRBC land enhanced forest activities and landscape levels the ground and accept the rationale for based programs funding management the Strategy Increase Forest Productivity in Pilot Sites Develop Framework for Direct Project Benefits to Improve Forest Resource Inventories Maximize Annual Return on Local Forest Managers and Local Stakeholders Develop Pilot Pilot Project Measured at Five Year Intervals Implementation of Plans and EFM Community Leading (Performance Driver) Investment at stand, forest Project EFM Strategy and Working Objectives Level Leading (Performance Driver) Leading (Performance • Forest resource inventories initiated, updated or Leading (Performance Driver) Groups • % of area planted with genetically improved Driver) Leading (Performance Driver) • Stakeholder representation, turnout, and input at local advisory (WG’s) • Forest resource database, resource recommended seedlings targets, performance measures, and • Local expenditures ($/yr) Lagging (Outcome) • Decrease expenditures on group meetings • % area planted comm plan developed within one year of Lagging (Outcome) • Types of resource inventories digitized and available for treatments/ha land base • Number and frequency of meetings • Area of incremental silv treatment i.e., pilot commencing • Direct and indirect local resource planning Lagging (Outcome) • Local managers sign-off on EFM strategy fertilizing contribution to local • Decrease in treatment costs Lagging (Outcome) • Disturbance rates and area by type (fire, Build & Implement EFM Strategy economy ($’s, jobs) Increase Biological Effectiveness and increase in revenue per • Public satisfaction with management plan (survey, media articles, insects, disease) • Forest mgmt strategy and • develop curriculum • Carry out field trials, initiate research and make ha number of complaints) • Interim measures leading towards increased recommendations (including Action Plan package suitable for grade recommendations on the timing location and intensity of •Annual report of implementation and effectiveness of EFM strategy productivity for implementation) are developed & schools silviculture practices to SC by local forestry manager Lagging (Outcomes) delivered as per the Pilot Project Plan Maintain Forest Biodiversity of Pilot Site • Average regen delay (years) • Complete EFM Strategy and commence implementation within 3 years Leading (Performance Driver) • Years to reach free growing status • maintenance of species diversity and seral stage • % increase in MAI (m3/ha) distribution as per land use objectives • % area of previously uneconomic or inoperable Lagging (Outcome) timber now accessed • Forest cover distribution at the landscape level (stand type; • % area of productive land lost through pattern (geographic); stage of development) harvesting (i.e., roads and landings) • Birds & mammals - presence/absence, distribution, Promote the Enhanced Forest Management Pilot Project and Communicate Project Learnings • Number of notifications, bulletins, articles re: EFM strategy • Develop and Implement Communication Plan Monitor Results Facilitate/Guide Local Strategies Review Evaluation Criteria Empower Local Managers Monitor Stakeholder Steering • WG’s report to SC on a bimonthly basis Involvement Committee • SC performs annual review of the EFM • Evaluates the biological, socio-economic, and community impact performance measures on an annual basis to ensure that they • Provide WG’s with guidance strategies performance measures as re: pilot project performance are appropriate and are an adequate measures of progress towards the project objectives & resources to enable the • Endorse pilot project WG (SC) • Working Groups receive timely feedback participants, public implemented by the Pilot Projects development of EFM strategy • Review interim measures of pilots for phases from SC re: issues raised for the Pilots involvement process, and leading up to an EFM strategy advisory group structure Promote the Enhanced Forest Management Pilot Project and Communicate Project Learnings • Number of policy recommendations put forward to and acted on by government • Ensure technical reports and strategy documents developed through the project are adequately distributed to government and industry decision makers • Provincial communications plan developed and implemented as per Terms of Reference • Ensure Pilot communication plans are developed and implemented as per Pilot Project Plans • Technical Initiate Research to Support Assist with Selection and Application of Timber Supply Models and Socio Economic Analysis Tools Implementation of Strategies • Ensures that the WG’s modelling approach is adequate to address the pilot project objectives Advisory • Number of research needs identified by • Ensure socio-economic analyses for each pilot are appropriate for meeting their objectives Committee WG’s and reviewed by the TAC (TAC) Ensure Proper Use of Models Provide ongoing advice and training where feasible, to pilot WG members to facilitate efficient use of various forest management models and to understand the modelling results. Provide Technical Advice to Pilot Project Local Working Groups • Advice given on time as requested by WG’s
  • 7. Enhanced Forest Management Pilot Project Performance Measures 5 4.2Linkage of Objectives and Measures The overall project objectives and performance measures for the overall project, the pilot projects and each committee are illustrated in Figure 2. A separate column has been developed for each major objective. The linkages between the overall project objectives and the committee and pilot project objectives are shown in each column. In order to track cause and effect, the pilot project measures are presented in terms of leading and lagging performance measures. Leading measures are performance drivers - they relate to the cause. Lagging measures are the outcome as a result of the performance driver. The specific links between the two sets of measures will need to be established as the pilot projects are implemented. 4.3Committee Objectives and Measures The following section provides more detailed information with respect to the work plan activities of the Steering Committee and the Technical Advisory committee and the linkage of performance measures to these activities. 4.3.1 Steering Committee Table 1 outlines Steering Committee objectives and the activities associated with these objectives that should be included in the Steering Committee work plan. The achievement of these activities as per the work plan schedule could constitute the performance measure. The target would be the anticipated date for completion of the activity. 4.3.2 Technical Advisory Committee Similar to the Steering Committee work plan, Table 2 outlines the objectives of the Technical Advisory Committee and the activities associated with these objectives that should be included in the Committee work plan. The achievement of these activities as per the work plan schedule could constitute the performance measure. The target would constitute the anticipated date of completion. 4.3.3 Pilot Projects Appendix A is a draft format for evaluating pilot project performance measures. The project objectives or goals are fairly well established and a sample of corresponding performance measures have been included. Each pilot site should select or add measures that will most accurately represent the pilots progress towards the objectives. Once established this table should become a reporting tool and ultimately be incorporated into the pilot management strategy document.
  • 8. Enhanced Forest Management Pilot Project Performance Measures Page 6 Table 1 - Steering Committee Objectives and Measures Objective Activity Performance Measure Output and/or Target Date (goal) (Process) TAC Invermere TFL 39 Burns Lake Monitor • Develop project timelines 1. Review and approve pilot project timelines N/A Approved April 1996 Approved Nov 1996 Approved July 1997 Progress and • Conduct bimonthly committee meetings 2. Meetings held as per schedule ºMeetings held on the second Wednesday bimonthly Results • Request reports from WGs and TAC on 3. WGs and TAC report to Steering Committee on ºRecord reports in minutes at bimonthly meeting key performance measures at bimonthly schedule meetings 4. Perform annual review of overall project progress ºHold reviews in January of each year against performance measures and timelines Empower Local • Endorse project work plan 1. Workplan approved Submitted March ‘97 Approved April 1996 Approved Nov. 1996 Approved Sept. 1997 Managers • Provide resources to support strategy 2. Each pilot site and TAC has sufficient resources to Committee formed Annual budget plus 3 Annual budget plus Annual budget plus development carry out workplan with annual budget FTEs contractors coordinator Promote the • Make presentations and provide project 1. Number of presentations and level of positive ºVan Isl. IAMC in Oct. 1996; FRBC Env. Comm. in May 1997; FRBC Board in Sept. 1997 Project updates to FRBC and FSSC response and Committee members promote project at other associations and committees they attend • Develop back grounder and update as 2.Provincial communications plan developed and ºProvincial backgounder released June 1996. Update due for release in December 1997 necessary implemented as per Terms of Reference • Develop generic presentation for project 3. Generic presentation prepared on schedule ºGeneric presentation produced in July 1996 and distributed to all committee members Communicate • Review and approve pilot 1. Pilot project communication plans approved as per N/A Approved July 1996 Partially approved Approved Sept 1998 Project communication plans the work plan schedule Nov 1996 Learnings • Technical reports and strategy 2. Technical reports and strategy documents None to Date 7 technical reports 2 technical reports None to Date documents developed by WGs and adequately distributed to government and reviewed and reviewed and reviewed industry decision makers published between released • Develop process for meeting with gov’t 3. Number of policy recommendations put forward to Sept ’97 and Feb ‘98 (distribution?) officials to recommend policy changes and acted on by government Monitor • Endorse local working group participants 1. Multidisciplinary working group N/A Advisory group Public has been Process will be Stakeholder • Review public involvement/advisory 2. Public involvement process implemented and with established; 2 news approached. Met underway by Sept Involvement Group process information transfer on a regular basis paper supplements with enviro groups 1998 Facilitate/Guide • Request reports from WGs and TAC 1. Working Groups report to Steering Committee on ºRecord reports in minutes at bimonthly meeting Local Strategy • Provide advice to WGs re: analysis and a bimonthly basis Development implementation of strategies 2. Working Groups receive timely feedback from ºTrack issues and resource requests in minutes for timely feedback Steering Committee re: issues raised • Assist with data and resource 3. Working group strategies presented to Steering N/A June 1998 March 1999 March 2000 acquisition Committee on schedule Review • Review criteria in terms of impacts on 1. Biological and socio-economic performance TAC developing SEA completed; SEA underway; SEA to commence Evaluation timber supply, environment and measures are appropriate and adequate measures soci-economic employing employing Fall 1998 Criteria employment of progress towards the project objectives indicators Woodstock silv Woodstock silv • Review resource scenarios as available 2. Resource scenarios reviewed as per pilot timeline optimization model optimization model • Review socio-economic indicators and 3. Socio-economic impact of enhanced forest financial evaluation principles practices are quantified
  • 9. Enhanced Forest Management Pilot Project Performance Measures Page 7 Table 2 - Technical Advisory Committee Objectives and Measures Objective Activities Performance Measures Targets (date) Selection and • Ensure that WG approach is adequate 1. Feedback to Working Groups on time Ongoing and application of to address pilot site objectives as per pilot project schedule reviewed each year timber supply and • Ensure socio-economic analysis for 2. Economic analysis updates and results socio-economic each pilot area is appropriate for provided on time as per pilot project analysis models meeting objectives schedule Ensure proper use • Provide ongoing training and advice 1. Positive feedback from pilot sites on Ongoing and of models as feasible service requested reviewed each year Provide technical • Provide advice to WGs re: data 1. Prompt response and assistance as Ongoing and advice Pilot requirements requested with positive feedback from reviewed each year Working Groups • Assist with defining alternative forest pilot sites as requested management scenarios to support local objectives Initiate research • Determine research needs 1. Number of research needs identified by Ongoing and to support Working Groups and reviewed by TAC reviewed each year implementation of strategies Share Learnings • Meet with government and industry 1. # Meetings conducted with government Ongoing and with other areas of officials in other areas of the province and industry as per TAC schedule reviewed each year the province
  • 10. Enhance Forest Management Pilot Project Performance Measures Appendix A-1 Table 3 - Pilot Project Evaluation Framework Project Objectives Strategy Performance Measures Target & Data Source/ Frequency of (Goal) (How to accomplish (How to track Acceptable Range Definition Reporting goal) achievement of goal) Increase forest Leading (Performance Driver) productivity in pilot • % of trees planted that are genetically improved site (measured at 5 • % area naturally regenerated year intervals) • % of area meeting free to grow requirements annually • Area of stands treated (fertilizing, spacing, brushing, etc.) • Area harvested by type of silviculture system (clearcut, selection, partial retention) • Disturbance rates and area by type (fire, insects, disease, logging, etc.) Lagging (Outcomes) • Average regen delay (years) • Years to reach free growing status • % increase in MAI (m3/ha) • m3 per ha harvested • % of fibre that meets utilization standards • NSR area as % of annual harvest area • % area of previously uneconomic or inoperable timber brought into production • % area of productive land lost through harvesting (i.e., roads and landings) Develop framework • Forest info system, resource targets and for Implementation performance measures developed within one of Plans and EFM year of pilot commencing Objectives Build EFM Strategy • Action plan for implementation developed on schedule • Database of implementation progress developed on schedule including − forest productivity − employment − forest practices Recommendations/reports presented to steering committee on schedule
  • 11. Enhance Forest Management Pilot Project Performance Measures Appendix A-2 Table 3 - Pilot Project Evaluation Framework Project Objectives Strategy Performance Measures Target & Data Source/ Frequency of (Goal) (How to accomplish (How to track Acceptable Range Definition Reporting goal) achievement of goal) Implement EFM Stakeholder buy-in, and Advisory Group and Strategy (within 3 Steering Committee approval years of commencing pilot) Direct some project Leading (Performance Driver) investments toward • Local expenditures ($/yr) the community • Decrease cutting permit approval process time Lagging (Outcome) • Direct and indirect contribution to local economy ($’s, jobs) • Number of local jobs created in short term and estimate of jobs over the longer term • Increase in local average % of year employed • secure 5 yrs of clear cutting permit approvals by 1999 • develop curriculum package suitable for grade schools Improve forest Leading (Performance Driver) resource • Forest resource inventories initiated, updated or inventories recommended Lagging (Outcome) • Types of resource inventories digitized and available for local resource planning • Ability to adequately measure other resource values to see if land use objectives are being maintained Optimize biological • Carry out field trials, initiate research and make effectiveness recommendations on the timing location and intensity of silviculture practices Maintain forest Leading (Performance Driver) biodiversity of the • maintenance of species diversity and seral stage
  • 12. Enhance Forest Management Pilot Project Performance Measures Appendix A-3 Table 3 - Pilot Project Evaluation Framework Project Objectives Strategy Performance Measures Target & Data Source/ Frequency of (Goal) (How to accomplish (How to track Acceptable Range Definition Reporting goal) achievement of goal) pilot site distribution as per land use objectives Lagging (Outcome) • Forest cover distribution at the landscape level (stand type; pattern (geographic); stage of development) • Birds and mammals - presence/absence, distribution, habitat supply Maximize annual Leading (Performance Driver) return on • Decrease expenditures on treatments/m3 cut investment at • Decrease expenditures on treatments/ha land stand/forest level base Lagging (Outcome) • Return on treatments − MAI − log value • operating earnings/m3 Local forest Leading (Performance Driver) managers and • Stakeholder representation and turnout at local advisory group meetings local • Stakeholder advisory committee input during stakeholders strategy development develop pilot • Number and frequency of meetings project EFM • Local managers sign-off on EFM strategy strategy • Number of notifications, bulletins, articles re: EFM strategy Lagging (Outcome) • Turnover rate and attrition of Working Group and advisory committee over the planning period • Public satisfaction with management plan (survey, media articles, number of complaints)
  • 13. Enhanced Forest Management Pilot Project Performance Measures Appendix B Participants and Interview Guide The following members of the Steering Committee were contacted for their comments on the EFMP project objectives and suggested performance measures: • Rick Brand • Gordon Baskerville • Stan Coleman • Greg Anderson • Barry Phillips • Bronwen Beedle • Dana Bonnieux • Dave Peterson • Don Laishley • Rick Jeffery The following questions served as an interview guide: Project Objectives 1. Five years from now, we will know that the EFM project has been a success if.... 2. What is your main expectation from the project? 3. How will other forest operations in the province benefit from the EFM project? 4. How will the government benefit from the EFM project? 5. What is forest productivity and how will we know if it has been increased through EFM? 6. What is biological effectiveness and how is it important to this project? 7. How does economic efficiency relate to the EFM project? 8. What does “local empowerment” mean and how do you know if it has been encouraged or achieved? 9. If land-use planning and zoning initiatives are successfully “operationalized”, it will be apparent because... The EFM Process 10. What are the key challenges of the Steering Committee and how will it know if it is doing an effective job? 11. What information does the Steering Committee need to review on a regular basis from the TAC and the Working Groups? 12. What are the key challenges of the TAC and how will it know if it is on track with achieving the EFM project objectives? 13. What information does the TAC need from the Working Groups? 14. What objectives and performance measures have been developed to track the success of the pilot site initiatives? (Examples…, discuss…)