Your SlideShare is downloading. ×
CES Toronto 2013 Engaging Practitioners in Evaluation using the Risk Based Compliance Framework
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

CES Toronto 2013 Engaging Practitioners in Evaluation using the Risk Based Compliance Framework

127
views

Published on

This is a paper presentated to CES Toronto 2013. It describes a case study of how a management process is being used to embed effective design of regulatory compliance approaches into practise, and …

This is a paper presentated to CES Toronto 2013. It describes a case study of how a management process is being used to embed effective design of regulatory compliance approaches into practise, and how it makes space for monitoring and evaluation. It highlights the importance of team formation in bringing expertise toegether from across the regulatory system (standard setting, monitoring and behaviour change). It also highlights some of the lessons from the initial application of the framework

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
127
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Risk Based Compliance Framework as anApproach to Engage Practitioners inEvaluationCanadian Evaluation Society Conference 2013Mark WhiteOntario Ministry of the Environment
  • 2. Purpose• Describe the Risk Based Compliance Framework• A case study in the use of a management process toengage practitioners in evaluation• Highlight our learning relevant to the themes ofengagement across boundaries and embeddingevaluation in the organization• It has to be needed... now!• Familiarity breeds... engagement• It helps to be good enough, not perfect• Adapt to the environment• Providing a trigger for evaluation2 Ministry of the Environment
  • 3. 3Risk Based Compliance Framework: OverviewContext•A healthy environment for a strong Ontario•Compliance approach = a combination oftools to monitor compliance and to influencebehaviourAim•To develop compliance approaches thatensure efficient and effective delivery ofregulatory objectivesKey Features•Integrates expertise across organizationalboundaries•Systematically considers influencesBenefits•Focus on evidence based decisions•Focus on communicating the benefits wedeliver3 Ministry of the Environment
  • 4. Risk based Compliance Framework: Steps4
  • 5. The Risk based Compliance Framework and Evaluation5 Ministry of the Environment
  • 6. Performance Monitoring6 Ministry of the Environment• Performance Monitoringunderpinned by logic model A conceptual framework formingthe basis for a managementscorecard Intuitive way of linking activitiesto regulatory objectives Tailored for each complianceapproach• Challenges Outcome measures Limited evidence base Cost effectiveness
  • 7. Engaging Practitioners in Evaluation7 Ministry of the Environment
  • 8. Theme 1: It has to be needed… now!8 Ministry of the Environment
  • 9. Theme 2: Familiarity breeds… Engagement•Relationship between evaluativeprocess and context is importantfor engagement Familiarity with concepts Familiarity with tasks Fit with process•Engagement on easier tasksprovides basis for more complexevaluative tasks9 Ministry of the Environment
  • 10. Theme 3: Good enough, not perfect10** Haynes, L. et al. (2012). Test, Learn, Adapt: Developing Public Policywith Randomised Controlled Trials, Cabinet Office. Behavioural Insights• Working together to improvethe framework Improves the product Improves engagement• Pilot project Test on a ‘Real’ problem Engages project team Engages management• Awareness raising as anopportunity to learn andadapt
  • 11. Theme 4: Adapt to the Environment11 Ministry of the EnvironmentDarwin’s finches**Wikimedia Commons (PublicDomain)• Adapting the framework tomeet the needs of thespecific circumstances Subject matter Tailoring the framework to context Importance of understandinginterdependencies betweencompliance tools Limitations in evidence base Users Integration of different subjectmatter expertise Decision required Most effective & efficientcombination of approaches acrosscompliance program
  • 12. Theme 5: Providing a ‘Trigger’ for Evaluation12 Ministry of the Environment• ‘Trigger’ points forevaluation are important Using a managementprocess to ensure evaluationis considered in a timelyfashion Consideration of what is goodpractice Development of Monitoring andEvaluation Plan prior toapproval
  • 13. Conclusion Evaluation across Boundaries– Functional & Professional Engaging Practitioners in Evaluation– Management process:o Driven by need to promote effectivenesso Approach tailored to the circumstances– Evaluation:o Design Effectivenesso Implementation Effectivenesso Outcome Effectiveness13 Ministry of the EnvironmentQuestions?mark.white@ontario.ca