Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PhD Confirmation PP


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

PhD Confirmation PP

  1. 1. James Herbert (Bachelor of Arts, Master of Criminal Justice)
  2. 2. EBP and Government EBP and Evaluation Definitions Evaluation Use From Use to Influence Research Questions & Aims Methodology Ethics Applications
  3. 3.  The Rising Prominence of Evaluation.  What Does Evaluation Actually Do?
  4. 4. “Evidence-based policy making is at the heart of being a  reformist government... We cannot afford a Public Service culture where all you do is tell the Government what it wants to hear” – Prime Minister Kevin Rudd addressing the Australian Public Service Commission. Restore Scientific  Integrity to the White House: Restore the basic principle that government decisions should be based on the best-available, scientifically-valid evidence and not on ideological predispositions.
  5. 5.  The Most Accessible Form of Evidence.  The Most Common Form of Evidence.  Practical and Operationally Oriented.
  6. 6.  Very Applied Research  Pragmatic  Focused on Operational Issues  Looks to Provide Direct Benefit and Value
  7. 7.  Dedicated Set of Resources and Activities Directed Towards Social Change  Programs Can Become Institutionalised Over Time
  8. 8.  The Degree to Which the Program can Continue to Function  Falls On a Continuum Between a Program Ending and the Program Expanding into Other Jurisdictions
  9. 9.  Instrumental Use Recommendations made by evaluators  are implemented Tends to not be direct, over long periods  of time with many indirect agents Not particularly common; politics tends  to supersede Example 
  10. 10.  Symbolic Use Evaluations are used to legitimise a  pre-empted course of action  Does not include cases where evaluation evidence is distorted, misused, or selectively disseminated  Example
  11. 11.  Conceptual Use Individual evaluations do not provoke  a change, evidence accumulates and causes an enlightenment  The most powerful way by which evaluations can affect change
  12. 12.  Imposed Use Legislative or regulatory requirement for  empirically tested programs United States Government Educational  Programs  Process Use The activity of engaging in evaluations  affects change separate to the outcome of the evaluation
  13. 13. Use has a long history in the study of  evaluation, however:  It lacks distinctiveness and firm definition  It fails to recognise change processes at the individual, interpersonal, and organisational levels  Use does not recognise the higher aim of evaluation as social betterment  Use suggests a directness and intentionality that rarely exists
  14. 14. Henry and Mark’s (2003) theory of change spans  from individual, interpersonal, and organisational levels providing a comprehensive definition The model enhances links to other more developed  areas of research Recognises the goal of evaluation as social  betterment Allows for the examination of influence through  different levels Can improve the validity of studies looking at the  outcomes of evaluation Allows for a more realistic assessment of what  evaluation actually does
  15. 15.  Henry & Marks (2003) provide a framework in which to understand change processes  Changes at different levels can act as a catalyst for further change  Examining ‘chains of influence’ may provide some insight into dynamic change processes  The narrative of the change  D.A.R.E. Example
  16. 16.  Social improvement is contingent on sustainability  The program sustainability model enhances the evaluation influence model
  17. 17.  Does evaluation influence how programs function? If so, how?  Does evaluation influence affect the sustainability of a social change program? If so, how?  What program characteristics affect evaluation influence?
  18. 18.  To discover what evaluation does for programs and the management of public/private/philanthropic resources  To find ways to improve the way evaluation is used in order to increase the efficiency and effectiveness of program resources
  19. 19.  Multiple Case Study Approach  4-7 Social Change Programs  Aim: To Generate and Test Theory  Replication Strategy  Explanatory & Chronological Analysis
  20. 20.  Social Change Programs that Have Been Evaluated in the Last 12 Months  Include cases with a diverse mix of characteristics and sustainability factors  Develop an extensive list of potential programs to include
  21. 21. Host Host Program Locatio Program Name Organisation Program Type Evaluation Funding Source Website Contact Organisation Description n Type Involves camps TaskForce and physical Head Office activities for Website Delinquency http://www.ta PO Box 369 Outdoor young people claims the Diversion. Donations, Bequests, Prahran Vic. Adventure Taskforce NGO aimed at VIC program has Adventure Fundraising. /ss_oat.htm 3181 Therapy developing skills, been Therapy. 9521 4000 confidence and evaluated. enquiries@tas pro-social interactions. University to school peer Ian Potter Foundation, mentoring. BP, Department of http://about. Science and Supporting Education and Training, star@central. Technology Murdoch Professional University science education. Education. WA Youth Affairs, au/star/peer_t Awareness University evaluation. Academic Woodside, Social utoring/peertu au Raising (STAR) support, positive Ventures Australia, Rio te.html role models and Tinto WA Future Fund. first hand advice. Swan Education District and the Beacon Academic and Website Foundation with the employment gives support of the WA and http://www.b Tiesha McGill Midland Education and Swan Education support for young comparative Australian eaconfoundati tiesha@beaco Indigenous Government employment WA District Indigenous people testing governments, local nfoundation.n Youth Project support. in the Midlands figures for employers and Beacon projects.php et area. outcomes. sponsors, Newmont, Readymix and Sinclair Knight Merz.
  22. 22. Evaluation Characteristics  Positive/Neutral/Negative Performance  Internal/External Evaluation  Sustainability Performance  Ceased/Reduced/Sustained/Expanded  Public Profile  Popular or Community Support Exists/Neutral/Unpopular or  Community Unsupportive Program Type   Justice/Education/Health/Community Location   WA/VIC/NSW/QLD Host Organisation/ Funding Bodies   State Gov/Federal Gov/NGO/University/Private Companies
  23. 23. Protocol: What will be done in each case  Interviews with program staff, evaluators, relevant  people within host organisation, relevant people within funding bodies  Analysis of key documents: Evaluation report, evaluation tender/scope documents  Triangulation of data  Member Checking
  24. 24. Findings for each case:   Evaluation influence (within program, host organisation, funding body)  Evaluation influence on sustainability of the program  Findings for across cases:  Possible common rules for how evaluation influence functions within programs  How different characteristics affect evaluation influence (program & evaluation characteristics)
  25. 25. Agreed terms and conditions between the  programs, host organisations, funding bodies and the researcher Consultation after data collection with participants and  organisations involved Voluntary participation  Due discretion with sensitive materials  (contracts, tenders etc) Available counselling and contact details of relevant  organisational support (internal witnesses officer, equity officer)
  26. 26. Improved understanding of the evaluation process and  the extent of its effects Improved understanding of how evaluation is used in  programs, different types of organisations, and funding bodies Improved understanding of what program  characteristics are associated with evaluation influence and how evaluation influence functions Recommendations on how researchers can maximise  the influence of their findings Recommendations on how to manage the effects of  evaluation