Primary Processes and the Pursuit of Quality: Past and Present Perspectives

  • 348 views
Uploaded on

This lecture discusses one of the primary processes: evaluation. The presenter discusses evaluation schemes and the criteria needed as well as how quality is related to primary processes.

This lecture discusses one of the primary processes: evaluation. The presenter discusses evaluation schemes and the criteria needed as well as how quality is related to primary processes.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
348
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
8
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Primary processes and the pursuit of quality: Past and present perspectives Bjørn Stensaker
  • 2. Past perspectives and debates….
    • Evaluation as a phenomenon in general emerging after WWII
    • a) Overarching belief: ”Social engineering”, positivism, quantification
    • b) Two forms of evaluation schemes:
    • - PPBS (future oriented, policy-relevant techniques: cost-benefit analysis, system analysis, etc)
    • - Social programme evaluation (the controllable experiment)
  • 3. Past perspectives and debates cont.
    • Critique of positivism (is man really rational? Is generalisation possible when the context change?
    • Change of perspective:
    • - from a belief that those in charge knows best..(”outside perspective”)
    • - to a belief grounded in enlighenment of those implementing reforms/practitioners (interpretative, hermeneutic perspectives -> extreme case: social constructivism) (”inside perspective”)
    • - from that time, one can identify a continuing debate between proponents of the two perspectives
  • 4. Current perspectives and debates
    • Central issue in the debate: how to deal with values? (what criteria should be used as ”benchmark”?)
    • - validisation: is it true?
    • - valorisation: is it good?
    • Dominant evaluation perspectives in use:
    • - Evaluation ”in context”
    • - From summative to formative evaluation
    • - Re-instate rationality (evidence-based approach)
    • - Pluralist approach (using competing theories and perspectives)
  • 5. Evaluation in Higher Education today
    • Evaluation schemes a reflection of a changing policy context:
    • - downsizing and transforming welfare states (relevance, ”the evaluative state”)
    • - resource problems (growth in number of HE-students) (efficiency)
    • - accountability issues (effectiveness)
    • Quality = relevance, efficiency and effectiveness put together?
  • 6. Evaluation: a key to understand the primary processes of HE
    • Evaluation used to:
    • - determine reputation (rankings..)
    • - resource allocation
    • - appraisal of knowledge (peer review)
    • - certification of students
    • - legitimation of academics
    • - ranking of students, academics and institutions
    • - promotion to academic posts
    • - etc, etc….
  • 7. The different underpinnings of peer review
    • Is peer review:
    • - based on general consensus of values and understandings (within the discipline)
    • - just a process to enforce the autonomy, creativity and integrity of the individual academic?
    • - a conservative process emphasising values of the past, unable to recognise and appriciate innovation?
    • - or, a power struggle between competitive groups within certain disciplines or fields
  • 8. The development of peer review:
    • ” extended peer review”
    • - new groups of academics involved (need for an ”outside view” of new academic spesialities/tribes and territories)
    • - people from business, industry or society more involved
    • - students are given a voice as members of evaluation panels
    • - ”the ordinary man” included in evaluations where ”big issues” are dealt with (technology assessment, etc)
    • More focus on legitimacy of the process? Evaluation as a political process
  • 9. From Method (evaluation) to Content (quality)
    • How to evaluate what academics do? (directly, indirectly, quantitative, qualitative)
    • What form of QA is most appropriate for HE? (control, enhancement, finding a balance?)
    • How can academic standards be secured and assessed? (examinations, relevance)
    • How can the outcome of HE be assessed? (employment, relevance, ”happiness”, democracy)
    • What are the links between QA and the improvement of teaching and learning? (is there a causal model, poor methodology)
  • 10. Studies of quality focuses on four central themes:
    • Course evaluation
    • Grading and outcomes
    • National monitoring practices
    • System standards
  • 11. Course evaluations
    • Are current course evaluation techniques effective?
    • - ”happy sheets” (problem with response rates)
    • - ”student engagement questionnaires”
    • What is measured in student assessment of teaching?
    • Shevlin et al (2000): is there a ”halo effect” in student evaluation of their teachers? Charismatic teachers receive better assessments…(but, can this research be trusted, or is the result dependent on the design of the study..?)
    • Wiers-Jenssen et al (2002): what trigger student satisfaction? Academic quality..(but also institutional size, buildings, etc..)
  • 12. Grading and outcomes
    • How do/should lectures assess their students?
    • - coursework receive higher grading than formal examinations
    • - differences between academic fields
    • - argument for external examiner schemes (Norway, Denmark and the UK)
    • - cultural differences between countries (Bologna)
    • - In general, grading not high on the institutional/public policy agenda (how are teachers trained in grading procedures…?)
  • 13. National monitoring practices (NMP)
    • Research and teaching monitored independently, and with very different consequences
    • What is the effect of NMP?
    • - shifts in power distribution
    • - bureaucratisation
    • - permeability (exposure of HE)
    • - public relations (active use of rankings to establish a positive image externally)
    • Very few studies show a link between NMP and improvement in teaching and learning
  • 14. System standards
    • What is the role of evaluation in HE?
    • - an instrument to instigate change?
    • - an instrument to monitor change?
    • Why this focus on ´quality´?
    • - how should ”good” QA-systems look like?
    • - what is the role of various actors in such systems?
    • - how can current QA-systems be transformed?
    • Conclusion: quality is an integrated part of the governance of the HE-sector