Anna Dickinson REF Update from HEFCE


Published on

Published in: Education, Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Anna Dickinson REF Update from HEFCE

  1. 1. The Research Excellence Framework Anna Dickinson REF Team DARTS June 2012
  2. 2. Presentation outline• Overview• Staff• Outputs• Impact• Environment• REF panels• Submissions
  3. 3. Overview: Purpose of the REF • The REF is a process of expert review • It replaces the RAE as the UK-wide framework for assessing research in all disciplines • Its purpose is: - To inform research funding allocations by the four UK HE funding bodies (approximately £2 billion per year) - Provide accountability for public funding of research and demonstrate its benefits - To provide benchmarks and reputational yardsticks
  4. 4. Overview: The assessment framework Overall quality Outputs Impact Environment Maximum of 4 outputs Impact template and Environment data and per researcher case studies template 65% 20% 15%
  5. 5. Overview: The REF process Criteria phase Submissions phase Assessment phase 2011 2012-13 2014 • Develop and publish • HEIs submit Codes of • Panels assess Guidance on submissions practice submissions (Jul) • Launch the REF • Publish outcomes • Develop, consult on and submissions system Dec 2014 publish Panel criteria (Jan 2012) • Submission deadline 29 Nov 2013
  6. 6. Overview: Guidance and criteria Comprehensive information and guidance is set out in: • Assessment framework and guidance on submissions (July 2011): - Sets out the information required in submissions and the definitions used • Panel criteria and working methods (Jan 2012): - Sets out how panels will assess submissions The above documents set out the official guidelines for the REF. These slides provide a summary of key points but do not provide or replace the official guidelines.
  7. 7. Overview: Submissions • Each HEI may submit in any or all of the 36 units of assessment (UOAs) • Each submission in a UOA provides evidence about the activity and achievements of a ‘submitted unit’ including: - Staff details (REF1a/b/c) - Research outputs (REF2) - Impact template and case studies (REF3a/b) - Environment data (REF4a/b/c) - Environment template (REF5) • A submitted unit may, but need not, comprise staff who work within a single ‘department’ or organisational unit
  8. 8. Overview: Assessment • Submissions will be assessed by 36 sub-panels working under the guidance of 4 main panels • Panels will carry out the assessment according to the published criteria and working methods Sub-panel responsibilities Main panel responsibilities • Contributing to the panel • Developing the panel criteria criteria and working methods and working methods • Assessing submissions and • Ensuring adherence to the recommending the outcomes criteria/procedures and consistent application of the overall assessment standards • Signing off the outcomes
  9. 9. Staff
  10. 10. Staff: Staff selection • HEIs are responsible for selecting eligible staff whose outputs are to be included in their REF submissions • Each HEI is required to develop, document and apply a code of practice on the fair selection of staff: - Demonstrating principles of transparency, consistency, accountability and inclusivity - Guidance on developing the codes has been based on good practice found in the 2008 RAE - The code must be submitted to the REF team, will be examined for adherence to the published guidance, and will be published at the end of the exercise
  11. 11. Staff: Individual staff circumstances • Up to four outputs must be listed against each individual • The number of outputs can be reduced without penalty where an individual’s circumstances have constrained their ability to work productively or produce four outputs in the REF period • We have sought to make these arrangements as clear and consistent as possible, with due regard to confidentiality
  12. 12. Staff: Clearly defined circumstances • These are circumstances involving a - Early Career clear ‘absence’ from work researchers • ‘Tariffs’ define the number of outputs that - Part-time working, may be reduced without penalty career breaks and • These will be applied consistently by all secondments REF sub-panels outside of HE • Circumstances can be combined up to a maximum reduction of three outputs - Periods of maternity, adoption and • Where an individual has a combination of additional paternity clearly defined and complex leave circumstances, these should be submitted collectively as ‘complex’
  13. 13. Staff: Complex circumstances - Disability - Ill health or injury - Mental health • For these circumstances a judgement is conditions needed about the appropriate reduction • The EDAP will consider all these cases - Additional on a consistent and confidential constraints related basis, and recommend the appropriate to bringing a child reductions to the Main Panel Chairs into the family • Sub-panels will be informed of the - Other caring decisions and will not have access to responsibilities further details - Gender • ECU has published worked examples reassignment ( - Other circumstances related to legislation
  14. 14. Outputs
  15. 15. Outputs: Research outputs • Panels will assess the quality of research outputs through a process of expert review • All forms of output that embody research will be assessed on an equal footing • Panels will assess the quality of outputs, not the contribution of individual researchers • A co-authored output may be listed against one or more individuals that made a substantial research contribution to it (no more than twice within the same submission) • Institutions may request ‘double-weighting’ for outputs of extended scale and scope
  16. 16. Outputs: Additional information • Several sub-panels will make use of citation data as a minor component to inform peer-review • HEIs will be provided access to the Scopus citation data (in the relevant UOAs) through the REF submission system • Several sub-panels invite additional information from the HEI to inform judgements • Panels will not use journal impact factors, rankings or lists or the perceived standing of the publisher
  17. 17. Outputs: Assessment criteria The criteria for assessing the quality of outputs are originality, significance and rigour* Four star Quality that is world-leading in terms of originality, significance and rigour Quality that is internationally excellent in terms of originality, Three star significance and rigour but which falls short of the highest standards of excellence Two star Quality that is recognised internationally in terms of originality, significance and rigour One star Quality that is recognised nationally in terms of originality, significance and rigour Quality that falls below the standard of nationally recognised Unclassified work. Or work which does not meet the published definition of research for the purposes of this assessment * Each main panel provides descriptive account of the criteria
  18. 18. Impact
  19. 19. Impact: Definition of impact • Impact is defined broadly for the REF: an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia • Panels recognise that impacts can be manifest in a wide variety of ways, may take many forms and occur in a wide range of spheres, in any geographic location • Panels provide examples of impact relevant to their disciplines, intended to stimulate ideas - not as exhaustive or prescriptive lists
  20. 20. Impact: Some examples of impact Enhanced professional Improved health or standards, ethics, guidelines Improved risk welfare outcomes or training management Public debate has been shaped or Improved More effective Improved business informed by research quality, accessibility or management or performance efficiency of a public service workplace practices A social enterprise Research has enabled initiative has beenChanges to the Production costs have stakeholders to challenge createddesign or delivery of reduced conventional wisdomthe school curriculum Improved forensic Enhanced Improved access to methods or expert Policy debate or decisions preservation, conservation or justice, employment systems have been influenced or presentation of cultural or education shaped by research heritage Improved management or Research has informed conservation of natural Organisations have Jobs have been public created or protected resources adapted to changing understanding, values, attitud cultural values es or behaviours Enhanced corporate The policies or activities of Changes to Levels of waste have social responsibility NGOs or charities have been legislation or reduced policies informed by research regulations New forms of artistic Changes in Enhanced technical A new product has expression or changes to professional practice standards or been commercialised creative practice protocols
  21. 21. Impact: Submission requirements • Sets out the submitted unit’s general Impact template approach to supporting impact from (REF3a) its research: • Approach to supporting impact during 20% of the the period 2008 to 2013 impact sub-profile • Forward strategy and plans • Specific examples of impacts already achieved, that were underpinned by Case studies the submitted unit’s research: (REF3b) • 1 case study per 10 FTE staff 80% of the submitted (plus 1 extra) impact • Impacts during 2008 to 2013; sub-profile underpinned by research since 1993
  22. 22. Impact: Case studies • Each case study should: - Clearly describe the underpinning research, who undertook it and when - Provide references to the research and evidence of quality - Explain how the research led/contributed to the impact - Clearly identify the beneficiaries and define the impact - Provide evidence/indicators of the impact - Provide independent sources of corroboration • All the material required to make a judgement should be included in the case study • Submitted case studies need not be representative of activity across the unit: pick the strongest examples
  23. 23. Impact: Assessment criteria The criteria for assessing impacts are reach and significance* Four star Outstanding impacts in terms of their reach and significance Three star Very considerable impacts in terms of their reach and significance Two star Considerable impacts in terms of their reach and significance One star Recognised but modest impacts in terms of their reach and significance The impact is of little or no reach and significance; or the impact Unclassified was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit * Each main panel provides descriptive account of the criteria
  24. 24. Environment
  25. 25. Environment: Environment template • Each submission to include a completed template: - Overview - Research strategy - People, including: - staffing strategy and staff development - research students - Income, infrastructure and facilities - Collaboration and contribution to the discipline or research base • The ‘panel criteria’ request specific types of evidence under each heading, and indicate how much weight they will attach to each component
  26. 26. Environment: Environment data • All submissions to include data on: - Research doctoral degrees awarded (REF4a) - Research income (REF4b) - Research income in-kind (REF4c) • Definitions are aligned with HESA returns; the data relate to the ‘whole unit’ - not just submitted staff • Some sub-panels request specific additional data, to be included within the environment template (REF5) • Data will be considered by panels alongside the narrative information provided in the relevant section of the environment template
  27. 27. Environment: Assessment criteria The criteria for assessing the environment are vitality and sustainability* Four star An environment that is conducive to producing research of world-leading quality, in terms of its vitality and sustainability An environment that is conducive to producing research of Three star internationally excellent quality, in terms of its vitality and sustainability An environment that is conducive to producing research of Two star internationally recognised quality, in terms of its vitality and sustainability An environment that is conducive to producing research of One star nationally recognised quality, in terms of its vitality and sustainability Unclassified An environment that is not conducive to producing research of nationally recognised quality * Each main panel provides a descriptive account of the criteria
  28. 28. Submissions
  29. 29. Overview: Multiple and joint submissions • Institutions will normally make one submission in each UOA they elect to submit in • Joint submissions are encouraged where this is an appropriate way of describing collaborative research • An institution may make multiple submissions in a UOA only by exception and with prior permission
  30. 30. Pre-submission• Codes of practice Submission by Response from funding body by 27 April 2012 6 July 2012 31 July 2012 12 October 2012• Requests for multiple submissions or case studies requiring security clearance Request by Response from REF team by 27 April 2012 8 June 2012 28 September 2012 9 November 2012 7 December 2012 18 January 2013• Survey of submission intentions Invitation and guidance Online survey open July 2012 Early Oct – early Dec 2012
  31. 31. Overview: The submission system • All submissions must be made through the REF submission system:  Pilot available to all HEIs: Sep 2012  Open for submissions: Jan–Nov 2013 • Each HEI to set up system users and user permissions • All data may be entered onto the system and/or bulk imported • HEI contacts have seen a demonstration of the system • User guidance and support will be provided
  32. 32. Access to REF4 data• We will provide HESA data on research income and doctoral degrees awarded in stages: - May 2012: Data for 2008-09, 2009-10 and 2010-11 - Apr 2013: Data for 2011-12 - Doctoral degrees data for 2012-13 will be available from HESA when the student record is submitted• Data on research income-in-kind will be provided by the Research Councils and health research funders• Institutions will need to allocate these data to the appropriate UOAs; or use their own sources• The submission system will validate submitted data against the data we provided, at HEI level• stem
  33. 33. Further information (includes all relevant documents)Enquiries from staff at HEIs should be directed to their nominated institutional contact (see for a list) Other enquiries to