Your SlideShare is downloading. ×
HANDBOOK FOR THE ASSESSMENT   OF AoC BEACON AWARDS          APRIL 2012                              1
CONTENTSIntroduction                                               Page 3Aim of the Beacon Awards                         ...
AoC BEACON AWARDS    HANDBOOK FOR THE ASSESSMENT OF AoC BEACON                    AWARDS                            Introd...
The final decision on the awards is made by the Beacon AwardsSteering Group. The assessment process is subject to qualitya...
Aim of the AoC Beacon AwardsThe aim of the AoC Beacon Awards programme is to highlight thebreadth and quality of education...
General criteria for Beacon Awards and the structure of                              applicationsThe Beacon Awards recogni...
SponsorshipAwards are made by around 15 sponsors; the awards are dividedinto three categories: Teaching and learning, curr...
COLLEGE TO REMOVE ANY REFERENCES IF THEY CAN BEIDENTIFIED.Submissions should be structured under the following headings: ...
Initiatives which have previously won a Beacon Award are noteligible for consideration for the same award a second time. A...
Assessors are appointed following registration and the approval ofthe Chief Assessor and Manager. Assessors are expected t...
assessors jointly assess each application and complete a standardselection analysis form (SAF)(see appendix 2). The SAF re...
Colleges should ensure that they present clearly to assessors how theproject meets the specific criteria for the award, as...
the President of the AoC Charitable Trust so that he or she candecide if any should receive a ‘President’s Award’.Detailed...
 “C” – project fails to meet important criteria and is not        suitable for shortlisting   The two assessors assigned...
 Completed SAFs should be returned to the Manager for all  Colleges by Wednesday 8th August 2012 for approval. The form  ...
improvement in submissions and the rationale for the final  shortlist. This part of the report should be sent to the Manag...
the Colleges consider to be excellent work. Assessors are not inspectors but are there to listen, to learn and to  ask qu...
have entered for the award or been shortlisted     Assessors should not tell Colleges what their recommendations      are...
 Assessors should recommend one winner. In the unusual case of  two applications being of absolutely equal merit the asse...
 if necessary, contacts assessors through the Manager for       clarification or further comment      produces a short c...
 Award winners will be announced at the AoC’s annual conference  between 20-22 November 2012.                            ...
Quality Assurance and Evaluation of AssessmentThe Chief Assessor is responsible to the Beacon Awards SteeringGroup for the...
Moderation is carried out in two stages:(i)    End of first stage assessment - moderation of initial assessment       (sho...
The form should be completed by the College and returned to theBeacon Awards Manager by 17 October 2012 . The forms aresum...
Appendix 2. Assessment Forms  Below are two forms which assessors should use to assist them in the  reporting of the asses...
Project outcomes: (identify the key benefits to learners and the value ofthe initiative to other Colleges)Overall Strength...
AoC BEACON AWARDS                                  SELECTION ANALYSIS FORM FOR SHORT-LISTING APPLICATIONSTitle and year of...
AoC BEACON AWARDS                             SELECTION ANALYSIS FORM (PART2 SPONSORS CRITERIA ASSESSMENT)Title and year o...
AoC BEACON AWARDS                                      SELECTION ANALYSIS FORM (PART3 SUMMARY ASSESSMENT)Title and year of...
Appendix 3. Sample confirmation letter to shortlisted CollegesContact name and addressDateDear (Contact)AoC Beacon Awards ...
you best with the minimum disruption to you and yourcolleagues.Please confirm your receipt of this letter and that you are...
Appendix 4. College Evaluation form                     AoC BEACON AWARDS: EVALUATION OF ASSESSMENT                TO BE C...
We were satisfied that:                                                                        Strongly          Agree    ...
Appendix 5. Assessors’ Fees and ExpensesAoC Beacon Assessors are entitled to claim a daily rate plus reasonableexpenses. A...
Appendix 6:    CHECK LIST OF DOCUMENTS TO BE SENT TO THE BEACON                             AWARDS OFFICE                 ...
 Current knowledge of the further education sector i.e. direct    experience of the sector outside the AoC Beacon Awards ...
Upcoming SlideShare
Loading in...5
×

AoC Beacon Assessor Handbook 2012/2013

591

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
591
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "AoC Beacon Assessor Handbook 2012/2013"

  1. 1. HANDBOOK FOR THE ASSESSMENT OF AoC BEACON AWARDS APRIL 2012 1
  2. 2. CONTENTSIntroduction Page 3Aim of the Beacon Awards Page 5General criteria for Beacon Awards and the structure of Page 6applicationsAssessment of Beacon Awards: overview of assessment Page 9Assessment process : procedures at each assessment stage- Stage 1 Shortlisting Page 13- Stage 2 Site visits Page 16- Stage 3 Final decisions Page 19Quality Assurance of Assessment Page 22Evaluation of Assessment Page 23Appendix 1. Members of the Beacon Awards Steering Page 24GroupAppendix 2. Assessment forms Page 25Appendix 3. Sample shortlist confirmation letter Page 30Appendix 4. Evaluation form Page 32Appendix 5.Assessors fees and expenses Page 34Appendix 6. Checklist of forms for assessors Page 35Appendix 7. Appointment of assessors Page 35 2
  3. 3. AoC BEACON AWARDS HANDBOOK FOR THE ASSESSMENT OF AoC BEACON AWARDS IntroductionThe Beacon Awards programme is governed by the AoC CharitableTrust which is a registered charity, and is administered by theAssociation of Colleges (AoC). The Beacon Awards Steering Groupprovides advice and guidance on the delivery and development ofthe programme. The Steering Group is made up of representativesfrom Colleges, funding bodies and sponsoring organisations. TheSteering Group, acting as the Assessment Board, makes the finaldecisions about award winners in the light of recommendationsfrom the team of assessors and the Chief Assessor.This handbook is intended to provide guidance on applying for anAoC Beacon Award and on the assessment for the awards. Itprovides details of the processes involved in the assessment ofsubmissions and of the conditions attached to the submission of theawards. It is intended that the guidance in this handbook willensure that parity is achieved between awards in terms of thestandard of the awards and in quality of the assessment of awards.Assessors are responsible to the Manager of the AoC CharitableTrust for analysing submissions, carrying out site visits, writingreports on these visits and making recommendations for awards.Individuals with suitable experience and current knowledge of thework of Further Education Colleges are invited to apply to join theregister of Beacon Assessors. Colleges winning a Beacon Award areinvited to nominate a member of staff associated with the winningsubmission to be considered for appointment as a Beacon Assessor. 3
  4. 4. The final decision on the awards is made by the Beacon AwardsSteering Group. The assessment process is subject to qualityassurance and to an evaluation.The Manager of the AoC Charitable Trust is Alice Thiagaraj – 2-5Stedham Place, London WC1A 1HU, Phone: 020 7034 9917, fax: 0207034 9950Email: alice_thiagaraj@aoc.co.ukThe Chief Assessor is Roger Marriott – phone: 01908 567374.Email: johnroger.marriott@btinternet.com 4
  5. 5. Aim of the AoC Beacon AwardsThe aim of the AoC Beacon Awards programme is to highlight thebreadth and quality of education in Colleges throughout the UK andincrease understanding of Colleges’ contribution to UK educationalskills policy and economic and social development.The awards: Recognise imaginative and exemplary teaching and learning practice in Colleges Draw attention to provision which encourages and supports learners to approach challenges positively and creatively Support learning and continuous improvement through the dissemination of award-bearing practice Promote and celebrate the further education sector’s commitment to pioneering approaches to equality and diversityThe programme represents partnership in action by providingsignificant benefits for the major players involved: An award acts as a development grant to help realise the fullpotential of a project based in a College of further education An award serves to promote the business needs and interests ofthe sponsor.The dissemination of the exemplary practice represented by theaward-winning projects is a major purpose of Beacon Awardsprogrammes. Project profiles of winning Colleges are written upand disseminated to the FE sector after their announcement. TheManager maintains a website for the awards which contains muchvaluable information on the awards scheme and lists of winningprojects. The AoC web site can be found at www.aoc.co.uk. 5
  6. 6. General criteria for Beacon Awards and the structure of applicationsThe Beacon Awards recognise exemplary initiatives (which can beprogrammes, courses, projects or other forms of teaching, learning,advice, guidance or support). The initiatives should recognise andpromote the interdependence of Colleges in the further educationsector and of business, professional and voluntary organisations totheir mutual advantage.General Beacon Award criteriaTo be eligible for an award, the initiative should show evidence ofimaginative yet sustainable teaching and learning practice or otherrelevant provision. The application can be for an initiative whichcan be a programme, course, project or other form of teaching,learning, advice, guidance or support.The application must fulfil the following criteria:(i) meet the specific requirements set out by the sponsors of the award to which the submission relates(ii) have regard to promoting equality and diversity in the delivery of the programme(iii) be subject to evaluation/ quality assurance to influence the continuing development of the initiative(iv) have been running for at least one academic year by 31 July 2012(v) have features which actively promote exemplary teaching and learning(vi) be of benefit to one or more groups of students or trainees who are identified and described in the application(vii) have wider relevance and applicability which would make it of value to other Colleges as an example of good practice or innovation 6
  7. 7. SponsorshipAwards are made by around 15 sponsors; the awards are dividedinto three categories: Teaching and learning, curriculum design anddevelopment; Leadership and quality improvement;Responsiveness, partnership and impact (see current prospectus fordetails of the awards).Awards are designed at the discretion of the sponsors either as UK-wide awards or as awards for Colleges in England, NorthernIreland, Scotland or Wales and take the form of monetary grants of,on average, between £3,000 and £5,000 which are awarded on theclear understanding that the total award monies will be set againstexpenditures incurred in taking forward the development anddissemination of each winning initiative.It is a condition of entry that each College gaining an awardundertakes to inform its local community about its achievement andabout the support and interest provided by the sponsor.Structure of the applicationSubmissions should address both the general criteria for BeaconAwards and any specific criteria of the sponsor. The submissionshould be no more than 3000 words and should come from a seniormember of staff who has had close contact with the initiative.Written evidence from beneficiaries should be included in the wordlimit. Reference can be made to other materials (i.e. multimediamaterial, College documents etc.) which the assessors can request,should they wish to do so. WHILST DETAILS OF THE COLLEGEMUST BE SUPPLIED ON THE APPLICATION FORM THEMAIN BODY OF THE SUBMISSION SHOULD BEANONYMOUS. SUBMISSIONS MAY BE RETURNED TO THE 7
  8. 8. COLLEGE TO REMOVE ANY REFERENCES IF THEY CAN BEIDENTIFIED.Submissions should be structured under the following headings: Project summary – a brief overview of the initiative including aclear description of target group Aims and objectives – how they were established and how theyhave resulted in the promotion of exemplary teaching and learning How the project meets the sponsor’s criteria – a section whichclearly addresses each of the sponsor’s criteria as set out on therelevant page in the Prospectus Project management – a description of how the project ismanaged including how quality assurance is used to improve theinitiative, and how the project meets equality and diversitystandards Outcomes and benefits to learners – how the initiative hasbenefited learners and others involved with the project – this shouldbe supported by written evidence from beneficiaries who may bestudents, trainees, employers or, in some cases, parents. This sectionshould also include data on enrolment, retention, achievement,progression and how this project shows a commitment to equalityand diversity. Dissemination and the future – key features of the project shouldbe highlighted demonstrating how it can benefit other Colleges andhow it will develop in the futureThe submission must be accompanied by a completed applicationform signed by the Principal/ Chief Executive of the College. 8
  9. 9. Initiatives which have previously won a Beacon Award are noteligible for consideration for the same award a second time. ACollege may apply for as many awards as it wishes but it maysubmit only one application per award. A College which has aproject which falls into two or more categories may submit thatproject for only one award.Applications are treated as strictly confidential to the SteeringGroup, Assessors and the Manager. Material from any application ismade public only with the express approval of the Collegeconcerned. All applications must be received by 4 July 2012.The Manager sends an acknowledgement for each application to thePrincipal/ Chief Executive. Colleges are subsequently contactedonly if the project is shortlisted.Award winning Colleges are permitted to use the Beacon Awardslogo, with the year of the programme beneath it, on College headedpaper and promotional material.It is expected that within approximately six months of theannouncement of the award, the winning College will organise alocal presentation ceremony in consultation with the sponsor toallow for maximum media coverage; the Beacon Awards office at theAoC must be informed of the arrangements. Assessment of Beacon Awards Overview of AssessmentAssessment of the applications for Beacon Awards is theresponsibility of specialist assessors co-ordinated by the Managerand supported by the Chief Assessor. Normally, two assessors areassigned to each award one of whom may represent the sponsor. ABeacon Assessor (appointed by the Manager) acts as the LeadAssessor for each award. 9
  10. 10. Assessors are appointed following registration and the approval ofthe Chief Assessor and Manager. Assessors are expected to havesubstantial experience of Further Education. Assessors are requiredto inform the Manager of any potential conflict of interest associatedwith the Awards they are assessing and/or the Colleges applying forthose Awards.The assessment process has three stages – short listing, anassessment visit to the College and the final decision by the BeaconAwards Steering Group. Assessors complete confidential writtenreports at each stage. The reports are subject to moderation andquality assurance by the Chief Assessor. Colleges short-listed (i.e.those visited) for an award are asked to complete an evaluation formproviding feedback on the arrangements for assessment.Assessors are expected to respond to queries from the ChiefAssessor on their reports and recommendations in order that theChief Assessor may present an overview of their reports andrecommendations to the Beacon Awards Steering Group.The Chief Assessor completes an annual report which makesrecommendations for the awards to the Beacon Awards SteeringGroup, and reports on the efficacy of the arrangements forassessment. The Beacon Awards Steering Group, acting as theassessment board, makes the final decision on awards and proposesa small number of awards to be considered for the PresidentsAward.Stage 1 Short listingThe initial assessment, leading to short listing, has regard to thegeneral criteria for Beacon Awards and the specific criteria for eachaward (see the current prospectus). The outcome of this stage is alist of applications for each award that best meet these criteria. Two 10
  11. 11. assessors jointly assess each application and complete a standardselection analysis form (SAF)(see appendix 2). The SAF requiresassessors to identify the extent to which an application meets thegeneral Beacon criteria and the specific criteria for the award.Assessors are required to include comments identifying points ofmerit and any shortcomings in the submission.The short-list is reviewed by the Chief Assessor and once approvedassessors contact the short-listed Colleges to arrange an assessmentvisit. Colleges which are not short-listed may contact the Managerafter the awards have been announced for feedback on theirsubmission.Stage 2 Assessment visitsAssessment visits take place in September and early October and areintended to match the ‘evidence on the ground’ at the shortlistedCollege with the application. Visits will normally involve attendingthe College for 4-5 hours and are conducted by two assessors. Thelead assessor will contact the College to outline the generalarrangements for the visit. Assessors should make clear anyrequirements for additional information prior to the visit. TheManager will confirm the arrangements for the visit in a letter to thePrincipal.Assessors are interested in exploring with the College, its staff,students and stakeholders the benefits of the project to learners.They will be mainly interested in the impact of the project on theexperience of the students, particularly any outcomes for learningincluding achievements in terms of skills and/or qualificationsgained. Assessors will also wish to explore how the project ismanaged including, its impact on equality and diversity, methods ofevaluation, quality assurance, the future development of theinitiative and its dissemination to other Colleges. 11
  12. 12. Colleges should ensure that they present clearly to assessors how theproject meets the specific criteria for the award, as this forms a keypart of the assessment report following the visit.Assessors will provide brief feedback to the College at the end of thevisit. This will normally identify particular aspects of good practiceand suggestions for areas for development where appropriate. Noindication of the outcome of the visit in terms of decisions on theaward can or should be made at this stage.Following the visit, assessors complete a visit report (see appendix 2)which provides a written assessment of the project. Following thevisits to all of the short-listed Colleges, assessors complete anoverview report and agree the recommendations for the award. Theoutcomes of this stage are judgements and recommendations forawards to the Manager of the AoC Charitable Trust.Sponsors may, with the agreement of the Manager, undertake Stages 1 and2 of the assessment process in respect of their own award(s) in conjunctionwith a Beacon Award assessor.Stage 3 Final decisionsThe Chief Assessor presents the assessors’ reports andrecommendations to the Beacon Awards Steering Group whichselects the award winners.All Beacon Awards are awarded or withheld at the sole and absolutediscretion of the Beacon Awards Steering Group and those acting onits behalf. No explanation is given following the making orwithholding of an award.The Chief Assessor and the Manager may identify a number ofwinning projects which he feels to be particularly outstanding orinnovative to be considered by the Steering Group for submission to 12
  13. 13. the President of the AoC Charitable Trust so that he or she candecide if any should receive a ‘President’s Award’.Detailed guidance for assessors on each of these stages is givenbelow.Guidance to Assessors on the procedures at each assessment stage Stage 1 Shortlisting  The assessment of applications for each award is undertaken by two assessors appointed by the Chief Assessor and the Manager.  Assessors should inform the Manager of any substantial involvement with particular Colleges e.g. as a member of staff, Governor, or as a result of continuing consultancy with a College which could present a potential conflict of interest.  One assessor in each pair assigned to an award acts as the Lead Assessor and reports to the Manager over the period of the assessment and at the conclusion of the first stage of assessment submits details of the agreed shortlist. Where one of the pair of assessors is a sponsor, the Beacon Assessor acts as the Lead Assessor.  Assessors should primarily assess the extent to which the submission satisfies the sponsor’s criteria completing the Selection Analysis Form (SAF). Assessors should also identify any possible shortcomings in meeting the AoC Beacon criteria. A grade should be allocated with indicative assessment  “A” – project meets all the criteria well and is suitable for shortlisting  “B” – project meets most of the criteria 13
  14. 14.  “C” – project fails to meet important criteria and is not suitable for shortlisting The two assessors assigned to each award are responsible to the Manager for assessing applications for that award and for drawing up an agreed shortlist, of no more than three applications for consideration at the next stage. The list of submissions for an assessment visit should be based on an assessment of those suitable for short-listing which best meet the criteria for the award. In general, where there are fewer than six submissions, only one College need be visited if no others meet the criteria. For between six and 15 submissions, two Colleges should be visited, above 15 submissions a maximum of three Colleges may be visited. Submissions from Colleges should be anonymous. Each College will be allocated a reference code which the assessors should use in correspondence with the Manager. If a submission has not been fully anonymised, assessors should note this on the SAF for feedback to the College. Submissions may be returned to the College for removal of any reference before being passed on to the assessors. Applications from two or more Colleges working in partnership or submissions from consortia of Colleges should be treated as any other application, although one College must agree to be the main point of contact to avoid confusion. Assessors should read submissions, complete the SAF and then discuss their findings with each other to agree which Colleges should be shortlisted for a visit. If the two assessors are unable to agree a shortlist, the views of the Manager and the Chief Assessor should be sought. It is expected that assessors complete the assessment and SAFs for at least five submissions a day. 14
  15. 15.  Completed SAFs should be returned to the Manager for all Colleges by Wednesday 8th August 2012 for approval. The form asks assessors to identify points of merit and areas for improvement in the submissions. The comments should be fairly extensive and should include an explicit reason for the decision to shortlist or not. This summary needs to be self explanatory and provide sufficient information to enable the Manager to give informed feedback to any non-shortlisted College. The Chief Assessor and/or Manager will review the SAFs as part of a moderation process. The Manager will inform the Lead Assessor of the College contact details on the approval of the shortlist. Once the shortlist for an award has been approved, the assessors should contact the relevant Colleges to arrange mutually convenient dates and times for visits. Assessors should make clear any requirements for additional information prior to the visit. The arrangements for the visit will be confirmed in writing to the Principal. Assessors should draft a letter (an example can be seen at appendix 3) for the College specifying the arrangements for the visit. When assessors wish to see a particular aspect of the project, or require additional information to be made available either prior to, or during the visit, they should ensure this request is included in the confirmation letter for the visit rather than ask to see it on the day. The letter confirming the arrangements for the visit is sent to the College by AoC Beacon Awards Manager on behalf of the assessors. Assessors must also complete an overview report of each award. This should be completed in two parts. Part 1 of the assessors’ overview report should identify any general areas for 15
  16. 16. improvement in submissions and the rationale for the final shortlist. This part of the report should be sent to the Manager with the SAFs and recommended shortlist. Part 2 of the report should be completed following the assessment visits. After the announcement of Award winners, Colleges are sent general feedback, by the AoC Beacon Awards Manager, on each of the awards they have applied for and are offered the option of specific feedback to be given verbally. Stage 2 Site visits The purpose of the site visits should be to validate the evidence provided in the submission, clarify any aspects of the submission, collect additional evidence and assess the extent to which the project meets the criteria for the award. The Chief Assessor may, by arrangement, join any site visit by assessors and may stand in for an assessor in cases of unavoidable absence. Any additional documents required by the assessors should be requested from the College in advance of the site visit by the assessors. Discussions with staff, students and other relevant parties in the course of site visits should focus on the benefits of the project to learners, particularly any outcomes for learning including achievements in terms of skills and/or qualifications gained. Assessors will also wish to explore and clarify areas of the project such as: its future development and how it will be sustained, how it is managed including, the impact on equality and diversity, evaluation and dissemination to other Colleges. Assessors should remember that all submissions are about what 16
  17. 17. the Colleges consider to be excellent work. Assessors are not inspectors but are there to listen, to learn and to ask questions, in other words, act as a ‘critical friend’. Assessors should bear in mind that Colleges go to considerable effort to ensure their visit showcases their work in the best light possible. They often arrange for external stakeholders and former students to be available to meet with them. Assessors should display every courtesy to the people they meet on the visit. They should endeavour to ensure the timetable for the visit is maintained and be appreciative of the time external stakeholders have taken to support the College and meet them. The following procedural points may be helpful to assessors:  a half day visit (4- 5 hours) is often sufficient for assessors to form a judgement on the quality of the project and avoid placing too great a demand on the College  it is useful for assessors to agree who should lead on each discussion point – this could be done prior to each visit by asking the College for the use of a room for about half-an-hour before each site visit  similarly, it is useful for assessors to request a room at the end of the visit so that they can have a short meeting in private to prepare a brief summary feedback on the visit  Assessors should give a short feedback (about 10 minutes) to Colleges following the visit. This should not be critical but recognise areas of good practice and suggest areas for development where appropriate  Assessors should not tell Colleges how many other Colleges 17
  18. 18. have entered for the award or been shortlisted  Assessors should not tell Colleges what their recommendations are The Lead Assessor should write to the College liaison officer after the visit thanking the College for all it has done to make the visit a success; the letter should be copied to the Principal of the College and to the Beacon Awards Manager. Assessors should produce a joint report of the visit which is sent to the Manager in electronic form (i.e. by email or on a USB stick). The visit report completed by the assessors must include the following: (see standard format for the visit report at appendix 2)- College details- Project summary- A summary of the visit- Assessment against the sponsor’s criteria (the main evidence for the decision on the award)- Project organisation and management- Project outcomes and benefits- Summary strengths and area for development- Recommendation (Overall judgement on the match to the sponsor’scriteria and recommendation for an award) The visit report should not be more than three sides of A4, in reasonable font size (i.e. no smaller than font size 10) and must be agreed by the two assessors. The copy of the assessors’ report given to the Steering Group will be anonymised so assessors are requested to make no references which could identify the College beyond the first section of the report. 18
  19. 19.  Assessors should recommend one winner. In the unusual case of two applications being of absolutely equal merit the assessors may refer to the Chief Assessor for approval to recommend joint winners. This must also be discussed with the Manager since there may be financial implications for the sponsor. Assessors must complete part 2 of the assessors’ overview report which should provide the rationale for recommendation and identify any areas for improvement in the assessment of awards. This report will be used to support the moderation and quality assurance of assessment. Assessors must keep to the final date for the return of notes to the Manager since this date is determined by the time scale needed to allow the Steering Group to make its decisions. The final date for receipt of all assessors’ feedback forms, overviews, reports and recommendations is Wednesday 17th October 2012. Stage 3 Final decisionsThe Beacon Awards Steering Group acting as the Assessment Boardmakes the final decisions about award winners in the light ofrecommendations from the team of assessors and the Chief Assessor.The Chief Assessor:  reads reports of all site visits made to short listed Colleges  reads the SAFs from assessors in order to gain an overview of the quality of non-shortlisted submissions and the reasons for not being shortlisted  reads a selection of the full submissions from Colleges 19
  20. 20.  if necessary, contacts assessors through the Manager for clarification or further comment  produces a short commentary on the recommendations made for each award  uses the commentaries to advise the Beacon Awards Steering Group in its decision making  produces an overview report for the Steering Group The Chief Assessor, together with the Manager, may identify a number of winning projects which they feel to be particularly outstanding or innovative in any given year to be submitted to the President of the AoC Charitable Trust so that he or she can decide upon one to receive a ‘Presidents’ Award’. Decisions about the AoC Beacon Award winners are taken by the Steering Group at its meeting in late October/early November each year. The Steering Group reserves the right not to make an award. The Steering Group does not give any explanation of its decisions nor does the Manager enter into any discussion about decisions or about individual submissions. In the situation where the Steering Group’s decision differs from the recommendation made by the assessors, the sponsor may be informed of this, but will not be entitled to appeal the decision. The Steering Group appreciates that sponsors wish to maximise the publicity in relation to their own winning College, however they must not contact the College they have recommended as the winner until this status has been confirmed at the announcement. 20
  21. 21.  Award winners will be announced at the AoC’s annual conference between 20-22 November 2012. 21
  22. 22. Quality Assurance and Evaluation of AssessmentThe Chief Assessor is responsible to the Beacon Awards SteeringGroup for the quality assurance of the assessment of the awards. Allassessors are required to attend training and new assessors aresupported by a Mentor. The Chief Assessor reviews the assessorsreports to ensure the reliability of assessment, the consistency ofassessment and parity between awards. Colleges complete anevaluation form following an assessment visit. The Collegeevaluations are used as a basis for identification of training needsand for further development and improvement in the assessment ofawards. An overview of the arrangements for quality assurance isgiven below:Moderation and quality assurance of assessmentThe Chief Assessor reviews all reports by assessors to ensure:  Assessment forms and reports are appropriately completed with clear identification of strengths and weaknesses in relation to the Beacon and sponsors criteria  The short-list of submissions is appropriate and supported by clear analysis in the assessment forms  Recommendations for the award are well supported by evidence and assessment recorded in the visit report  The overview report provides a clear summary analysis of strengths and weaknesses of submissions, identifies areas for improvement in submissions and gives a clear rationale for the recommendation for the award  Assessment is consistent between submissions and is comparable in the standard across all awards. 22
  23. 23. Moderation is carried out in two stages:(i) End of first stage assessment - moderation of initial assessment (short-listing) of submissions: For each award a sample of submissions and SAFs is read and the assessors overview report (part 1) reviewed. The Chief Assessor considers the appropriateness of the completion of SAFs and of the rationale for the shortlist. The outcomes of the moderation/review is fed back to assessors and reported to the Steering Group as part of the annual Chief Assessors report.(ii) End of second stage assessment - review of assessors’ recommendations: For each award the assessors visit reports and overview reports are read. Where there is a lack of clarity in the visit reports the submissions are reviewed and in cases where the reports for the winning College(s) are found to be insufficiently detailed assessors will be contacted for further information or clarification. The review considers the appropriateness of the visit report, and of the assessors justification of the award. The outcomes of the review are reported to the Steering Group as part of the annual Chief Assessors report. In cases where the Chief Assessor finds the reports anomalous the final decision on the award will be referred for review by the Steering Group.Feedback is provided to assessors at each stage of moderation andall assessors receive a copy of the Chief Assessor’s annual report.EvaluationTo further support improvement and development of assessmentshort-listed Colleges have opportunity to evaluate and comment,formally on the arrangements for assessment. The evaluation formcan be found at Appendix 4. 23
  24. 24. The form should be completed by the College and returned to theBeacon Awards Manager by 17 October 2012 . The forms aresummarised by the Manager and reviewed by the Chief Assessor. Asummary report resulting from the analysis is included in the ChiefAssessor’s annual report to the Steering Group and forms the basisof recommendations for changes and improvements to assessmentThe forms will be copied to the Beacon Assessors for the award butotherwise treated as confidential. Where evaluation consistentlygives rise to concerns over assessment for a particular award orregarding the conduct of specific assessors the Chief Assessor willconduct a review of the arrangements for assessment andrecommend suitable actions to the Beacon Awards Manager.Appendix 1. Members of the Beacon Awards Steering GroupDame Pat Bacon Chair, AoC Beacon AwardsLesley Davies Deputy Chief Executive and Director of Policy, AoCHaydn Edwards Colleges Wales RepresentativeSteve Frampton Principal, Portsmouth College – Sixth Form College RepresentativeLiz Green Head of Post-19 Qualifications, OCRJoan Herron Southern Regional College – Northern Ireland RepresentativeRobin Landman Chief Executive, Network for Black ProfessionalsRoger Marriott Chief Assessor, AoC Beacon AwardsKaren Murray Head of Framework (Quality Assurance and Intervention), YPLAMichael Group Marketing, City & GuildsOsbaldestonJudith Stradling Deputy Principal, City of Bristol College – GFE RepresentativeAlice Thiagaraj Manager, AoC Charitable Trust 24
  25. 25. Appendix 2. Assessment Forms Below are two forms which assessors should use to assist them in the reporting of the assessment process. The first is a template for assessors to complete following the site visit of a shortlisted College.VISIT REPORT – TEMPLATEVisit Report AoC Beacon Award (title of award) 2012College:Title of Project:Date of Visit:Assessors:1. Project Summary:( Please bear in mind that this section may be sharedwith the Beacon Awards Steering Group who will not have seen the originalapplication, so include a brief summary of the project as a whole as well asidentifying any unusual or innovative features of the project.)2. Summary of Activities during Visit: (a brief summary of the meetingsand activities of the visit)3. Assessment Notes: (assessment against sponsor’s criteria) Identify strengths and weaknesses against each of the sponsors criteria4. Summary Assessment: Project organisation and management: (identify key features that support the success of the project or that need to be further developed) 25
  26. 26. Project outcomes: (identify the key benefits to learners and the value ofthe initiative to other Colleges)Overall Strengths and Areas for development: (summarise the mainkey strengths and areas for development)5. Recommendation (overall judgement on the match of the project tosponsors criteria and the recommendation of the assessors for an award) ************************************************The following is the Selection Analysis Form to assist assessors in theshortlisting process. This form is available in electronic format andwill be emailed to assessors for use each year. 26
  27. 27. AoC BEACON AWARDS SELECTION ANALYSIS FORM FOR SHORT-LISTING APPLICATIONSTitle and year of Award:College code:Title of Project:ASSESSOR: BEACON CRITERIA Full Part Not COMMENTSTo what extent are the Beacon criteria metby the submission (identify specificshortcomings)It is subject to evaluation/quality assuranceto influence continuing development of theinitiativeIt promotes equality and diversity in thedelivery of the programmeIt has been running for at least oneacademic year by 31 July 2012It has features which actively promoteexemplary teaching and learningIt is of benefit to one or more groups ofstudents or trainees which are identifiedand described in the submissionIt has wider relevance and applicabilitywhich makes it of value to other Colleges asan example of good practice or innovation Assessment Grade: 27
  28. 28. AoC BEACON AWARDS SELECTION ANALYSIS FORM (PART2 SPONSORS CRITERIA ASSESSMENT)Title and year of Award:College code:Title of Project:ASSESSOR: SPONSOR CRITERIA Full Part Not COMMENTSAgainst each criteria set out by the sponsorassess the extent to which the criteria aremet. Assessment Grade:Note : This part of the form should be continued as required. 28
  29. 29. AoC BEACON AWARDS SELECTION ANALYSIS FORM (PART3 SUMMARY ASSESSMENT)Title and year of Award:College code:Title of Project:ASSESSOR:Please identify the following for feedback to Colleges should it be requested (bullet points).What features of the initiative areof merit or of interest?What aspects of theinitiative/application could beimproved, should the College wishto re-submit in the future? OVERALL GRADE:Assessors’ Decision: SHORT-LISTED / NOT SHORT-LISTED (delete as appropriate)Comments if any: 29
  30. 30. Appendix 3. Sample confirmation letter to shortlisted CollegesContact name and addressDateDear (Contact)AoC Beacon Awards 2012/13 – (Title of Beacon Award)Ref: (code)Further to our assessor’s recent telephone call, I have pleasure inconfirming that your College has been shortlisted for the (Award).The two assessors assigned to the award are (Lead Assessor) andhis/her co-assessor (Sponsor Assessor) from (Sponsor). Asdiscussed, they would like to visit you on (date) from (time) until(time).The assessors will wish to engage in discussions with managers,stakeholders, partners and staff involved in delivering theinitiative. They would also like an opportunity to talk to learners.It would be helpful if you would draw up a programme toinclude evidence of the following:(list evidence required)Where possible, please include points that are additional to thoseoutlined in your submission as well as incorporating any otheraspect of your work which you would like the assessors to see.It would be helpful if you could allow 15 minutes for assessors tomeet privately on arrival and at the end of the programme so thatthey can provide feedback to the College at the end of the visit.Apart from this you should feel free to structure the visit as suits
  31. 31. you best with the minimum disruption to you and yourcolleagues.Please confirm your receipt of this letter and that you are happywith the arrangements.I understand that you will send a programme and location detailsto the assessors prior to their visit. Their contact details are:Lead assessor address, number and emailSponsor assessor address, number and email(Please not that x assessor has the following dietaryrequirements/needs car parking space – delete if not applicable.)I enclose the handbook for the assessment of AoC Beacon Awardswhich gives further detail on what the assessors will be lookingfor in their visit as well as an evaluation of assessment form whichI should be grateful if you could complete after your visit hastaken place and returned to me by Wednesday 17 October 2012 atthe latest.Congratulations on reaching this stage of the assessment process.I appreciate this is a busy time and I hope that as littleinconvenience as possible will be caused. I will be sending a copyof this letter to the Principal. If you have any queries relating tothe visit, do not hesitate to contact me.Yours sincerelyAlice ThiagarajCc Principal
  32. 32. Appendix 4. College Evaluation form AoC BEACON AWARDS: EVALUATION OF ASSESSMENT TO BE COMPLETED BY COLLEGES FOLLOWING AN ASSESSMENT VISITThe purpose of this questionnaire is to gain feedback from Colleges of the assessment arrangements for the AoC Beacon Awards.Findings will be used in the overall evaluation of assessment and the improvement of practice.Name of organisation: (Please circle one number) 1 Sixth Form College 2 General FE College 3 Tertiary College 4 Land Based College 5 Art & Design College 6 Independent Specialist College 7 Other: Please specifyDate of assessment visit:Beacon AwardTitle of the project/submissionFor the remainder of the questionnaire, please circle one number and provide any additional comments as appropriate.(Where a statement is not applicable please circle Box 5.)We were satisfied with: Strongly Agree Disagree Strongly No Opinion Agree Disagree or N/A1. The notice given for the visit 1 2 3 4 5Comments2. The quantity of documentation requested 1 2 3 4 5 prior to and during the visit.Comments3. The way the college’s documentation was used during the visit 1 2 3 4 5Comments4. The conduct of assessors and their management of the meetings. 1 2 3 4 5Comments
  33. 33. We were satisfied that: Strongly Agree Disagree Strongly No Opinion Agree Disagree or N/A5. Assessors had the professional 1 2 3 4 5 competence to assess the submissionComments6. The assessors considered the evidence 1 2 3 4 5 appropriatelyComments7. The feedback at the end of the visit 1 2 3 4 5 week comprised a fair reflection of the project/submission.CommentsPlease indicate by ticking the appropriate box below whether there were more gains from the assessment in helping the organisation/overall provisionto improve, or more negative effects. 8. The gains in helping to improve the provision outweigh negative effects 8.1 The gains and negative effects are equally balanced 8.2 The negative effects outweigh the gains 8.3Thank you for taking the time to complete this form. If you have any further comments, which you believe would help improve the assessment,please supply them below. In particular, is it helpful to know what went well during the assessment and what did not go so well.Assessment would be improved by:Form completed by (please enter name and position in College )......................................................................Please return this form to Alice Thiagaraj, Manager AoC Beacon Awards, 2-5 Stedham Place London WC1A 1HU
  34. 34. Appendix 5. Assessors’ Fees and ExpensesAoC Beacon Assessors are entitled to claim a daily rate plus reasonableexpenses. A day’s work should be calculated as being 7.5 hours, in linewith the contracted working day for AoC employees.Visits should normally be completed in one day including travel. Howeverwhere travel would result in a working day longer than 9 hours it isreasonable for assessors to claim an overnight stay.Visit reports should be completed within 0.5 of a day. Assessors shouldnormally aim to complete a visit including travel, any overnight stays, andreport writing within 1.5 days.Administration may be claimed for each award. Assessors should aim toallocate no more than 0.5 days per award for administration to cover thetime to arrange visits, write letters and complete the overview report.Expenses may be claimed according to AoC policy but assessors areencouraged to seek the cheapest (acceptable) accommodation.AoC Beacon Assessors are asked to bear in mind that the Beacon Awardsoperate through the AoC Charitable Trust which is a Registered Charityand the assessment, administration, events, PR and publications all need tobe covered by the sponsors’ donations. As such, the Trust greatlyappreciates the efforts Beacon Assessors make to keep their expenses to aminimum.Assessors are asked to submit an invoice to cover their fees once they havereturned all their completed forms and reports to the AoC Beacon Awardsoffice at the end of the process. Expenses may be claimed for throughoutthe process using the expenses claim form which can be emailed to BeaconAssessors. An example of the claim form appears below. Expenses claimforms can be emailed back to the office, provided scanned copies of allreceipts are also attached, or sent by post with the original receipts.
  35. 35. Appendix 6: CHECK LIST OF DOCUMENTS TO BE SENT TO THE BEACON AWARDS OFFICE Date sent to Beacon Office Selection Analysis Forms Visit letter to college Site visit reports Award overview reportsincluding Recommendations of assessors Copy of letter of thanks sent to the College’s liaison officerAppendix 7: Appointment and training of Beacon Assessors.Appointment of Beacon Assessors:Individuals with suitable experience and current knowledge of the work ofFurther Education Colleges are invited to apply to join the register ofBeacon Assessors. Colleges winning a Beacon Award are invited tonominate a member of staff associated with the winning submission to beconsidered for appointment as a Beacon Assessor.Appointment is made following assessment, by the Beacon Awardsmanager and the Chief Assessor, of the match of the applicants experienceand skills to the specification included in the job description (available online at www.aoc.co.uk/beaconawards/assessment). The register ismaintained by the Beacon Awards Manager. The Selection criteria for AoCBeacon Assessors contained in the job description are :Applicants should demonstrate evidence of:  Leadership and management within the further education sector
  36. 36.  Current knowledge of the further education sector i.e. direct experience of the sector outside the AoC Beacon Awards within the last 3 years  In-depth knowledge of a specific curriculum or development area  Evidence of substantial teaching experience in the FE sector  Project management skills including the ability to effectively and efficiently manage resources  Competent written skills  Ability to consolidate and analyse information to make an objective assessment of provision and outcomes against the agreed criteria.Assessors are required to re-register after a period of three years and toattend the one day training for assessors on re-appointment. To be re-registered assessors will be required to submit to the Beacon Awardsmanager a summary of their professional development and directexperience of the sector outside the AoC Beacon Awards within theprevious 3 yearsTraining of AssessorsBeacon assessors are required to attend the assessor training at least everythree years. Sponsor’s assessors should also attend when possible but willnot be reimbursed for their expenses or attendance.New Beacon Assessors are allocated a mentor and required to attend theone day training conference. The role of mentor is to provide support andguidance especially during the first two visits made by a new assessor. Thementor will accompany both visits; on the first the mentee will mainlyobserve but should lead the second visit and be responsible for writing thereport of that visit. On completion of the second visit the mentee is requiredto complete a self-assessment form which should be agreed by the mentorfollowing a review of the report. On subsequent visits the new assessor isexpected to take a full role in the assessment processes.

×