Star Rating for
Language Teachers
Research project conducted at
International House London
By Lara Panzini
©Eaquals Eaquals International Conference | Prague | 26-28 April 2018 #eaquals18prague
Introduction
Is star rating a feasible option?
A familiar instrument to communicate quality of service.
Key issues: value of rating and what students want.
©Eaquals #eaquals18prague
Method
Primary Research
• Online surveys for teachers and students
• Face to face interviews with academic managers
Secondary Research
• Analysed existing methods to measure quality in the
hotel industry and their recommendations.
• Popular examples of star rating in the educational world.
©Eaquals #eaquals18prague
Secondary research
How is the quality of hotels measured?
• Classification
• Grading
• Rating
©Eaquals #eaquals18prague
Secondary research
How popular is rating in education?
• Milingual has over 10.000 users since 2015
• Ratemyteachers has 17.000.000 annual users since
2011
• Ratemyprofessors has over 17.000.000 ratings since
2016
©Eaquals #eaquals18prague
Secondary research
How popular is rating in education?
• Milingual has over 10.000 users since 2015
• Ratemyteachers has 17.000.000 annual users since
2011
• Ratemyprofessors has over 17.000.000 ratings since
2016
©Eaquals #eaquals18prague
Secondary research
What do they look like? This is Milingual.
©Eaquals #eaquals18prague
Secondary research
What do they look like? This is Rate your lecturer.
©Eaquals #eaquals18prague
Primary research – response rate
Who took part in the online survey?
Student population: general group courses, 12 questions.
Teachers: permanent, 17 questions.
©Eaquals #eaquals18prague
Primary research – Students’ survey
1. What do you associate STAR RATING with?
2. Please STAR RATE your current teacher.
3. What did you consider when star rating your teacher? Which qualities and skills?
Please add the 3 most important to you.
4. Please rate your current teacher
5. Would you stay or would you go?
6. At the end of your language course, what is most important to you? Please put most
important first, least important last.
7. When choosing a language school, how important are the following?
8. Would star rating of teachers on a school's website help you chose between
schools?
9. Are you male or female?
10. What is your age?
11. What is your first language?
12. What is the reason for studying at International House?
©Eaquals #eaquals18prague
Primary research – Teachers’ survey
1. What do you associate STAR RATING with?
2. What do you think about star rating of language teachers?
3. When you study a language, what qualities do you look for in a language teacher? Please list top 3.
4. When you are a student, how important is for you to like your teacher?
5. What does it mean "like your teacher"?
6. If you didn't like your teacher, would you continue studying with him/her or would you change?
7. Now as a language teacher. What, in your opinion, could a student evaluate?
8. When students complete their feedback form at the end of their course, what do they think about?
Please add 3 key areas.
9. Which teacher's qualities and skills are most important to students? Please add 3.
10. At the end of a language course, what is most important to a student? Please put most important first,
least important last.
11. When a student chooses a language school, how important do you think these are?
12. Do you think that STAR RATING of teachers on a school's website could help students chose between
schools?
13. As a language teacher, what do you look for in students' feedback?
14. How does it help you?
15. Are you male or female?
16. What is your first language?
17. What is your age?
©Eaquals #eaquals18prague
Primary research
Students and Teachers: Q1 “What do you associate
STAR RATING with?
Objectivity: measuring something specific
Subjectivity: measuring experience
68% of students selected “Subjectivity”
82% of teachers selected “Subjectivity”
©Eaquals #eaquals18prague
Primary research
When a student chooses a language school, how
important do you think these are?
©Eaquals #eaquals18prague
Primary research
Students’ Survey: Q2 Please STAR RATE
your current teacher.
©Eaquals #eaquals18prague
Primary research
Teachers’ Survey: Q2 What do you think about star
rating of language teachers?
©Eaquals #eaquals18prague
Primary research
Q6/Q10 At the end of your language course, what is
most important to you/student? Please put most
important first, least important last.
©Eaquals #eaquals18prague
Primary research
SQ8/TQ12 Would star rating of teachers on a school's
website help you chose between schools?
©Eaquals #eaquals18prague
Primary research
Here are the main findings from the academic
managers’ interview and the online surveys completed
by teachers and students.
a. Rating is subjective.
b. Students are experienced raters.
c. Teachers fear rating may allow or encourage unfair feedback.
d. All participants believe that progress is the most important factor
when evaluating a course.
e. Students believe that a school should guarantee excellent
teaching.
f. Students would find rating helpful if supported by reviews.
Managers also agreed with this combination.
©Eaquals #eaquals18prague
Conclusion
Star rating is a feasible system to share an overall
customer experience and our students’ perception of
the value of rating is accurate.
A fully integrated system is what customers would find
most useful when choosing a school.
©Eaquals #eaquals18prague
Recommendations for IHL
©Eaquals #eaquals18prague
International House London guarantees excellent teaching by …
EAQUALS rating
IHWO
Students’ Reviews
25/12/2017 Charmaine: It was very interactive and really enjoyed the games.
07/08/2017 Bryan Rossi-Anderson: The way in which Ferruccio used innovative
means to teach us using contemporary Italian film.
27/06/2017 Kayleigh Ryan: The teacher, Bruno. He makes it very interactive and
interesting. Thoroughly enjoy the class, it's different every week.
What would have we scored in 2017?
©Eaquals #eaquals18prague
Thank you.
©Eaquals #eaquals18prague
Bibliography
©Eaquals #eaquals18prague
BBC (2006). Teachers 'bullied by online grading'. BBC UK. [online] Available at:
http://news.bbc.co.uk/1/hi/uk/6139626.stm [Accessed 5 Mar. 2017].
Burgess, T. (2001). A general introduction to the design of questionnaires for survey research..
[Guide] University of Leeds, Leeds.
ChrisParrTHE (2013). Rate your lecturer website scores poorly with lecturers. [online] Available at:
https://www.timeshighereducation.com/news/rate-your-lecturer-website-scores-poorly-with-
lecturers/2004760.article [Accessed 4 Mar. 2017].
Dayan, O., Struwig, M. and Hensens, W. (2010). GUEST-REVIEW CRITERIA ON TRIPADVISOR
COMPARED TO CONVENTIONAL HOTEL-RATING SYSTEMS TO ASSESS HOTEL QUALITY.
Amsterdam: EuroCHRIE, pp.1-11.
Dayan, O., Struwig, M. and Hensens, W. (2012). Do social media display correct conventional hotel
ratings?. Research in Hospitality Management, [online] 1(1), pp.17-9. Available at:
http://www.ajol.info/index.php/rhm/article/view/75098 [Accessed 4 Mar. 2017].
De Vaus, D. (2001). Research design in social research. Thousand Oaks, CA: SAGE Publications.
Easterby-Smith, M., Thorpe, R., Jackson, P. and Lowe, A. (2008). Management research (SAGE
series in management research). 3rd ed. Los Angeles: Sage Publications.
Bibliography
©Eaquals #eaquals18prague
ECC-Net (2009). [online] Available at:
http://ec.europa.eu/consumers/ecc/docs/hotel_establishment_classification_EU_en.pdf [Accessed 5 Mar.
2017].
Editorial, R. (2007). UK union condemns "rate my teacher" web sites. [online] Available at:
http://uk.reuters.com/article/life-britain-teachers-dc-idUKL1588395620071115 [Accessed 5 Mar. 2017].
ESRC (2015). ESRC framework for research ethics updated January 2015. [online] Available at:
http://www.esrc.ac.uk/files/funding/guidance-for-applicants/esrc-framework-for-research-ethics-2015/
[Accessed 4 Mar. 2017].
Go Lingual (2015). Grupos de Conversación en bares. [online] Available at: http://milingual.com [Accessed 30
Jan. 2017].
Harris, S. (2014). Pupils who troll staff: Children as young as four insult teachers. Daily Mail. [online] Available
at: http://www.dailymail.co.uk/news/article-2609200/Primary-pupils-troll-staff-Children-young-four-taking-
Twitter-Facebook-RateMyTeacher-insult-teachers-survey-reveals.html [Accessed 5 Mar. 2017].
Harrison, C. (2007). Tip sheet on question wording. Program on Survey Research. Harvard University, pp.1-4.
Henry, G. (1990). Practical sampling. 10th ed. United States: Sage Pubns, Thousand Oaks, California, U.S.A.
Hensens, W., Srugwig, M., & Dayan, O. 2010. Guest-Review Criteria on TripAdvisor Compared to
Conventional Hotel-Rating Systems to Assess Hotel Quality. Passion for Hospitality Excellence:
European Council on Hotel, Restaurant & Institutional Education 2010 Conference Proceedings.
Amsterdam, 25-28 October: 1-12.
How's my feedback?. (2011). How's my feedback?. [online] Available at:
http://www.howsmyfeedback.org/category/ratemyteachers/ [Accessed 5 Mar. 2017].
Bibliography
©Eaquals #eaquals18prague
Madariaga, L., Nussbaum, M., Burq, I., Marañón, F., Salazar, D., Maldonado, L., Alarcón, C. and
Naranjo, M. (2017). Online survey: A national study with school principals. Computers in Human
Behavior, [online] 74, pp.35-44. Available at:
http://file:///C:/Users/larap/AppData/Local/Temp/Rar$DIa0.490/Online-survey-A-national-study-with-
school-principals_2017_Computers-in-Human-Behavior.pdf [Accessed 18 Apr. 2017].
MORI, I. (n.d.). The national student survey 2017. [online] Available at:
http://www.thestudentsurvey.com/ [Accessed 25 Jan. 2017].
Neuman, W. (2005). Social research methods: Qualitative and quantitative approaches. 5th ed.
Boston: Pearson Education (US).
Nulty, D. (2008). The adequacy of response rates to online and paper surveys: What can be done?.
Assessment & Evaluation in Higher Education, [online] 33(3), pp.301-314. Available at:
https://www.uaf.edu/files/uafgov/fsadmin-nulty5-19-10.pdf [Accessed 14 Feb. 2017].
Orna, E. and Stevens, G. (1995). Managing information for research. Buckingham, United Kingdom:
Open University Press.
Panzini, L. (2017). Figure 1: Minimum sample. [image].
Patankar, F. (2013). Interview with rate your lecturer Co-Founder and director. [online] Available at:
http://www.careergeekblog.com/2013/06/11/interview-with-rate-your-lecturer-co-founder-and-director/
[Accessed 4 Mar. 2017].
Bibliography
©Eaquals #eaquals18prague
RateMyProfessors (2016). RateMyProfessors.com – find and rate your professor or campus. [online] Available
at: https://www.ratemyprofessors.com [Accessed 4 Mar. 2017].
RMT Acquisition, LLC (2017). Rate My Teacher. [online] Available at: http://uk.ratemyteachers.com/
[Accessed 25 Jan. 2017].
Rumsey and Rumsey, S. (2008). How to find information: A guide for researchers. 2nd ed. Maidenhead,
England: Open University Press.
Saunders, M., Lewis, P., Thornhill, A. and Thorn.., A. (2009). Research methods for business students (5th
edition). 5th ed. New York: Financial Times Prentice Hall.
Sterling Ideas Limited (n.d.). Rate your lecturer. [online] Available at: http://rateyourlecturer.co.uk/ [Accessed
12 Feb. 2017].
The Research Ethics Guidebook. (2017). Building ethics into the research design. [online] Available at:
http://www.ethicsguidebook.ac.uk/Practitioner-research-and-dual-roles-187 [Accessed 1 Feb. 2017].
TripAdvisor. (2017). TripAdvisor: Read reviews, compare prices & book. [online] Available at:
https://www.tripadvisor.co.uk [Accessed 4 Mar. 2017].
Vine, P. (1981). Hotel classification — art or science?. International Journal of Tourism Management, 2(1),
pp.18-29.
Wai-Ching, L. (2001). How to design a questionnaire. Student BMJ, 9, pp.187-189.
World Tourism Organization (UNWTO) (2014). Online guest reviews and hotel classification systems an
integrated approach. [online] Available at:
http://cf.cdn.unwto.org/sites/all/files/pdf/online_guest_reviews_and_hotel_classification_sytems_an_integrated_
approach.pdf [Accessed 5 Mar. 2017].

Lara Panzini: Star Rating for Language Teachers

  • 1.
    Star Rating for LanguageTeachers Research project conducted at International House London By Lara Panzini ©Eaquals Eaquals International Conference | Prague | 26-28 April 2018 #eaquals18prague
  • 2.
    Introduction Is star ratinga feasible option? A familiar instrument to communicate quality of service. Key issues: value of rating and what students want. ©Eaquals #eaquals18prague
  • 3.
    Method Primary Research • Onlinesurveys for teachers and students • Face to face interviews with academic managers Secondary Research • Analysed existing methods to measure quality in the hotel industry and their recommendations. • Popular examples of star rating in the educational world. ©Eaquals #eaquals18prague
  • 4.
    Secondary research How isthe quality of hotels measured? • Classification • Grading • Rating ©Eaquals #eaquals18prague
  • 5.
    Secondary research How popularis rating in education? • Milingual has over 10.000 users since 2015 • Ratemyteachers has 17.000.000 annual users since 2011 • Ratemyprofessors has over 17.000.000 ratings since 2016 ©Eaquals #eaquals18prague
  • 6.
    Secondary research How popularis rating in education? • Milingual has over 10.000 users since 2015 • Ratemyteachers has 17.000.000 annual users since 2011 • Ratemyprofessors has over 17.000.000 ratings since 2016 ©Eaquals #eaquals18prague
  • 7.
    Secondary research What dothey look like? This is Milingual. ©Eaquals #eaquals18prague
  • 8.
    Secondary research What dothey look like? This is Rate your lecturer. ©Eaquals #eaquals18prague
  • 9.
    Primary research –response rate Who took part in the online survey? Student population: general group courses, 12 questions. Teachers: permanent, 17 questions. ©Eaquals #eaquals18prague
  • 10.
    Primary research –Students’ survey 1. What do you associate STAR RATING with? 2. Please STAR RATE your current teacher. 3. What did you consider when star rating your teacher? Which qualities and skills? Please add the 3 most important to you. 4. Please rate your current teacher 5. Would you stay or would you go? 6. At the end of your language course, what is most important to you? Please put most important first, least important last. 7. When choosing a language school, how important are the following? 8. Would star rating of teachers on a school's website help you chose between schools? 9. Are you male or female? 10. What is your age? 11. What is your first language? 12. What is the reason for studying at International House? ©Eaquals #eaquals18prague
  • 11.
    Primary research –Teachers’ survey 1. What do you associate STAR RATING with? 2. What do you think about star rating of language teachers? 3. When you study a language, what qualities do you look for in a language teacher? Please list top 3. 4. When you are a student, how important is for you to like your teacher? 5. What does it mean "like your teacher"? 6. If you didn't like your teacher, would you continue studying with him/her or would you change? 7. Now as a language teacher. What, in your opinion, could a student evaluate? 8. When students complete their feedback form at the end of their course, what do they think about? Please add 3 key areas. 9. Which teacher's qualities and skills are most important to students? Please add 3. 10. At the end of a language course, what is most important to a student? Please put most important first, least important last. 11. When a student chooses a language school, how important do you think these are? 12. Do you think that STAR RATING of teachers on a school's website could help students chose between schools? 13. As a language teacher, what do you look for in students' feedback? 14. How does it help you? 15. Are you male or female? 16. What is your first language? 17. What is your age? ©Eaquals #eaquals18prague
  • 12.
    Primary research Students andTeachers: Q1 “What do you associate STAR RATING with? Objectivity: measuring something specific Subjectivity: measuring experience 68% of students selected “Subjectivity” 82% of teachers selected “Subjectivity” ©Eaquals #eaquals18prague
  • 13.
    Primary research When astudent chooses a language school, how important do you think these are? ©Eaquals #eaquals18prague
  • 14.
    Primary research Students’ Survey:Q2 Please STAR RATE your current teacher. ©Eaquals #eaquals18prague
  • 15.
    Primary research Teachers’ Survey:Q2 What do you think about star rating of language teachers? ©Eaquals #eaquals18prague
  • 16.
    Primary research Q6/Q10 Atthe end of your language course, what is most important to you/student? Please put most important first, least important last. ©Eaquals #eaquals18prague
  • 17.
    Primary research SQ8/TQ12 Wouldstar rating of teachers on a school's website help you chose between schools? ©Eaquals #eaquals18prague
  • 18.
    Primary research Here arethe main findings from the academic managers’ interview and the online surveys completed by teachers and students. a. Rating is subjective. b. Students are experienced raters. c. Teachers fear rating may allow or encourage unfair feedback. d. All participants believe that progress is the most important factor when evaluating a course. e. Students believe that a school should guarantee excellent teaching. f. Students would find rating helpful if supported by reviews. Managers also agreed with this combination. ©Eaquals #eaquals18prague
  • 19.
    Conclusion Star rating isa feasible system to share an overall customer experience and our students’ perception of the value of rating is accurate. A fully integrated system is what customers would find most useful when choosing a school. ©Eaquals #eaquals18prague
  • 20.
    Recommendations for IHL ©Eaquals#eaquals18prague International House London guarantees excellent teaching by … EAQUALS rating IHWO Students’ Reviews 25/12/2017 Charmaine: It was very interactive and really enjoyed the games. 07/08/2017 Bryan Rossi-Anderson: The way in which Ferruccio used innovative means to teach us using contemporary Italian film. 27/06/2017 Kayleigh Ryan: The teacher, Bruno. He makes it very interactive and interesting. Thoroughly enjoy the class, it's different every week.
  • 21.
    What would havewe scored in 2017? ©Eaquals #eaquals18prague
  • 22.
  • 23.
    Bibliography ©Eaquals #eaquals18prague BBC (2006).Teachers 'bullied by online grading'. BBC UK. [online] Available at: http://news.bbc.co.uk/1/hi/uk/6139626.stm [Accessed 5 Mar. 2017]. Burgess, T. (2001). A general introduction to the design of questionnaires for survey research.. [Guide] University of Leeds, Leeds. ChrisParrTHE (2013). Rate your lecturer website scores poorly with lecturers. [online] Available at: https://www.timeshighereducation.com/news/rate-your-lecturer-website-scores-poorly-with- lecturers/2004760.article [Accessed 4 Mar. 2017]. Dayan, O., Struwig, M. and Hensens, W. (2010). GUEST-REVIEW CRITERIA ON TRIPADVISOR COMPARED TO CONVENTIONAL HOTEL-RATING SYSTEMS TO ASSESS HOTEL QUALITY. Amsterdam: EuroCHRIE, pp.1-11. Dayan, O., Struwig, M. and Hensens, W. (2012). Do social media display correct conventional hotel ratings?. Research in Hospitality Management, [online] 1(1), pp.17-9. Available at: http://www.ajol.info/index.php/rhm/article/view/75098 [Accessed 4 Mar. 2017]. De Vaus, D. (2001). Research design in social research. Thousand Oaks, CA: SAGE Publications. Easterby-Smith, M., Thorpe, R., Jackson, P. and Lowe, A. (2008). Management research (SAGE series in management research). 3rd ed. Los Angeles: Sage Publications.
  • 24.
    Bibliography ©Eaquals #eaquals18prague ECC-Net (2009).[online] Available at: http://ec.europa.eu/consumers/ecc/docs/hotel_establishment_classification_EU_en.pdf [Accessed 5 Mar. 2017]. Editorial, R. (2007). UK union condemns "rate my teacher" web sites. [online] Available at: http://uk.reuters.com/article/life-britain-teachers-dc-idUKL1588395620071115 [Accessed 5 Mar. 2017]. ESRC (2015). ESRC framework for research ethics updated January 2015. [online] Available at: http://www.esrc.ac.uk/files/funding/guidance-for-applicants/esrc-framework-for-research-ethics-2015/ [Accessed 4 Mar. 2017]. Go Lingual (2015). Grupos de Conversación en bares. [online] Available at: http://milingual.com [Accessed 30 Jan. 2017]. Harris, S. (2014). Pupils who troll staff: Children as young as four insult teachers. Daily Mail. [online] Available at: http://www.dailymail.co.uk/news/article-2609200/Primary-pupils-troll-staff-Children-young-four-taking- Twitter-Facebook-RateMyTeacher-insult-teachers-survey-reveals.html [Accessed 5 Mar. 2017]. Harrison, C. (2007). Tip sheet on question wording. Program on Survey Research. Harvard University, pp.1-4. Henry, G. (1990). Practical sampling. 10th ed. United States: Sage Pubns, Thousand Oaks, California, U.S.A. Hensens, W., Srugwig, M., & Dayan, O. 2010. Guest-Review Criteria on TripAdvisor Compared to Conventional Hotel-Rating Systems to Assess Hotel Quality. Passion for Hospitality Excellence: European Council on Hotel, Restaurant & Institutional Education 2010 Conference Proceedings. Amsterdam, 25-28 October: 1-12. How's my feedback?. (2011). How's my feedback?. [online] Available at: http://www.howsmyfeedback.org/category/ratemyteachers/ [Accessed 5 Mar. 2017].
  • 25.
    Bibliography ©Eaquals #eaquals18prague Madariaga, L.,Nussbaum, M., Burq, I., Marañón, F., Salazar, D., Maldonado, L., Alarcón, C. and Naranjo, M. (2017). Online survey: A national study with school principals. Computers in Human Behavior, [online] 74, pp.35-44. Available at: http://file:///C:/Users/larap/AppData/Local/Temp/Rar$DIa0.490/Online-survey-A-national-study-with- school-principals_2017_Computers-in-Human-Behavior.pdf [Accessed 18 Apr. 2017]. MORI, I. (n.d.). The national student survey 2017. [online] Available at: http://www.thestudentsurvey.com/ [Accessed 25 Jan. 2017]. Neuman, W. (2005). Social research methods: Qualitative and quantitative approaches. 5th ed. Boston: Pearson Education (US). Nulty, D. (2008). The adequacy of response rates to online and paper surveys: What can be done?. Assessment & Evaluation in Higher Education, [online] 33(3), pp.301-314. Available at: https://www.uaf.edu/files/uafgov/fsadmin-nulty5-19-10.pdf [Accessed 14 Feb. 2017]. Orna, E. and Stevens, G. (1995). Managing information for research. Buckingham, United Kingdom: Open University Press. Panzini, L. (2017). Figure 1: Minimum sample. [image]. Patankar, F. (2013). Interview with rate your lecturer Co-Founder and director. [online] Available at: http://www.careergeekblog.com/2013/06/11/interview-with-rate-your-lecturer-co-founder-and-director/ [Accessed 4 Mar. 2017].
  • 26.
    Bibliography ©Eaquals #eaquals18prague RateMyProfessors (2016).RateMyProfessors.com – find and rate your professor or campus. [online] Available at: https://www.ratemyprofessors.com [Accessed 4 Mar. 2017]. RMT Acquisition, LLC (2017). Rate My Teacher. [online] Available at: http://uk.ratemyteachers.com/ [Accessed 25 Jan. 2017]. Rumsey and Rumsey, S. (2008). How to find information: A guide for researchers. 2nd ed. Maidenhead, England: Open University Press. Saunders, M., Lewis, P., Thornhill, A. and Thorn.., A. (2009). Research methods for business students (5th edition). 5th ed. New York: Financial Times Prentice Hall. Sterling Ideas Limited (n.d.). Rate your lecturer. [online] Available at: http://rateyourlecturer.co.uk/ [Accessed 12 Feb. 2017]. The Research Ethics Guidebook. (2017). Building ethics into the research design. [online] Available at: http://www.ethicsguidebook.ac.uk/Practitioner-research-and-dual-roles-187 [Accessed 1 Feb. 2017]. TripAdvisor. (2017). TripAdvisor: Read reviews, compare prices & book. [online] Available at: https://www.tripadvisor.co.uk [Accessed 4 Mar. 2017]. Vine, P. (1981). Hotel classification — art or science?. International Journal of Tourism Management, 2(1), pp.18-29. Wai-Ching, L. (2001). How to design a questionnaire. Student BMJ, 9, pp.187-189. World Tourism Organization (UNWTO) (2014). Online guest reviews and hotel classification systems an integrated approach. [online] Available at: http://cf.cdn.unwto.org/sites/all/files/pdf/online_guest_reviews_and_hotel_classification_sytems_an_integrated_ approach.pdf [Accessed 5 Mar. 2017].

Editor's Notes

  • #2 SLIDE 1 I’m the academic manager of the Modern languages department at IHL where we offer 6 main languages for evening and Saturday group classes (once or twice a week); we also offer 121 and closed groups on demand and deliver courses either at the school, in company, on line and abroad.   This research project, conducted in 2017 while studying at Birkbeck University of London, aims to investigate if transferring the rating system from a service industry like hotels to teaching languages is feasible. Hands up if you think it’s a good idea. (Count hands)
  • #3 SLIDE 2 My research question was “Is star rating a feasible option?” At IHL teachers’ performance is regularly assessed internally by students and academic managers and externally by inspectors. The potential benefits to the schools are to introduce an instrument to communicate the quality of its services using a tool that is familiar to our potential customers. A key issue, in transferring the rating system from a service industry like hotels to teaching languages, is to understand the value of rating and what students see important when evaluating their learning experience.
  • #5 SLIDE 4 From the service industry we learn that the main methods by which a hotel’s quality can be assessed are classification, grading and rating, where rating is used to measure the hotel experience overall. The first two methods are based on mainly tangibility and objectivity, the latter on the guest experience and as such it is subjective (Vine, 1981). In further education inspections make use of similar tools to classify or grade schools so their results could be communicated through a star rating style. Classification: The more common classification systems include "star" rating, letter grading, from "A" to "F", such as hotels and motels. Systems using terms such as Deluxe/Luxury, First Class/Superior, Tourist Class/Standard, and Budget Class/Economy are more widely accepted as hotel types, rather than hotel standard. Grading: in the UK the AA (Automobile Association) and the national tourist boards. What can cause confusion is that all three can use stars.
  • #6 SLIDE 5 Students’ tendency to rate their learning experience was confirmed in my secondary research when examining websites created to star rate teachers and lecturers. They are easy to find on the internet and they prove popular with students. Rating a service is something most people have become accustomed to. In fact, for example, TripAdvisor claims to have “over 500 million candid traveller reviews” on their website (TripAdvisor: Read reviews, compare prices & book, 2017).  
  • #8 Milingual Explain Correct Knowledge of topic Experience
  • #9 SLIDE 8 Rate your lecturer Teaching ability Feedback Intranet support Office hours/ approachability SLIDE 6-8 It seems that the categories students are asked to rate are related to their learning experience more than an objective evaluation of the tutor’s knowledge of the subject, teaching techniques or range of techniques. This shows they are mostly in line with the original use of rating which is meant to measure the experience and not objective or tangible categories. So, in theory, this choice should make them fair, like we have seen for hotels customer feedback. However, over the years these websites attracted negative press as reported by the BBC (BBC, 2006), the Daily Mail (Harris, 2014), Reuters (Editorial, 2007) and others. These websites, like Trip Advisor, base their rating system mainly on the student learning experience but not only. Each site offers clears guideline on how it should be used and claims to have filters and rules against offensive and inappropriate comments, still students’ review are not always constructive. “Would you be happy for students to evaluate your teachers under these headings?” “Do you already use similar categories in your feedback?”
  • #10 SLIDE 9 One of the limitations of researching in this sector (EFL and ML) is that English is used as medium so the language and length of the surveys had to be kept simple and short. Age may also be a factor that has an impact on the results of the surveys at IHL: 56% of teachers were age 30-39, and that was the youngest band represented in the survey where the majority of students were age 21-29.
  • #13 SLIDE 12 The survey confirms that all participants’ perception of rating is in line with its definition “it is based on clients’ experience and as such it is subjective (Vine, 1981)” and that they wouldn’t rely on grading for objectivity.
  • #14 SLIDE 13 Schools like International House London would benefit from sharing how they perform in terms of teaching and learning. In fact the most important criteria for choosing a school is excellent teaching, however traditional accreditations and inspections are also on top of the list. This is interesting because teaching, and learning, could be considered the essence of any school. Although they can happen independently from each other – excellent learning does not depend on excellent teaching, in fact learning may happen despite terrible teaching or even lack of it - they are the manifestations of any course and, once again, this consumer behaviour is in line with what the WTO found: “Consumers often use hotel classifications as a filter mechanism, with guest reviews used to make a final selection.” (World Tourism Organization (UNWTO), 2014b). The quality of teaching is measured by inspections and so is students’ progress: school would benefit by sharing this with potential customers.
  • #15 SLIDE 14 Students, when asked to rate their teacher without any guidance to what they should consider, rated our teachers very highly, with just 12.73 below “very good”.
  • #16 SLIDE 15 This couldn’t be more different from teachers expectations: teachers were clear in expressing their fear that rating may allow or even encourage unfair feedback showing an underlying distrust in the method: they focused on the preconceptions that it is a bad idea because students would rate the experience and not the qualities of a good teacher. From the survey one can observe that Teachers are not open to have their teaching skills evaluated by students as they are not knowledgeable. But if it’s true that grading requires knowledge as it’s based on objectivity, rating evaluates personal experience so surely students can evaluate whether the teaching suits them or not.
  • #17 SLIDE 16 In the survey, all participants believe that progress is the most important factor when evaluating a course. Progress is not included in any rating system found in this research as this is something measurable and objective, making the available rating websites coherent with the concept of rating. Measuring learning and progress is part of all major inspections, so an integrated approach, like the one used by Trip Advisor, would insure the inclusion of this very important aspect. This is also in line with the most popular reason for choosing to study at IHL : To improve my level: 51.16% To prepare for an exam: 15.12% To see if I like learning a language: 12.79% For work: 12.79% To socialise: 8.14%
  • #18 SLIDE 17 In fact, students would find rating helpful on a school’s website, to which academic mangers added “if supported by reviews”. In fact “An integrated approach is recommended by the United Nation World Tourist Organization whereby guest reviews add a quality dimension to hotel classifications, thereby refining the classification.” (World Tourism Organization (UNWTO), 2014b). In addition, Hensens proved that, even though some reviews may be false, the quantity of these is negligible making the rating system reliable (Hensens, Struwig, and Dayan, 2010).
  • #19 SLIDE 18 The first and most obvious finding to emerge from this study is that rating is a successful system to share an overall customer experience: it does not sale itself as an objective measurement of quality not it is perceived as one. Our students’ perception of the value of rating is accurate. However, the small sample size of this study did not allow for definite conclusions. f. The second important finding is that a fully integrated system is what customers would find most useful when choosing a school.
  • #20 SLIDE 19 The findings of this study have a number of practical implications: the website and the publicity material could include: 1) How IHL guarantees excellent teaching to its students. 2) An integrated star rating system: Accrediting body should work with schools to translate their grades into star rating. Schools should offer up to date students’ reviews.
  • #21 SLIDE 20 3) A star rating system for language teachers, grouped by language, not as individuals. Good and bad reviews should be added to the website on a regular basis (they need to be recent) and good filters used to insure individual teachers are protected from inappropriate or offensive reviews.