Statistic report-bib satisfaction-internet version


Published on

This survey was done on Bachelor of Int'l Business students at the HTW-Berlin (College of Applied Sciences, Berlin) about their satisfaction with the programme and possible areas of difficult / improvement. Alina Sachapow, Vanessa Günther, Kasey Navita Phifer.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • We didn‘t really have a hypothesis and neither could we measure something easily such as brand awareness. Rather, we had a single objective: measuring the satisfaction of the students with their program. It was difficult to be objective when evaluating our own program. We spent many hours on developing the questions.When I show you the survey, I‘ll go into more detail about criticism we found with the questions.Although we didn‘t have a hypothesis per se, we did have a few biases. We thought, for example, that students with job experience might have higher expecttions for the program. We thought that students who have studied at other unis may have more specific (and possibly more demanding) expectations. In order to see if this was true, we included some questions about demographics at the end of the survey.
  • You all have seen the paper version and so can compare the formats and how easy it is to fill out.In terms of thinking about the things that could influence people, we thought about if it would be better to include the htw logo to the survey in order to make it look more professional, but when we originally wantetd to distribute the same surveys to hwr students, they could have felt that this survey has a competitive character and students could be afraid of answering honestly, because they do not want to downgrade their programme. So the second thought was to include both logos on the front page, but then the competitive character would have come out even more. So our conclusion was, that the only way of receiving honest answers would be to have two different layouts: a htw-survey with htw logo and another one for the hwr with their logo on it.
  • Many students might not have come to the program with any expectations or prior work experience. Therefore, we wanted to know what kind of comparison students have in their mind‘s for what they want from their study program.
  • We received a lot of feedback that this question was too vague. One good suggestion was that we should make the question a bit longer and more complex: „what percentage of your teachers would you rate as being: (1) highly competent (2) competent (3) average (4) difficult to understand (5) not helpful at all.
  • A few people didn‘t fill out their age. We thought that age is a crucial element of one‘s expectations, that older people would have more job or uni experience and therefore have higher expectations from the program.We also thought about asking the students for their GPA (grade point average), but thought that many may not know it and also the questionarre would be too lengthy.The gender probably wasn‘t an important question to ask, and we allowed the option of „uncertain“ just to see how many people would take the survey seriously.The mother tongue could play an important role in students‘ understanding of the lectures, but some felt that it singled them out and invalidated their anonymity. We should have had listed the options as „german, english, or other“.
  • The paper survey was two pages long, 18 questions in total.We liked the idea of a paper survey because people are more likely to fill it out when a classmate asks, rather than click on a link during one‘s free time (or forgetting to take the survey). We could also make sure that only HTW BIB students took it and not erasmus or other, for example. We also got valuable verbal feedback which we surely would not have gotten in the online form (example coming later).
  • Statistic report-bib satisfaction-internet version

    1. 1. VS. Student Satisfaction with the International Business B.A. Programme Vanessa Günther - K. Navi Phifer - Alina Sachapow -------------------------- 3 February 2011Applied Statistics & Research Methods – Mr. Holger Lütters – HTW Berlin
    2. 2. Outline1. Press Release2. Management Summary 2.1 Preparation/Hypothesis 2.2 Objectives 2.3 Research Design 2.4 Methodology 2.5 Workpackages3. Topic Relevance 2
    3. 3. Outline4. Findings 4.1 Students Expectations 4.2 Best Subjects and Why 4.3 Worst Subjects and Why 4.4 Satisfaction and Education 4.5 Satisfaction and Age 4.6 Satisfaction and Job Experience 4.7 Teachers´ Abilities 4.8 Programme Criticism 4.9 Programme Improvements 4.10 Positive Remarks 3
    4. 4. Outline5. Conclusion6. Challenges7. Criticism and Possible Improvement 7.1 Survey Design 7.2 Question Formulation 7.3 Process 4
    5. 5. 1.Press ReleaseTeachers Held Largely Responsible for Student Satisfaction,concludes an independent study of Berlin’s College of Applied Science (HTW)students.The push for international competitiveness is becoming stronger and moreintense in Germany, forcing universities to rise up to the challenge in orderto stay on top. The implementation of completely internationally orientedstudy programmes are met with a chorus of praise from both political andacademic circles.But just how effective are these programmes? By measuring studentsatisfaction, a study completed by HTW students explores the practicalapplication of international courses at the HTW both in the classroom andin real life – and the results were quite unexpected: regardless of age orprior life experience, TEACHERS were mainly responsible for studentsatisfaction and the necessary skills they gained (or did not learn) for theirfuture careers. 5
    6. 6. 2. Management SummaryOverall, HTW Students were pleasantly satisfied with their BIB programme; much criticism for improvement was given, along with feedback that satisfaction with teachers played the largest role in satisfaction with the programme (the teachers‘ ability to teach, variety of material and methods of testing, practical application of subject, etc.)Factors such as age and previously completed degrees of higher education, surprisingly played no conclusive role in determining student satisfaction with the HTW‘s BIB programme. 6
    7. 7. 2.1 Preparation/Hypotheses• Research Question: Are the students of the BIB programme satisfied?• Hypotheses: • Students with job experience might have higher expectations for the programme • Students who have studied at other universities may have more specific (and possibly more demanding) expectations • Age and therefore different life experiences could influence how satisfied people are with the programme 7
    8. 8. 2.2 ObjectivesFind out about:• Expectations of the students• Satisfaction with: • Fellow students • Course material • Subjects • Teachers • Administration 8
    9. 9. 2.3 Research Design• Conclusive Research Design: • Objective: Test Hypotheses • Characteristics: • Information necessary: Are BIB students satisfied? Why and why not? • Interviewees: BIB students of all semesters currently studying at HTW • Sample is representative -> about 20 students of every semester covered 9
    10. 10. 2.3 Research Design• Research Process: • Develop a survey with all the important criteria to be able to judge satisfaction • Make a Paper & Pencil and a online version to collect primary data • Ask students in person with the Paper & Pencil version to fill out the surveys • Forward the online version to the 7th semester students • Evaluation and reporting of data 10
    11. 11. 2.3 Research Design• Findings: Should be helpful to draw conclusions about the programme and possible improvements of it Reveal possible areas of improvement or clarification of the research process itself (i.e. Does the survey accurately represent students’ opinions? Does the data clearly outline which areas of the program are working well or poorly?)• Outcome: Findings can hopefully be used to improve the programme 11
    12. 12. 2.4 Methodology• Data Collection: Quantitative survey• 18 questions Questions about satisfaction or dissatisfaction General information (age, gender etc) All questions were not obligatory (some left blank) 10 scale questions (poorly to well and sometimes vice versa) 8 open end questions• 2 versions: Paper & Pencil (handed out in person ) Online via Questfox (sent through email & Facebook links → impersonal) 12
    13. 13. 2.5 Division of Workpackages • Division of labour: Development Creating Preparation Distribution of of Questions Questfox before Survey distributionAlina X X XNavi X X X XVanessa X X X 13
    14. 14. 2.5 Division of Workpackages Typing in the PSPP & Report & Presentation data creating of Evaluation of statistical results tables, chartsAlina X X XNavi X X X XVanessa X X X 14
    15. 15. 3. Topic Relevance• Being students of this programme, we find this survey to be highly relevant!• The idea of a satisfaction survey came to us because there was so much trouble last semester with an unqualified teacher (XXXX, in XXX Dept.). The third semester students and their foreign exchange classmates rallied together to petition the unfair results of the exam. 15
    16. 16. 3. Topic Relevance• Now this semester, we are still hearing some complaints about teachers or subjects and so we thought, „Are students truly unhappy? Or are they just ready to complain about any subject they don‘t like or any class which is difficult?“• Does student satisfaction even have any merit? Do students know what they‘ll need for their future careers, or do they only know how to write exams and memorize formulas? This survey strives to remove the subjective „I hate math“ and put forth an objective reasoning for students‘ expectations and satisfaction with the programme. 16
    17. 17. 4. Findings • Over 25 countries were represented • Twelve people chose to withhold their nationality • Roughly 63% of the BIB students (58 of 92 responses given) are of German nationality • Approximately one-fifth of BIB students are from developing countries 17
    18. 18. 4. Findings Gender of HTW Students Surveyed• Number of Respondants:• Female – 64• Male – 29• „Uncertain“ – 3• No Answer – 8 18
    19. 19. 4. FindingsStudents Surveyed and Their Respective Semester 35 30 No. of Student Responses 25 No Answer 20 1 2 15 3 10 5 7 5 0 Semester 19
    20. 20. 4.1 Student ExpectationsTop 5 Expectations of the programme:1. English Language (44 votes)2. Internationality & Business topics (27 votes each)3. Good career opportunities (16 votes)4. Location Berlin (12 votes)5. Experience abroad (11 votes) 20
    21. 21. 4.1 Student ExpectationsOur hypotheses: After thinking more thoroughly about their expectations and satisfaction, students would change their opinion (namely, that they would be less satisfied). TRUE 21
    22. 22. 4.1 Student ExpectationsQuestion #2: Overall, how well have your expectations for this programme been met?Questions #11: Generally speaking, what is your level of satisfaction with your study programme? 22
    23. 23. 4.1 Student Expectations Student Satisfaction with the HTW BIB Programme1st Time Asked 6 - Well 5 4 3 22nd Time Asked 1 - Poorly 0 10 20 30 40 23
    24. 24. 4.2 Best Subjects and WhyBest subjects: – Macroeconomics with Professor XXX (26 votes) – Microeconomics with Professor XXX (23 votes) – Business Law 1 with Professor XXX (18 votes) – Introduction to Business with XXX (12 votes) – Financial Accounting with Professor XXX (8 votes) 24
    25. 25. 4.2 Best Subjects and Why • Reason for best subjectsSubject teacher Other reasonEconomics Fair, organized, great language skills, theory and practice interesting well combined, very good teacher, huge efforts to subject, good explain content, good material, always prepared, always participation able to answer, competent, motivated opportunities, Import structured, transparent and fair ant for future, plenty assessment, friendly, helpful, understandable of examplesBusiness Law 1 Competent, great knowledge, knows how to teach, cares relevant for future about students, able to teach dry topics in an amazingly career, challenging interesting mannerIntroduction to good teacher, interesting subject, good teaching style, interesting, mostbusiness cool, creative interactive, relevant for future career, not only theoreticalFinancial Good explanation, good structure, good teaching style,Accounting stays in timetable, good speed 25
    26. 26. 4.3 Worst Subjects and Why• Worst subjects: – Research Methods with Professor XXX (12 votes) – Marketing with XXX (11 votes) – Mathematics with XXX (9 votes) – Mathematics with Professor XXX and Corporate Finance with XXX (each 7 votes) – Communication Skills with Professor XXX (6 votes) 26
    27. 27. 4.3 Worst Subjects and Why • Reasons for worst subjectsSubject Teacher Other ReasonResearch Methods Bad teacher, does not know Nothing learnt, irrelevant, what he is talking about, no information partly only in guidance, unjust, German available incomprehensible gradingMarketing Confused, no knowledge about subject, could not answer questions, unorganized, unfair, learnt nothing, subjective evaluation of students, no interest in student’s progress, unclear expectations, uncomfortable atmosphere 27
    28. 28. 4.3 Worst Subjects and WhySubject Teacher Other ReasonMathematics Does not explain well, very impatientMathematics Professor not able to too demanding, very explain, not able to work difficult, very fast with class, not able to catch attentionCorporate Finance Bad language skills, little knowledge about subject, can’t answer questions, doesn’t pay attention to presentations, speaks only RussianCommunication Skills Bad structure, bad very theoretical, absolutely feedback, poor language unnecessary and boring skills 28
    29. 29. 4.4 Satisfaction and EducationOur hypothesis: Students who have completed study programmes (either Bachelor‘s or Master‘s degrees) are less satisfied with the programme than students with no prior university experiences (since they with no experience have nothing to compare with). 29
    30. 30. 4.4 Satisfaction and EducationNumber of students (frequency) who have completed Bachelor‘s 5degree:Number of students who have completed Master‘s degree: 2Total number of students surveyed (N)*: 104Relative frequency of students with degrees of higher education 0.067(Baachelor‘s & Master‘s): The relative frequency is not large enough to provide an accurate, unbiased answer to our hypothesis. *We used the total number of students rather than those who marked „BA, MA, school,“ because logically ALL students must have attended school and received their A-levels before being admitted to the HTW – despite the fact that only 63 students marked „school“. 30
    31. 31. 4.5 Satisfaction and AgeOur hypothesis: Age and therefore different life experiences could influence how satisfied people are with the programme. Conclusion: too difficult to measure • Students ages 20 – 26 nicely dispersed / represented • Only one student age 30 • No students older than 30 31
    32. 32. 4.5 Satisfaction and Age 32
    33. 33. 4.6 Satisfaction and Job ExperienceOur hypothesis: Students with job experience might have higher expectations for the programme and thus a resulting lower level of satisfaction. (slightly) TRUE • 32 Students had part-time work experience • 34 Students had full-time work experience • Some marked both full and part-time • Some marked multiple options other than work 33
    34. 34. 4.6 Satisfaction and Job Experience HTW BIB Student Satisfaction No Work35.00% Experience30.00% (Full & Part-25.00% time) Work Experience20.00%15.00% Total responses without work experience: 12610.00% Total responses of full and part- time work: 66 5.00% Respondants ticking more than one option (i.e. „international 0.00% experience“ and „school“) are 1 - Poorly 2 3 4 5 6 - Well Phifer, Sachapow, Günther 2011 34
    35. 35. 4.7 Teachers´AbilitiesAre students whose mother tongue is English more or less satisfied with their teachers‘ language abilities? Surprisingly, German students held a more negative opinion of their teachers‘ English abilities. This could be due to the fact that most English- speaking students come from African countries, or that most of the native English speakers are in the first and second semester. Or perhaps English-speaking students are just nicer.  35
    36. 36. 4.7 Teachers´Abilities HTW BIB Student Satisfaction with Teachers‘ Language 25 20Number ofRespondents 15(frequency) 10 5Bilingual 0resonpentswere counted Germantwice English Other 2 3 4 5 6 - Bad enough to make you cry 1 - Fantastic 36
    37. 37. 4.7 Teachers´Abilities 37
    38. 38. 4.8 Programme CriticismQuality of teachers• There should be a better quality of teachers and• Higher entrance reuqirements for teachers• More professors would be nice, not only teachers• Better evaluation of teachers• Teachers should be obliged to make TOEFL too• Some teachers as in Introduction to Business are too unorganized 38
    39. 39. 4.8 Programme CriticismSubjects• Some topics feel like a waste of time• Too much learning by heart for exams• There should be a broader variety of languages• Although 2/3 of the course selected HR, XXX failed to manage that everyone who wanted gets a HR topic. There were too many Marketing topics and too few HR topics• Learning target is often not transparent, seems like only filling out the time• Some subjects as research Methods 1 are useless, one doesn’t learn anything 39
    40. 40. 4.8 Programme CriticismAdministration• There was never the chance to complain anywhere, because nobody wanted to help students, „they“ always just tell us how great they are• Students should be better informed about AWE, everyone has different information on that• Unorganized, uncoordinated, nobody takes responsibility, people in charge are not approachable• In the administration nobody has really a clue of how the system is working and if one needs certain information, one has to talk to many people and does not know more in the end. 40
    41. 41. 4.8 Programme CriticismAdministration• Mrs. XXX is rude• XXX is not able to argue on a high level with good arguments, he cannot communicateGeneral Criticism• Some students hate the comparison of their semester to other semesters• Student was not able to get credits from his previous study program for this course because of the language, but they think that those classes were useless here and they did not learn anything 41
    42. 42. 4.9 Programme ImprovementsThings that should be improved• It is not helpful to be forced to do presentations in EVERY class, sometimes it is better to get something explained by the teacher; in the end one has to double and triple check because the presentations are not sufficient, sometimes those presentations take too much time of the class, which could have been used better. If presentations still necessary, then it should rather be done like in economics where professor XXX has his actual lectures and lets students present on related topics so that everybody can understand what is done. 42
    43. 43. 4.9 Programme Improvements• In the 4th semester there should be more options, not only marketing and HR• There are very good and very bad teachers, that is why it is very difficult to assess the program• Adding and elective C „finance“ in the 5th and 7th semester would be really interesting• Having some more online classes would be nice• Some more events to encourage the interaction between Germans and international students would be nice• There is someone needed in mathematics who pushes the students practically 43
    44. 44. 4.9 Programme Improvements• Would be nice to learn more about ethics and about criticism of capitalism• More scholarships should be available for good students to be able to concentrate more on their studies• More co-work of students from the first and higher semesters appreciated• Would be nice to be able to select some courses which are relevant for specialization and skip some irrelevant courses as regional studies• Selection of courses should be a little more flexible (although hard because of the English language)-> would like to take some more courses from other disciplines as law 44
    45. 45. 4.10 Positive Remarks• It is getting better each year• Ms. XXX is a real enrichment for the programme, she does her job really well, although there are many incompetent people around and above her• The staff in the international office is very competent and friendly• Great program, much practice, not too theoretical• It is a nice study program due to its internationality• It is nice that the university tries to make everything online• Some teachers are very good 45
    46. 46. 5. Conclusion• Overall satisfaction was good• Students mostly complained about poor teaching quality (inability to explain, moving too quickly) and subjects which were „pointless“ and „useless“ → graphs still look quite positive because people had no choice but to choose the average quality• [What to change, if we could go back in time] We should have had one hypothesis and based the survey around it (more in-depth)• [What to change, if doing another survey in the future ] Students felt much more comfortable giving feedback and criticism to us in person (perhaps easier to say than to write, or it felt more „off the record“)... Maybe it is better to do personal interviews (qualitative instead of quantitative surveys), especially when students care about their programme and the response rate would be high 46
    47. 47. 5. Conclusion• Evaluation of best and worst subjects is difficult → subjects from 5th and 7th semesters have less chances to be chosen due to the fact that less people attended these classes so far (e.g. Business Ethics) → sometimes we just didnt know which teacher taught when...(e.g. if a seventh semester student says that they liked marketing most, that is hard to find out which teacher they had AND it is also not really comparable to later marketing classes because of the different teacher)• Hard to judge quality of the whole programme → too few students who went through all of it (7th semester or alumnis) → results concerning the level of satisfaction would be completely different if there would have been more higher semester students or alumnis been answering (the programme is very new and faced a lot of problems in the early stages; many people worked hard on the improvement of it already)• Cannot form direct linkage between work experience and satisfaction with our data (needs to be more precise and detailed in certain areas) 47
    48. 48. 6. Challenges• Finding time for everyone in the group to meet (different schedules, living in different locations)• Limited number of questions – Wanted to keep survey as short as possible (should fit on one piece of paper front and back)• Accurate representation – Limited quantity of data because there is no 4th and 6th semester at the moment and the alumnis are not represented – 7th semester students had a different version of the survey (online); the different environment could have impacted their answers 48
    49. 49. 6. Challenges• Timing of survey – Survey done before exam time (students are stressed or scared to criticize teachers or won’t criticize teachers until their marks are received)• Topic too broad – Satisfaction is hard to judge because there are many different factors influencing it – More detail is required to be precise about satisfaction and possible bias of the students… but number of questions and time were limited• Learning PSPP & Questfox• Navi and the American way of thinking 49
    50. 50. 7. Criticism• Survey design (Screenshots of the online survey on the next couple of slides)• Question formulation• Process 50
    51. 51. The Online Survey 51
    52. 52. 52
    53. 53. 53
    54. 54. 54
    55. 55. 55
    56. 56. 7.1 Survey Layout discussionIn case we would have done the comparison:• Putting the logo of the HTW on the front page would make the survey look more professional → but when we originally wantetd to distribute the same surveys to HWR students, they could have felt that this survey has a competitive character and students could be afraid of answering honestly ( nobody would want to downgrade his programme)• We then thought about including both logos on the front page → but that would still cause competition between the two.• So our conclusion was, that the only way of receiving honest answers would be to have two different layouts: a HTW-survey with HTW logo and another one for the HWR with their logo on it. 56
    57. 57. 7.1 Evaluation of Survey Design• Paper & Pencil + Must not be filled out sequentially + Possibly higher response rate + Verbal feedback - Illegible handwriting - Not all questions answered - It is necessary to type up all of the questions - Some students did not attend the classes when we distributed the survey 57
    58. 58. 7.1 Evaluation of Survey Design• Online + Link forwarded to 7th semester students → (otherwise they could not have been reached) + Easier data compilation (Questfox) + 0-drop-out rate from online survey - Not in school environment - Easier to just “click away” - Possible fraud - Time consuming to create an online survey additionally 58
    59. 59. 7.1 Evaluation of Survey DesignOf both forms, overall...• High flexibility of data collection because two different versions• Stimuli used (only personal): Distribution of sweets, opportunity to improve the programme (chance that results will be published)• Little control over the environment (only possible in choosing different courses where to ask the students) 59
    60. 60. 7.1 Evaluation of Survey Design Summarized• Response rate (personal) High (because they could not run away!) High completion rate (100% except a few answers left blank)• Response rate (online): Higher than expected, 100% completion (no drop-outs)• Anonymity: No student ID or name• Speed: It took about 15 minutes to distribute, let it be filled out and collecting it in every class• Bias towards interviewers: Little because we are also BIB students• Costs: Only printing costs 60
    61. 61. 7.2 Question Formulation• Teacher evaluation questions were too general because there are very good and very bad teachers, so the average will always be chosen• One suggestion was that we should make the questions a bit longer and more complex: e.g.:„what percentage of your teachers would you rate as being: (1) highly competent (2) competent (3) average (4) difficult to understand (5) not helpful at all 61
    62. 62. 7.2 Question Formulation• The mother tongue could play an important role in students‘ understanding of the lectures, but some felt that it singled them out and invalidated their anonymity• We should have had listed the mother tongue options as „German, English, or other“ - to make sure that anonymity is given• When we asked about the age, people also were concerned about their anonymity →We should have used age brackets instead to avoid this from happening (age 20 and under, ages 21 – 24, ages 25 – 28, etc.) 62
    63. 63. 7.2 Question Formulation• When we asked about the age, people also were concerned about their anonymity →We should have used age brackets instead to avoid this from happening (age 20 and under, ages 21 – 24, ages 25 – 28, etc.)• In some questions our data might not be accurate because software like questfox and pspp cannot detect spelling mistakes → (e.g. nationality,…) we could have offered again options to tick the box instead• We should have been more precise e.g. when choosing best or worst subject we should have asked the students to include the name of the teachers of the class because it is hard to find out which teacher had which class 63
    64. 64. 7.2 Question Formulation• When people had to tick what they did before studying at HTW in order to find out, if that might have influenced their expectations and therefore their level of satisfaction with the programme→ we did not offer enough choices like Diploma or studies that were not finished → people had to cross other, eventhough they might be highly qualified in some other fields of study• We should have offered a box like some colleges to also be able to recognize these kinds of qualifications 64
    65. 65. 7.3 Process• We did not get any answer from the HWR → we should have contacted them much earlier (as soon as we decided on the topic)• Dependent on other lecturers in terms of reaching them and to get permission to run surveys during their classes → we should have started to prepare this earlier• Better organisation/structure (complete Questfox before starting to type information into it) 65
    66. 66. Thank You HTW-BerlinCollege of Applied Sciences, Berlin Kasey Navita Phifer 66