Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to We Don't Know What We Think (Darrel Lawlor, Michelle Vanginneken)
Similar to We Don't Know What We Think (Darrel Lawlor, Michelle Vanginneken) (20)
More from COHERE2012
More from COHERE2012 (12)
Recently uploaded
Recently uploaded (20)
We Don't Know What We Think (Darrel Lawlor, Michelle Vanginneken)
- 3. Year 2004 2014 Enrollment 547 4362 Courses 22 112
- 5. Our Big Underlying Question:
- 6. 1. Demographics -- how are we doing overall? 2. Mobile devices 3. Expectations 4. Major assessments 5. Motivation to learn 6. Why choose online? 7. Learning preferences
- 7. 0% 2% 81% 17% Female Male
- 8. 47% 19% 9% 13% 11% 1% 18-24 25-31 32-38 39-45 Over 45 Undisclosed
- 9. 22% 16% 54% 8% Location Rural Area Urban Area (not Regina) Regina Other
- 10. 36% 18% 26% 19% 1% 1 2 3 4 or more Unspecified
- 12. -- 50% said it doesn’t matter
- 13. MORE COURSES OFFERED IN SUMMER Faculties do not want online competing with face to face
- 14. Our Big Underlying Question:
- 15. Failing website
- 16. Satisfied 54% Somewhat satisfied 33% Somewhat dissatisfied c Dissatisfied 7% 6%
- 17. How Are We Doing?
- 18. available to answer questions within 24 hours 54% approachable and offers multiple ways to connect respond but timing could be better (within 48) 45% 26% frustrating and always late with responses (after 48) instructor did not engage in my online course at all I feel uncomfortable asking questions, so I don’t ask 12% 9% 7%
- 19. How Are We Doing?
- 20. Very important Important Somewhat important Not important No answer: 1% 64% 25% 6% 4%
- 21. How Are We Doing?
- 22. Excellent Good Fair Poor 43% 43% 11% 3% No answer: 1%
- 23. How Are We Doing?
- 24. All the time Sometimes Never 24% 52% 23% No answer: 1%
- 26. 60% 50% 40% 30% 20% 10% 0%
- 27. 0% 20% 40% 60% 80% Completing Quizzes Submitting Assignments Videos Accessing PowerPoint Accessing PDF Documents Reading Course Content Accessing Readings Posting to Discussions Reading Discussion Forums Email
- 28. What is UR Courses Mobile NOT Effective at doing? 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Email Reading Discussion Forums Accessing Readings Reading Course Content Posting to Discussions Accessing PDF Documents Accessing Videos Accessing PowerPoint Completing Quizzes Submitting Assignments
- 33. 12% 18% 29% 9% 4% 11% 11% 2% 4% Harder than a face-to-face course A course with defined start and end dates Self-paced - a course where you can start and finish when you want Personalized learning with opportunities to choose what you will do with little or no interaction with others Fully interactive and collaborative (i.e. with student-to-student interaction and instructor-to-student interaction) A course that follows a weekly schedule Other: Please specify
- 35. 58% selected: “In lieu of a final exam, a combination of quizzes and assignments.”
- 36. Several graded quizzes 48% One final exam 15% In lieu of a final exam a combination of quizzes and assignments 58% Major individual project 27% Term paper 36% Team presentation 4%
- 37. “We must find ways to get our students to use and practice over time the new course material that we ask them to learn. Traditional methods of letting students listen passively to lectures and then use a short burst of intensive study to cram for a test ...does not result in students forming new neural networks” (p. 19). Terry Doyle (2012)
- 38. “By definition, a teaching approach that allows for cramming fails to meet the definition of learning. Cramming results in a hollow victory for students. They often earn a passing grade but they are unable to recall or use most of the information even a week later” (Bjor, 1994; Ebbinghaus, 1913) (p. 20). Terry Doyle (2012)
- 41. Grades 66% Apply knowledge to real world Choice of assignments Timely feedback Participation by instructor Engaging with other students 74% 42% 46% 37% 28%
- 43. Extremely valuable a Valuable Somewhat valuable Not valuable 14% 30% 33% 21%
- 45. Extremely valuable a Valuable Somewhat valuable Not valuable 8% 34% 38% 17%
- 47. 75% selected: “Flexibility of location and time.”
- 49. Videos a 65% Downloadable text (i.e. PDF) Powerpoint (text only) Powerpoint (with audio) Online conversations Text within URCourses Audio files Live real-time chats 60% 43% 43% 26% 23% 21% 21%
- 50. Michelle van Ginneken michelle.vanGinneken@uregina.ca Darrel Lawlor Darrel.lawlor@uregina.ca
Editor's Notes
- Darrel/Michelle slide: introductions In this session we will show results from an electronic survey that was completed by online students at the University of Regina and we will compare our assumptions and practices to what students say and wherever possible show related research that backs up our findings. So the first question is: Do we really know what we think we know? Educators often think they know what is best for their students but then, when you talk to students, you find the opposite. When we hear from students themselves, what we think can be at odds with the reality of the student experience. MV – learning can be uncomfortable (like a really hard chair with a tack in it) What motivates students to learn? Are students motivated by grades or something else? Who are our online students? Why do they take online courses? What are they doing with their mobile devices? What do they have to say about their online learning experiences? Are there areas of contradiction between our practices as educators and student habits? Are there identifiable themes that we can use to improve how we teach? What should, or could, we be doing differently? Are there things we do to help students that actually drive them to crazy?
- Michelle slide Centre for Continuing Education, more specifically ‘Flexible Learning Division’ This fall we ran 34 online courses with 1332 enrolments Average enrolment is 39 students We are a cost-recovery unit which offers several delivery methods: Fully online, Blended Televised F2F at Saskatchewan Regional Colleges Evenings Weekends & Summer courses We have 6 FTE & 3 part-time instructional designers. 2 dedicated to the Faculty of Nursing & 1 at First Nations University of Canada We currently use Moodle as our Learning Management System for our online courses - called ‘URCourses’ .
- Michelle slide Then 2004 we ran 22 online course with 547 enrolments 2014 we ran 112 online course with 4362 enrolments
- Darrel slide Year 1 we held focus groups Year 2 we held focus groups Year 3 we conducted a survey based on the themes arising from years prior
- Michelle slide We chose an open ended question for Y1 & Y2 student focus groups. used the feedback gathered to develop a survey In this session we’ll examine student feedback with an eye for revealing discrepancies between what we think and what students know.
- Darrel Slide A coop student – a fourth-year Arts student majoring in English, named Ben Woolhead – looked at the data and put the basic findings into nice charts that we could use. With the help of our coop student, Ben Woolhead, we broke the results of the survey into 7 focus areas: 1. Demographics of survey participants (michelle) -- and we will look at how we’re doing overall 2. Mobile devices & their usage (Michelle) 3. Student expectations 4. Major Assessments 5. Motivation to learn 6. Why choose online? 7. Learning preferences. We begin with the demographics of our survey participants to see if we have a fair representation.
- Michelle slide One full year of online students were sent the survey – 1234 individual students 454 responded 38% Overwhelmingly female responses 81% institutionally we have about 60% female
- Michelle slide About 50% are 18-24 Other 50% were varying ages - 25 and over Our participants have mostly taken 1 (36%) or 4+ (26%) online courses We had a balanced amount of students from all 10 Faculties. ½ our students stated they were Y1 & Y2 within their studies. This may not be a true reflection but rather a result of most of our online offerings fall within introductory & 2nd year courses.
- Michelle slide 22% were from a rural area 16% were from an urban area that was not Regina 54% were from Regina. 8% were from other locations (such as?)
- Michelle slide 36% of respondents had taken 1 online course And 26% had taken 4 or more online courses. And we had a balanced amount of students from all 10 Faculties. ½ our students stated they were Y1 & Y2 within their studies. Please note that this may not be a true reflection of our student body, but a result of most of how our online offerings are mostly within the introductory 100 and second year 200 level courses. So, now we have a fairly good idea of who the students who completed this survey were. Let’s move to the real heart of the data.
- Darrel slide Interestingly enough, 6% of our participants didn't realize they had registered for an online course! The U of R registration system does not provide easy to course searching for online, blended and non-traditional courses, Visually, there isnt any cues to distinguish. Just the word web in the room # area.
- Darrel slide 50% said it doesn’t matter
- Darrel slide Do you prefer to take an online course in the Spring, Summer, Fall or Winter? Over 50% percent didn’t have a preference and choice ‘It Doesn’t Matter’. And Spring Summer only account for 6%. The desire is not there for the summer months. Ironically, these are our highest enrolment terms. Sometimes students are forced to take classes during this time as there are not other face-to-face offerings during this time. Faculties do not like online competing with face-to-face.
- Michelle slide We chose an open ended question for Y1 & Y2 student focus groups. used the feedback gathered to develop a survey In this session we’ll examine student feedback with an eye for revealing discrepancies between what we think and what students know.
- Darrel slide 66% of students have never travelled to our website for information, online course lists, or preparatory course documents.
- Darrel Slide 54% were Satisfied 33% were somewhat satisfied 7% were somewhat dissatisfied 6% were dissatisfied So, one could say, based on this survey at least, we’re doing not bad. Pretty good. Room for improvement for the 13% dissatisfied.
- Michelle slide This question pertains to online instructors. We’ve identified as a best practice the critical importance of the interaction of the instructor with the student. So, how are we doing? Your experience with online instructors has been...
- Michelle slide They are often available when I have questions in the course (within 24 hours) – 54% They are approachable and offer multiple ways to connect if questions arise – 45% They respond to questions but the timing could be better (within 48 hours) – 26% I find them frustrating and always late with responses (after 48 hours) – 12% My instructor did not seem to engage in my online course at all – 9% I feel uncomfortable asking questions, so I don’t ask the instructor questions – 7%
- Darrel slide How important is it for you to have an engaged instructor (or Teaching Assistant)?
- Darrel slide Very important to most. The people who don’t care so much wanted a self-directed course. Incidentally, 6 % of respondents wanted a self-directed online course. SUMMARY: 38.9% of the people who thought that an engaged instructor was NOT IMPORTANT expected a self-directed course (compare to 29% out of everyone expected self-directed). Same with Somewhat Important answerers (37.9%).
- Michelle slide Question: Overall, was your instructor proficient in UR Courses?
- Michelle slide A fairly good response – looks like only a handful of people found their instructor had difficulties with UR Courses. Unfortunately, judging from comments (throughout the survey), there are a few very bad online instructors.
- Darrel slide
- Darrel slide 76% of those surveyed do at least sometimes actively seek out fully online courses. Interestingly, that leaves almost a quarter of respondents saying they never seek out online courses. Incidentally, in a related question we found that very few students actually go to the Flexible Learning Division web site within CCE to find information about online courses. The main source continues to be UR Self Service.
- Michelle slide What mobile devices are students using?
- Michelle slide No surprises. iPhone, android phone and iPad are the main mobile devices used by students.
- Michelle slide Email, reading forums, posting in forums, accessing readings. Further to this we found that we do a pretty good job with email and forums. But, we were most interested in areas that we’re not doing a good job.
- Michelle slide According to students UR Courses is not effective for submitting assignments, completing quizzes, accessing powerpoints, accessing videos, accessing PDF documents, posting discussions, reading content. What does this mean? This may not be of huge concern because students already said they are not in the habit of doing these things with their mobile devices but, nevertheless, we should make students aware of what Mobile devices are good at and not so good at. Mobile is less on our radar now because the upgraded Moodle we now have adjusts to mobile formats.
- Darrel slide Let’s take a look at student expectations. What percentage of students thought that online would be easier than face to face?
- Darrel slide 12% thought online would be easier than face to face
- Michelle slide What percentage of students thought online would be harder than face to face?
- Michelle slide 18% thought online would be harder than face to face but that’s just part of the story. Let’s dig into student expectations.
- Darrel/michelle slide It seems that students aren’t quite sure what to expect when they sign up to study online. It’s a gamble. They told us that in in our face to dace focus group discussions as well. Seems students aren’t sure what to think about online courses. We do our best to inform them but students really won’t know what to expect because A) not all online courses are the same and B) you can provide orientations but until you actually take an online course, you’re not sure what it is. Let’s stop for a moment to consider what this all means. Let’s explode the ideas presented in this graph to see where it takes us. Let’s tease with some ideas and unpack the data. There’s a lot to think about here. 11% of students thought that online would be “Personalized learning with opportunities to choose what you will do with little or no interaction with others” AND very few students - 4% - expected online learning to be interactive and collaborative and yet, we like to think that they are. We focus on collaboration and communication between and among students when we design online courses BUT that’s not what students expect or want, at least according to our focus group conversations. If we think that courses should be collaborative, why don’t students? To me the biggest factor here is the notion by students that courses are “Self-paced.” 29% of students expected their online course to be self-paced and they could start and finish when they wanted. 29% said that. Why would they think that? What does our division name do to that notion. We’re the Flexible learning division. This isn’t to say that we should give what students expect, but would it help students to know what to expect. They walk into a face-to-face classroom and they have a pretty clear idea of what to expect. Students have consistently told us that forums and workload are problematic for them. Is pacing something we should be more aware of? (4% no answer 2% other)
- Michelle slide Let’s look at how students want to be assessed. We asked: What is your preference for major assessment in online courses? What do you think is the number one way students would like to be assessed?
- Michelle slide 58% selected “In lieu of a final exam, a combination of quizzes and assignments.” What other forms of assessment are preferred?
- Michelle slide Alrighty then - as we go through this, I want you to think about what your practice is as an educator and what we do as institutions to assess our students. Does what we do relate to what we know about how people learn? What percentage of students said they’d like “Several graded quizzes”? What percentage of students said “One final exam”? What do you think? Why do we have one final exam? Is it the best way for students to learn? In lieu of a final exam a combination of quizzes and assignments Major individual project Term paper Team presentation The least favourite options: Team presentation and One final exam. Some of the comments included: “not group projects (especially those requiring meeting); difficult to write in-person finals for some students.” DL: How does the student preference for, in lieu of a final exam, a combination of quizzes and multiple assignments” compare with current research into the brain and how people learn.
- Darrel slide Terry Doyle - pictured here speaking to Faculty on April 18, 2012 – said, in his book Learner-centered teaching Putting the research on learning into practice: Permit me to read the quote, ““We must find ways to get our students to use and practice over time the new course material that we ask them to learn. Traditional methods of letting students listen passively to lectures and then use a short burst of intensive study to cram for a test ...does not result in students forming new neural networks” (p. 19). In other words, big all inclusive final exams are not the way to go if you want students to remember what they cram into their brain. He further said, Image source: http://www.stanly.edu/news/headlines/1178-dr-terry-doyle-speaks-at-scc.html
- Darrel slide “By definition, a teaching approach that allows for cramming fails to meet the definition of learning. Cramming results in a hollow victory for students. They often earn a passing grade but they are unable to recall or use most of the information even a week later” MV: Scaffolding assignments to mitigate. Image source: http://www.stanly.edu/news/headlines/1178-dr-terry-doyle-speaks-at-scc.html
- Michelle slide How does the above finding relate to student motivation? We asked students “What motivates you to learn?”
- Michelle slide Grades? Applying knowledge to real world ? Having a choice of assignments Timely feedback Participation by instructor Engaging with other students Other 2% So, what is the #1 thing that motivates students (at least, according to this survey)? Applying knowledge to real world situations. How does that relate to our means of major assessment? The Highest response: Applying knowledge to real world situations. Interesting note: all first and second year students answered this way. Lowest answer: Engaging with other students. Other responses: graduation; finishing degree. (Forums take time. Forum complaints came up in focus groups.) AND YET, WE CONTINUE TO MAKE FORUMS SO IMPORTANT. In the focus groups of the past students consistently said that they often learn anything in the forums. They just go through the motions to satisfy course requirements. Students frequently say they are more interested in what the instructor has to say than other students.
- Darrel slide Do you find participation in mandatory discussion forums a valuable way to learn?
- Darrel slide People are kind of on the fence here. By the numbers, most did feel that the forums were at least somewhat valuable, but the 59 comments tell a different tale. Almost every single comment was some sort of complaint or bad review about discussion forums. MV: - Question why you have a weekly forum (less is more) Major concerns: lack of participation; too much participation (hard to keep up/read all posts); often one real answer – first to posts gets the best marks; not enough instructor participation (here to learn from instructors, not students); forced discussion leads to tailored answers and discussions to please the instructor/get marks; can be time consuming for very little benefit.
- Michelle slide Do you find participation in quizzes a valuable way to learn?
- MV Slide Interestingly enough, the answers for this question are similar to those concerning forums. Some comments: if answers are given, then yes (seems obvious that answers should be given); don’t waste time on something not worth marks; depends on type of course.
- Darrel slide Why did you choose to take a course online?
- Darrel slide Not surprisingly 75% selected “Flexibility of location and time.” What other forms of assessment are preferred?
- MV slide If you could choose how to best learn course material, what methods would you choose? (check all that apply)
- Michelle & Darrel slide Interestingly enough, the answers for this question are similar to those concerning forums. Some comments: if answers are given, then yes (seems obvious that answers should be given); don’t waste time on something not worth marks; depends on type of course. What alternate things are you doing – in terms of quizzes - in online courses? MV Importance is to have variety. (HERE IT SEE IT VIDEO)
- MV slide If you could choose how to best learn course material, what methods would you choose? (check all that apply)