Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (19)
Similar to Indicators research expo
Similar to Indicators research expo (20)
Recently uploaded
Recently uploaded (20)
Indicators research expo
- 1. Enabling comparisons of LMS usage across institutions, platforms and time http://indicatorsproject.wordpress.com indicatorsproject@gmail.com
- 3. systems (LMS) are software systems that synthesize the functionality of computer- mediated communications software and online methods of delivering course activities and materials
- 4. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 5. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 6. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 7. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 8. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 9. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 10. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 11. LMS come in many different flavours http://farm1.static.flickr.com/113/278221145_758080af62.jpg
- 13. Learning management systems have become almost ubiquitous as a technical solution to e- learning within universities (Beer, Jones & Clark 2009) http://farm3.static.flickr.com/2042/2217375343_c55801ed85_b.jpg
- 14. The rapid adoption of LMS has occurred in a vacuum of research into their teaching and learning effectiveness (Lopes, 2008)
- 15. Rates of attrition for online students range between 20-50% higher than on- campus students (Dawson, Macfadyen, & Lockyer, 2009) http://farm3.static.flickr.com/2106/2477266021_17875c4ac3.jpg
- 16. Distance learning can be difficult http://farm1.static.flickr.com/30/60732740_61abdc2f9e.jpg
- 17. increasingly independent and perhaps isolated forms of study (Coates, James & Baldwin 2005) http://farm1.static.flickr.com/3/3500076_8bcfce68eb.jpg
- 18. increasingly independent and perhaps isolated forms of study (Coates, James & Baldwin The LMS2005)learning is a environment that demands a certain level of self discipline (Douglas & Alemanne, http://farm1.static.flickr.com/3/3500076_8bcfce68eb.jpg
- 19. The LMS influences students’ confidence and motivation with their learning, both of which can influence their levels of engagement (Beer 2010) http://farm3.static.flickr.com/2790/4065213505_4a02dfd033.jpg
- 20. Learning Online No one can hear you scream http://farm4.static.flickr.com/3651/3561662932_e04d335aa4.jpg
- 23. How are students using LMS? http://farm5.static.flickr.com/4023/4425849701_d89b8aacea.jpg
- 24. How are students using LMS? How are teachers using LMS? http://farm5.static.flickr.com/4023/4425849701_d89b8aacea.jpg
- 25. LMS have the ability to capture extensive amounts of data about individual user and designer behaviour (Heathcoate & Dawson, 2005).
- 26. What can LMS data tell us about successful students? http://www.flickr.com/photos/carbonnyc/143186839/
- 27. What can LMS data tell us about unsuccessful students? http://farm3.static.flickr.com/2123/2308371224_60e0cda6e8.jpg
- 28. What can LMS data tell us about the link between teacher behaviours and student success?
- 29. What is the Indicators project? http://farm4.static.flickr.com/3182/2949435839_a53d509915.jpg
- 30. The Indicators is a research project aims to build on and extend prior work in the analysis of usage data from Learning Management Systems (Beer,Jones & Clark 2009) http://farm3.static.flickr.com/2248/2253657555_9306374c9f.jpg
- 31. Project achievements to date 2009 CQUniversity teaching and learning grant Paper accepted. 2009 Refereed Ascilite2009 2009 Data used to assist in the Moodle transition 2010 Masters thesis on student engagement 2010 Masters thesis relating to staff engagement 2010 2 x papers recently submitted to be refereed for Moodlemoot Australia Journal paper in progress http://farm4.static.flickr.com/3200/3000043603_a1061664b4.jpg
- 33. We are looking for patterns in the data http://farm3.static.flickr.com/2672/3684420623_796346b171.jpg
- 34. The Indicators project is looking at two aspects of the data http://farm2.static.flickr.com/1275/1133080323_6a15a78e4a.jpg
- 35. What the student does! http://farm2.static.flickr.com/1275/1133080323_6a15a78e4a.jpg
- 36. What the student does! What the teacher does! http://farm2.static.flickr.com/1275/1133080323_6a15a78e4a.jpg
- 41. Your turn! • Assuming that click count is related to engagement, place the following groups in order of their levels of engagement online. • Under 20s • Under 30s • Under 40s • Over 40s
- 43. What do academics do on the LMS? http://farm4.static.flickr.com/3353/3541911418_2f629f37d8.jpg
- 44. Staff interaction Crucial for students (Fresen 2007) http://farm4.static.flickr.com/3422/3177467913_1a4567c0aa.jpg
- 45. Quality of teacher presence (Dawson & McWilliam 2008) http://farm1.static.flickr.com/155/349762358_309e5d9f52.jpg
- 46. An academic’s approach to teaching + Use = involvement
- 48. Table 1: Dimensions delimiting approaches to online teaching – (Gonzalez, 2009, p311) Informative/individual learning Communicative/ focuses Networked learning Small range on media and tools Wide range of media and focused used to support learning tasks and tools used to support Intensity of use activities (mainly sources of learning tasks and information with small opportunities activities (with emphasis for interaction and communication) on interaction and Web pages with information. Web pages with communication) Resources information. Lecture Lecture notes. Links to websites. notes. Links to web sites. Select and present information Design spaces for sharing Role of the Discussion boards. Chat. and communication. teacher Blogs. Spaces for sharing. Support the process Animations. Videos. Still Role of the Study individually information Participate in a process of images. students provided knowledge building
- 49. Course Coordinators posting to forums ! http://smartblogs.com/socialmedia/files/2009/04/forums-300x299.jpg
- 50. An academic’s number of forum messages compared to the university average (all courses) !
- 51. Student Hits for Course 1 - 2005-2009 ! !
- 52. Student Hits for Course 2 - 2005-2009 ! !
- 53. Student Hits for Course 3 - 2005-2009 ! !
- 54. Student Hits for All Courses 3 - 2005-2009 !
- 58. How do we make use of the vast amount of data captured by an LMS? http://farm4.static.flickr.com/3363/3199296759_e5130dc6c1.jpg
Editor's Notes
- Before explaining the Indicators project it may be best to give some background on what learning management systems are, and how they are being used in higher education.
- Learning management systems are typically enterprise web systems that allow teachers to upload course content, they facilitate online discussion forums and provide mechanisms for assessment upload most provide some sort of gradebook functionality.
- There are a wide variety, perhaps hundreds, of LMS platforms in the world today ranging from large commercial LMS like Blackboard to the very popular open source LMS Moodle.
- Blackboard was one of two LMS at CQUniversity until the start of 2010. The other LMS was a home grown system called Webfuse.
- in 2009 Moodle was piloted as a single replacement LMS for CQUni. In 2010 Moodle became CQUniversity’s single LMS.
- This is an example of a course the students will see when they log into Moodle.
- Like CQUniversity almost all universities are using a learning management system or LMS for delivering online courses.
- However it has really only been in the last ten years that LMS have gained widespread adoption and use and research into their teaching and learning effectiveness is still in its infancy.
- This is despite poor rates of student attrition for online students.
- any learning by distance can be difficult and one reason is difficult is due to the isolation from other learner.
- Online learning via an LMS can lead to feelings of isolation.
- and it requires a degree of discipline on behalf of the student to persist with their online studies as there are fewer external sources of motivations when compared to face-to-face classrooms.
- Student confidence and motivation with their learning is linked to their level of engagement with their course material, peers and teachers.
- Because of these and other factors, students can feel disconnected from their university learning community.
- and often the teachers are unaware of this occurring as they don’t have regular face to face contact with the student where they could sense a flagging level of commitment within the student.
- The teachers cannot see the glint in their eyes and therefore are not getting any feedback on the students commitment to their studies. So in terms of LMS and its influence on university teaching and learning. We set about finding out how both the students and staff are using the LMS.
- We really don’t know how the students are using the LMS
- and we are often unsure how the teachers are using the LMS to deliver their course materials and activities.
- Fortunately the typical LMS records an enormous amount of log data. They record the mouse clicks of each and every user in the system and this data can be mined for information that can help inform and improve teaching and learning.
- For example we can look at the online clicks and patterns for students who received passing grades and
- compare them to students who weren’t so successful. And hopefully the difference can tell us something that we can use to improve the way that we are delivering our online courses.
- We can also investigate the captured data to see what the link is between staff behaviours and student success.
- How do we do it?
- Assuming that student grade is a reasonable indicator of student success this graph shows a correlation between the number
- This graph shows the correlation between student engagement, indicated by the number of hits on the LMS and staff participation on LMS discussion forums.
- This study focussed on how an academic interacts with the LMS, how the LMS is used by students, and how these interactions create involvement.
- Fresen (2007) in researching web-based learning identified staff interaction with students as one of the key factors in student engagement. Dawson and McWilliam (2008, p. 27) also point out that it is not only staff interaction that is crucial, “the quantity of ‘teacher presence’ and quality of ‘teacher presence’ are influencing factors in developing and maintaining student online engagement.”
- Dawson and McWilliam (2008, p. 27) also point out that it is not only staff interaction that is crucial, “the quantity of ‘teacher presence’ and quality of ‘teacher presence’ are influencing factors in developing and maintaining student online engagement.”
- The method taken in this study is that an academic’s approach to teaching + Use = involvement (a key factor in student engagement according to Krause, 2005).
- Gonzales (2009) examined a small number of academic member teaching online postgraduate courses and applied Kember and Kwan’s (2000) Teaching Approach, and Robert’s (2003) study on online teaching of campus students to characterise the academic’s conceptions of teaching postgraduate courses at an Australian university. Kember and Kwan’s (2009) approach is based on two broad, though linked, teaching points of reference: teacher centred/content-oriented’ which include two conceptions of teaching (‘teaching as imparting information’ and ‘teaching as transmitting structured knowledge’); and ‘student-centred/learning-oriented’, which also includes two conceptions of teaching (‘teaching as facilitating understanding’ and ‘teaching as promoting intellectual development/conceptual change’) (p. 300) Based on this information, Gonzalez reached the conclusion that there are two broad approaches to teaching, what he classed as “‘informative/individual learning focused’ and ‘communicative/networked focused’” (see Table 1 below). What Gonzales created was a model/framework that could classify academics of online education as similar to, or different from on campus educators. What was not discussed in this paper was how such a framework could be used to develop pedagogical strategies that could enhance an academic’s teaching so that they could move from a content oriented approach to a learning oriented approach.
- where the simple query was run asking the question, “How many academics post to discussion forums?” 73% of Course Coordinators have no postings to the forums. The figure is staggering even taking into account that some courses are placeholders for Honours, and Masters subjects. The sheer volume of Course Coordinators that do not post even when provided with a forum is an interesting fact. Of course, more research would have to be done to see if any internal or external factors are involved, but it appears to be very significant in light of teacher involvement.
- I examined three courses - The result (Figure 2) indicates that there was a high level of communication occurring in the course Forums that the academic coordinates compared to the average for all academics across all courses at ‘Some University’.
- I examined three courses - The result (Figure 2) indicates that there was a high level of communication occurring in the course Forums that the academic coordinates compared to the average for all academics across all courses at ‘Some University’. What is intriguing about this data from this one course is that there seems to be a direct link between student hits and grade (Beer et al, 2009; Dawson & McWilliam, 2008), both in content and in the forums. While this is one of the first year courses it highlights the need for students to engage with the content and the forums, though it seems as though the higher the grade the better the engagement with content and with other students and staff via the forums.
- So what is heppening here? While the trends here demonstrate some interesting patterns, especially the dip with the Credit students, identifying the reasons behind these patterns require additional research methods, including surveys. However, the suggested pattern reveals that there is a link between student hits and student grades (Beer et al, 2009; Dawson & McWilliam, 2008) though the link is much weaker in this Course than in the previous course. Of course, whether this link is causal requires more research. This is a good example of the complexity of using the "LMS indicators" to measure the performance of staff, there are other factors that need to be taken into consideration which is outside of the scope of this study but interesting for future research.
- Agin a strange pattern though the overall indication is that there is a loose correlation between grade and forums
- Dawson and McWilliam (2008) argue that grade seems to be an indicator of involvement and although they do not state that there is a causal relationship, something is occurring. Aggregating all of the academic’s courses into one data set demonstrates that there is a relationship between students’ use of the LMS features and their final grade. Examining the total of all these courses across all terms that they have run on Blackboard (Figure 6) demonstrates that there is a direct link between student hits and grades (Beer et al, 2009; Dawson & McWilliam, 2008) and between student engagement with the forums and grades, which is indicative of involvement with the courseware.
- 67 % of students receiving a yellow or red warning improved their effort and grade. (Purdue University 2009)