Your SlideShare is downloading. ×
0
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
The indicators project - ASCILITE version
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The indicators project - ASCILITE version

1,505

Published on

A reworked version of presentation for ASCILITE'09 paper (http://tr.im/FD7a).

A reworked version of presentation for ASCILITE'09 paper (http://tr.im/FD7a).

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,505
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
13
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • To the rescue come academic analytics and related work. Folk doing studies of LMS usage through a variety of means. All aiming to give us better data on which to make decisions. All finding certain patterns, correlations and/or relationships
  • But this work is not with out its limitations. There is room for more work, or at least we thought so. Explain each in turn
  • There are significant limitations on what this information can tell on it’s own. Needs to be applied intelligently in a contextual and contingent way. Needs combination with other methods.
  • Let’s talk about the CQU context
  • Quickly talk about the different types of students. Explain the reason for mentioning this is that this is different from purely on-campus. Mixed delivery mode might reveal mixed results?
  • Make the point that most of CQU’s data for decision making shares certain characeteristics with other institutions
  • We embarked on the indicators project, with a specific aim.
  • We’re still taking baby steps and looking for interesting patterns to see if our idea is worthy of moving forward.
  • So now we look at some of the interesting patterns, lets see if you agree. Level of feature adoption
  • Explain what we’re using to measure participation
  • We’re still taking baby steps and looking for interesting patterns to see if our idea is worthy of moving forward.
  • Some quick reflections
  • We’re still taking baby steps, still learning, still wobbling. Need some help.
  • It’s a big task, complex combination of factors, educational, technical, statistical, political, social….
  • There are significant limitations on what this information can tell on it’s own. Needs to be applied intelligently in a contextual and contingent way. Needs combination with other methods.
  • But there’s enough of a hint here to indicate that there are rewards.
  • Transcript

    • 1. Twitter: #eair<br />The Indicators ProjectIdentifying effective learning: adoption, activity, grades and external factors<br />Colin BeerDavid JonesKen Clark<br />http://tr.im/FvRQ<br />
    • 2. Surprising patterns in CQU LMS data<br />Different from established patterns<br />Due to limitations in current work?<br />Apparent benefit in LMS usage analysis that is<br />Cross-LMS<br />Cross-institutional<br />Longitudinal<br />Want to play?<br />http://www.flickr.com/photos/anders-vindegg/3408838186/<br />
    • 3. Overview<br />Context<br />Level of feature adoption<br />Link between participation & grades<br />Reflection and Future work<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 4. Overview<br />Context<br />Level of feature adoption<br />Link between participation & grades<br />Reflection and Future work<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 5. The data that we, educators gatherand utilize is all but garbage.<br />(Wiley, 2009)<br />http://www.flickr.com/photos/dnorman/251646154/<br />
    • 6. (Dawson, 2009; Dawson et al, 2009)<br />(Bakharia et al, 2009)<br />(Dawson et al, 2008)<br />(Black et al, 2008)<br />(Dawson, 2006)<br />(Heathcoate et al, 2005)<br />(Malikowski, 2008; Malikowski, et al, 2007;Malikowski et al, 2006)<br />(Romero et al, 2007)<br />(Griffiths, 2007)<br />(Campbell et al, 2007)<br />(Hornik et al, 2008)<br />http://www.flickr.com/photos/mikebaird/2087879492/<br />
    • 7. http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 8. Limited views<br />LMS reporting tools<br />Only what is in the LMS<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 9. Limited views<br />Some surveys<br />LMS reporting tools<br />Human recollection<br />Only what is in the LMS<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 10. Limited views<br />Some surveys<br />LMS reporting tools<br />Human recollection<br />Only what is in the LMS<br />Error prone<br />Manual checks<br />Time-consuming<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 11. Limited views<br />Some surveys<br />LMS reporting tools<br />Human recollection<br />Only what is in the LMS<br />Error prone<br />Manual checks<br />Time-consuming<br />Limited longitudinal data<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 12. Limited views<br />Some surveys<br />LMS reporting tools<br />Human recollection<br />Only what is in the LMS<br />Error prone<br />Manual checks<br />Time-consuming<br />Limited longitudinal data<br />Limited cross-LMS comparisons<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 13. Limited views<br />Some surveys<br />LMS reporting tools<br />Human recollection<br />Only what is in the LMS<br />Error prone<br />Manual checks<br />Time-consuming<br />Limited longitudinal data<br />Limited cross-LMS comparisons<br />Limited cross-institutional comparisons<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 14. “LMS” situation<br />Students<br />
    • 15. Pre-2010 “LMS”<br />Webfuse 1997-2009<br />WebCT 1999-2004<br />Blackboard 2004-2009<br />
    • 16. Post-2010 “LMS”<br />Webfuse 1997-2009<br />Moodle….<br />WebCT 1999-2003/4<br />Blackboard 2004-2009<br />
    • 17. Students - 2009<br />4436<br />CQ on-campus<br />
    • 18. Students - 2009<br />8444<br />4436<br />CQ on-campus<br />Distance education<br />
    • 19. Students - 2009<br />8444<br />7962<br />4436<br />CQ on-campus<br />Distance education<br />AIC on-campus<br />
    • 20. The data that we, educators gatherand utilize is all but garbage.<br />(Wiley, 2009)<br />http://www.flickr.com/photos/dnorman/251646154/<br />
    • 21. Enabling comparisons of LMS usageacross institutions, platforms and time<br />http://indicatorsproject.wordpress.com<br />
    • 22. http://www.flickr.com/photos/david_jones/36379215/<br />
    • 23. Overview<br />Context<br />Level of feature adoption<br />Link between participation & grades<br />Reflection and Future work<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 24. There are more similarities than differences among LMS<br />(Black et al, 2007)<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 25. Level 1: Most used<br />Level 2: Moderately used<br />Level 3: Rarely used <br />More than 50% of courses<br />More than 20% of courses<br />(much) less than 20% of courses<br />CreatingClassInteractions<br />EvaluatingStudents<br />TransmittingContent<br /><ul><li> Chat
    • 26. Discussion forum
    • 27. Email
    • 28. Quizzes
    • 29. Drop box
    • 30. Content/files
    • 31. Announcements
    • 32. Gradebook</li></ul>ComputerBasedInstruction<br />Evaluatingcourse and instructors<br /><ul><li> Some quizzes
    • 33. Adaptive release
    • 34. Surveys</li></ul>Move to features in unexplored categories, until each category is considered for different learning needs<br />(Malikowski et al, 2007)<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 35. CQU<br />Bb<br />Wf<br />What we did<br />LMS logs<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 36. LMS Independent Data<br />CQU<br />Bb<br />Wf<br />LMS logs<br />What we did<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 37. % of courses adopting “Malikowski features”<br />LMS Independent Data<br />(Malikowski et al, 2007)<br />CQU<br />Bb<br />Wf<br />LMS logs<br />What we did<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 38. # course sites per “LMS”<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 39. Malikowskitop & bottomrange<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 40. Blackboard<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 41. Webfuse<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 42. http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 43. http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 44. http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 45. http://www.flickr.com/photos/wadem/2730257498/<br />
    • 46. The particular trajectory of emergence is not wholly determined either by the intentions of the human actors or by the material properties of technology, but rather by the interplay of the two<br />(Jones, 1999)<br />http://www.flickr.com/photos/wadem/2730257498/<br />
    • 47. Why?<br />Future Work<br />How<br />Will Moodle be different?<br />What about other institutions?<br />Are Malikowski ranges out of date?<br />When is a feature adopted?<br />…….<br />http://www.flickr.com/photos/nedrichards/3234490934/<br />
    • 48. Overview<br />Context<br />Level of feature adoption<br />Link between participation & grades<br />Reflection and Future work<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 49. The results indicated a significant difference between low and high performing students in terms of the quantity of online session times attended during the course<br />(Dawson et al, 2008)<br />Discussion forum activity has been demonstrated to be a sound indicator of future student academic performance.<br />(Dawson, 2009; Morris et al, 2005)<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 50. What about<br />CQU context<br />Different student types?<br />Different levels of staff interaction?<br />(Fresen, 2007)<br />Different staff academic background?<br />http://www.flickr.com/photos/jamuraa/813966437/<br />
    • 51. Feature usage<br />LMS Independent Data<br />(Malikowski et al, 2007)<br />CQU<br />Bb<br />Wf<br />LMS logs<br />What we did<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 52. Feature usage<br />LMS Independent Data<br />(Malikowski et al, 2007)<br />Student records<br />CQU<br />Bb<br />Wf<br />LMS logs<br />What we did<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 53. Feature usage<br />Participation, grades,external factors<br />LMS Independent Data<br />(Malikowski et al, 2007)<br />Student records<br />CQU<br />Bb<br />Wf<br />LMS logs<br />What we did<br />http://www.flickr.com/photos/chokingsun/3495110670/<br />
    • 54. Participation<br />Student/courses<br />N = 510,158<br />Hits<br />Discussion forum<br />Entire course site<br /># replies<br /># posts<br />Just course forum<br />http://www.flickr.com/photos/adactio/1259245482/<br />
    • 55. http://www.flickr.com/photos/david_jones/36379215/<br />
    • 56. Mixed design ANOVA<br />All reported effects are significant<br />Significant interaction effectbetween grades and participation<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 57. Distance education students<br />Hits on course site<br />Hits on course forum<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 58. CQ on-campus students<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 59. AIC students<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 60. FLEX<br />CQ<br />AIC<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 61. Distance education students<br /># of replies<br /># of posts<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 62. CQ on-campus students<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 63. AIC students<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 64. The results indicated a significant difference between low and high performing students in terms of the quantity of online session times attended during the course<br />(Dawson et al, 2008)<br />Discussion forum activity has been demonstrated to be a sound indicator of future student academic performance.<br />(Dawson, 2009; Morris et al, 2005)<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 65. http://www.flickr.com/photos/wadem/2730257498/<br />
    • 66. What about<br />CQU context<br />Different student types?<br />Different levels of staff interaction?<br />(Fresen, 2007)<br />Different staff academic background?<br />http://www.flickr.com/photos/jamuraa/813966437/<br />
    • 67. Course groups based on staff hits<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 68. High staff participation courses<br />Hits<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 69. High staff participation courses<br />Replies & posts<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 70. Super low staff participation courses<br />Hits<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 71. Super low staff participation courses<br />Replies & posts<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 72. Posts comparisons<br />High<br />Superlow<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 73. The results indicated a significant difference between low and high performing students in terms of the quantity of online session times attended during the course<br />(Dawson et al, 2008)<br />Discussion forum activity has been demonstrated to be a sound indicator of future student academic performance.<br />(Dawson, 2009; Morris et al, 2005)<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 74. http://www.flickr.com/photos/wadem/2730257498/<br />
    • 75. Examination of super-low<br />Future Work<br />- Course size<br />Forum content<br />Networks of interaction<br />(Bakharia et al, 2009; Dawson, 2009)<br />http://www.flickr.com/photos/nedrichards/3234490934/<br />
    • 76.
    • 77. What about<br />CQU context<br />Different student types?<br />Different levels of staff interaction?<br />(Fresen, 2007)<br />Different staff academic background?<br />http://www.flickr.com/photos/jamuraa/813966437/<br />
    • 78. Course offerings with<br />n=167<br />Staff with grad certs in L&T<br />n=4<br />Significant input from instructional designer<br />n=196<br />Staff who have received teaching awards<br />N=362<br />Staff teaching education courses<br />n=3288<br />None of the above<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 79. None of the above<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 80. Grad cert in L&T<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 81. Teaching awards<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 82. Education discipline<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 83. Instructional design<br />http://www.flickr.com/photos/orinrobertjohn/3321067724/<br />
    • 84. Overview<br />Context<br />Level of feature adoption<br />Link between participation & grades<br />Refleection and Future work<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 85. Some interesting patterns<br />Useful idea<br />Needs more work<br />http://www.flickr.com/photos/mikebaird/2985066755/<br />
    • 86. Research<br />T&L Practice<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />LMS Independent Data<br />Institution A<br />Institution B<br />Institution C<br />LMS 1<br />LMS 2<br />LMS<br />LMS<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 87. Research<br />T&L Practice<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />LMS Independent Data<br />Enabling<br />Institution A<br />Institution B<br />Institution C<br />LMS 1<br />LMS 2<br />LMS<br />LMS<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 88. Research<br />T&L Practice<br />What<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />LMS Independent Data<br />Enabling<br />Institution A<br />Institution B<br />Institution C<br />LMS 1<br />LMS 2<br />LMS<br />LMS<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 89. Avg course hits<br />By age<br />DE under grad students<br />http://www.flickr.com/photos/wolfgangstaudt/2279651479/<br />
    • 90. Why<br />Research<br />T&L Practice<br />What<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />LMS Independent Data<br />Enabling<br />Institution A<br />Institution B<br />Institution C<br />LMS 1<br />LMS 2<br />LMS<br />LMS<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 91. Why<br />Research<br />T&L Practice<br />How<br />What<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />Useful Data/Information<br />LMS Independent Data<br />Enabling<br />Institution A<br />Institution B<br />Institution C<br />LMS 1<br />LMS 2<br />LMS<br />LMS<br />http://www.flickr.com/photos/atbaker/1577665185/<br />
    • 92. http://www.flickr.com/photos/david_jones/36379215/<br />
    • 93. http://www.flickr.com/photos/dmatos/540688952/<br />
    • 94. http://www.flickr.com/photos/mwichary/2188958154/<br />
    • 95. http://www.flickr.com/photos/sylvar/1138341328/<br />
    • 96. http://www.flickr.com/photos/fncll/145149313/<br />
    • 97. http://indicatorsproject.wordpress.com<br />http://tr.im/FvRQ<br />http://www.flickr.com/photos/jamuraa/813966437/<br />
    • 98. Bakharia, A., E. Heathcote, et al. (2009). Social networks adapting pedagogical practice: SNAPP. Same places, different spaces. Proceedings ascilite Auckland 2009. Auckland: 49-51.<br />Black, E. W., K. Dawson, et al. (2008). &quot;Data for free: Using LMS activity logs to measure community in online courses.&quot; Internet and Higher Education11(11): 65-70.<br />Black, E., D. Beck, et al. (2007). &quot;The other side of the LMS: Considering implementation and use in the adoption of an LMS in online and blended learning environments.&quot; Tech Trends51(2): 35-39.<br />Campbell, J., P. DeBlois, et al. (2007). &quot;Academic analytics: A new tool for a new era.&quot; EDCAUSE Review 42(4): 40-42.<br />Coates, H., R. James, et al. (2005). &quot;A Critical Examination of the Effects of Learning Management Systems on University Teaching and Learning.&quot; Tertiary Education and Management 11(1): 19-36.<br />Dawson, S. (2009). &quot;&apos;Seeing&apos; the learning community: An exploration of the development of a resource for monitoring online student networking.&quot; British Journal of Educational Technology: in press.<br />Dawson, S., L. Macfadyen, et al. (2009). Learning or performance: Predicting drivers of student motivation. Same places, different spaces. Proceedings ascilite Auckland 2009, Auckland.<br />Dawson, S., E. McWilliam, et al. (2008). Teaching smarter: How mining ICT data can inform and improve learning and teaching practice. Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. Melbourne: 221-230<br />Dawson, S. (2006). &quot;Online forum discussion interactions as an indicator of student community.&quot; Australian Journal of Educational Technology22(4): 495-510.<br />Fresen, J. (2007). &quot;A taxonomy of factors to promote quality web-supported learning.&quot; International Journal on E-Learning6(3): 351-362.<br />Griffiths, M. E. (2007). Patterns of user acitivity in the Blackboard course management system across all courses in the 2004-2005 academic year at Brigham Young University. Department of Instructional Psychology and Technology., Brigham Young. Master of Science: 117.<br />Heathcoate, L. and S. Dawson (2005). &quot;Data Mining for Evaluation, Benchmarking and Reflective Practice in a LMS.&quot; E-Learn 2005: World conference on E-Learning in corporate, government, healthcare and higher education.<br />Hornik, S., C. S. Saunders, et al. (2008). &quot;The Impact of Paradigm Development and Course Level on Performance in Technology-Mediated Learning Environments.&quot; Informing Science. The international journal of an emerging transdiscipline11(11).<br />Jones, M. (1999). Information systems and the double mangle: Steering a course between the scylla of embedded structure and the charybdis of strong symmetry. Information Systems: Current Issues and Future Chalenges. T. Larsen, L. Levine and J. DeGross. Laxenburg, Austria, IFIP: 287-302.<br />
    • 99. Malikowski, S., M. Thompson, et al. (2007). &quot;A model for research into course management systems: bridging technology and learning theory.&quot; Journal of Educational Computing Research36(2): 149-173.<br />Malikowski, S., M. Thompson, et al. (2007). &quot;A model for research into course management systems: bridging technology and learning theory.&quot; Journal of Educational Computing Research36(2): 149-173.<br />Malikowski, S., M. Thompson, et al. (2006). &quot;External factors associated with adopting a CMS in resident college courses.&quot; Internet and Higher Education9(3): 163-174.<br />Morris, L., C. Finnega, et al. (2005). &quot;Tracking student behavior, persistence, and achievement in online courses.&quot; Internet and Higher Education8(3): 221-231.<br />Romero, C. b., S. n. Ventura, et al. (2007). &quot;Data mining in course management systems: Moodle case study and tutorial.&quot; ScienceDirect Computers and Education (51 (2008)): 368-384.<br />Salmon, G. (2005). &quot;Flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions.&quot; ALT-J, Research in Learning Technology13(3): 201-218.<br />Wiley, D. (2009). &quot;The LHC and education.&quot; Retrieved 29 November, 2009, from http://opencontent.org/blog/archives/1098.<br />

    ×