Your SlideShare is downloading. ×
Elearning evalfinal
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Elearning evalfinal

169
views

Published on

Published in: Education, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
169
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Evaluation of theCurrent LearningManagementSystem at JMUThe Students’Perspective onBlackboardPresented ByEric JacobsMichael MayJason PooleJessica Robinson
  • 2. Introduction This evaluation plan was prepared by Eric Jacobs, Michael May, Jason Poole, and Jessica Robinson who are graduateIntroduction students in the Educational Technology department atReview of Evaluation James Madison University. Coordinating efforts byProcess Andrea Adams of the Center for Instructional Technology (CIT), and by Dr. Michele Estes of the Educational Technology department at James Madison University have helped advise this evaluation plan. Background Blackboard is a learning management system that has the ability to act as the central hub for the curriculum. This has become increasingly apparent as distance learning programs depend on the features of this evaluand to manage course content and interact with students in a web-based environment. Blackboard’s diffusion wide, claiming more than 1,600 institutions using their LMS (Blackboard, 2011).
  • 3. PurposePurpose of • To determine the needs that students have forevaluation learning management system.Review of EvaluationProcess • To identify the technologies that faculty and students are currently using in teaching and learning • To identify the technologies that faculty and students are no long using in teaching and learning. Stakeholders • Primary Stakeholders • Andrea Adams • Dr. Michele Estes • Secondary Stakeholders • Evaluation team • Students of JMU
  • 4. Decisions to be addressedPurpose of • To what degree does Blackboard meet theevaluation expectations and learning needs of undergraduateReview of Evaluation and graduate students at JMU?Process • Do graduate and undergraduate students feel that Blackboard is a viable solution as a learning management system for JMU over the next 5 years? • To what degree do the features in Blackboard meet the technological needs of the students at JMU?
  • 5. Needs relatedEvaluation What are students needs in regards to using a learning management system in their coursework?questions What expectations do students have about learning managementReview of EvaluationProcess systems? Technology related What technologies are students using both in Blackboard and in their day to day lives? Do students feel that the technology in Blackboard is beneficial to their learning? Features related What features of Blackboard do students find obsolete or in need of updating? What features in Blackboard do students engage with most often? What features in Blackboard do students feel are the most beneficial?
  • 6. Methods ofevaluationReview of Evaluation MethodsProcess • Cross-sectional Survey • Census • Interview
  • 7. Sample for census survey • Students in high Blackboard use coursesThe Sample • 2,000 students or 10% of JMUReview of Evaluation enrollmentProcess • Expected return rate 50% Sample for census survey • ~20,000 students • Expected return rate ~2% Sample for interviews • 2 undergraduate • 2 graduate • Purposive sample of Ed Tech Students
  • 8. Instruments InterviewsReview of EvaluationProcess • Recorded using Elluminate Live! Surveys • Created and administered using Qualtrics.
  • 9. Limitations • Time Constraints • Return rate of survey dataEvaluation plan • Proximity of group members from JMU and each otherReview of EvaluationProcess Logistics & Timeline • Interviews and survey data completed in November Budget
  • 10. InterviewSummary #1 Positives - • Access to grades • The more information on Blackboard the better • E-mails of announcementsInterviewee Data • Organization of content • Discussion boardsGender: FemaleProgram of Study:Communications Major, NegativesEducational Media Minor • Announcements when viewed onClass: Junior BlackboardGeneral Info: Student has • Instructors that don’t updatetaken hybrid courses and iscurrently enrolled in an online Blackboard beyond syllabuscourse. Experience usingBlackboard in high school.
  • 11. InterviewSummary #2 Positives - • Internal Blackboard email tools • Access to everything in central locationInterviewee DataGender: FemaleProgram of Study:Educational Technology NegativesClass: Graduate • Dislike wiki featureGeneral Info: All classeswere taken on Blackboard.
  • 12. InterviewSummary #3 Positives - • All important documents in one place • Online grade book • Roster for contacting classmatesInterviewee DataGender: FemaleProgram of Study: MiddleSchool Education NegativesClass: Junior • Lots of features she doesn’t know howGeneral Info: Overall positive to useexperience with Blackboard • Inconsistency among JMU faculty
  • 13. InterviewSummary #4 Positives • Blackboard and Elluminate are perfect coupling - • Greater connection with course and instructor • E-mail features, discussion board, grade featureInterviewee Data • Links and ResourcesGender: FemaleProgram of Study:Educational Technology NegativesClass: Graduate • Instructor implementationGeneral Info: Overall positive • Instructor Usageexperience using • Collaboration -- Real-time lackingBlackboard. Almost allcourses using Blackboard. • Group sharing is “clunky” • Synchronous info sharing (IM, chat)
  • 14. 1. Please select one: Male or FemaleGenderCensus SurveyData # Answer Response % 1 Male 57 27% 2 Female 151 73% Total 208 100%
  • 15. 2. Please select one: Specific Educational StandingEducationalStandingCensus SurveyData # Answer Response % 1 Freshman 51 25% 2 Sophomore 27 13% 3 Junior 41 20% 4 Senior 48 23% 5 Graduate Student 41 20% Total 208 100%
  • 16. 3. Please select one: Program of StudyCensus SurveyDataCensus SurveyData # Answer Response % 1 College of Buisness 25 12% 2 College of Communications 14 7% 3 College of Education 29 14% 4 College of Health and Human Services 31 15% 5 College of Humanities and Social Sciences 44 21% 6 College of Science, Technology, Engineering 47 23% and Mathematics 7 College of Visual and Performing Arts 8 4% 8 Undecided 10 5% Total 208 100%
  • 17. 4. Based on your course work at JMU, please describeCourse your course interactions with Blackboard.interactions withBlackboardCensus SurveyData # Answer Response % 1 I have taken a fully online 57 27% course that used Blackboard 2 I have taken a hybrid/blended 54 26% course that meets online and in the classroom that used Blackboard 3 I have taken face-to-face 191 92% classroom courses that utilized Blackboard 4 I have taken face-to-face 125 60% classroom courses that do not utilize Blackboard
  • 18. 5. Which technologies play a role in your daily communications?CommunicationToolsCensus SurveyData # Answer Response % 1 social networks 177 85% 2 text messaging 196 94% 3 instant messaging 53 25% 4 email 200 96% 5 other 19 9%
  • 19. 6. What technologies do you regularly use?Regularly usedtechnologiesCensus SurveyData # Answer Response % 1 smartphone 109 55% 2 tablet 14 7% 3 laptop/desktop 194 98% 4 Other 29 15%
  • 20. 7. Would you like to see Blackboard have moreWould like social social features for learning?features inBlackboardCensus SurveyData # Answer Response % 1 yes 64 32% 2 no 57 29% 3 unsure 76 39% Total 197 100%
  • 21. 8. Blackboard fits my expectations of how a learning management system should work.Blackboard fitsexpectationsCensus SurveyData # Answer Response % 1 Strongly Disagree 4 2% 2 Disagree 11 6% 3 Neither Agree nor 46 23% Disagree 4 Agree 114 58% 5 Strongly Agree 22 11% Total 197 100%
  • 22. 9. Overall, how would you describe your experience using Blackboard?Overall experiencewith BlackboardCensus SurveyData # Answer Response % 1 negative 3 2% 2 somewhat negative 15 8% 3 neutral 30 15% 4 somewhat positive 89 45% 5 positive 60 30% Total 197 100%
  • 23. 10. Overall, how difficult is Blackboard to use?How difficult isBlackboardCensus SurveyData # Answer Response % 1 Very Difficult 2 1% 2 Somewhat Difficult 20 10% 3 Somewhat Easy 113 57% 4 Very Easy 62 31% Total 197 100%
  • 24. 11. Please describe your satisfaction level ofSatisfaction with Blackboard as a reliable service.BlackboardreliabilityCensus SurveyData # Answer Response % 1 Very Dissatisfied 2 1% 2 Dissatisfied 13 7% 3 Neutral 33 17% 4 Satisfied 122 62% 5 Very Satisfied 26 13% Total 196 100%
  • 25. 12. Please describe the difficulty in locating information within Blackboard.Difficulty locatinginfo in BlackboardCensus SurveyData # Answer Response % 1 Very Difficult 2 1% 2 Difficult 24 12% 3 Neutral 62 32% 4 Easy 90 46% 5 Very Easy 18 9% Total 196 100%
  • 26. 13. How often do you access Blackboard?How often do youaccess BlackboardCensus SurveyData # Answer Response % 1 Never 0 0% 2 Less than Once a Month 0 0% 3 Once a Month 3 2% 4 2-3 Times a Month 3 2% 5 Once a Week 11 6% 6 2-3 Times a Week 72 37% 7 Daily 107 55% Total 196 100%
  • 27. 14. Please indicate the importance of the listed features for a learning management systemImportance offeaturesCensus SurveyData # Question Not Very Not Important Very Responses Mean Important Important Important 1 Email 16 13 80 78 187 3.18 2 Course 2 5 64 116 187 3.57 Annoucem ents 3 Social 51 86 42 8 187 2.04 Media
  • 28. 15. Please indicate the importance of the listed features for a learning management systemImportance offeaturesCensus SurveyData # Question Not Very Not Important Very Responses Mean Important Important Important 1 Discussion 16 33 94 44 187 2.89 Boards 2 Blogs 34 101 47 5 187 2.12 3 Wiki 41 96 42 8 187 2.09
  • 29. 16. Please indicate the importance of the listed features for a learning management systemImportance offeaturesCensus SurveyData # Question Not Very Not Important Very Responses Mean Important Important Important 1 My Grades 2 1 22 162 187 3.84 2 Test 5 18 58 106 187 3.42 3 Rubrics and 1 10 59 117 187 3.56 Learning Outcomes
  • 30. 17. Please indicate the importance of the listed features for a learning management systemImportance offeaturesCensus SurveyData # Question Not Very Not Important Very Responses Mean Important Important Important 1 Assignment 2 1 41 143 187 3.74 s 2 Calendar 6 33 72 76 187 3.17 3 Groups 13 52 90 32 187 2.75 4 Journals 26 79 57 25 187 2.43 5 Learning 16 61 76 34 187 2.68 Module 6 Survey 22 72 66 27 187 2.52 7 Podcasts 36 83 52 15 186 2.25
  • 31. 18. Which features do you access MOST often in Blackboard?Most oftenaccessed featuresCensus SurveyData
  • 32. 18. Which features do you access MOST often in Blackboard?Most oftenaccessed featuresCensus SurveyData
  • 33. 19. Which features do you access LEAST often in Blackboard?Least oftenaccessed featuresCensus SurveyData
  • 34. 19. Which features do you access LEAST often in Blackboard?Least oftenaccessed featuresCensus SurveyData
  • 35. 20. Blackboard meets my needs in my current courses.Importance offeaturesCensus SurveyData # Answer Response % 1 Strongly Disagree 6 3% 2 Disagree 13 7% 3 Agree 108 58% 4 Strongly Agree 59 32% Total 186 100%
  • 36. Open comments sectionCommentsCensus Survey • Received 35 commentsData • 16 comments directly related to instructor knowledge and implementation of Blackboard • 8 comments related to amount of tools available vs. the few tools used. • Other less frequent comments • Smartphone support • Interface or navigation confusion • Dislike changes from previous versions
  • 37. Example commentsCommentsCensus Survey “I think that blackboard does not need to be overloaded withData features. From my experience, most of the features in blackboard are not used because they are not very important for a classroom setting. Blackboard is useful because it allows an instructor to communicate with his or her students and students to communicate with their teacher. There is no need to improve blackboard or add more features to it because it already does what it does well. In fact, I find that having too many features makes programs stressful and difficult to use because they demand too much of users. I dont want a comprehensive life planner and entertainment center with blackboard. It already does what I ask of it: It provides a way for my teachers to communicate with me about upcoming assignments and grades. It does this very well.”
  • 38. Example commentsCommentsCensus Survey “In my experience, use of blackboard as a learningData management tool largely depends upon the instructor. Only a few of my classes have used discussion board and only one has used Elluminate Live! For most classes, blackboard serves as a way to distribute class documents, announcements, and contact information. Therefore, most of my use of blackboard has been with simple tools, which is really all I need blackboard for.” “Some features are redundant and confusing, such as Journals and Online Journal. There is a lack of uniformity between how different professors use different tools. Its very clunky and clumsy. I wish it had a more seamless user interface, like facebook.”
  • 39. Conclusions & ConclusionsRecommendations • Students are overall pleased with Blackboard as a tool. • Many of Blackboards features are unused in courses. • Students feel instructors us of Blackboard is primary to its success in courses. Recommendations Option 1 Keep Blackboard as JMU’s learning management system and focus attention on course implementation and faculty training. Option 2 Investigate alternative tools that better implement the functions most used by instructors in their courses.
  • 40. Reflections ofevaluation team Positives - • Recording interviews using Elluminate • Data collection using Qualtrics Negatives • Mass student email process • Timing of survey • Evaluators distance from JMU campus