Unleashing Analytics in the Classroom - Using Blackboard Data to maximize your course design and student success.

1. Unleashing Analytics in
the Classroom
Jessica Rodriguez
Gus Roque

2. Resources

4. ● High Level Analytics
● Course Level Analytics:
o Impact on Course Design
o Impact on Student Success & Performance
● Blackboard Analytics
● Q&A

5. What days of the week are the majority
of users logging into Blackboard?
A. Early in the week (M/Tu/W)
B. Later in the week (W/Th/F)
C. Weekend (S/Su)

7. Bb Mobile App (Spring 2015)

8. Bb Mobile App (Spring 2014 to Spring 2015)

9. Bb Mobile App (Spring 2014 to Spring 2015)

10. Logins
Activity
Grades
Participation

11. Analytics &
Course Design

12. Course Design
First, Consider the Purpose of Measuring Student
Engagement.
– Identifying which activities not only engage students but also
produce quality analysis, is critical to designing and adapting online
courses.
– What are the patterns of performance you are trying to evaluate?

13. Course Design
─ When are students logging into my course?
Patterns of Performance
─ Which course resources/tools are being
used most frequently?
─ Which discussions boards generate the
most traffic?
─ What are the patterns of performance in
an online assessment?
─ What are some of my opportunities for
improvement?

14. Course Design
Course Reports
– Course Activity Overview
– Student Overview for Single Course
When are Students Logging into My Course?

15. Course Design

16. Course Design

17. Course Design
– Posting Announcements/Messages
– Materials/Resources being Accessed
[Homepage]
– Time Allotted for Module Content
Completion
How does this Information
Affect my Course Design?

18. Course Design
California State University identified that
the more ‘views’ or visits to the course
home page, the higher the final grade.
(Whitmer, Fernandes, & Allen, 2012).
What material do you house within your
course homepage?

19. Course Design
Which Course Resources/Tools are Being used
Most Frequently?
Course Reports
─ Overall Summary of User Activity

20. Course Design

21. Course Design
– Content Presentation (Tool Modality)
• Video clips, posted documents, etc.
– Supplemental Resources
– Group Tools
– Student Support Cases
How does this Information
Affect my Course Design?

22. Course Design
Course Reports
─ User Activity in Forums
Which Discussion Boards Generate the
Most Traffic?

23. Course Design

24. Course Design
– Number of views on discussion boards can
be an indicator of level of interest of a given
topic.
– Hot Topic / In the News
How does this Information
Affect my Course Design?

25. Course Design
What are the Patterns of Performance in an Online
Assessment?
Item Analysis
─ By Question

26. Course Design
A. Filter the question table by question type, discrimination category, and difficulty category.
B. Investigate a specific question by clicking its title and reviewing its Question Details page.
C. Statistics for each question are displayed in the table

27. Course Design
– Re-evaluate questions that may be
misleading and/or ambiguous
– Pre-work for assessments
– Development of supplemental
resources
– Create varied forms of assessment
How does this Information Affect my Course Design?

28. Course Design
• Enterprise Surveys
– Enterprise Surveys should be used to collect data
on student satisfaction about course items and
reflect on the changes based on the collected data.
What are Some of My Opportunities
for Improvement?

29. Course Design
What are Enterprise Surveys?
• Enterprise surveys are delivered at the system level.
• This system level tool does not replace the course level surveys tool
available to instructors in their courses
• You can send your survey out to multiple sets of recipients and analyze the
results.
• Survey results are compared using response periods, which are sets of
recipients and a time frame that you specify.
• You can also compare by enrollments, membership, demographics, or by a
specific survey question.

30. Course Design
Enterprise Surveys Deployed in Course

31. Course Design
How does this Information Affect my Course Design?
• Ease of Navigation
• Create actionable items based on feedback (e.g. extra time on exam)
• Are the resources within the course sufficient?
• Is there sufficient peer to peer interaction/collaboration?
• Multimedia preference: Pre-recorded lectures versus live lectures
• Are the course and professor expectations clear?
• Determine what day/time works best for live virtual meetings every week.

32. Course Design
Sample Questions

33. Course Design
Instructor Dashboard

34. Analytics &
Student Success

36. Key Question
● How can I identify students who are at risk?

37. Retention Center

38. Retention Center

39. Retention Center
● Missed Deadlines
● Grades
● Course Activity
● Course Access

40. Retention Center

41. Retention Center

42. Key Question
● How can I identify my top performers?

43. Key Question
● How can I identify my top performers?
● Why?
● Send them kudos for being top performers
● Ask them to serve as peer mentors
● Interview them and find out what makes them
successful; share with the class

44. Retention Center

45. Retention Center

46. Retention Center

47. Key Question
● How do I know if a particular student has
been investing time and energy into my
class?

48. Student Activity
• Find out how much time the student is
spending in the course
• Evaluation > Course Reports > Student
Overview for Single Course

50. Course Activity
• Find out how their time spent in the course
compares to other students.
• Evaluation > Course Reports > Course
Activity Overview

51. Course Average

52. Resources

53. Key Questions
● How do I identify students at-risk?
● Retention Center
● How do I identify my top performers?
● Retention Center with customized rules
● How do I know if a particular student has
been investing time and energy into my
class?
● Course Reports:
● Student Overview for Single Course
● Course Activity Overview

54. Blackboard
Analytics

55. ● More robust reporting & analytics tool
● Interactive reports
● Export to multiple formats (XML, CSV, Excel,
TIFF, Word)
Bb Analytics

56. Bb Analytics

57. • This report provides:
– Comparative information against other courses
in the same academic department
– How the students in the course are performing
compared to the average of all students enrolled
in the same course
Course At a Glance

58. Course At a Glance

59. Course At a Glance

60. • This report plots students enrolled in a
course against two variables: the Grade
Center External Grade and Course Accesses.
• This report can help analyze how the activity
of students does (or doesn’t) relate to the
grade recorded in the Grade Center.
Activity & Grade Scatter Plot

61. Grade Scatter Plot

62. • This report might be useful to find students
who fit certain activity and performance
profiles
– Instructors can find students that, for example,
are very active in a course, but who are
struggling from a grade perspective
Activity Matrix

63. Activity Matrix

64. • This report provides a list of students in a course
with submission information for each student.
• This report can be used to identify students who
are not engaged by displaying trends and numbers
of submissions compared to the average across all
students in that course.
• Students below the average may not be sufficiently
engaged in the course and may need assistance.
Course Submission Summary

65. Submissions Summary

66. Blackboard
Analytics:
Student Report

67. Student Report

68. Roll Out Plan
• Spring 2015:
– Implementation & Testing
• Summer 2015:
– Pilot Group
• Fall 2015:
– Full Roll Out

69. Resources
● http://go.fiu.edu/analytics
o Copy of the slides
o Digital version of conference poster
o Cheat Sheet on navigating course reports
o Cheat Sheet for Bb Analytics reports
o Sample ‘End of Survey’ questions

70. • Jessica Rodriguez, Senior Instructional Designer
– rodrigje@fiu.edu
• Gus Roque, Senior Instructional Designer
– groque@fiu.edu
– -@RoqueGus
Contact your Instructional Designer!!!
Contact Us

71. Q&A