We discuss the technical infrastructure needed to capture student data in an open learning environment (beyond the LMS), our iterative design process along with dashboard prototypes, and our dashboard evaluation results from focus groups and a survey. Visit BobBodily.com for more information about my research.

1. Designing, Developing, and
Evaluating a Real-Time
Student Dashboard
Robert Bodily, Charles Graham, Ben Mackley, and Tarah Kerr
Brigham Young University

2. Technical
Infrastructure
Learning Tools
Interoperability (LTI)
Experience API (xAPI)
Learning Record Store (LRS)

3. Design Process: Prototype 1

4. Design Process: Prototype 2

5. Design Process: Prototype 3

6. Focus Groups Methods
Four focus groups of 4-6 students
Approximately 60 minutes each
Audio was recorded, transcribed, coded using open coding protocol
Semi-structured format

7. Focus Group Results
Good features - reflection
Bad features - confusing
Requested features - compare to class
Frequency of Use - right before an exam
Course Synchronization - centralized location
Comparison between dashboards

8. End of Course Dashboard Evaluation
Survey Methods
Survey sent to all consenting students
70 student responses

9. End of Course Dashboard Survey Results
Some students did not even know they had access to a dashboard tool
How can we improve student awareness of the system?
Dashboards were not used very much.
Why didn’t student use the dashboard more often?
Dashboard ratings on:
User-friendly
Interesting/Engaging
Useful

10. Future Research
What is the effect of our real-time dashboard on student behavior, achievement,
or skills?
How can we increase student use of our dashboard system?
What would an instructor or administrator dashboard look like?
How can we make them actionable?