Jan-Willem Doornenbal, Blended Learning Coordinator & Chris Groot, Data Engineer; Amsterdam University of Applied Sciences: Basing Course Design on Student Engagement Evidence Using Brightspace Data
Basing Course Design on Student Engagement Evidence Using Brightspace Data
1. Names:Jan-Willem Doornenbal and Chris Groot
Date: 16 May 2022
Evidence Based Course Design (JWD)
Educational Data Platform (CG)
On: D2L Benelux/EU Connection 2022, Nijmegen
Basing Course Design on Student
Engagement Evidence using
Brightspace Data
2. Basing Course Design on Student Engagement Evidence usingBrightspaceData
2
Jan-Willem Doornenbal
Evidence Based Course Design
3. Basing Course Design on Student Engagement Evidence usingBrightspaceData
3
• How does educational design lead to study results?
• From self reflection to self reflection+
• Educational design: from Brightspace data to a dashboard
• Student behaviour: from Brightspace data to a dashboard
Where are we heading?
6. Titel presentatie
6
We use Brightspace data from
Announcements, Assignment summary,
Course awards, Content objects, Forums
topics, grade objects, LTI links,
Organizationalunits, Release conditions
objects, Quiz objects, Rubric objects,
Survey Objects.
And “Cosmos” (our course creation and
enrolment system) data for programs and
year.
Design data used from BSp
7. Basing Course Design on Student Engagement Evidence usingBrightspaceData
7
ORGANISE COUNTING – QUERY CONCEPTUALISING - SCORE MAX
→100
Topics, numberper course CountFROMContentObjects GROUPBYOrgIDWHERE
Type=TopicANDVisible ANDNOTdeleted
0,5 poIntpertopictill max.Thenmax 10
Accessible webpages(both
good for accessibilityand
for offeringinfoin chunks)
COUNT type=webpagebasedonFile-extensionfrom
ContentObjects.Location==>Share of webpagesintopics:
divide the numberof webpages bythe numberof topics
10* score, withmax 5. (thismeans
maxscore whenhalf ormore of your
topicsisa webpage)
5
EVALUATE →100
Grades-tool inuse COUNT fromGrades WHERE NOT DeletedGROUPBY
tool_name (=Quiz/Discussion/Dropbox/External/blanco)
3 points if agrade-itemispresentfora
quiz,3 for dropbox,3 fordiscuss,3 for
external and3 forblanco.
15
Assignments
SubmissionType
Countassignments,GROUPBYsubmission_types (file/text/in
person/onpaper);numberof differentsubm.types
2 if 3or4; 1pt if 2; 0pt if 0or1 2
INVOLVE/SHARE →100
Announcements COUNT FROMAnnouncementsWhere OrgID=x AND
IsDraft=FALSEANDDeletedDate=null andisGlobal=FALSE
2*numbertill Max 15
ACTIVATE →100
Reflection/activationof
studentsw/ lessonmaterial
COUNT FROMLTILinks Where OrgID=x ANDLTI-type=FBF-
interactive doc/video/audioANDVisible
5pt per LTILinkof thistype till Max 25
Design data counted and conceptualised
8. Titel presentatie
8
Other reference
groups
Manually
editing
scores
Suggestions
based on score:
* you could add some
announcements
* You could do
interactive stuff
Accountabilityfor
scores:
* you scored 10/15 on
Quizzes, because you
had 2. Feedback is …
Viewing permissions based course
permissions and direct links
Dashboard visualisation
9. Basing Course Design on Student Engagement Evidence usingBrightspaceData
9
STUDENT ACTIVITY COUNTING – QUERY CONCEPTUALISING - SCORE
Topics visited Count FROM Content User Progress How manyon eachday fromStart course
till Endcourse – weight1
Topics completed CountFROMContentUserCompletion How manyon eachday fromStart course
till Endcourse – weight1
Assignmentssubmitted Documentssubmitted How manyon eachday fromStart course
till Endcourse – weight20
Assignmentssubmittedontime Difference Due date andSubmissiondate How manydaysearlyor late – weight1
pt/dayearly
Assignmentfeedbackread Difference feedbackpublisheddate andFeedback
readdate
5 pointsif readonpublishingdate,counting
down
Discussionposts written Postswritten How manyon eachday fromStart course
till Endcourse – weight5
Discussionposts read Postsread How manyon eachday fromStart course
till Endcourse – weight2
Quizzescompleted Quizattempts How manyon eachday fromStart course
till Endcourse – weight20
LTI-linkslaunched LTI launches How manyon eachday fromStart course
till Endcourse – weight10
Student use data counted
10. Basing Course Design on Student Engagement Evidence usingBrightspaceData
10
Student engagement curve dashboard
Curves per
section/group:
* differentiation
between lecturers
Curves per final
result:
* relation between
engagementand
result
11. Basing Course Design on Student Engagement Evidence usingBrightspaceData
11
Chris Groot
Educational Data Platform
12. Basing Course Design on Student Engagement Evidence usingBrightspaceData
12
• Security
• Flexibility
• Scalable
• Financially acceptable
• According to current and future law development
• Less dependence of external companies, own development
• Data science and AI
• Adding new data sources fast
• Giving students ownership of their own data
The journey of modern data projects
13. Basing Course Design on Student Engagement Evidence usingBrightspaceData
13
• Different source systems with the "truth"
• Brightspace (DLO)
• SIS (Student information system)
• Cosmos (Modern app to connect Brightspace and SIS)
• Many other data sources
• Flexibility of software and hardware choices (scalable)
• Easy to adopt withing the organisation (learning curve)
• Affordable (no "big" license fees)
Project educational data platform
14. Basing Course Design on Student Engagement Evidence usingBrightspaceData
14
• Data lake (storage)
• Databricks (compute)
• Managed Apache Spark clusters
• Data handling service
• Scheduling
• Monitoring
• Keyvault (handling secrets like passwords)
• Power BI
• Active Directory
• Firewall (inbound, outbound)
• VPN
Reference architecture
