SMWCon Fall 20

1. Semantic MediaWiki for
Assessment in Education
Lloyd Rutledge, Open Universiteit
SMWCon Fall 2016, September 29th, 2016

2. OU courses with wiki-based assessment
• Model-Driven Development (MDD)
– Exam on paper about data modeling
– Cathedron tool for self-study and preparation
• Development Practicum (Ontwikkelpracticum)
– MDD techniques and tools using Cathedron
– Figures from 5 orders delivered as databases
• Semantic Web
– Data modeling, but different than for databases
– Figures from 5 assignments delivered as datasets
• Graduate thesis researching Semantic Web and Semantic Wikis
Pagina 2

3. Pagina 3

4. Model-driven development
from OU course T37121 Model-driven development workbook

6. MDDexamquestion

7. MDDexamanswer

8. MDDassessmentinput

9. MDDassessmentinput
Groupingoftestconcepts
Pulldown for comment options
Other Form templates for additional information

10. MDDassessmentoutput

11. MDDassessmentoutput

12. MDD assessment summary
• Student delivers exam on paper
• Grader assesses “pile” of exams
• Semantic Forms replaces ink on paper
• Pulldowns per point select comment and points
• Wiki adds points to calculate grade
• Wiki generates administrative overview spreadsheet
• Wiki digitally stores comments per exam
Pagina 12

14. Practicum Route

15. Practicum assignments
• 4: Create a data model with rules using Cathedron
– Student: Liver database with model and rules
– Teacher dogfood: SQL queries for data structure and rules
• 5: Convert big old database to model from 4
– SQL queries for old data in new structure
• 4 & 5: Queries generate readable analysis and wiki form code
• To do
– 6 - Student programs rules: even with queries
– 2 & 3: Student analyzes of data structure in document table

16. SQL maps data patterns to comments
Pagina 16
SELECT COUNT(*) AS "TGL: CNT_achternaam" FROM Contactpersoon WHERE achternaam != '';
SELECT COUNT(*) AS "TGL: CNT_voorletters" FROM Contactpersoon WHERE voorletters != '';
SELECT COUNT(*) AS "TGL: CNT_voornaam" FROM Contactpersoon WHERE voornaam!= '';
SELECT COUNT(*) AS "TGL: CNT_tussenvoegsels" FROM Contactpersoon WHERE tussenvoegsel!= '';
SELECT COUNT(*) AS "TGL: CNT_geslacht" FROM Contactpersoon WHERE geslacht != '';
SELECT COUNT(*) AS "TGL: CNT_M" FROM Contactpersoon WHERE geslacht = 'M';
SELECT COUNT(*) AS "TGL: CNT_V" FROM Contactpersoon WHERE geslacht = 'V';
SELECT COUNT(*) AS "TGL: CNT_O" FROM Contactpersoon WHERE geslacht != 'M' AND geslacht != 'V';
SELECT COUNT(*) AS "TGL: CNT_telefoon" FROM Klant_nfa WHERE nfa_type='telefoon' OR nfa_type='Telefoon';
SELECT COUNT(*) AS "TGL: CNT_email" FROM Klant_nfa WHERE nfa_type='e-mail' OR nfa_type='e-Mail';
SELECT COUNT(*) AS "TGL: CNT_mobiel" FROM Klant_nfa WHERE nfa_type='mobiel' OR nfa_type='Mobiel';
SELECT COUNT(*) AS "TGL: CNT_xport_datum_2099" FROM Opdracht WHERE datum_transport='01-01-2099';
SELECT COUNT(*) AS "TGL: CNT_Spoed" FROM Opdracht WHERE spoed_='J';
SELECT COUNT(*) AS "TGL: CNT_Spoed_in_note" FROM opdrachtnotitie WHERE tekst LIKE '%spoed%';
SELECT COUNT(*) AS "TGL: CNT_Verzekerde" FROM opdracht WHERE verzekerde_waarde IS NOT NULL;
SELECT COUNT(*) AS "TGL: CNT_Gewenst" FROM opdracht WHERE Datum_transport_gewenst IS NOT NULL;
SELECT COUNT(*) AS "TGL: CNT_Nmrs_w_b" FROM klant_nfa WHERE nfa_type = 'telefoon' AND nummer_adres
SELECT COUNT(*) AS "TGL: CNT_datum_blokkering" FROM Klant WHERE datum_blokkering IS NOT NULL;
SELECT COUNT(*) AS "TGL: CNT_blokkeringsreden" FROM Klant WHERE blokkeringsreden IS NOT NULL;
SELECT COUNT(*) AS "TGL: CNT_Inv_klant" FROM opdracht WHERE klant='3974' OR klant='4877' OR klant='678

17. SQL results help grader
Pagina 17

18. SQL results converted to SMW template code
Pagina 18
s/^ *TGL: # Value rules n====* *n *[1-9].*$/|OPR 4 rules value geen=No/p
s/^ *TGL: # Value rules n====* *n *0.*$/|OPR 4 rules value geen=Yes/p
s/^ *TGL: # att'ed Rules n====* *n *[1-9].*$/|OPR 4 rules geen atts=No/p
s/^ *TGL: # att'ed Rules n====* *n *0.*$/|OPR 4 rules geen atts=Yes/p
s/^ *TGL: CNT_voorletters n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voorletters=No/p
s/^ *TGL: CNT_voorletters n==* n *0[^0-9]*$/|OPR 5 Klant CP voorletters=Yes/p
s/^ *TGL: CNT_voornaam n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voornaam=No/p
s/^ *TGL: CNT_voornaam n==* n *0[^0-9]*$/|OPR 5 Klant CP voornaam=Yes/p
s/^ *TGL: CNT_tussenvoegsels n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP tussenvoegsels=No/p
s/^ *TGL: CNT_tussenvoegsels n==* n *0[^0-9]*$/|OPR 5 Klant CP tussenvoegsels=Yes/p
s/^ *TGL: CNT_geslacht n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP geslacht=No/p
s/^ *TGL: CNT_geslacht n==* n *0[^0-9]*$/|OPR 5 Klant CP geslacht=Yes/p
s/^ *TGL: CNT_telefoon n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel nfa=No/p
s/^ *TGL: CNT_telefoon n==* n *0[^0-9]*$/|OPR 5 Klant Tel nfa=Yes/p
s/^ *TGL: CNT_email n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant Email at=No/p
s/^ *TGL: CNT_email n==* n *0[^0-9]*$/|OPR 5 Klant Email at=Yes/p
s/^ *TGL: CNT_mobiel n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel mobile=No/p
s/^ *TGL: CNT_mobiel n==* n *0[^0-9]*$/|OPR 5 Klant Tel mobile=Yes/p

19. SQL results converted to SMW template code
Pagina 19
s/^ *TGL: # Value rules n====* *n *[1-9].*$/|OPR 4 rules value geen=No/p
s/^ *TGL: # Value rules n====* *n *0.*$/|OPR 4 rules value geen=Yes/p
s/^ *TGL: # att'ed Rules n====* *n *[1-9].*$/|OPR 4 rules geen atts=No/p
s/^ *TGL: # att'ed Rules n====* *n *0.*$/|OPR 4 rules geen atts=Yes/p
s/^ *TGL: CNT_voorletters n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voorletters=No/p
s/^ *TGL: CNT_voorletters n==* n *0[^0-9]*$/|OPR 5 Klant CP voorletters=Yes/p
s/^ *TGL: CNT_voornaam n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voornaam=No/p
s/^ *TGL: CNT_voornaam n==* n *0[^0-9]*$/|OPR 5 Klant CP voornaam=Yes/p
s/^ *TGL: CNT_tussenvoegsels n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP tussenvoegsels=No/p
s/^ *TGL: CNT_tussenvoegsels n==* n *0[^0-9]*$/|OPR 5 Klant CP tussenvoegsels=Yes/p
s/^ *TGL: CNT_geslacht n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP geslacht=No/p
s/^ *TGL: CNT_geslacht n==* n *0[^0-9]*$/|OPR 5 Klant CP geslacht=Yes/p
s/^ *TGL: CNT_telefoon n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel nfa=No/p
s/^ *TGL: CNT_telefoon n==* n *0[^0-9]*$/|OPR 5 Klant Tel nfa=Yes/p
s/^ *TGL: CNT_email n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant Email at=No/p
s/^ *TGL: CNT_email n==* n *0[^0-9]*$/|OPR 5 Klant Email at=Yes/p
s/^ *TGL: CNT_mobiel n==* n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel mobile=No/p
s/^ *TGL: CNT_mobiel n==* n *0[^0-9]*$/|OPR 5 Klant Tel mobile=Yes/p

20. Pagina 20
Practicum assessment input

21. Pagina 21
Practicum assessment inputOrdergiventhat
Helpsgrading
Some values automatically
prefilled by SQL queries

22. Pagina 22
Practicum assessment output

23. Pagina 23
Practicum assessment output
Generated email to student

24. Pagina 24
Comment page

25. Query for overview of grade for assignment
Pagina 25

26. Assignment grade analysis

27. Practicum assessment summary
• Student delivers assignments as databases
• Digital deliverable enables some automation of comments
• Semantic Forms buttons enable quick observations
• SMW query results generate feedback e-mail
• Enables new feedback more flexibly than MDD
– New comments from new insights during grading easier
– Drawback is grade not definite but relative
– Each comment plus or minus points not portion of grade
Pagina 27

28. Pagina 28

32. Semantic Web
• Development Practicum techniques adapted
– SPARQL scripts, not SQL
– Testing wiki remains the same
• Linked with research topic
– Semantic wiki's research focus
– Semantic wiki’s turned out not to be very suitable for student
assignments

33. Masters thesis research
• Making Semantic MediaWiki’s that provide …
– Assessment based on Rubrics
– Administration of educational program policy
– All apply Semantic Web logic and academic domain models
• Generating form property clustering and order from …
– Simple ontology heuristics, compared to text processing
– Complex ontology heuristics and data analysis
• Intelligent feedback from patterns in data models
– For business rules in non-wiki tool
– Principles apply here too
Pagina 33

34. Sqwizzki — The Quizzing Semantic Wiki

35. Summary about SMW for assessment
• Semantic Forms enables quick input
• Forms automatically prefilled from digital deliverables
– Then grader: confirms, corrects, adds more
• Feedback for students automatically generated
• Learning analytics via SMW queries
• Also: generated links to SPARQL endpoint
– General student data, such as grades in other courses
– XLST-template generates XHTML from result table
Pagina 35

36. Questions
Pagina 36