Adolfo Ruiz Calleja "Using social and semantic tech"

1. SEEK-AT-WD: A social-semantic infrastructure for educators to discover and select ICT tools in the Web of Data
Adolfo Ruiz-Calleja
08th, October 2014

2. Outline
GSIC @ University of Valladolid
Introduction:
Context and problem definition
Approach: a social-semantic solution in the Web of Data
Contributions:
SEEK-AT-WD infrastructure
Applications  demo!!
Evaluation
Conclusions
2

3. Outline
GSIC @ University of Valladolid
Introduction:
Context and problem definition
Approach: a social-semantic solution in the Web of Data
Contributions:
SEEK-AT-WD infrastructure
Applications  demo!!
Evaluation
Conclusions
3

4. GSIC @ University of Valladolid
4
Scripting
Tool integration
frameworks
Orchestration
Augmented
reality
Semantic search
Web of Data
Social Network
Analysis
Monitoring
Learning flow
Cloud computing
E-Portfolios
CSCL-EREM
CSCL

5. Outline
GSIC @ University of Valladolid
Introduction:
Context and problem definition
Approach: a social-semantic solution in the Web of Data
Contributions:
SEEK-AT-WD infrastructure
Applications  demo!!
Evaluation
Conclusions
5

6. Marie
Offers tools
Marie
Google
Context: Educational ICT tools registries
6
Google
Docs
MediaWiki
Hot
potatoes
Group
Scribbles
Wikimedia
foundation
Learning situation
Educational ICT tool pool
SELECT...tool...write ... collaboration... peer review...
1.- Google Docs
2.- MediaWiki
Activities
People
Google
Docs
?
I need a tool for my students to write a document collaboratively in a peer review activity
Query: write, collaboration, peer review
Results
Educational ICT tool search system
(semantic or keyword-based)
Registry of
educational ICT tools
(semantic or not)

7. Administers and
publishes tool
descriptions
Registry
administrator
Marie
Offers
tools
Marie
Google
7
Registry of
educational ICT tools
(semantic or not)
Google
Docs
MediaWiki
Hot
potatoes
Group
Scribbles
Wikimedia
foundation
Learning situation
Educational
ICT tool pool
Activities
People
Google
Docs
Educational ICT tool search system
(semantic or keyword-based)
(Non-semantic)
(Semantic)
Sisoft
Context: “Web 1.0 approach”
 The institution assumes all the sustainment effort
 No information about the educational use of the tools

8. Publish tool
descriptions
Publish tool
descriptions
Educators
community
Administers
Registry
administrator
Marie
Offers
tools
Marie
Google
8
Google
Docs
MediaWiki
Hot
potatoes
Group
Scribbles
Wikimedia
foundation
Learning situation
Educational
ICT tool pool
Activities
People
Google
Docs
Social annotation
application
Social annotation
application Educational ICT tool search system
(semantic or keyword-based)
Social registry of
educational ICT tools
(semantic or not) (Semantic)
(Non-semantic)
Context: “Social Web approach”
 The cold-start problem
 The isolation of registries

9. This thesis proposal
9
Federate a social registry of educational ICT tools with third parties
–Import tool descriptions from other registries
–Allow third parties to freely use its data
–Avoids the isolation of educational ICT tool registries
Create an initial collection of ICT tool descriptions using third-party data
–Palliate the cold-start problem

10. 10
Linked Data
Use of semantic technologies to publish and federate data on the Web
–In addition to enable semantic searches
Use of open standards
–RDF [W3C04] to structure the data
–SPARQL [W3C08] to query the data
Data is federated reusing and linking third parties URIs and vocabularies
Linked Data [BL06]: “Set of best practices
for publishing and connecting structured
data on the Web” [Biz09]

11. The Web of Data
11
Web of Data: “Extension of the Web with a global
data space based on open standards” [Hea11]

12. The Web of Data: descriptions of educational ICT tools
12
Domain-specific datasets
Cross-Domain datasets
Few hundreds of descriptions
Small educational communities
No information about educational use
Several thousands of descriptions
Huge general purpose communities
No educational vocabularies

13. Administers
Registry
administrator
Educators
community
Web community
Marie
Offers
tools
Marie
Google
13
Social-Semantic registry
of educational ICT tools
in the Web of Data
Semantic educational ICT
tool search system
Google
Docs
MediaWiki
Hot
potatoes
Group
Scribbles
Wikimedia
foundation
Learning situation
Educational
ICT tool pool
Activities
People
Google
Docs
Social annotation
application
ICT tool
descriptions
Web of Data
Are
federated
A social-semantic approach in the WoD (I)
Publish tool
descriptions

14. 14
Approach
Expected sustain team
Expected advantages
Social- Semantic in the Web of Data
Registry administrator
+
Community of users
+
Web community
Issue 1: Is the information currently published on the Web of Data useful to inform educators about ICT tools?
Issue 2: Does a social-semantic approach allow the collection of educational ICT tool descriptions in a sustainable way?
-Use of third-party non-educative data to palliate the cold-start problem
-Use of the tool descriptions updated by the web community
-Still enable educators to publish and enrich tool descriptions
Limitations for the sustainment of educational ICT tool descriptions
Approach
Sustain team
Problems detected
Web 1.0
Registry administrator
-The institution assumes all the effort
-Few tool descriptions
-Not educational experiences using tools
-The isolation of registries
Social Web
Registry administrator
+
Community of users
-The cold-start problem
-The isolation of registries
A social-semantic approach in the WoD (II)

15. Outline
GSIC @ University of Valladolid
Introduction:
Context and problem definition
Approach: a social-semantic solution in the Web of Data
Contributions:
SEEK-AT-WD infrastructure
Applications  demo!!
Evaluation
Conclusions
15

16. A social-semantic infrastructure
16
 Offer a collection of educational ICT tool descriptions
 Publish educational ICT tool descriptions as Linked Open Data
 Provide semantic structure to the tool descriptions using an
educational ontology
 Allow educators to share their experiences
 Automatically obtain tool descriptions from the Web
Social- semantic
infrastructure
obtain ICT tool descriptions
Educational data
consuming applications
Make ICT tool
Educators
community
Enrich ICT tool descriptions
Educational
annotation applications
publish information
about ICT tools
Web community
(tool providers,
Wikipedia publishers...)
Web of Data publish
data
ICT tool
descriptions
Obtain
descriptions
Educational
ontology
descriptions available
as Linked Open Data

17. SEEK-AT-WD requirements
17
Offer a collection of educational ICT tool descriptions
Publish educational ICT tool descriptions as Linked Open Data
Provide semantic structure to the tool descriptions using an educational ontology
Allow educators to share their experiences
Automatically obtain tool descriptions from the Web
Web community(tool providers, Wikipedia publishers...) Educators communityWeb of Datapublish data ICT tool descriptionsEducational data consuming applicationsObtain descriptionsConvert descriptionsMake descriptions availableEnrich ICT tool descriptions publish information about ICT tools Educational annotation applications obtain ICT tool descriptions SEEK-AT-WDSEEK OntologyLinked Open Data interfaceData publicationinterface

18. SEEK Ontology: Conceptualization
18
Reused ontologies:
–Ontoolcole [VG08]
–Review vocabulary [Hea07]
–DBpedia ontology [Aue07], FOAF, DC... and others
Educational context
–[Kur08] [IEE02] [Jov07] [JA06]...
–Competency questions

19. SEEK Ontology: Example (I)
19
“I need free tools that allow the construction of multimedia documents”
“I need a latex editor for Windows”
“I need a tool to support the synchronous communication in an on-line debate”
Some information needs that can be
expressed using the SEEK Ontology

20. SEEK Ontology: Example (II)
20
Tool:GoogleDocs
EducationalReview:Review000
Label
Google Docs
Publisher
Marie
Description
“Is a free office suite…”
Text
“A very useful tool that…”
Type
Asynchronous text editor
Rating
4
Type
Group tool
MaxRating
5
Supports task
Writing
MinRating
1
Supports task
Commenting
HasEducationalContext
ContextAr
Manages artifact
Text document
EducationalContext:ContextAr
Manages artifact
Spreadsheet
Course
Computers Architecture
Operating system
Web application
AreaOfKnowledge
Engineering
License
Propietary software
LearningGoal
RequirementAnalysis
Developer
Google
TeachingTechnique
Peer review
Has review
Review000
DeliveryMode
Blended
Was used in context
ContextAr
VLE
Moodle

21. SEEK-AT-WD infrastructure:
Software architecture
21
Publish
data
Web of Data
SEEK-KB
Convert
descriptions
Crawler
SPARQL
endpoint
Web
interface
Data publication interface
Educators
community
Web community
(tool providers,
Wikipedia publishers...)
SEEK-AT-WD
Educational data consuming applications
Educational annotation
applications
Obtain descriptions
Make descriptions
available
Enrich ICT tool
descriptions
Obtain ICT
tool descriptions
Publish information
about ICT tools

22. SEEK-AT-WD infrastructure: Obtaining ICT tool descriptions
22
Crawling algorithm
–DBpedia and Factforge are accessed
–Follow your nose [Hea11]
–Automatically discover data sources
Alignment to the SEEK Ontology
–Ontology mappings [Cho06]
–Exploiting the links of the Web of Data

23. SEEK-AT-WD infrastructure: Implementation
23
114 formal relationships stated between the SEEK Ontology and DBpedia vocabulary
–599 additional relationships crawling the Web
–No need to manually create additional mappings to extract data from Factforge
SEEK crawler was implemented as a Java application
SEEK-KB was implemented as an RKBexplorer registry
–Includes web and SPARQL interfaces
–Robust and reliable registry
–Available at: http://seek-rkbexplorer.com

24. SEEK-AT-WD dataset
24
Retrieved from
ICT tool descriptions
Educational contexts
Educational reviews
Web of Data
7935
0
0
Community of educators
49
68
176
TOTAL
7984
68
176
September 2013

25. Applications that interact with SEEK-AT-
WD
25
Web community
(tool providers,
Wikipedia publishers...)
Educators
community
Web of Data publish
data
ICT tool
descriptions
Educational data
consuming applications
Obtain
descriptions
Convert
descriptions
Make
descriptions available
Enrich ICT
tool descriptions
publish information
about ICT tools
Educational
annotation applications
obtain ICT tool descriptions
SEEK-AT-WD
SEEK
Ontology
Linked Open
Data interface
Data publication
interface
Educators
community
U-Seek
Make
descriptions available
Enrich ICT
tool descriptions
obtain ICT tool
descriptions
SEEK
Ontology
Open
Data publication
interface
We-Share

26. Applications: DEMO
26

27. Outline
GSIC @ University of Valladolid
Introduction:
Context and problem definition
Approach: a social-semantic solution in the Web of Data
Contributions:
SEEK-AT-WD infrastructure
Applications  demo!!
Evaluation
Conclusions
27

28. Evaluation issues and methodology
28
Is it possible to
develop a sustainable
registry of educational
ICT tools?
Mixed methods [Tas03][Cre03]
CSCL-EREM [JA09]
Adaptation of the mixed evaluation method by [Mar03]
I 2: Does a social-semantic approach allow the collection of educational ICT tool descriptions in a sustainable way?
I 1: Is the information already published on the Web of Data able to support educators when discovering ICT tools?
ID
Happening
Date
1
Tool descriptions from the Web
Spring 2011
2
SEEK-AT-WD administration
January 2012 – March 2013
3
Data published on We-Share
Winter 2013
1
2
3

29. Evaluation issues and methodology
29

30. Evaluation issues and methodology
30

31. Aim: analyze whether the data available on the Web is useful for educators to discover ICT tools
Preliminary dataset of educational ICT tools from the Web of Data
–Ontoolcole-based dataset [VG08]
–2753 different ICT tool descriptions
31
Tool descriptions from the Web: Happening description (I)

32. Evaluation
Tool descriptions from the Web: Happening description (II)
 [WOZ-x] Wizard of Oz experiment [Kel84]
–12 participants (43 questions)
–Reuse of 33 questions from Ontoolcole evaluation [VG08]
 [EXP-y] Expert panel with 8 TEL practitioners and researchers
 [INT-z] Interviews to 4 participants
49

33. Tool descriptions from the Web: Selected findings and evidence
33
It is possible to generate a useful dataset of educational ICT tools out of data from the Web
–A dataset with 2753 tool descriptions was automatically created
–[INT-1] The most important thing I would expect to retrieve is a short description of each tool
–[INT-2] If a search engine starts with such amount of content, it obviously encourages you to use it
Mappings add a significant value
–[EXP-1]An important added value is that it is possible to answer Ontoolcole questions using external non- educative data

34. Tool descriptions from the Web: Selected findings and evidence
34
Most educators’ information needs are satisfied using external data
–[WOZ-1] Median of results = 134 (SIQR = 58)
–[WOZ-2] 27 out of 32 information needs were satisfied with 5 random results
The results retrieved by the repository are precise enough
–[WOZ-3] 57% of satisfactory results
–[INT-3] I got 54 results for my query. If 10 out of these 54 satisfy my query I think it would be enough

35. 35
Evaluation
SEEK-AT-WD administration: Happening description
Aim: discuss whether the web community can contribute to SEEK-AT-WD sustainment and whether its administration effort may hinder its sustainability
[DAG-x] Analysis of the data gathered by SEEK-AT- WD
–March 2013
[ADL-y] Analysis of SEEK-AT-WD administration log
–Log from January 2012 to March 2013

36. SEEK-AT-WD administration: Selected findings and evidence (I)
36
SEEK-AT-WD automatically generates a dataset that covers almost all SEEK Ontology categories
–[DAG-1] 3/46 tool categories could not be mapped to concepts defined in external vocabularies
–[DAG-2] Several tens of tool descriptions in each category
–[DAG-3] +6000 tool descriptions imported (March 2013)
–[DAG-4] Noisy data appears (as expected)

37. SEEK-AT-WD administration: Selected findings and evidence (II)
37
The web community contributes to the sustainment of SEEK-AT-WD
–[DAG-5] New tool descriptions are imported (e.g. Google Drive or Academic Room)
The SEEK-AT-WD administration tasks can reasonably be done by a technical administrator
–[ADL-1] SEEK-AT-WD administration tasks took 24 hours during the first 15 months
–[ADL-2] Flexibility of the SEEK Ontology and mappings

38. Data published on We-Share: Happening description
Aim: evaluate the possibility of collecting useful data using We-Share
23 participants with experience using ICT in classroom
–No previous knowledge of the applications
[UWS-x] Use of We-Share to publish data
[COM-x] Comparison of two versions of U-Seek
–Based on the TREC workshops [Lag98]
–Using and not using social information
–Metrics: satisfaction on the tools discovered and confidence on the tools selected
[QUE-Y] Questionnaires
38

39. Data published on We-Share: Selected findings and evidence (I)
39
 U-Seek and We-Share allow educators to interact with SEEK-AT-WD
–[COM-1][USW-1] No important usability problems were detected due to the origin of the data they manage
–[COM-2][USW-2] Most participants agreed that these are useful and easy-to-use applications
–[COM-3] In 80 out of 82 observations using U-Seek, the participants satisfied their information needs
–[QUE-1] The main problem for me is the classification of the taxonomies […] But maybe I can solve it with a bit of practice

40. Data published on We-Share: Selected findings and evidence (II)
40
 Educators find useful the data added using We- Share
–[COM-4] No significant difference on the satisfaction of the tools discovered
–[COM-5] Significant difference on the confidence of the tools selected when social information is available (90% confidence interval)
–[QUE-2]At first, I did not understand the exact meaning of the term ‘educational context’. After watching some other reviewed tools, it became clearer
–[QUE-3] I discovered the option ‘Has reviews’!!! (very useful!!!)

41. Wrap up: Issue 1
41

42. Wrap up: Issue 1
42
 The data published on the Web is useful to palliate the cold-start problem
–Educators can discover tools using data from the Web once related to an educational vocabulary
–The support provided by this data to the educators can be improved
 Educators can take advantage of the tool descriptions available on the Web of Data using interactive applications
Is the information already published on the Web of Data able to support educators when discovering ICT tools?

43. Wrap up: Issue 2
43

44. Wrap up: Issue 2
44
 SEEK-AT-WD automatically updates its dataset obtaining information from the Web of Data
 SEEK-AT-WD administration tasks are reasonable to be assumed by a technical administrator
 Educators could be expected to sustain SEEK-AT- WD data
–We-Share is perceived as a useful and easy-to-use application
–The information published on We-Share improves the utility of SEEK-AT-WD
Does a social-semantic approach allow the collection of educational ICT tool descriptions in a sustainable way?

45. Wrap up: Some caveats
45
…but there are some (expected) caveats
–Three happenings with a limited number of participants
•But triangulation increases the trustworthiness of the findings and conclusions
–The SEEK Ontology had an important impact on the results obtained
•It does not describe purpose-specific tools
•Its users are higher-education educators
–Limited amount of social-information available
•Although enough to increase the utility of SEEK-AT-WD

46. Outline
GSIC @ University of Valladolid
Introduction:
Context and problem definition
Approach: a social-semantic solution in the Web of Data
Contributions:
SEEK-AT-WD infrastructure
Applications  demo!!
Evaluation
Conclusions
46

47. Conclusions (I)
47
 There are data sustainability problems in current registries of educational ICT tools
 SEEK-AT-WD follows a social-semantic approach
–Takes advantage of the Linked Open Data to create and update a collection of ICT tool descriptions
–Allows the community of educators to further enrich its dataset
 U-Seek and We-Share are two applications that allow educators to interact with SEEK-AT-WD
–Educators find them useful and easy to use

48. Conclusions (II)
48
SEEK-AT-WD presents sustainability advantages derived from following a social-semantic approach
–Advantages for educators
•Palliate the cold-start problem
•Obtain updated descriptions of ICT tools
•Obtain information published by other communities …and also
•Semantic searches are allowed
•Share their experience using tools in the classroom

49. Conclusions (III)
49
SEEK-AT-WD presents sustainability advantages derived from following a social-semantic approach
–Advantages for registry administrators
•Can reasonably be administered by a technician
–Advantages for the web community
•Third parties can make use of the data managed by SEEK-AT-WD
•Open educational-specific data on the Web

50. Conclusions (IV)
50
 Problems derived from the social-semantic approach
–Importation of noisy data from the Web
•Convenient to curate SEEK-AT-WD dataset
•The precision of the search results decreases
–Educators need to understand SEEK Ontology
–Other problems related to the creation of a community of users
•Engagement

51. SEEK-AT-WD: A social-semantic infrastructure for educators to discover and select ICT tools in the Web of Data
Adolfo Ruiz-Calleja
08th, October 2014