SlideShare a Scribd company logo
1 of 34
Download to read offline
 

Using Linked Data in Learning Analytics
LAK 2013 tutorial
EvaluaHon	
  of	
  Linked	
  Data	
  tools	
  for	
  Learning	
  AnalyHcs
	
  
Hendrik	
  Drachsler	
  (@hdrachsler,	
  drachsler.de)	
  
(CELSTEC,	
  Open	
  Universiteit	
  Nederland,	
  NL)	
  
Eelco	
  Herder	
  
(L3S	
  Research	
  Center,	
  DE)	
  
Mathieu	
  d’Aquin	
  (@mdaquin,	
  mdaquin.net)	
  
(Knowledge	
  Media	
  InsHtute,	
  The	
  Open	
  University,	
  UK)	
  
Stefan	
  Dietze	
  	
  
(L3S	
  Research	
  Center,	
  DE)	
  
Example of scientific competitions

	
  

What are the evaluation criteria of Robot Wars?
Criteria:
•  Damage
•  Aggression
probabilistic combination of
•  Control
– Item-based method
– User-based method
•  Applause
– Matrix Factorization
– (May be) content-based method

2
RecSysTEL Evaluation criteria
1. Accuracy
2. Coverage
3. Precision
4. Recall

1. Accuracy
2. Coverage
3. Precision
4. Recall

1. Effectiveness of learning
2. Efficiency of learning
3. Drop out rate
4. Satisfaction

Combine approach by
Drachsler et al. 2008

1. Reaction of learner
2. Learning improved
3. Behaviour
4. Results

Kirkpatrick model by
Manouselis et al. 2010
3
TEL RecSys::Review study

	
  

Conclusions:
Half of the systems (11/20) still at design or prototyping
stage only 9 systems evaluated through trials with human
users.
Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H. G. K., & Koper, R. (2011).
Recommender Systems in Technology Enhanced Learning. In P. B. Kantor, F. Ricci,
L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 387-415).
Berlin: Springer.
4
 

The TEL recommender
research is a bit like this...
We need to design for each domain an appropriate
recommender system that fits the goals and tasks
"

5
 

TEL recommender
experiments lack results
“The performance
transparency and
of different research
standardization.
efforts in recommender
They need tohardly
systems are be
repeatable to test:
comparable.”

•  Validity
(Manouselis et al., 2010)
•  Verification
•  Compare results

Kaptain Kobold

http://www.flickr.com/photos/
kaptainkobold/3203311346/

6
 

Data-driven Research and Learning Analytics"

EATELHendrik Drachsler (a), Katrien Verbert (b)"
"
(a) CELSTEC, Open University of the Netherlands"
(b) Dept. Computer Science, K.U.Leuven, Belgium"
"

7	


7
 
TEL RecSys::Evaluation/datasets

	
  

"
Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G.,
Lindstaedt, S., Stern, H., Friedrich, M., & Wolpers, M. (2010). Issues and Considerations
regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning.
Presentation at the 1st Workshop Recommnder Systems in Technology Enhanced Learning
(RecSysTEL) in conjunction with 5th European Conference on Technology Enhanced Learning
(EC-TEL 2010): Sustaining TEL: From Innovation to Learning and Practice. September, 28,
2010, Barcelona, Spain."
9
"
dataTEL evaluation model
5. Dataset Framework
Formal

Data A

Datasets

Data B

Informal
Data C

Algorithms:
Algoritmen A
Algoritmen B
Algoritmen C

Algorithms:
Algoritmen D
Algoritmen E

Algorithms:
Algoritmen B
Algoritmen D

Models:
Learner Model A
Learner Model B

Models:
Learner Model C
Learner Model E

Models:
Learner Model A
Learner Model C

Measured attributes:
Attribute A
Attribute B
Attribute C

Measured attributes:
Attribute A
Attribute B
Attribute C

Measured attributes:
Attribute A
Attribute B
Attribute C

17

42

10
dataTEL evaluation model
5. Dataset Framework
Formal

Datasets

Informal

In Data A
LinkedUp we have Data B opportunity to apply a
the
Data C
structured approach to develop a
community accepted evaluation framework.
Algorithms:
Algorithms:
Algorithms:
Algoritmen A
Algoritmen B
Algoritmen C

Algoritmen D
Algoritmen E

Algoritmen B
Algoritmen D

Learner Model A
Learner Model B

Learner Model C
Learner Model E

Learner Model A
Learner Model C

Measured attributes:
Attribute A
Attribute B
Attribute C

Measured attributes:
Attribute A
Attribute B
Attribute C

Measured attributes:
Attribute A
Attribute B
Attribute C

1.  Top-Down by a literature study
Models:
Models:
2.  Bottom-up by Models: with experts in the field
GCM

17

42

11
12
Development	
  of	
  the	
  Evalua=on	
  Framework	
  
	
  

	
  

P3: Exit and
Sustainability

P1: Initialisation

P2: Establishment
and Evaluation

M0-M6: Preparation

M7-M18: Competition cycle M18-M24: Finalising
Comp
etition

EF proposal

Expert
validation

Draft

New
versio
n

3x

Final
release
of EF

Revie
w of
EF

Refin
ement
of EF

Literature review
Group Concept
Cognitive Mapping Mapping

Documentation
Dissemination
Practical experiences
and refinement

Hendrik Drachsler

25 February 2013

13
Group	
  Concept	
  Mapping	
  	
  

	
  

•  Group Concept Mapping resembles the
Post-it notes problem solving technique
and Delphi method
•  GCM involves participants in a few
simple activities (generating, sorting
and rating of ideas) that most people are
used to.
GCM is different in two substantial ways:
1. Robust analysis (MDS and HCA)
GCM takes up the original participants contribution and then quantitatively
aggregate it to show their collective view (as thematic clusters)
2. Visualisation
GCM presents the results from the analysis as conceptual maps and other graphical
representations (pattern matching and go-zones).

Hendrik Drachsler

25 February 2013

14
 

Group	
  Concept	
  Mapping	
  	
  
brainstorm
•  innovations in way network is delivered
•  (investigate) corporate/structural alignment
•  assist in the development of non-traditional partnerships (Rehab with the
Medicine Community)
•  expand investigation and knowledge of PSN'S/PSO's
•  continue STHCS sponsored forums on public health issues (medicine
managed care forum)
•  inventory assets of all participating agencies (providers, Venn Diagrams)
•  access additional funds for telemedicine expansion
•  better utilization of current technological bridge
•  continued support by STHCS to member facilities
•  expand and encourage utilization of interface programs to strengthen the
viability and to improve the health care delivery system (ie teleconference)
•  discussion with CCHN

sort
Decide how to
manage multiple
tasks.
20

Manage resources effectively.
4

Work quickly
and effectively
under pressure
49

Organize the work
when directions are
not specific.
39

e
he

Ra

...organize the
issues...

gS
tin

2
1

a
an

g

ime
et

ive
e ct
eff

ly

es

rc
ou
res

ly.
tive

ec
eff

n
ic.
s.
atio
ecif
a sk
rm t.
t sp
le t
info tan
no
of por
re
ultip
a
e
ud is im age m
ns
2
4
ultit hat
ctio
an
3
ire
am ew
5
om
nd
id
an
wt
he
4
Sc dec
ho
kw
d
e
r
3
an
5 3
cid
wo
nd
De
he
2
na
ely
4
et
4
ctiv
atio
niz
3
ffe
ga
5
orm
Or
ee
2
inf nt.
4
tim
of ta
5
ge
1
r
de
3
na
5
itu impo
Ma
ult
2
4
a m at is
1
1
3
an e wh
5
Sc cid
2
4
de
3
1
3
5
2
4
1
3
2
1
5

1

t

1

M

Ma

na

ge

rate
Hendrik Drachsler

25 February 2013

15
Group	
  Concept	
  Mapping	
  	
  

	
  

Hendrik Drachsler

25 February 2013

16
Group	
  Concept	
  Mapping	
  	
  

	
  

Hendrik Drachsler

25 February 2013

17
Group	
  Concept	
  Mapping	
  	
  

	
  

Hendrik Drachsler

25 February 2013

18
 

Group	
  Concept	
  Mapping	
  	
  
•  innovations in way network is delivered
•  (investigate) corporate/structural alignment
•  assist in the development of non-traditional partnerships (Rehab with the
Medicine Community)
•  expand investigation and knowledge of PSN'S/PSO's
•  continue STHCS sponsored forums on public health issues (medicine
managed care forum)
•  inventory assets of all participating agencies (providers, Venn Diagrams)
•  access additional funds for telemedicine expansion
•  better utilization of current technological bridge
•  continued support by STHCS to member facilities
•  expand and encourage utilization of interface programs to strengthen the
viability and to improve the health care delivery system (ie teleconference)
•  discussion with CCHN

…”map” the issues...

organize
sort
Decide how to
manage multiple
tasks.
20

Manage resources effectively.
4

Work quickly and
effectively under
pressure
49

Organize the work
when directions are not
specific.
39

Ra

ely
.
et
he effectiv effectively
s
g S time
tin Manage resource
1

2

1

Ma

5

4
1

5

4
2

1

3

1

3

5

4
2

na

2

3

4
2
1

3
2

1

3

2

1

rate

d
an
.
ation
ks.
ecific
inform
t sp
le tas
no
e of rtant.
ltip
are
mu
ltitudimpo
ns
ge
na
a muat is
ectio
ma
an
dir
Sc e wh w to
en
cid
wh
de
e ho
rk
cid
wo
d
De
e the
ely
n an
niz
tio
ectiv
ga
ma
eff
Or
or
e
inf
tim
5
ge
e of rtant.
na
5
ltitudimpo
Ma
4
a muat is
1
an wh
5
Sc cide
4
de
3
3
5
4
3
2

Technology

ge

Information Services

4
5

3

Community & Consumer Views

Regionalization

Management
Financing

STHCS as model

Hendrik Drachsler

25 February 2013

19
 

Group	
  Concept	
  Mapping	
  	
  
•  innovations in way network is delivered
•  (investigate) corporate/structural alignment
•  assist in the development of non-traditional partnerships (Rehab with the
Medicine Community)
•  expand investigation and knowledge of PSN'S/PSO's
•  continue STHCS sponsored forums on public health issues (medicine
managed care forum)
•  inventory assets of all participating agencies (providers, Venn Diagrams)
•  access additional funds for telemedicine expansion
•  better utilization of current technological bridge
•  continued support by STHCS to member facilities
•  expand and encourage utilization of interface programs to strengthen the
viability and to improve the health care delivery system (ie teleconference)
•  discussion with CCHN

Information Services

organize

Technology

sort
Decide how to
manage multiple
tasks.
20

Community & Consumer Views

Manage resources effectively.
4

Work quickly and
effectively under
pressure
49

Organize the work
when directions are not
specific.
39

Ra

ely
.
et
he effectiv effectively
s
g S time
tin Manage resource
1

2

1

Ma

5

4
1

5

4
2

1

3

1

3

5

4
2

na

2

3

4
2

3

1

2

1

d
an
.
ation
ks.
ecific
inform
t sp
le tas
no
e of rtant.
ltip
are
mu
ltitudimpo
ns
ge
na
a muat is
ectio
ma
an
dir
Sc e wh w to
en
cid
wh
de
e ho
rk
cid
wo
d
De
e the
ely
n an
niz
tio
ectiv
ga
ma
eff
Or
or
e
inf
tim
5
ge
e of rtant.
na
5
ltitudimpo
Ma
4
a muat is
1
an wh
5
Sc cide
4
de
3
3
5
4
3
2

ge

4
5

3

3

2

1

Regionalization

rate

map

Information Services

Technology

Community & Consumer Views

Regionalization

Financing
Management
Financing

Management

Mission & Ideology

STHCS as model

...prioritize the issues...
Hendrik Drachsler

25 February 2013

20
 

Group	
  Concept	
  Mapping	
  	
  

D2.1 Evaluation Criteria and Methods
•  Invited 122 external experts
•  56 experts contributed 212 indicators for the
evaluation framework
•  After cleaning -> 108 indicators remained
•  26 experts sorted on similarity in meaning
•  26 experts rated on priority and
applicability

Hendrik Drachsler

25 February 2013

21
Plus Minus Interesting rating

	
  

Look at and listen to the presentation
of the Evaluation Framework
Meanwhile…create notes on
P:
M:
I:

Plus
Minus
Interesting

Write down everything that comes to your mind, generate
as many ideas as possible, do not filter your ideas.
Group	
  Concept	
  Mapping	
  	
  

	
  

A point map

Hendrik Drachsler

25 February 2013

23
Group	
  Concept	
  Mapping	
  	
  

	
  

A cluster map 15

Hendrik Drachsler

25 February 2013

24
Group	
  Concept	
  Mapping	
  	
  

	
  

A cluster map 6

Hendrik Drachsler

25 February 2013

25
Group	
  Concept	
  Mapping	
  	
  

	
  

Clusters’ labels

Hendrik Drachsler

25 February 2013

26
Group	
  Concept	
  Mapping	
  	
  

	
  

Rating Map Priority

Hendrik Drachsler

25 February 2013

27
 

Group	
  Concept	
  Mapping	
  	
  
Rating Map Applicability

Hendrik Drachsler

25 February 2013

28
Group	
  Concept	
  Mapping	
  	
  

	
  

Hendrik Drachsler

25 February 2013

29
Group	
  Concept	
  Mapping	
  	
  

	
  

Hendrik Drachsler

25 February 2013

30
WP2: Literature review

	
  
WP2: Literature review

	
  

1. Literature review of suitable evaluation approaches and criteria
2. Review of comprising initiatives such as LinkedEducation, MULCE, E3FPLE and
the SIG dataTEL

	
  
WP2: Literature review

	
  
 

Many thanks for your attention!
This silde is available at:
http://www.slideshare.com/Drachsler
Email:

hendrik.drachsler@ou.nl

Skype:

celstec-hendrik.drachsler

Blogging at: http://www.drachsler.de
Twittering at: http://twitter.com/HDrachsler

Hendrik Drachsler

25 February 2013

34

More Related Content

What's hot

Almaden presentation 15-dec-2015
Almaden presentation 15-dec-2015Almaden presentation 15-dec-2015
Almaden presentation 15-dec-2015Paul Courtney
 
THIC MedIX Summer 2015 Poster
THIC MedIX Summer 2015 PosterTHIC MedIX Summer 2015 Poster
THIC MedIX Summer 2015 PosterDiana Zajac
 
Humanitarian_IM_Research_July2015
Humanitarian_IM_Research_July2015Humanitarian_IM_Research_July2015
Humanitarian_IM_Research_July2015Katie Whipkey
 
‘Smart’ Taxonomy- & Ontology- Enabled Resources for Taxonomy Bootcamp
‘Smart’ Taxonomy- & Ontology- Enabled Resourcesfor Taxonomy Bootcamp‘Smart’ Taxonomy- & Ontology- Enabled Resourcesfor Taxonomy Bootcamp
‘Smart’ Taxonomy- & Ontology- Enabled Resources for Taxonomy BootcampDeborah McGuinness
 
Developing the Informatics Workforce for Scotland's Health and Social Care
Developing the Informatics Workforce for Scotland's Health and Social CareDeveloping the Informatics Workforce for Scotland's Health and Social Care
Developing the Informatics Workforce for Scotland's Health and Social CareCILIPScotland
 

What's hot (8)

Almaden presentation 15-dec-2015
Almaden presentation 15-dec-2015Almaden presentation 15-dec-2015
Almaden presentation 15-dec-2015
 
The Sustainable Value of Open Data
The Sustainable Value of Open DataThe Sustainable Value of Open Data
The Sustainable Value of Open Data
 
THIC MedIX Summer 2015 Poster
THIC MedIX Summer 2015 PosterTHIC MedIX Summer 2015 Poster
THIC MedIX Summer 2015 Poster
 
Humanitarian_IM_Research_July2015
Humanitarian_IM_Research_July2015Humanitarian_IM_Research_July2015
Humanitarian_IM_Research_July2015
 
‘Smart’ Taxonomy- & Ontology- Enabled Resources for Taxonomy Bootcamp
‘Smart’ Taxonomy- & Ontology- Enabled Resourcesfor Taxonomy Bootcamp‘Smart’ Taxonomy- & Ontology- Enabled Resourcesfor Taxonomy Bootcamp
‘Smart’ Taxonomy- & Ontology- Enabled Resources for Taxonomy Bootcamp
 
Linxen&pimmer
Linxen&pimmerLinxen&pimmer
Linxen&pimmer
 
Data and communication of research: incentives and disincentives
Data and communication of research: incentives and disincentivesData and communication of research: incentives and disincentives
Data and communication of research: incentives and disincentives
 
Developing the Informatics Workforce for Scotland's Health and Social Care
Developing the Informatics Workforce for Scotland's Health and Social CareDeveloping the Informatics Workforce for Scotland's Health and Social Care
Developing the Informatics Workforce for Scotland's Health and Social Care
 

Viewers also liked

Tech Success: Web/2.0 startup HOWTO
Tech Success: Web/2.0 startup HOWTOTech Success: Web/2.0 startup HOWTO
Tech Success: Web/2.0 startup HOWTOlyncis
 
Greater Pittsburgh Arts Council 2008 Annual Meeting
Greater Pittsburgh Arts Council 2008 Annual MeetingGreater Pittsburgh Arts Council 2008 Annual Meeting
Greater Pittsburgh Arts Council 2008 Annual Meetingtwilhelm
 
WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3
WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3
WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3togelius
 
Sample Works
Sample WorksSample Works
Sample Worksxyndz
 
Power Point
Power PointPower Point
Power PointMJH123
 
Lyddie: Unit2 lesson5
Lyddie:  Unit2 lesson5Lyddie:  Unit2 lesson5
Lyddie: Unit2 lesson5Terri Weiss
 
Burrowing Owl by Aaliyah
Burrowing Owl by AaliyahBurrowing Owl by Aaliyah
Burrowing Owl by Aaliyahvebrya
 
Leadership.Mena
Leadership.MenaLeadership.Mena
Leadership.Menaagek2005
 
Bobcat by Noah
Bobcat by NoahBobcat by Noah
Bobcat by Noahvebrya
 
Garden 5 9 08 Presentation7
Garden 5 9 08 Presentation7Garden 5 9 08 Presentation7
Garden 5 9 08 Presentation7themir
 
Why Perl, when you can use bash+awk+sed? :P
Why Perl, when you can use bash+awk+sed? :PWhy Perl, when you can use bash+awk+sed? :P
Why Perl, when you can use bash+awk+sed? :PLuciano Rocha
 
Presentatie UBN Forum Web 2.0
Presentatie UBN Forum Web 2.0Presentatie UBN Forum Web 2.0
Presentatie UBN Forum Web 2.0Jonna
 
From research student to academic: thinking about and preparing for academic ...
From research student to academic: thinking about and preparing for academic ...From research student to academic: thinking about and preparing for academic ...
From research student to academic: thinking about and preparing for academic ...Joss Winn
 
新聞整理五則
新聞整理五則新聞整理五則
新聞整理五則guest370229
 
Reaching net-generation learners with social technologies
Reaching net-generation learners with social technologiesReaching net-generation learners with social technologies
Reaching net-generation learners with social technologiesguestba21f9
 
Muskrats by Morgan
Muskrats by MorganMuskrats by Morgan
Muskrats by Morganvebrya
 
JSX の現在と未来 - Oct 26 2013
JSX の現在と未来 - Oct 26 2013JSX の現在と未来 - Oct 26 2013
JSX の現在と未来 - Oct 26 2013Kazuho Oku
 

Viewers also liked (20)

Tech Success: Web/2.0 startup HOWTO
Tech Success: Web/2.0 startup HOWTOTech Success: Web/2.0 startup HOWTO
Tech Success: Web/2.0 startup HOWTO
 
h
hh
h
 
Greater Pittsburgh Arts Council 2008 Annual Meeting
Greater Pittsburgh Arts Council 2008 Annual MeetingGreater Pittsburgh Arts Council 2008 Annual Meeting
Greater Pittsburgh Arts Council 2008 Annual Meeting
 
WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3
WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3
WCCI 2008 Tutorial on Computational Intelligence and Games, part 2 of 3
 
Sample Works
Sample WorksSample Works
Sample Works
 
Power Point
Power PointPower Point
Power Point
 
Zendframework Parte2
Zendframework    Parte2Zendframework    Parte2
Zendframework Parte2
 
Lyddie: Unit2 lesson5
Lyddie:  Unit2 lesson5Lyddie:  Unit2 lesson5
Lyddie: Unit2 lesson5
 
Burrowing Owl by Aaliyah
Burrowing Owl by AaliyahBurrowing Owl by Aaliyah
Burrowing Owl by Aaliyah
 
Leadership.Mena
Leadership.MenaLeadership.Mena
Leadership.Mena
 
香港六合彩
香港六合彩香港六合彩
香港六合彩
 
Bobcat by Noah
Bobcat by NoahBobcat by Noah
Bobcat by Noah
 
Garden 5 9 08 Presentation7
Garden 5 9 08 Presentation7Garden 5 9 08 Presentation7
Garden 5 9 08 Presentation7
 
Why Perl, when you can use bash+awk+sed? :P
Why Perl, when you can use bash+awk+sed? :PWhy Perl, when you can use bash+awk+sed? :P
Why Perl, when you can use bash+awk+sed? :P
 
Presentatie UBN Forum Web 2.0
Presentatie UBN Forum Web 2.0Presentatie UBN Forum Web 2.0
Presentatie UBN Forum Web 2.0
 
From research student to academic: thinking about and preparing for academic ...
From research student to academic: thinking about and preparing for academic ...From research student to academic: thinking about and preparing for academic ...
From research student to academic: thinking about and preparing for academic ...
 
新聞整理五則
新聞整理五則新聞整理五則
新聞整理五則
 
Reaching net-generation learners with social technologies
Reaching net-generation learners with social technologiesReaching net-generation learners with social technologies
Reaching net-generation learners with social technologies
 
Muskrats by Morgan
Muskrats by MorganMuskrats by Morgan
Muskrats by Morgan
 
JSX の現在と未来 - Oct 26 2013
JSX の現在と未来 - Oct 26 2013JSX の現在と未来 - Oct 26 2013
JSX の現在と未来 - Oct 26 2013
 

Similar to Evaluation of Linked Data tools for Learning Analytics

Poster: Perspectives on Increasing Competency in Using Digital Practices and ...
Poster: Perspectives on Increasing Competency in Using Digital Practices and ...Poster: Perspectives on Increasing Competency in Using Digital Practices and ...
Poster: Perspectives on Increasing Competency in Using Digital Practices and ...Katja Reuter, PhD
 
Australia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityAustralia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityTERN Australia
 
eLSE2019-presentationTPACK
eLSE2019-presentationTPACKeLSE2019-presentationTPACK
eLSE2019-presentationTPACKDana Craciun
 
Research and Deployment of Analytics in Learning Settings
Research and Deployment of Analytics in Learning SettingsResearch and Deployment of Analytics in Learning Settings
Research and Deployment of Analytics in Learning SettingsKatrien Verbert
 
Bridging the missing middle for al_tversionfinal_14_08_2014
Bridging the missing middle for al_tversionfinal_14_08_2014Bridging the missing middle for al_tversionfinal_14_08_2014
Bridging the missing middle for al_tversionfinal_14_08_2014debbieholley1
 
First eStandards conference Panel of the European SDO Platform
First eStandards conference Panel of the European SDO PlatformFirst eStandards conference Panel of the European SDO Platform
First eStandards conference Panel of the European SDO Platformchronaki
 
On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...Grial - University of Salamanca
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning AnalyticsVitomir Kovanovic
 
Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...
Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...
Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...Maulik Kamdar
 
3rd Workshop on Social Information Retrieval for Technology-Enhanced Learnin...
3rd Workshop onSocial  Information Retrieval for Technology-Enhanced Learnin...3rd Workshop onSocial  Information Retrieval for Technology-Enhanced Learnin...
3rd Workshop on Social Information Retrieval for Technology-Enhanced Learnin...Hendrik Drachsler
 
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...Health Informatics New Zealand
 
Age well template lunch and learn (1)
Age well template lunch and learn (1)Age well template lunch and learn (1)
Age well template lunch and learn (1)AchXu
 
CORBEL West-Life webinar slides
CORBEL West-Life webinar slidesCORBEL West-Life webinar slides
CORBEL West-Life webinar slidesCORBEL
 
The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...
The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...
The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...Mark Hawker
 
Effective utilisation and allocation of health resources
Effective utilisation and allocation of health resourcesEffective utilisation and allocation of health resources
Effective utilisation and allocation of health resourcesHealth Informatics New Zealand
 
Data, Data Everywhere: What's A Publisher to Do?
Data, Data Everywhere: What's  A Publisher to Do?Data, Data Everywhere: What's  A Publisher to Do?
Data, Data Everywhere: What's A Publisher to Do?Anita de Waard
 
L yuan alt c 3
L yuan alt c 3L yuan alt c 3
L yuan alt c 3cetisli
 
Nasrullah khilji icickm 2017 hong kong presentation
Nasrullah khilji icickm 2017 hong kong presentationNasrullah khilji icickm 2017 hong kong presentation
Nasrullah khilji icickm 2017 hong kong presentationUniversity of Bedfordshire
 

Similar to Evaluation of Linked Data tools for Learning Analytics (20)

Poster: Perspectives on Increasing Competency in Using Digital Practices and ...
Poster: Perspectives on Increasing Competency in Using Digital Practices and ...Poster: Perspectives on Increasing Competency in Using Digital Practices and ...
Poster: Perspectives on Increasing Competency in Using Digital Practices and ...
 
Australia's Environmental Predictive Capability
Australia's Environmental Predictive CapabilityAustralia's Environmental Predictive Capability
Australia's Environmental Predictive Capability
 
eLSE2019-presentationTPACK
eLSE2019-presentationTPACKeLSE2019-presentationTPACK
eLSE2019-presentationTPACK
 
Research and Deployment of Analytics in Learning Settings
Research and Deployment of Analytics in Learning SettingsResearch and Deployment of Analytics in Learning Settings
Research and Deployment of Analytics in Learning Settings
 
Bridging the missing middle for al_tversionfinal_14_08_2014
Bridging the missing middle for al_tversionfinal_14_08_2014Bridging the missing middle for al_tversionfinal_14_08_2014
Bridging the missing middle for al_tversionfinal_14_08_2014
 
First eStandards conference Panel of the European SDO Platform
First eStandards conference Panel of the European SDO PlatformFirst eStandards conference Panel of the European SDO Platform
First eStandards conference Panel of the European SDO Platform
 
On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...On data-driven systems analyzing, supporting and enhancing users’ interaction...
On data-driven systems analyzing, supporting and enhancing users’ interaction...
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning Analytics
 
Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...
Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...
Elsevier's Healthcare Knowledge Graph: An Actionable Medical Knowledge Platfo...
 
3rd Workshop on Social Information Retrieval for Technology-Enhanced Learnin...
3rd Workshop onSocial  Information Retrieval for Technology-Enhanced Learnin...3rd Workshop onSocial  Information Retrieval for Technology-Enhanced Learnin...
3rd Workshop on Social Information Retrieval for Technology-Enhanced Learnin...
 
Sirtel Workshop
Sirtel WorkshopSirtel Workshop
Sirtel Workshop
 
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
Aleksandar Zivaljevic - Annotation of clinical datasets using openEHR Archety...
 
Age well template lunch and learn (1)
Age well template lunch and learn (1)Age well template lunch and learn (1)
Age well template lunch and learn (1)
 
CORBEL West-Life webinar slides
CORBEL West-Life webinar slidesCORBEL West-Life webinar slides
CORBEL West-Life webinar slides
 
COVID-19 Evidence Network to support Decision-making (COVID-END): Resources t...
COVID-19 Evidence Network to support Decision-making (COVID-END): Resources t...COVID-19 Evidence Network to support Decision-making (COVID-END): Resources t...
COVID-19 Evidence Network to support Decision-making (COVID-END): Resources t...
 
The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...
The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...
The Future: Overcoming the Barriers to Using NHS Clinical Data For Research P...
 
Effective utilisation and allocation of health resources
Effective utilisation and allocation of health resourcesEffective utilisation and allocation of health resources
Effective utilisation and allocation of health resources
 
Data, Data Everywhere: What's A Publisher to Do?
Data, Data Everywhere: What's  A Publisher to Do?Data, Data Everywhere: What's  A Publisher to Do?
Data, Data Everywhere: What's A Publisher to Do?
 
L yuan alt c 3
L yuan alt c 3L yuan alt c 3
L yuan alt c 3
 
Nasrullah khilji icickm 2017 hong kong presentation
Nasrullah khilji icickm 2017 hong kong presentationNasrullah khilji icickm 2017 hong kong presentation
Nasrullah khilji icickm 2017 hong kong presentation
 

More from Hendrik Drachsler

Trusted Learning Analytics Research Program
Trusted Learning Analytics Research ProgramTrusted Learning Analytics Research Program
Trusted Learning Analytics Research ProgramHendrik Drachsler
 
Smart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao ParganaSmart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao ParganaHendrik Drachsler
 
Verhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning AnalyticsVerhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning AnalyticsHendrik Drachsler
 
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...Hendrik Drachsler
 
E.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic datasetE.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic datasetHendrik Drachsler
 
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...Hendrik Drachsler
 
Towards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning AnalyticsTowards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning AnalyticsHendrik Drachsler
 
Fighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-levelFighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-levelHendrik Drachsler
 
LACE Project Overview and Exploitation
LACE Project Overview and ExploitationLACE Project Overview and Exploitation
LACE Project Overview and ExploitationHendrik Drachsler
 
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the ConsistentDutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the ConsistentHendrik Drachsler
 
Recommendations for Open Online Education: An Algorithmic Study
Recommendations for Open Online Education:  An Algorithmic StudyRecommendations for Open Online Education:  An Algorithmic Study
Recommendations for Open Online Education: An Algorithmic StudyHendrik Drachsler
 
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...Hendrik Drachsler
 
DELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning AnalyticsDELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning AnalyticsHendrik Drachsler
 
The Future of Big Data in Education
The Future of Big Data in EducationThe Future of Big Data in Education
The Future of Big Data in EducationHendrik Drachsler
 
The Future of Learning Analytics
The Future of Learning AnalyticsThe Future of Learning Analytics
The Future of Learning AnalyticsHendrik Drachsler
 
Six dimensions of Learning Analytics
Six dimensions of Learning AnalyticsSix dimensions of Learning Analytics
Six dimensions of Learning AnalyticsHendrik Drachsler
 
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store - Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store - Hendrik Drachsler
 

More from Hendrik Drachsler (20)

Trusted Learning Analytics Research Program
Trusted Learning Analytics Research ProgramTrusted Learning Analytics Research Program
Trusted Learning Analytics Research Program
 
Smart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao ParganaSmart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao Pargana
 
Verhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning AnalyticsVerhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning Analytics
 
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
 
E.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic datasetE.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic dataset
 
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
 
Towards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning AnalyticsTowards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning Analytics
 
Trusted Learning Analytics
Trusted Learning Analytics Trusted Learning Analytics
Trusted Learning Analytics
 
Fighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-levelFighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-level
 
LACE Project Overview and Exploitation
LACE Project Overview and ExploitationLACE Project Overview and Exploitation
LACE Project Overview and Exploitation
 
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the ConsistentDutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
 
Recommendations for Open Online Education: An Algorithmic Study
Recommendations for Open Online Education:  An Algorithmic StudyRecommendations for Open Online Education:  An Algorithmic Study
Recommendations for Open Online Education: An Algorithmic Study
 
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
 
DELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning AnalyticsDELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning Analytics
 
LACE Flyer 2016
LACE Flyer 2016 LACE Flyer 2016
LACE Flyer 2016
 
The Future of Big Data in Education
The Future of Big Data in EducationThe Future of Big Data in Education
The Future of Big Data in Education
 
The Future of Learning Analytics
The Future of Learning AnalyticsThe Future of Learning Analytics
The Future of Learning Analytics
 
Six dimensions of Learning Analytics
Six dimensions of Learning AnalyticsSix dimensions of Learning Analytics
Six dimensions of Learning Analytics
 
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store - Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
 
Ethics privacy washington
Ethics privacy washingtonEthics privacy washington
Ethics privacy washington
 

Recently uploaded

How to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineHow to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineCeline George
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationdeepaannamalai16
 
ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6Vanessa Camilleri
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxAnupam32727
 
An Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERPAn Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERPCeline George
 
Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Celine George
 
Comparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptxComparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptxAvaniJani1
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmStan Meyer
 
6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroom6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroomSamsung Business USA
 
Narcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfNarcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfPrerana Jadhav
 
Scientific Writing :Research Discourse
Scientific  Writing :Research  DiscourseScientific  Writing :Research  Discourse
Scientific Writing :Research DiscourseAnita GoswamiGiri
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQuiz Club NITW
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Projectjordimapav
 
Integumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptIntegumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptshraddhaparab530
 

Recently uploaded (20)

How to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command LineHow to Uninstall a Module in Odoo 17 Using Command Line
How to Uninstall a Module in Odoo 17 Using Command Line
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentation
 
prashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Professionprashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Profession
 
ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6
 
Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...
Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...
Plagiarism,forms,understand about plagiarism,avoid plagiarism,key significanc...
 
Spearman's correlation,Formula,Advantages,
Spearman's correlation,Formula,Advantages,Spearman's correlation,Formula,Advantages,
Spearman's correlation,Formula,Advantages,
 
Mattingly "AI & Prompt Design" - Introduction to Machine Learning"
Mattingly "AI & Prompt Design" - Introduction to Machine Learning"Mattingly "AI & Prompt Design" - Introduction to Machine Learning"
Mattingly "AI & Prompt Design" - Introduction to Machine Learning"
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
 
Chi-Square Test Non Parametric Test Categorical Variable
Chi-Square Test Non Parametric Test Categorical VariableChi-Square Test Non Parametric Test Categorical Variable
Chi-Square Test Non Parametric Test Categorical Variable
 
An Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERPAn Overview of the Calendar App in Odoo 17 ERP
An Overview of the Calendar App in Odoo 17 ERP
 
Mattingly "AI & Prompt Design: Large Language Models"
Mattingly "AI & Prompt Design: Large Language Models"Mattingly "AI & Prompt Design: Large Language Models"
Mattingly "AI & Prompt Design: Large Language Models"
 
Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17
 
Comparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptxComparative Literature in India by Amiya dev.pptx
Comparative Literature in India by Amiya dev.pptx
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and Film
 
6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroom6 ways Samsung’s Interactive Display powered by Android changes the classroom
6 ways Samsung’s Interactive Display powered by Android changes the classroom
 
Narcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdfNarcotic and Non Narcotic Analgesic..pdf
Narcotic and Non Narcotic Analgesic..pdf
 
Scientific Writing :Research Discourse
Scientific  Writing :Research  DiscourseScientific  Writing :Research  Discourse
Scientific Writing :Research Discourse
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Project
 
Integumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptIntegumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.ppt
 

Evaluation of Linked Data tools for Learning Analytics

  • 1.   Using Linked Data in Learning Analytics LAK 2013 tutorial EvaluaHon  of  Linked  Data  tools  for  Learning  AnalyHcs   Hendrik  Drachsler  (@hdrachsler,  drachsler.de)   (CELSTEC,  Open  Universiteit  Nederland,  NL)   Eelco  Herder   (L3S  Research  Center,  DE)   Mathieu  d’Aquin  (@mdaquin,  mdaquin.net)   (Knowledge  Media  InsHtute,  The  Open  University,  UK)   Stefan  Dietze     (L3S  Research  Center,  DE)  
  • 2. Example of scientific competitions   What are the evaluation criteria of Robot Wars? Criteria: •  Damage •  Aggression probabilistic combination of •  Control – Item-based method – User-based method •  Applause – Matrix Factorization – (May be) content-based method 2
  • 3. RecSysTEL Evaluation criteria 1. Accuracy 2. Coverage 3. Precision 4. Recall 1. Accuracy 2. Coverage 3. Precision 4. Recall 1. Effectiveness of learning 2. Efficiency of learning 3. Drop out rate 4. Satisfaction Combine approach by Drachsler et al. 2008 1. Reaction of learner 2. Learning improved 3. Behaviour 4. Results Kirkpatrick model by Manouselis et al. 2010 3
  • 4. TEL RecSys::Review study   Conclusions: Half of the systems (11/20) still at design or prototyping stage only 9 systems evaluated through trials with human users. Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H. G. K., & Koper, R. (2011). Recommender Systems in Technology Enhanced Learning. In P. B. Kantor, F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 387-415). Berlin: Springer. 4
  • 5.   The TEL recommender research is a bit like this... We need to design for each domain an appropriate recommender system that fits the goals and tasks " 5
  • 6.   TEL recommender experiments lack results “The performance transparency and of different research standardization. efforts in recommender They need tohardly systems are be repeatable to test: comparable.” •  Validity (Manouselis et al., 2010) •  Verification •  Compare results Kaptain Kobold
 http://www.flickr.com/photos/ kaptainkobold/3203311346/ 6
  • 7.   Data-driven Research and Learning Analytics" EATELHendrik Drachsler (a), Katrien Verbert (b)" " (a) CELSTEC, Open University of the Netherlands" (b) Dept. Computer Science, K.U.Leuven, Belgium" " 7 7
  • 8.  
  • 9. TEL RecSys::Evaluation/datasets   " Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G., Lindstaedt, S., Stern, H., Friedrich, M., & Wolpers, M. (2010). Issues and Considerations regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning. Presentation at the 1st Workshop Recommnder Systems in Technology Enhanced Learning (RecSysTEL) in conjunction with 5th European Conference on Technology Enhanced Learning (EC-TEL 2010): Sustaining TEL: From Innovation to Learning and Practice. September, 28, 2010, Barcelona, Spain." 9 "
  • 10. dataTEL evaluation model 5. Dataset Framework Formal Data A Datasets Data B Informal Data C Algorithms: Algoritmen A Algoritmen B Algoritmen C Algorithms: Algoritmen D Algoritmen E Algorithms: Algoritmen B Algoritmen D Models: Learner Model A Learner Model B Models: Learner Model C Learner Model E Models: Learner Model A Learner Model C Measured attributes: Attribute A Attribute B Attribute C Measured attributes: Attribute A Attribute B Attribute C Measured attributes: Attribute A Attribute B Attribute C 17 42 10
  • 11. dataTEL evaluation model 5. Dataset Framework Formal Datasets Informal In Data A LinkedUp we have Data B opportunity to apply a the Data C structured approach to develop a community accepted evaluation framework. Algorithms: Algorithms: Algorithms: Algoritmen A Algoritmen B Algoritmen C Algoritmen D Algoritmen E Algoritmen B Algoritmen D Learner Model A Learner Model B Learner Model C Learner Model E Learner Model A Learner Model C Measured attributes: Attribute A Attribute B Attribute C Measured attributes: Attribute A Attribute B Attribute C Measured attributes: Attribute A Attribute B Attribute C 1.  Top-Down by a literature study Models: Models: 2.  Bottom-up by Models: with experts in the field GCM 17 42 11
  • 12. 12
  • 13. Development  of  the  Evalua=on  Framework       P3: Exit and Sustainability P1: Initialisation P2: Establishment and Evaluation M0-M6: Preparation M7-M18: Competition cycle M18-M24: Finalising Comp etition EF proposal Expert validation Draft New versio n 3x Final release of EF Revie w of EF Refin ement of EF Literature review Group Concept Cognitive Mapping Mapping Documentation Dissemination Practical experiences and refinement Hendrik Drachsler 25 February 2013 13
  • 14. Group  Concept  Mapping       •  Group Concept Mapping resembles the Post-it notes problem solving technique and Delphi method •  GCM involves participants in a few simple activities (generating, sorting and rating of ideas) that most people are used to. GCM is different in two substantial ways: 1. Robust analysis (MDS and HCA) GCM takes up the original participants contribution and then quantitatively aggregate it to show their collective view (as thematic clusters) 2. Visualisation GCM presents the results from the analysis as conceptual maps and other graphical representations (pattern matching and go-zones). Hendrik Drachsler 25 February 2013 14
  • 15.   Group  Concept  Mapping     brainstorm •  innovations in way network is delivered •  (investigate) corporate/structural alignment •  assist in the development of non-traditional partnerships (Rehab with the Medicine Community) •  expand investigation and knowledge of PSN'S/PSO's •  continue STHCS sponsored forums on public health issues (medicine managed care forum) •  inventory assets of all participating agencies (providers, Venn Diagrams) •  access additional funds for telemedicine expansion •  better utilization of current technological bridge •  continued support by STHCS to member facilities •  expand and encourage utilization of interface programs to strengthen the viability and to improve the health care delivery system (ie teleconference) •  discussion with CCHN sort Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 e he Ra ...organize the issues... gS tin 2 1 a an g ime et ive e ct eff ly es rc ou res ly. tive ec eff n ic. s. atio ecif a sk rm t. t sp le t info tan no of por re ultip a e ud is im age m ns 2 4 ultit hat ctio an 3 ire am ew 5 om nd id an wt he 4 Sc dec ho kw d e r 3 an 5 3 cid wo nd De he 2 na ely 4 et 4 ctiv atio niz 3 ffe ga 5 orm Or ee 2 inf nt. 4 tim of ta 5 ge 1 r de 3 na 5 itu impo Ma ult 2 4 a m at is 1 1 3 an e wh 5 Sc cid 2 4 de 3 1 3 5 2 4 1 3 2 1 5 1 t 1 M Ma na ge rate Hendrik Drachsler 25 February 2013 15
  • 16. Group  Concept  Mapping       Hendrik Drachsler 25 February 2013 16
  • 17. Group  Concept  Mapping       Hendrik Drachsler 25 February 2013 17
  • 18. Group  Concept  Mapping       Hendrik Drachsler 25 February 2013 18
  • 19.   Group  Concept  Mapping     •  innovations in way network is delivered •  (investigate) corporate/structural alignment •  assist in the development of non-traditional partnerships (Rehab with the Medicine Community) •  expand investigation and knowledge of PSN'S/PSO's •  continue STHCS sponsored forums on public health issues (medicine managed care forum) •  inventory assets of all participating agencies (providers, Venn Diagrams) •  access additional funds for telemedicine expansion •  better utilization of current technological bridge •  continued support by STHCS to member facilities •  expand and encourage utilization of interface programs to strengthen the viability and to improve the health care delivery system (ie teleconference) •  discussion with CCHN …”map” the issues... organize sort Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Ra ely . et he effectiv effectively s g S time tin Manage resource 1 2 1 Ma 5 4 1 5 4 2 1 3 1 3 5 4 2 na 2 3 4 2 1 3 2 1 3 2 1 rate d an . ation ks. ecific inform t sp le tas no e of rtant. ltip are mu ltitudimpo ns ge na a muat is ectio ma an dir Sc e wh w to en cid wh de e ho rk cid wo d De e the ely n an niz tio ectiv ga ma eff Or or e inf tim 5 ge e of rtant. na 5 ltitudimpo Ma 4 a muat is 1 an wh 5 Sc cide 4 de 3 3 5 4 3 2 Technology ge Information Services 4 5 3 Community & Consumer Views Regionalization Management Financing STHCS as model Hendrik Drachsler 25 February 2013 19
  • 20.   Group  Concept  Mapping     •  innovations in way network is delivered •  (investigate) corporate/structural alignment •  assist in the development of non-traditional partnerships (Rehab with the Medicine Community) •  expand investigation and knowledge of PSN'S/PSO's •  continue STHCS sponsored forums on public health issues (medicine managed care forum) •  inventory assets of all participating agencies (providers, Venn Diagrams) •  access additional funds for telemedicine expansion •  better utilization of current technological bridge •  continued support by STHCS to member facilities •  expand and encourage utilization of interface programs to strengthen the viability and to improve the health care delivery system (ie teleconference) •  discussion with CCHN Information Services organize Technology sort Decide how to manage multiple tasks. 20 Community & Consumer Views Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Ra ely . et he effectiv effectively s g S time tin Manage resource 1 2 1 Ma 5 4 1 5 4 2 1 3 1 3 5 4 2 na 2 3 4 2 3 1 2 1 d an . ation ks. ecific inform t sp le tas no e of rtant. ltip are mu ltitudimpo ns ge na a muat is ectio ma an dir Sc e wh w to en cid wh de e ho rk cid wo d De e the ely n an niz tio ectiv ga ma eff Or or e inf tim 5 ge e of rtant. na 5 ltitudimpo Ma 4 a muat is 1 an wh 5 Sc cide 4 de 3 3 5 4 3 2 ge 4 5 3 3 2 1 Regionalization rate map Information Services Technology Community & Consumer Views Regionalization Financing Management Financing Management Mission & Ideology STHCS as model ...prioritize the issues... Hendrik Drachsler 25 February 2013 20
  • 21.   Group  Concept  Mapping     D2.1 Evaluation Criteria and Methods •  Invited 122 external experts •  56 experts contributed 212 indicators for the evaluation framework •  After cleaning -> 108 indicators remained •  26 experts sorted on similarity in meaning •  26 experts rated on priority and applicability Hendrik Drachsler 25 February 2013 21
  • 22. Plus Minus Interesting rating   Look at and listen to the presentation of the Evaluation Framework Meanwhile…create notes on P: M: I: Plus Minus Interesting Write down everything that comes to your mind, generate as many ideas as possible, do not filter your ideas.
  • 23. Group  Concept  Mapping       A point map Hendrik Drachsler 25 February 2013 23
  • 24. Group  Concept  Mapping       A cluster map 15 Hendrik Drachsler 25 February 2013 24
  • 25. Group  Concept  Mapping       A cluster map 6 Hendrik Drachsler 25 February 2013 25
  • 26. Group  Concept  Mapping       Clusters’ labels Hendrik Drachsler 25 February 2013 26
  • 27. Group  Concept  Mapping       Rating Map Priority Hendrik Drachsler 25 February 2013 27
  • 28.   Group  Concept  Mapping     Rating Map Applicability Hendrik Drachsler 25 February 2013 28
  • 29. Group  Concept  Mapping       Hendrik Drachsler 25 February 2013 29
  • 30. Group  Concept  Mapping       Hendrik Drachsler 25 February 2013 30
  • 32. WP2: Literature review   1. Literature review of suitable evaluation approaches and criteria 2. Review of comprising initiatives such as LinkedEducation, MULCE, E3FPLE and the SIG dataTEL  
  • 34.   Many thanks for your attention! This silde is available at: http://www.slideshare.com/Drachsler Email: hendrik.drachsler@ou.nl Skype: celstec-hendrik.drachsler Blogging at: http://www.drachsler.de Twittering at: http://twitter.com/HDrachsler Hendrik Drachsler 25 February 2013 34