This document summarizes a presentation on using readership statistics from online reference managers like Mendeley to create overview visualizations of academic fields. It notes that traditional visualizations based on citations only show the past due to delays in citations accumulating. Readership data is available earlier and can show more recent topics and diverse areas of a field. The presentation describes a prototype visualization of the educational technology field created from Mendeley data that validated well against other literature analysis methods. It concludes that readership statistics provide a more recent and diverse view of a research field than citations alone.

1. gefördert durch das Kompetenzzentrenprogramm
Web Science Webinar
12 June 2013
know-center.tugraz.at
Head Start: Improving Academic
Literature Search with Overview
Visualizations based on
Readership Statistics
Peter Kraker, Kris Jack, Christian Schlögl,
Christoph Trattner, Stefanie Lindstaedt

2. © Know-Center 2011
2
Motivation
Overview of a research field

3. © Know-Center 2011
3
Motivation
Information overload
Information overload is NOT a contemporary problem in
science
Science has been growing exponentially for
the last 400 years (Price 1963)
Number of papers (Larsen/von Ins 2010)
Number of researchers (NSB 2010)
Problem
Missing overview of research fields
 Approach: Visualizations
Price 1963
Extended by Leydesdorff (2008)

4. © Know-Center 2011
4
Exemplary visualization of the research field
“Educational Technology”

5. © Know-Center 2011
5
The „traditional“ way of creating overview
visualizations
Basis: Citations
Co-citations as a measure of subject similarity (Small 1973)
Problem: Citations take very long to appear in meaningful
quantities (~3-5 years)
 Visualizations actually a look into the past!

6. © Know-Center 2011
6
A new approach
Visualisations based on the readership of publications
Assumptions: Publications that are often read together, are
of a similar subject (Rowlands & Nicholas 2007, Bollen &
van de Sompel 2008)
With online reference management systems (e.g.
Mendeley), we can measure readership
Readership statistics are much earlier available than
citations

7. © Know-Center 2011
7
Online reference management in Mendeley
Paper 1 Paper 2 30
Paper 1 Paper 3 100
Paper 2 Paper 3 5

8. © Know-Center 2011
8
Data & Method
Research field of Educational Technology as a use case
Data from Mendeley
91 publications from users in Educational Technology
7,414 user libraries
19,402 co-occurrences
Methods
Multidimensional Scaling (MDS)
Force-directed Layout
Hierarchical Clustering
Naming approach using OpenCalais and Zemanta
Stress: 0,2; R2: 0,86

9. © Know-Center 2011
9
Results
Prototype in Mendeley Labs
http://labs.mendeley.com/headstart

10. © Know-Center 2011
10
Results
Validation through literature comparison
Qualitative comparison with other forms of literature
analysis of the field of Educational Technology
10 studies: citations, words, qualitative content analysis,
Delphi study, larger streams
• Comparison to citations
• Recent topics are
better covered
• More and more
diverse areas
• Most research areas
from qualitative analyses
are covered as well
• Qualitative studies offer a
more differentiated image
• Areas that are mostly
influenced by computer
science are missing (e.g.
“Adaptive Hypermedia”)

11. © Know-Center 2011
11
Conclusions and Future Work
Visualizations based on readership statistics offer a more
recent, and a more diverse representation of the field than
visualizations based on citations
Automatization is possible with a few manual interventions
But: Characteristics of readers have an influence on the
visualization
Future Work
Expert interviews
Extension to Web Science (based
on the work of Clare Hooper)
Adaptivity/Personalization

12. © Know-Center 2011
12
References
Bollen, J., & Van de Sompel, H. (2006). Mapping the structure of science
through usage. Scientometrics, 69(2), 227–258.
Kraker, P., Körner, C., Jack, K., & Granitzer, M. (2012). Harnessing User
Library Statistics for Research Evaluation and Knowledge Domain
Visualization. Proceedings of the 21st International Conference
Companion on World Wide Web (pp. 1017–1024). Lyon: ACM.
Leydesdorff, L. (2008). Journals as retention mechanisms of scientific
growth - Research Trends. Research Trends, 7, 6–7.
Price, D. J. D. S. (1963). Little science, big science (p. 118). Columbia Univ.
Press.
Rowlands, I., & Nicholas, D. (2007). The missing link: journal usage
metrics. Aslib Proceedings, 59(3), 222–228.
Small, H. (1973). Co-citation in the scientific literature: A new measure of
the relationship between two documents. Journal of the American Society
for information Science, 24(4), 265–269.
Images on slides 4, 5, 6, 7, and 11 by Maxi Schramm.

13. gefördert durch das Kompetenzzentrenprogramm
Web Science Webinar
12 June 2013
know-center.tugraz.at
Thank you very much for your
attention!
pkraker@know-center.at
http://science20.wordpress.com
http://twitter.com/PeterKraker