Your SlideShare is downloading. ×
Harnessing User Library Statistics for Research Evaluation and Knowledge Domain Visualization
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Harnessing User Library Statistics for Research Evaluation and Knowledge Domain Visualization


Published on

Presented at the WWW'2012 Workshop on Large Scale Network Analysis:

Presented at the WWW'2012 Workshop on Large Scale Network Analysis:

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. WWW„2012 Workshop on Large Scale Network Analysis www.know-center.at16/04/2012Harnessing User LibraryStatistics for ResearchEvaluation and KnowledgeDomain VisualizationPeter Kraker (Know Center Graz)Christian Körner (TU Graz)Kris Jack (Mendeley London)Michael Granitzer (University of Passau) Funded by
  • 2. IntroductionInformation overload is not a contemporary issueScience has been growing exponentiallyfor the last 400 years (Price 1961) Papers (Larsen/von Ins 2010) Scientists (NSF 2010)Problems Lacking overview of (sub-)disciplines Simultaneous discoveries Repeated research 2 Price 1961 © Know-Center 2011 extended by Leydesdorff (2008)
  • 3. Quantitative Analysis of Science based on Large Scale Citation NetworksResearch Evaluation – Knowledge DomainImpact Factor Visualization Chen & Carr 1999 3 © Know-Center 2011
  • 4. Quantitative Analysis of Science based on Usage DataProblems of citation-based approaches Citations take a long time to become available (~ 3-5 years) Corpus has to be limited  results differ (Meho & Yang 2004)Possible solution: usage data References are earlier available Not restricted to formal communicationExamples Click data/download data (e.g. PLoS, arXiv) Social reference management: “Add to library” (e.g. bibsonomy, Mendeley, Zotero) 4 © Know-Center 2011
  • 5. MendeleySocial reference Crowdsourced Mendeleymanagement platform research cataloge Organizing personal 1.5 million users research library 50 million unique articles Creating user profile Reading and annotating of PDFs Sharing of references/PDFs 5 © Know-Center 2011
  • 6. Empirical StudiesTwo studies in the main areas of quantitative analysis ofscience Large-scale impact factor analysis Exploratory knowledge domain visualization of the emerging research field of Technology Enhanced LearningBasis: User library statistics from MendeleyMeasures: Occurrences and co-occurrences of referencesin user libraries 6 © Know-Center 2011
  • 7. Biology Data Comp. Sc. MedicineEngineering None Social Sc. Psychology Education Business Physics Electrical ChemistryEnvironment Economics Arts Humanities EarthManagement MaterialsMathematics Linguistics Law Design Philosophy Astronomy Sports 437,812 users Snapshot from March 2011 18,080,679 unique documents 7 © Know-Center 2011
  • 8. Study 1 – Large Scale Analysis of Journal ImpactMRank: Measuring journal impact with number of readersResearch question: “How do library occurrences reflecttraditional measures of impact based on citations?”Measures Occurrences/unique occurrences Authority score (Kleinberg)External validation using SCIMago (based on Scopus) Total number of documents Citations per document (Impact factor)Method Ranking journals for each measure 8 © Know-Center 2011 Calculating Spearman correlations between rankings
  • 9. ResultsUnique occurrences of publications in Mendeley 2010 xTotal number of documents in SCIMago 2010 Overall Biology Comp. Sc. Arts N=3806 N=508 N=225 N=116 Corr. 0.70 0.76 0.57 0.28Mendeley library statistics of publications from 2008 and2009 xCitations per document (impact factor) from SCIMago 2010 Authority Score Occurrence Overall 0.64 0.53 Biology 0.60 0.56 Comp. Sc. 0.60 0.59 9 Arts 0.52 0.30 © Know-Center 2011
  • 10. Study 2 – Knowledge Domain Visualization of Technology Enhanced LearningCo-citation as a measure of subject similarity (Small 1973)Research question: “Can we use library co-occurrences tovisualize a research field that is not yet covered bytraditional subject descriptors?”Data Researchers from computer science: 35,560 user libraries and 1,964,367 articles.Method Identify libraries from the field by filtering resarch interests Calculate co-occurrences of most occurring papers Perform multi-dimensional scaling and hierarchical clustering 10 © Know-Center 2011
  • 11. Results Multidimensional Scaling Hierarchical ClusteringAH: Adaptive HypermediaGL: Game-based LearningCC: Citation Classics 11MC: Miscellaneous Publications from TEL © Know-Center 2011OD: Publications from Other Disciplines
  • 12. Conclusions & Future WorkResults are encouraging… Significant relationship between library statistics and the impact factor Meaningful results in knowledge domain visualization…but need further validation Longer periods of time, beyond Scopus (social media)Outlook Including more information from the user profile (discipline, location, academic status) Getting even closer to readership: incorporating click data Implementing knowledege domain visualizations in 12 Mendeley © Know-Center 2011
  • 13. Thank you for your attention! Peter Kraker @PeterKraker© Know-Center 2011 gefördert durch das Kompetenzzentrenprogramm