WWW„2012 Workshop on Large Scale Network Analysis                                                    www.know-center.at16/...
IntroductionInformation overload is not a contemporary issueScience has been growing exponentiallyfor the last 400 years (...
Quantitative Analysis of Science based on  Large Scale Citation NetworksResearch Evaluation –                    Knowledge...
Quantitative Analysis of Science based on   Usage DataProblems of citation-based approaches   Citations take a long time t...
MendeleySocial reference                 Crowdsourced Mendeleymanagement platform              research cataloge   Organiz...
Empirical StudiesTwo studies in the main areas of quantitative analysis ofscience    Large-scale impact factor analysis   ...
Biology                                                                                                    Data  Comp. Sc....
Study 1 – Large Scale Analysis of Journal   ImpactMRank: Measuring journal impact with number of readersResearch question:...
ResultsUnique occurrences of publications in Mendeley 2010 xTotal number of documents in SCIMago 2010                  Ove...
Study 2 – Knowledge Domain Visualization of   Technology Enhanced LearningCo-citation as a measure of subject similarity (...
Results                                          Multidimensional Scaling                                           Hierar...
Conclusions & Future WorkResults are encouraging…   Significant relationship between library statistics and the   impact f...
www.know-center.at                     Thank you for your                     attention!                     Peter Kraker ...
Upcoming SlideShare
Loading in …5
×

Harnessing User Library Statistics for Research Evaluation and Knowledge Domain Visualization

1,115 views

Published on

Presented at the WWW'2012 Workshop on Large Scale Network Analysis: http://largenetwork.org

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,115
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Harnessing User Library Statistics for Research Evaluation and Knowledge Domain Visualization

  1. 1. WWW„2012 Workshop on Large Scale Network Analysis www.know-center.at16/04/2012Harnessing User LibraryStatistics for ResearchEvaluation and KnowledgeDomain VisualizationPeter Kraker (Know Center Graz)Christian Körner (TU Graz)Kris Jack (Mendeley London)Michael Granitzer (University of Passau) Funded by
  2. 2. IntroductionInformation overload is not a contemporary issueScience has been growing exponentiallyfor the last 400 years (Price 1961) Papers (Larsen/von Ins 2010) Scientists (NSF 2010)Problems Lacking overview of (sub-)disciplines Simultaneous discoveries Repeated research 2 Price 1961 © Know-Center 2011 extended by Leydesdorff (2008)
  3. 3. Quantitative Analysis of Science based on Large Scale Citation NetworksResearch Evaluation – Knowledge DomainImpact Factor Visualization Chen & Carr 1999 3 © Know-Center 2011 http://www.scimagojr.com
  4. 4. Quantitative Analysis of Science based on Usage DataProblems of citation-based approaches Citations take a long time to become available (~ 3-5 years) Corpus has to be limited  results differ (Meho & Yang 2004)Possible solution: usage data References are earlier available Not restricted to formal communicationExamples Click data/download data (e.g. PLoS, arXiv) Social reference management: “Add to library” (e.g. bibsonomy, Mendeley, Zotero) 4 © Know-Center 2011
  5. 5. MendeleySocial reference Crowdsourced Mendeleymanagement platform research cataloge Organizing personal 1.5 million users research library 50 million unique articles Creating user profile Reading and annotating of PDFs Sharing of references/PDFs 5 © Know-Center 2011 http://www.mendeley.com/research-papers/
  6. 6. Empirical StudiesTwo studies in the main areas of quantitative analysis ofscience Large-scale impact factor analysis Exploratory knowledge domain visualization of the emerging research field of Technology Enhanced LearningBasis: User library statistics from MendeleyMeasures: Occurrences and co-occurrences of referencesin user libraries 6 © Know-Center 2011
  7. 7. Biology Data Comp. Sc. MedicineEngineering None Social Sc. Psychology Education Business Physics Electrical ChemistryEnvironment Economics Arts Humanities EarthManagement MaterialsMathematics Linguistics Law Design Philosophy Astronomy Sports 437,812 users Snapshot from March 2011 18,080,679 unique documents 7 © Know-Center 2011
  8. 8. Study 1 – Large Scale Analysis of Journal ImpactMRank: Measuring journal impact with number of readersResearch question: “How do library occurrences reflecttraditional measures of impact based on citations?”Measures Occurrences/unique occurrences Authority score (Kleinberg)External validation using SCIMago (based on Scopus) Total number of documents Citations per document (Impact factor)Method Ranking journals for each measure 8 © Know-Center 2011 Calculating Spearman correlations between rankings
  9. 9. ResultsUnique occurrences of publications in Mendeley 2010 xTotal number of documents in SCIMago 2010 Overall Biology Comp. Sc. Arts N=3806 N=508 N=225 N=116 Corr. 0.70 0.76 0.57 0.28Mendeley library statistics of publications from 2008 and2009 xCitations per document (impact factor) from SCIMago 2010 Authority Score Occurrence Overall 0.64 0.53 Biology 0.60 0.56 Comp. Sc. 0.60 0.59 9 Arts 0.52 0.30 © Know-Center 2011
  10. 10. Study 2 – Knowledge Domain Visualization of Technology Enhanced LearningCo-citation as a measure of subject similarity (Small 1973)Research question: “Can we use library co-occurrences tovisualize a research field that is not yet covered bytraditional subject descriptors?”Data Researchers from computer science: 35,560 user libraries and 1,964,367 articles.Method Identify libraries from the field by filtering resarch interests Calculate co-occurrences of most occurring papers Perform multi-dimensional scaling and hierarchical clustering 10 © Know-Center 2011
  11. 11. Results Multidimensional Scaling Hierarchical ClusteringAH: Adaptive HypermediaGL: Game-based LearningCC: Citation Classics 11MC: Miscellaneous Publications from TEL © Know-Center 2011OD: Publications from Other Disciplines
  12. 12. Conclusions & Future WorkResults are encouraging… Significant relationship between library statistics and the impact factor Meaningful results in knowledge domain visualization…but need further validation Longer periods of time, beyond Scopus (social media)Outlook Including more information from the user profile (discipline, location, academic status) Getting even closer to readership: incorporating click data Implementing knowledege domain visualizations in 12 Mendeley © Know-Center 2011
  13. 13. www.know-center.at Thank you for your attention! Peter Kraker @PeterKraker http://science20.wordpress.com pkraker@know-center.at© Know-Center 2011 gefördert durch das Kompetenzzentrenprogramm

×