Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multiple bibliometric indicators approach to Croatian open access (OA) journals

602 views

Published on

Talk given at LIBRARIES IN THE DIGITAL AGE (LIDA) 2014: ASSESSING LIBRARIES AND LIBRARY USERS AND USE (Part II: Altmetrics - new methods in assessing scholarly communication and libraries: issues applications, results) at Zadar, Croatia, 16 - 20 June 2014, University of Zadar, Zadar, Croatia (http://www.unizd.hr/)
Web site: http://ozk.unizd.hr/lida/

Published in: Engineering, Technology, Education
  • Be the first to comment

  • Be the first to like this

Multiple bibliometric indicators approach to Croatian open access (OA) journals

  1. 1. Multiple Bibliometric Indicators Approach to Croatian Open Access (OA) Journals Franjo Pehar, Jadranka Stojanovski Department of Information Sciences, University of Zadar Libraries in the Digital Age 2014
  2. 2. “Better to understand a little than to misunderstand a lot.” Anonymous writer
  3. 3. Rise of metrics ● importance and usage of citation metrics has increased substantially in recent years ● the most popular metric indicator is still JIF ● ...initially intended as a way to help libraries to select the journals to be acquired… ● ...and to help academics to ‘filter content’ ● used to measure academic achievement o at individual, project (grant) and institutional level
  4. 4. ... ● academic career ● research project assessment ● institution ranking ● funding decisions ● … ● new situation has drawn significant criticism o practices that encourage conformity in research o “salami-slicing” or incremental publishing o unequal treatment of researchers from low-cited disciplines
  5. 5. Eugene Garfield at LIDA 2004 “JIF is not created for all the purposes that some people want to use them for. A lot of people who are using JIF are doing it for advertising purposes.” http://www.garfield.library.upenn.edu/ papers/medicalwritingv8(1)1999.html “It is crucial to know how citations are connected, how they are related.” “The real problem is when you start using JIF as the substitute for citation analysis of individuals. I have always said that you should not use JIF to evaluate a person or department. If I manage to get my paper published in Nature, does that guarantee that it is a great paper? Even if the paper is never cited?”
  6. 6. http://researchimpact.files.wordpress.com/2011/10/research-impact-cartoon.jpg
  7. 7. What is a citation? ● simple line in the reference list (not) pointing to the paper ● paper which act as a foundation for our own work ● paper which help us to broaden our knowledge ● tool to confirm that we prepared well prior to our own research ● tool to flatter someone ● tool to “self-cite” at different levels ...
  8. 8. Citation as a metric tool ● many factors can influence the rate of citation o age of article, type of research conducted, the subject area o objectivity of citations? real benefit? (erroneous references with hundreds, thousand citations) ● tools for evaluation and assessment are evolving constantly, giving rise to new metrics o different aspects of citation and publication behaviour
  9. 9. Journals ● fee and free journals ● „OA journal” as a synonym for APC model ● journals? - or more portals ● very prominent journals = high JIF journals = very for-profit journals ● „international journals” and „local journals” ● „regional journals”
  10. 10. Journals ● journal as an old concept from the printed world ● journal as (still) main channel of scholarly communication ● journals slowing down a possible development of science (?), not implementing ICT advantages and possibilities… ● ...or implementing it at very slow pace
  11. 11. What’s about journals from the scientific semi-periphery?
  12. 12. Narrower context ● Croatia o 4.5M population, 11k scholars, 7 universities, 25 research institutes, 25k papers per year and 1300 islands ● Croatian journals o challenges for small language and semi-peripheral science context
  13. 13. Croatian journals in numbers
  14. 14. Academic career policies in Croatia “papers must be published in journals with JIF above average for the specific subject category” “at least two-thirds of the papers must be published in the journals listed in SCIE” “paper is rated regarding ratio of the JIF and median of the subject category, and the minimal value must be 0.5 “the value of the qualitative criteria Q is calculated as a sum of the ratios of JIF and median JIF of the subject category, multiplied by the factor Fd of the candidate” “Example: Journal of Engineering Mechanics in 2011 has JIF 0.99, subject category is Engineering – Mechanical with median JIF 0.743 …” “in the A1 category are the papers published in journals with JIF in Q1, Q2, i Q3 of the subject category ...in the A2 category are papers from the journals in Q4…” “in the A1 category are the papers published in journals indexed by Web of Knowledge...if the JIF falls in the first 50% of the subject category...then the paper is counted as two papers”
  15. 15. Hmmmm…. are Croatian journals important? is it reasonable just to transfer the Croatian journal funds to APC or fee based prominent and popular “international” journals? what justification do we have to publish Croatian scholarly journals?
  16. 16. HRCAK - improved visibility, readability...popularity...citations
  17. 17. HRCAK by disciplines
  18. 18. Objective of the study ● Which aspects of journal usage metrics differ from citations metrics ● In particular, following issues were be addressed: o Croatian OA journals usage o Relationship of full-text paper downloads and citation counts o Comparison between full paper downloads and citations counts relative to different subject areas
  19. 19. Materials and methods ● HRCAK usage statistics o Cover journal's web page visits o Issue web page visits o Article (bibliographic records) web page visits o Full-text (PDF) downloads o Unique visits from the same IP address during the period of 30 minutes
  20. 20. ... ● For the purpose of the study o Usage data for 115,458 full-text papers (documents) acquired from 340 unique OA journal titles  March 2007 to the May 1st 2014  Article page visits  Full-text downloads
  21. 21. 674 downloads 1293 visits
  22. 22. Research sample ● 47 OA journals (28,121 papers) available in HRCAK repository and indexed by o Web of Science™ Core Collection (Thomson Reuters)  28,725 indexed papers o Scopus (Elsevier)  28,953 indexed papers
  23. 23. Preprocessing challenges ● Assembling a single data source based on three data sets o lack of (P)ID’s for matching documents from different data sets ● Titles were reduced to the same form ● Levenshtein distance algorithm to determine similarity between paper titles (2 iterations) o 15,023 (53%) papers in Web of Science and 16,592 (57%) papers in Scopus matched with HRCAK titles o 13,127 (ca. 46%) unique papers merged from all three data sets ● 2 journals dropped-out
  24. 24. ... ● Initial cleaning of usage data o validity, reliability, transparency ... ● Missing data o HRCAK page visit & download dates/years, more reader demo-geographic information ● Matching/merging procedure o two journal titles completely lost o further improvements possible > expected matching rate ca. 85% (cat(ti ... jn … is … vl … pg)
  25. 25. Results ● download and citation frequencies (March 2007 - May 2014) Matched articles HRCAK visits HRCAK Downloads Scopus citations WOS citations 13,127 2,084,174 4,174,888 38,106 30,324
  26. 26. Visits vs Downloads Article page visits: 1-250=84%; Downloads: 1-250=59%, 251-500=22%
  27. 27. cor(log(1+HRVisits, HRDown), method=”spearman”)
  28. 28. Possible reasons …
  29. 29. ...
  30. 30. ...
  31. 31. Scopus vs Web of Science citations Scopus: 0=41%, 1-5=44%; Web of Science: 0=47%, 1-5=42%
  32. 32. cor(log(1+HRDown, ScopusCit, WoSCit), method=”spearman”)
  33. 33. Visits, downloads and citations relative to different subject fields Subject area No. journals % No. articles % Biomedicine and health 10 22.2% 5856 44.6% Biotechnical sciences 5 11.1% 1420 10.8% Humanities 7 15.6% 527 4.0% Sciences 8 17.8% 2037 15.5% Social sciences 6 13.3% 1365 10.4% Technical sciences 9 20.0% 1922 14.6% Total 45* 100.0% 13127 100.0%
  34. 34. Median number of visits and downloads & Mean number of citations Median number of HRCAK visits and downloads relative to six subject areas Mean number of Scopus and Web of Science citations relative to six subject areas
  35. 35. Key results ● Different usage (reader) and impact (author) characteristics ● Very strong, strong and low correlations o Scopus and WoS citation counts o article page visits and downloads o downloads and citation counts ● But, with significant differences between and within six subject areas
  36. 36. Conclusion ● Important to have different (MULTIPLE) but complementary measures for better reflecting the multifaceted and multidimensional character of scientific work (output, impact, activities …)
  37. 37. Acknowledgment • HRCAK team (University Computing Centre) • Tomislav Jagušt (Faculty of Electrical Engineering and Computing, University of Zagreb)
  38. 38. Bieber’s response to yesterday's accusation

×