Quantifying the impacts of scholarship
•
1 like•765 views
A presentation delivered online to the Mountain Plains Management Conference at Cedar City, UT on Oct. 18, 2013. Presented by: Jon Ritterbush of the Calvin T. Ryan Library at the University of Nebraska-Kearney.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to Quantifying the impacts of scholarship
Similar to Quantifying the impacts of scholarship (20)
Recently uploaded
Recently uploaded (20)
Quantifying the impacts of scholarship
- 1. Quantifying the impacts of scholarship by Jon Ritterbush E-Resources & Serials Librarian Calvin T. Ryan Library University of Nebraska-Kearney Mountain Plains Management Conference October 18, 2013
- 2. A few housekeeping items • A link to these slides will be posted at: guides.library.unk.edu/scholarship • Q & A period following today’s presentation • Tweet a question during today’s presentation to #MPMCimpact
- 3. What are Impact Factors (IFs)? • Quantitative values assigned to academic journals – not to authors’ individual articles – based on citations to a journal over a specified time period • Idea originated with Dr. Eugene Garfield in 1955 • IFs are published annually in Journal Citation Reports (JCR) - a Thomson Reuters database ($$$ - check your library) - Provides IFs for 8,200+ science and 2,900+ social science journals - Includes 2-year and 5-year IF values, with other journal metrics
- 4. Why do IFs matter? • Used by scholars to help decide where to publish an article for maximum exposure • Used by librarians to evaluate journal titles for acquisition or cancellation • May be used by some accreditation boards to evaluate a program’s scholarly productivity • May be used in faculty evaluations for promotion, tenure, or grants
- 5. How are IFs calculated? The 2011 2-year impact factor for Academy of Management Review is calculated by: # of times AMR articles published in 2009 and 2010 were cited in indexed journals1 during 2011 # of “citable” 2 AMR articles published in 2009 and 2010 1. Only journals indexed by JCR are counted. Book chapters, proceedings, and journals not indexed by JCR are not counted in this numerator. 2. Only research articles and reviews are counted as citable works. Letters to the editor or other commentaries are not counted in this denominator.
- 6. How are IFs calculated? The 2011 2-year impact factor for Academy of Management Review is calculated by: Note: a journal’s IF will vary each year!
- 7. Who else provides IF data? Eigenfactor (free) • Its “article influence” (AI) score measures the average influence of each article in a journal during 5 years following publication - The AI score is somewhat comparable to JCR’s impact factors - AI scores are weighted, so that journals are ranked as more influential if cited by other influential journals • Limited to/Ranks the same journals indexed in JCR • Unlike JCR, journals in Eigenfactor are assigned to only one subject category • “Self-citations” to the same journal are removed! • Freely available at eigenfactor.org
- 8. Who else provides IF data? SCImago Journal Rank - SJR (free) • Based on citations in Elsevier’s Scopus database to articles published over a 3-year period • Impact factors are weighted based on rankings • Self-citations to the same journal are removed • Freely available at scimagojr.com - Note: SJR results use European/international numerical notations for decimals and the thousands separator.
- 9. Who else provides IF data? • Google Scholar Metrics (free) - Indexes journal and conference papers, theses and dissertations, academic books and other content from publishers, societies, and open repositories. Court opinions and patents are also included. - Uses h5-index for journal rankings: Equal to h articles published in last 5 years, which have been cited elsewhere at least h times. - See bit.ly/14pgXmL for business journal rankings • Publishers’ descriptions of journals (free) - Google the name of the journal to find publisher’s website listing subscription info, author instructions, and sometimes the journal’s IF
- 10. Criticisms of IFs in general • Some disciplines take longer to build upon and cite new research • IFs vary between subject disciplines -- an IF for a chemistry journal is not comparable to the IF of a business journal. • Publication changes can affect IF values • Not all scholarly journals receive IF values; new journals are at some disadvantage • The quality and impact of an author’s work cannot be truly assessed by a journal’s IF
- 11. Individual measures of scholarship Two common methods: • Counting citations to one’s published research • h-index Other new methods: • Views/downloads of articles - Rarely available for public access from publishers - Some publishers may make this information available to authors (e.g. Nature, Springer) • Altmetrics
- 12. Who provides citation counts? • Some academic library databases - Social Science Citation Index - Scopus - EBSCO’s Academic Search Premier, Business Source Premier, Communication & Mass Media Complete, PsycINFO - ProQuest’s ABI/Inform among others • Google Scholar (free)
- 13. Citation counts in Business Source If available at your library...
- 14. Citation counts in Google Scholar Start at http://scholar.google.com Search by an article title, or use Advanced Scholar Search to search by author
- 15. Citation Count Comparison Number of Articles Indexed / Number of Citations to those Articles Social Science Citation Index Dr. Sample EBSCO Business Source Premier Google Scholar 7 / 24 11 / 28 21 / 75
- 16. h-index h-index: Equal to h papers which have been cited elsewhere at least h times. Paper Cited by My first paper 14 My second paper 6 My third paper 4 My fourth paper 2 My fifth paper 0 h-index = 3
- 17. Tracking your citations in Google Scholar Google Scholar’s “My Citations” • Free with Google account • Add, edit, or ignore citations within your profile • More details at Google: see bit.ly/YzJ2BY
- 18. “Publish or Perish” software • Freeware for analyzing GS citations, available for Windows and MacOS • See http://www.harzing.com/pop.htm
- 19. Additional readings at bit.ly/11wjLiK • Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals. College & Research Libraries News, 68(5), 314–316. Retrieved from http://crln.acrl.org/content/68/5/314.full.pdf+html • Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 9093. Retrieved from http://garfield.library.upenn.edu/papers/jamajif2006.pdf • Levine-Clark, M. & Gil, E. L. (2009). A comparative citation analysis of Web of Science, Scopus, and Google Scholar. Journal of Business & Finance Librarianship, 14(1), 32-46. doi:10.1080/08963560802176348 • Monastersky, R. (2005). The number that’s devouring science. Chronicle Of Higher Education, 52(8), A12-A17. • Rossner, M., Epps, H. V., & Hill, E. (2007). Show me the data. The Journal of Cell Biology, 179(6), 1091–1092. Retrieved from http://jcb.rupress.org/content/179/6/1091 • Testa, J. (2012). The Thomson Reuters journal selection process. Retrieved from http://thomsonreuters.com/products_services/science/free/essays/ journal_selection_process/
- 20. Questions? #MPMCimpact For a brief slidecast and more links on scholarly communication, go to: guides.library.unk.edu/scholarship Thank you! Jon Ritterbush ritterbushjr@unk.edu Skype & Twitter: @loperlibrarian
Editor's Notes
- A presentation delivered online to the Mountain Plains Management Conference at Cedar City, UT on Oct. 18, 2013. Presented by: Jon Ritterbush of the University of Nebraska-Kearney’s Calvin T. Ryan Library.
- Dr. Garfield set out to identify source journals for inclusion in what would eventually become the print Science Citation Index.JCR published since 1975. JCR is updated once a year – usually in June/July.
- In England, hiring panels routinely consider impact factorsAccording to Spanish law, researchers are rewarded for publishing in journals ranking in the upper third of IF listings.In some Chinese universities, graduate students in physics must place at least two articles in journals with a combined IF of 4 to earn their PhDs
- The next two slides offer an example of how a 2-year Impact Factor are calculated by Journal Citation Reports.Bear in mind: Not all journals are indexed in JCR! And other scholarly sources are not either! Titles are added, and sometimes removed from JCR based on selection criteria.
- Remember the IF will vary each year, and it is not cumulative!“Of 38 million items cited from 1900-2005, only 0.5% were cited more than 200 times. Half were not cited at all...” according to Dr. Garfield in an article on the “History and Meaning of the Journal Impact Factor” published in JAMA 2006.
- Article Influence scores are normalized so that the mean article in the entire Thomson Journal Citation Reports (JCR) database has an article influence of 1.00. There are some key difference with JCR, such as the inclusion of subject categories and removal of self-citations. “Self-citations” are articles citing other articles from the same journal. Some scholars have alleged that self-citation practices lead to over-inflated journals IF values.
- Scopus indexes more titles than (about 21,000 titles versus 11,000 in JCR).SCImago uses a 3-year period, instead of 2- or 5-years.They weight their IF values based on rankings, similar to Eigenfactor, and also removed self-citations in the calculations.
- Because Google Scholar is crawling so many sources automatically, it can cover a wide body of literatureExample: In GS – Academy of Management Journal currently has an h5-index of 69 which means in the last 5 complete years (2008-2012), 69 articles have at least 69 citations each...Talk more about h-index values in a few minutes, as it relates to individual productivityGS also provides business journal rankingsJournal of Management – is an example of publisher-listed IF.
- Title changes: After a title change, two JCR years must pass before the new title fully replaces the previous title in JCR. This would also impact Eigenfactor. And changes can even influence Google Scholar indexing and hence its metrics! Likewise, if a journal cuts back from publishing 6 times a year to 4 times a year, the impact factor may rise temporarily.As mentioned before, new journal titles may take awhile to be included in JCR, if ever!
- Keep in mind that EBSCO databases are looking at specific slices of scholarly literature – typically scholarly journals and selected trade publications within a particular subject discipline. These citation counts will often be lower than what is retrieved using Google Scholar!
- It is imperative if you search the SSCI database that you use the “Cited Reference Search” option!!! Using this, I found more citations for Dean Burkink (7 items with 24 citations, instead of 4 and 17 using the “Author search” option.The differences are linked to what is in each source’s database. SSCI is the smallest of the three sources in this table, so it’s not a terrible surprise that they came up with the lowest numbers.
- In this example, an author has three papers that have been cited at least three times, yielding an h-index of 3.
- Creating a GS “My Citations” profile requires a Google accountWorth noting that an article with 18 citations was published in the Journal of Web Librarianship. This peer-reviewed journal has been published by Taylor & Francis since 2007, but still isn’t indexed by JCR! While that journal doesn’t have an impact factor, it doesn’t mean that my article isn’t having an impact!Meanwhile, my article published in the Journal of Academic Librarianship has only been cited once. The journal is in JCR, and is regarded among academic librarian as a leading publication in the field.Publishing in a high-impact factor journal may not always translate into more views/downloads, nor into more citations.
- This didn’t work so great for my name. It pulled up all of the database/website reviews I had written (none of which had been cited). It also pulled in results for lots of other Ritterbushes with different first initials – searching on “J* Ritterbush” It also appeared to treat my title and place of employment as articles(?!) and it pulled in some webpage research guides I’ve created too.