Downloaded 62 times
Uploaded byLinda Galloway
Citation Metrics: Established and Emerging Tools
Citation metrics tools can be used to measure the impact of scholarly work. Established tools like Scopus, Web of Science, and Google Scholar provide citation counts and metrics like an author's h-index. Emerging altmetrics tools measure impact through social media mentions, saves, downloads, and other non-traditional sources. While still new, altmetrics provide a broader view of impact beyond citations alone. Librarians should be aware of both established and emerging citation metrics tools to help faculty evaluate research.
More Related Content
Similar to Citation Metrics: Established and Emerging Tools
More from Linda Galloway
Citation Metrics: Established and Emerging Tools
- 1. Citation Metrics: Established andEmerging Tools Linda Galloway Syracuse University Library Syracuse, NY galloway@syr.edu
- 2. Citation metrics - Tools to gauge scholarly impact • Quantity – Publications (tweets, blog posts, mentions…) • Quality – Citations (saves, downloads, likes…) • Funding – Research funds, collaborations • Technology transfer – Patents, start-ups, etc. Focus today is on individual author metrics
- 3. Why should librarianscare? • Build relationships with faculty • Learn about faculty research interests • Assist in evaluation of departments, programs, and faculty • Accreditation efforts • Marketing for academic programs
- 4. Scholars care because… •Citation metrics are used in promotion and tenure decisions • Prestige and influence – Within a field – Within an institution – Comparison to peer and aspirational institutions • Vanity – Individual and institutional
- 5. Established Tools Scopus Web of Science Google Scholar
- 6. Consider Before Beginning: •Cost of subscription databases • Ease of use • Time frames – Citations to past year’s work (2011), – Citations to author’s work in past 5 years (07-11) – Citations to author’s work in past 10 years (02-11) • Skewed towards STEM fields • Don’t compare across databases!
- 7. Citation Metrics forIndividual Faculty Members Typically, assess scholarly impact by looking at: • Works published • Citations to works published • Publication influence (h-index, journal prestige, etc. )
- 8. Scopus or Webof Science? Scopus Web of Science SciVerse Scopus is the world’s largest abstract Web of Science consists of nine databases and citation database of peer-reviewed containing information gathered from thousands of scholarly journals, books, book literature. series, reports, conferences, and more. • Contains 47 million records, 70% with abstracts • It fully covers over 12,000 major journals. • Nearly 19,500 titles from 5,000 publishers • Create a visual representation of citation worldwide relationships with Citation Mapping • Capture citation activity and trends • Includes over 4.9 million conference graphically with Citation Report papers • Use the Analyze Tool to identify trends • Interoperability with Engineering Village & and patterns Reaxys • Provides 100% Medline coverage SU subscription: • Science Citation Index Expanded (1899- Subscription includes: present) • 26 million records with references back to • Social Sciences Citation Index (1898- 1996 (of which 78% include references). present) • 21 million records pre-1996 which go back • Arts & Humanities Citation Index (1975- present) as far as 1823. As of 10/2012
- 9. Cited Reference Search in Scopus Tips: Create an account to save your work Check within each author group for publications Merge author profiles, if necessary
- 11. Cited Reference Search in Web of Science Tips: Create an account to save your work If you have a very prolific author, ask for assistance with article identification By default, citation counts are for All Years – you must modify for your chosen parameters Read the ‘Cited Reference Search’ how-to and follow the directions closely
- 12. Must always ‘FinishSearch!
- 14. Citation Mapping inWeb of Science Visually Demonstrate Author/Article Influence
- 15. Google Scholar Citations Tips: Public profiles are available in Google Can search for an author from within your own profile page
- 16. Cited Reference Search in Google Scholar Citations • Author needs to set up their profile (using their Google account); Google Scholar will harvest related data. • Authors can add articles, groups of articles, edit entries, etc. • Profiles are private unless author elects to make public • Quality control: “To be eligible for inclusion in Google Scholar search results, your profile needs to be public and needs to have a verified email address at your university”
- 17. Comparison Times cited H-Index Scopus 135 7 Web of Science 85 11 Google Scholar 279 10 Times cited = number of documents published from 2007-current* that have cited this author's work H index = Number of author's articles that have been cited at least this many times (during time span indicated) *Normally, would choose the past 5 years, i.e. 2007-2011. However, to compare more equitably to Google Scholar, used 2007- current (25 Nov 2012)
- 18. Author Disambiguation Identification Schemes ORCID, about.orcid.org • Scopus – Scopus Author • Not for profit Identifier • Create registry of • Web of Science – unique identifiers for ResearcherID individual researchers • Google Scholar Profiles • Open and transparent linking between ORCID • Institutional ID and other ID schemes • Many other identifiers… • Many vendors, institutions are members
- 19. altmetrics is thecreation and study of new metrics based on the Social Web for analyzing, and informing scholarship Altmetrics: A manifesto http://altmetrics.org/manifesto/
- 20. What can bemeasured? “Evidence of Use” – http://impactstory.org • # of Tweets • # of “Saves” in online reference managers • Scholarly (and popular) blog interest and activity • Activity in social networking platforms, tools • Etc!!
- 21. Tracking Mentions (oruploads/downloads, etc.) in Social Media citeulike is a free service for Mendeley is a combination of a desktop managing and discovering application and a website which helps you scholarly references manage, share and discover both content and 6,402,349 articles - 3,030 added contacts in research. today. ImpactStory is a website that makes it quick and easy to view the impact of a wide range of Zotero is a free, easy-to-use tool to help research output. It goes beyond traditional you collect, organize, cite, and share measurements of research output -- citations your research sources. It lives right to papers -- to embrace a much broader where you do your work—in the web evidence of use across a wide range of browser itself. scholarly output types.
- 22.
- 23. Altmetric Limitations • New •Time frame – some new tools cannot search old mentions, tweets, etc. • Rely on user generated metadata • Should social media mentions be given the same weight as scholarly article citations? • Can these tools be easily manipulated to raise significance of an article?
- 24. Citation Metrics: Established and emerging tools • Established tools – Web of Science and Scopus – Widely recognized and trusted – Pricey – Require some level of expertise to use – Google Scholar is becoming an established tool • Emerging tools – Altmetrics – New, many promising platforms and services – Librarians & information professionals should be aware – When a few tools are established, trusted and verified ==>> acceptance
- 25. Thank you! More information: http://researchguides.library.syr.edu/citationmetrics References 1. Eysenbach, G. (2011). Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), e123. doi:10.2196/jmir.2012 2. Google. (n.d.). Google Scholar Citations Help. Retrieved November 19, 2012, from http://scholar.google.com/intl/en/scholar/citations.html#overview 3. ImpactStory. (n.d.). ImpactStory: faq. Retrieved November 19, 2012, from http://impactstory.org/faq 4. Galloway,L. & Rauh, A. (2012). Subject Guides. Citation Metrics. Web of Science. Retrieved November 16, 2012, from http://researchguides.library.syr.edu/citationmetrics 5. ORCID Inc. (2012). ORCID. Retrieved November 6, 2012, from http://about.orcid.org/ 6. Priem, J., Groth, P., & Taraborelli, D. (2012). The Altmetrics Collection. (C. A. Ouzounis, Ed.)PLoS ONE, 7(11), e48753. doi:10.1371/journal.pone.0048753 7. Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2011, September 28). altmetrics: a manifesto – altmetrics.org. Retrieved November 16, 2012, from http://altmetrics.org/manifesto/
Editor's Notes
- #2 HI, I’m really glad to be here today and excited to talk about citation metrics. Using these tools and teaching others how to calculate individual author metrics has been a very interesting part of my job and has helped me connect with the faculty I serve.So today, as I mentioned, we are going to be talking about individual author or researcher citation metrics.
- #3 Scientists’ contributions to their field are reflected in their publications; for many scholars, publishing is important for peer recognition, promotion, and tenure. At this point in time, scholarly communication via academic journals remains the most widely accepted method of disseminating information. And quantifying the scholarly output from journals remains the time-honored method of determining productivity.So traditionally, when we look at citation metrics we evaluate the quality and quantity of publications, funding received, research collaborations and technology transfer. In purple, next to the traditional metrics, are the more ephemeral types of scholarly output ranging from tweets to mentions in social media to saves in online reference managers.And again, while there are many uses for citation metrics, today we are focusing on individual author metrics.
- #4 I’m a subject specialist librarian and fairly new to Syracuse University. One of the most effective ways that I have to get to know my constituents is by reviewing their publications – it helps me to get a good sense of the type of research they do.
- #5 Faculty and researchers care about this because this data is used in promotion and tenure decisions. And this also applies to librarians, of course, some of whom are faculty but if we publish, we want to see that our work is useful.Scholars may also like to gauge their impact on a particular field by using citation metrics. These data can also be used to compare various departments within an institution, although I would caution anyone who compares the citation patterns of different disciplines. Practitioners in some fields publish and cite much more than others. Metrics can also be used to compare to peer and aspirationalinstituions.And some people just like to see how much they are cited.
- #6 The established tools that I’ll briefly talk about today are Scopus, Web of Science and Google Scholar, which of course is a newer tool but uses primarily conventional data to calculate metrics. A note of caution: the data is skewed toward STEM (science, technology, engineering, mathematics) fieldsAnd, Books and book chapters not well covered by these tools
- #7 Don’t compare across tools – content coverage is different, metrics different and/or calculated differently. Be consistent in the work that you do with your faculty.Useful data can be found in each tool but direct comparisions across databases are problematic. These resources use different pools of data, date ranges and may interpret citations differently. Correct attribution of authorship can also cause reporting errors.
- #8 Traditional Citation Metrics measure:Number of published worksCitations to published worksPublication influencePublishing in prestigious journalsH-index – number of an author’s publications (h) that have at least (h) citations (so if an author has an h-7, they have 7 articles with at least 7 citations each) Other indicies
- #12 Citation analyses in WoS require more expertise and more time, in my experience. In my last position, I ran individual author citation metrics on about 130 faculty members. This work takes a LONG time in WoS. I would suggest working closely with authors if you are using WoS.
- #13 Need to look through all the citations to an author in order to make sure they are correctly attributed, and then ‘Finish Search’Want to know how many citations have SE Parks’ articles received in the past 5 years (2007-2011).For Parks, se = had to look through 51 citations to articles , however WoS did index some conference proceedings not found in Scopus.
- #14 Check/modify date range – make sure you know what date ranges mean (a little confusing in WoS)For example, ‘Cited Years’ in main search fields refers to the years that SE Parks’ articles were authored.But in our case, we are interested in citations to SE Parks’ work during the time frames we specified.
- #15 Citation mapping feature in WoS is very useful and provides excellent bibliometric data and a nice visual representation of an article’s (by extension, authors’) influence.
- #16 Google Scholar citationsCitations to articles publishedH-indexI10-index - i10-index is the number of publications with at least 10 citations. The second column has the "recent" version of this metric which is the number of publications that have received at least 10 new citations in the last 5 years.An author chooses to make their profile public - can find then in GoogleCan search for an author from within your own profile page
- #18 Will always find differences – function of data/publications included, date ranges and other variables – some that you can control and some that you can’t. Correct attribution of scholarly works is a concern, particularly with common names or Asian names. I’ll talk about this a bit more in the next slide.In S.E. Parks’ case – missing citations to two 2007 book chapters – not indexed in Scopus, but some conference proceedings are indexed in WoSMuch more content – and types of content – is included in Google Scholar. Also, in GS you “get what you get” – can’t change date ranges or work with the citation metrics. Perhaps, though, GS captures a more complete picture of a scholar’s output.
- #19 ORCID - Open Researcher ID - is an initiative to provide researchers and scholars with a persistent, unique identifier. This will enable individuals to get recognized for all their scholarly output, in both established and emerging media. With broad-based support from publishers, academic institutions, and funders, ORCID registration and services are free to individuals. Sign up here: http://about.orcid.org/.
- #20 Supplement to traditional peer-reviewed metrics“Crowdsourced peer-review”Many tools currently available:Google Scholar Profile/citations, Mendeley, Total-Impact, ReaderMeter, Plus, many more!
- #21 What can Altmetrics measure?“Evidence of Use” – Impactstory.org Rapidly gauge use and influence of an article – for example# of Tweets (has been found to predict citation rates)# of saves in online reference manager (Mendeley, Zotero) librariesScholarly blog interest and activity
- #22 At this point, there are many tools and resources to explore. Some of the most popular are on this slide. I personally like Mendeley as sort of a “readers guide” site – on the next slide you’ll see what I mean. Zotero is a fabulous citation management tool with awesome sharing and collaborative functions. And then there is ImpactStory (formerly total-impact) that allows users to create a report of impacts from many resources including articles, blog posts, datasets and much more. Altmetric toolsEmerging area with many emerging tools and platformsDescribing some of the most well known and more established tools
- #23 Here’s an example of crowdsourced peer-review – I’ve looked at the Marine Biology group in Mendeley, there are almost 90,000 articles in this group and the most popular papers are displayed on the home page of this discipline.
- #24 AnnieLogistical limitationsNew tools – still under developmentSome new tools can’t look at older tweets that happened before the tool was developedRely on user generated metadata – indexes get it wrong too but have some QC, for something to “count” when you tweet about something, how often do you have the Orcid ID, DOI or PubMed ID ?Larger questions to grapple with?Should social media mentions be given the same weight as scholarly article citations?Can these tools be easily manipulated to raise significance of an article?
- #25 ConclusionIndividual researcher metrics are moving beyond H-indices and Times CitedAltmetric tools can and should complement traditional metric toolsField is changing with technology and adoption of social networking for scientists It is a really interesting time to be involved with citation metrics. As we grapple with the importance and impact of social media on scholarly communication, and as adoption of different forms of communication becomes more widespread, the measurement of influence will need to change. I think it’s a great time for librarians to become familiar with traditional tools and help their faculty to begin to embrace altmetrics which allow for much more rapid measure of scholarly influence.