Measuring Scholarly Impact: Citation Metrics 2012

Building Your Academic Profile (Part 2):Understanding the Role of Citation Analysis & Impact Factors in Your Academic Career Sheryl Adam | Dean Giustini | UBC Librarians

Related workshops: Web of Science/Journal Citation Reports Scholar, Blogger, Tweeter, Author: Building Your Academic Profile

Measuring research impact:introduction to citation metrics in scholarship Sheryl Adam | Dean Giustini | UBC Librarians

Agenda – 1 hrI. IntroductionII. Overview of issues | toolsIII. Practice time for Google scholar | Web of ScienceIV. Discussion / Q & A

Goals• Overview of journal impact vs. article impact metrics vs. scholar impact "citation metrics“• Pros & cons | how to calculate impact factors for journals (& scholars)• How to manage your indexed publications (or unindexed)• Leave workshop knowing a bit about Web of Science & Google scholar

Publish or Perish?In the digital age its increasingly "Get cited or perish"Departments track citation counts for individuals/subgroupsNoticeable increase in requests for citation metrics from faculty,especially for tenure and promotion & graduate studentsEmphasis on quantitative data & evaluating output to make decisions inacademic unitsCitation counts used in tenure & funding decisions by institutions

Why look at this issue?• To examine humanities-social sciences vs. sciences viewpoints• To consider a broad range of “usage” and “metrics”• To discuss what shapes success (or lack thereof)• Does “value” shift across disciplines? • connection between impact | quality | importance?• Tools, how they deal with citations, articles-authors & books• Which tools cover books? • Monographs still benchmark in humanities/social science

Why evaluate ‘impact’?See journal impact factors based on "usage" (measured by citations,primarily to journal articles, but also some book chapters)Rank or visualize impact of publications, scholars & journalsDiscover new research areas and trends by mappingSee which journals are most important, and publish in those journals

How is citation analysis done?Citation, journal & author metrics are part of field called bibliometricsBibliometrics is "discipline of measuring performance of a researcher,collection of articles, journal, research discipline or institution“. It involves"application of statistical analyses to study patterns of authorship,publication, and literature use" (Lancaster 1977)What is being counted? # papers published per individual, institutionalaggregated # of citations, cites per paper, cites per journal, cites per book,cites per book chapter …& so onOther metrics: number of patents, number of graduate students,conference papers, monographs, etc.

Research impact in STEMWeb of Science and Scopus (UBC does not subscribe to Scopus)For example, determine researcher’s output of articles from 2005 to presentBook chapters (see other tools) find social scientists’ workSmall part of humanities scholars’ workLook at journal impact factor (IF) in Journal Citation ReportsDetermine "research impact"/ figure out H-indexGoogle scholar can be used

Research impact in Hum-SocSciWeb of Science and Scopus• index only a small part of social scientists’ work , a very small part of a humanities scholar’s output of journal articles: and a limited number of booksTo determine HSS research impact, you need to use tools that find citations inbooks and to books, including GoogleBooks, HathiTrust and other tools .For example, in Google Books, the book to journal citations ratio, roughly:Philosophy 4:1 |Sociology, Fine Arts 3:1 | Psychology 1.5:1 | Physics .001:1Example(s): Jon Beasley-Murray; W. Peter Ward

Why do you need impact?Prepare for confirmation, promotion, tenureKnow who is citing your work; find collaboratorsExciting to see how others use your researchGet ego boost when someone reads your paperGenerate new ideas or reframe existing

Traditional Impact MeasuresJournal impact factor (Web of Science , Eigenfactor)Individual scholar impact: h-index (Web of Science, also Google Scholar)Individual scholar impact: i10-index (Google Scholar)Individual article impact: Citation count(Web of Science; Google Scholar)InCites - individual and institutional dataArticle acceptance rate- low acceptance rate = high status journal?

Where is citation data? Google Scholar : only source with broad coverage, some limitations Web of Science : “Gold Standard” Eigenfactor.org : non – commercial alternative to Web of Science Discipline specific citation sources, e.g. Citeseer (computer science), RePEc (Economics)More info about discipline-specific sources athttp://en.wikipedia.org/wiki/Google_Books andhttp://hlwiki.slais.ubc.ca/index.php/Citation_metrics_%E2%80%93_Where_to_find_them%3F

Google scholar covers?• All major & mid-size academic publishers/societies • Elsevier, Springer, Wiley, Science, Nature, APS, ACS, IEEE, ACM, IOP, OUP, Sage, JSTOR, Taylor & Francis, Highwire, LWW…• Most smaller publishers & journals• All major public A&Is (open search tools) • PubMed, INIST, ADS, DTIC, ERIC, AGRIS, WHO…• Google Books, US/CAN/EU/WO Patents• Repositories: arXiv, PubMed, RePec, SSRN

Google scholar citations feature• Simple, open way to track citations to author’s articles• A form of user-generated evaluation • Determine citation metrics • Do it for most articles in most journals • Create automatically maintained profile• Public profiles appear in any GS search results display• Takes ~10 mins to set up feature (need a Google mail account)• Automatically finds all (most?) papers by an academic • Sometimes, even ones forgotten (or, hidden)

Critique of impact factorsAll have data errors; in fact, data is less complete & accurate than youd thinkWeb of Science relies on journals but some areas (e.g., philosophy) incompleteImpact metric formulas are based on science citation patterns most articles cited within a few years of publicationHSS disciplines are cited over longer periods, e.g., relevant longer Years to zero cites: Biomed= 7-8 | Soc=12.5 | History, Philosophy=39 yearsDisciplines are divided in ways that don’t make sense to scholarsCanadian journals are missing from lists; critical to publish about Canada, etc.Gaming the system; manipulated by journal editors; self-citations

Critique of impact factorsStray citations are common; different “author initials” lead to confusionWoS has poor aggregation of minor variations of same titles80% of citations to “Baden-Fuller” entered as "Badenfuller“ 80-90% of citations to “van Raan” entered as “Vanraan” Both receive 100-150 citations to correct name even though their names were correctly referenced (incorrectly entered by WoS)Require formal request to WoS to merge variations with master recordLanguage bias: WoS has limited coverage of non-English sources (note origins inscience) e.g. For example, journals in CAIRN not included

Sources & use of data for impactGoogle scholar is useful for individuals | CAUTION: some counts are inflatedWeb of Science – InCites are useful for universities (UBC does not have Scopus)H-index is convenient summary of quantity & impact (productivity & citedness)H-index of 7 = 7 papers have at least 7 citations each total # citations is a fair way to assess individual impact adjust for co-authorship & age (citations in recent years count more) focus on citations “per paper” may discourage new scholars adjust by years (cites/year) age of papers (AWCR; age-weighted citation rate) and # of authors (AWCRpA)

Q&AWhat else can scholars do to get more citations to their publications?Will publishing in an open access journal help to improve my citation counts?Do you promote your research? Are you aware of the major communicationchannels in your field? If not, why not?Speak to a subject specialist librarian

Thank you~ Instructors: Sheryl Adam & Dean Giustini Resources for workshop • February 19th powerpoints & handout • Citation metrics – Where to find them? • Workshop: Graduate Student Workshop Series Citation MetricsSee also Altmetrics | Author impact metrics | Bibliometrics | Citation management | Utopia Documents | Web of Science vs. Scopus | Webometrics

http://blogs.ubc.ca	769
https://twitter.com	2
http://translate.googleusercontent.com	2
https://si0.twimg.com	1
http://kred.com	1
http://search.daum.net	1

Measuring Scholarly Impact: Citation Metrics 2012

by Dean Giustini on Oct 24, 2012

Accessibility

Upload Details

Usage Rights

6 Embeds 776

Statistics

Measuring Scholarly Impact: Citation Metrics 2012 Presentation Transcript