Academics must provide evidence to demonstrate the impact and outcomes of their scholarly work. This webinar, presented by librarians, will help faculty explore various forms of documentary evidence to support their case for excellence. Sponsored by the IUPUI Office of Academic Affairs. Note: The webinar included demonstrations of Web of Science & Scopus, which the slides do not reflect.

1. IUPUI OFFICE OF
ACADEMIC AFFAIRSAltmetrics
PRESENTERS:
Heather Coates, Assistant Librarian, University Library
Willie Miller, Assistant Librarian, University Library

2. Citation Metrics
Willie Miller
Assistant Librarian, Liaison to the
Schools of Informatics, Journalism,
University Librarian

3. Altmetrics
Heather Coates
Digital Scholarship & Data
Management Librarian, Liaison to
the Richard M. Fairbanks School
of Public Health, University Library

4. Traditional Metrics
Team Science & Collaborative
Translational Research
March 5, 2015
Willie Miller, MLS Informatics and Journalism Librarian

5. Traditional Metrics: Levels
• Article-level: citation counts
• Journal-level: impact factor
• Author-level: h-index

6. Sources of Data
• Web of Science
– Journal Citation Reports (JCR)
• Scopus
• Google Scholar Citation

7. WEB OF SCIENCE
For author-level, article-level, and journal level metrics (JCR)

8. SCOPUS
For author-level, article-level, and institution-level metrics

9. GOOGLE SCHOLAR CITATIONS
For author-level and article-level metrics

10. Citation Metrics: Tracking
March 5, 2015

11. Citation Metrics: Tracking
• Team outputs
• # of team publications
• Journals for team publications
• High profile citation sources – seminal article, leading
researcher in the field, cited in a new policy, etc.
• Personal outputs
• Contribution toward key publications or other outputs
• Clearly identify ouputs created as part of a team versus
individual or smaller group work
March 5, 2015

12. Citation Metrics: Presentation
March 5, 2015
5 Top-ranked Journals
Journal
No. of
Publications
Journal Impact Factor
(2013)
5-yr Impact Factor
The Lancet Neurology 3 21.823 24.075
American Journal of Psychiatry 3 13.559 15.062
Brain 3 10.226 10.846
Neurology 11 8.303 8.375
Biological Psychiatry 5 9.472 10.347

13. Presentation: Examples
March 5, 2015
Team H-index Values
Author H-Index Team Rank
Ira Shoulson 54 1
Elizabeth H. Aylward 53 2
*Jane S. Paulsen 45 3
Julie C. Stout 42 4
Douglas R. Langbehn 33 5
Kevin M. Duff 25 6

14. Altmetrics
Team Science & Collaborative
Translational Research
March 5, 2015
Heather Coates, MLS, MS Digital Scholarship & Data Management Librarian

15. Evaluating Impact
Team Science & Collaborative Translational Research
March 5, 2015

16. March 5, 2015 https://becker.wustl.edu/impact-assessment

17. March 5, 2015
Impact Indicators
Research Output
& Activities
Biological materials, collaborations, data, databases, repositories,
techniques & procedures, grey literature, invention disclosures,
mobile apps, patents, trainees, etc.
Advancement of
Knowledge
Books/chapters (inclusion in bibliographies, library ownership,
textbook use), change in understanding (paradigm shift, lead to
new approach), citations (first & second generation citations,
countries and institutions represented), conference themes, new
centers/institutes
Clinical
Implementation
(or TRIP)
Biological materials, study cited in clinical decision aid,
clinical/practice guidelines, diagnostic application, instruments,
quality measure guidelines (gov’t or NPO), reporting requirements
Legislation &
Policy
Committee participation, study cited in guidelines, study cited in
policy, study cited in enactment of standards
Economic
Benefit
Findings cited in reduced costs for delivery of healthcare services,
findings result in enhancement of existing resources and expertise,
license agreements for use of IP, spinoff or startup company
Community
Benefit
Public awareness of risk factors, patient decision materials, cited in
public/private insurance benefit plan

18. Altmetrics
Team Science & Collaborative Translational Research
March 5, 2015

19. Traditional measures of impact
• Proxy for expert evaluation
• Typically citation-based
• Levels of evidence
• Journal-level
• Article-level
• Scholar-level
• How can you use these in your dossier?
• What is the value of these metrics?
March 5, 2015
Altmetrics typically
measure impact of
individual products

20. Citation-based Metrics
March 5, 2015
Journal Level
Metrics
Output/Article
Level Metrics
Author Level
Metrics

21. Altmetrics
March 5, 2015
Web Metrics
Social Media
Metrics
View Use
Share Recommend Discuss

22. Altmetric Manifesto
NO ONE CAN READ EVERYTHING. We rely on filters to
make sense of the scholarly literature, but the
narrow, traditional filters are being swamped.
However, the growth of new, online scholarly tools
allows us to make new filters; these altmetrics
reflect the broad, rapid impact of scholarship in this
burgeoning ecosystem. We call for more tools and
research based on altmetrics…
March 5, 2015

23. Altmetrics
• Supplement traditional metrics
• Scope extends beyond the formal published scholarly
record (journals & books)
• Timeframe: immediate to short-term impact
• Sources: focus is on social media/engagement
• DOES capture qualitative data (ex: blog snippets,
tweet content)
March 5, 2015

24. Types of scholarly outputs
• Articles
• Books & chapters
• Conference presentations, panels, etc.
• Blogs & blog posts
• Grey literature, conference materials, white papers,
unpublished reports
• Learning objects, instructional content, assessment tools,
online courses, innovative use of technology, syllabi, etc.
• Impact on diverse populations or communities; impact on
the community through media, changes in policy, law, or
programs
March 5, 2015

25. Altmetric Sources
• Publishers (e.g., PLoS, PubMed, etc.)
• Aggregation services
– Impact Story: http://impactstory.org/
– Altmetric.com: http://www.altmetric.com/
– Plum Analytics: https://plu.mx/g/samples
• Subject repositories
– PubMedCentral (PMC)
– arXiv
– SSRN
• Institutional repositories
– IUPUI ScholarWorks: http://scholarworks.iupui.edu
– IUPUI DataWorks: http://dataworks.iupui.edu
March 5, 2015

26. Aggregator Sources
• News media
• Social media platforms
• Blogs
• Twitter
• Reference managers (e.g., Mendeley)
• other online conversations captured include
YouTube, Wikipedia, Reddit, F1000, Q&A, Policy
Docs
• NOT server-side views & download data
March 5, 2015

27. March 5, 2015

28. March 5, 2015

29. March 5, 2015

30. March 5, 2015

31. March 5, 2015

32. March 5, 2015

33. ImpactStory: Example
March 5, 2015

34. Plum Analytics Example
March 5, 2015

35. March 5, 2015
The problem is not a lack of
data, but how to evaluate
and make sense of it.

36. Using Altmetrics
What is your case? What statements do you need to
support?
• Identify priorities
• Choose tools/platforms
• Gather
• Select
• Store
• Visualize
March 5, 2015

37. Gathering Data
• Gather manually
– Free
– Time-consuming
– Messy & redundant
– Tailored to your scholarship & argument
– Include unusual or field-specific sources
• Use an aggregator
– Minimal cost (individual)
– Less time-consuming
– Generic, broad presentation
– Heavy emphasis on major social media channels
March 5, 2015

38. Gathering Data: Altmetric.it
• Install Altmetric.it Bookmarklet
– http://www.altmetric.com/bookmarklet.php
– Drag the Altmetric.it icon to your browser bookmarks toolbar
• Get altmetrics for your outputs
– Go to URL for your article/blog post/column/code/etc
– Click on the Altmetric.it shortcut
– Gather data from the donut in a spreadsheet
• Note: reliant on unique identifiers like DOI, handles, arXiv ID, SSRN ID, PMC
ID, etc. but NOT limited to articles
March 5, 2015

39. Gathering Data: Scopus
• Register for an ORCID (unique author ID)
• Sign up for a Scopus account
• Link your ORCID to your Scopus account
• Maintain an accurate list of articles associated with your
profile
• Gather data for each article (track in a spreadsheet)
March 5, 2015

40. Manual Tracking: Example
March 5, 2015
NOTE: The usage, views, and mentions data are fabricated for this example.

41. Presentation: Examples
March 5, 2015
Top 5 Articles
Title Year Citations Usage Views Mentions
Apathy is not depression 1998 351 1947 4208 0
*A new model for prediction of the age of
onset and penetrance for Huntington's
disease based on CAG length 2004 303 2691 5712 2
Generalized cognitive deficits in
schizophrenia: A study of first- episode
patients 1999 303 1658 3309 0
*Venezuelan kindreds reveal that genetic and
environmental factors modulate Huntington's
disease age of onset 2004 288 2147 4348 0
*Detection of Huntington's disease decades
before diagnosis: The Predict-HD study 2008 234 5819 12381 9
*Product of the Huntington Study Group
NOTE: The usage, views, and mentions data are fabricated for this example.

42. Presentation: Examples
March 5, 2015
Top 5 Non-Article Outputs
Title Impact Evidence
Huntington Study Group website Community
Receives more than 2,000 unique visitors
per month.
HSG participant database Community 4300 registered potential participants
Research sites Community 63 research sites in 11 countries
Physician's Guide to the Management
of Huntington Disease
Clinical
Implementation/TRIP Downloaded more than 2,400 times
UHDRS
Clinical
Implementation/TRIP Downloaded more than 13,000 times
NOTE: The evidence details are fabricated for this example.

43. Sources for Altmetrics by product
• Articles
• Publisher website
• PubMedCentral
• SSRN/arXiv/PMC
• IUPUI ScholarWorks
• Presentations
• Slideshare
• IUPUI ScholarWorks
• Blogs & blog posts
• Wordpress
• Google Analytics
• Twitter
• Data
• Figshare
• DataDryad
• Data Journals
• IUPUI DataWorks
March 5, 2015

44. Using Altmetrics
• Complementary - Portray just part of the picture
– More immediate measure of impact
– Measure of impact outside your discipline and academia (public policy,
practice, community engagement, public awareness, etc.)
– Reflect engagement with and impact upon a broader audience,
beyond academia and your own area of expertise
• Flavors of impact (Priem et al, 2012)
– “popular hit” – highly tweeted and shared on non-academic social
media sites
– “expert pick” – good F1000 ratings and subsequent citations, few
shares or social media mentions
March 5, 2015

45. Resources
• ImpactStory Guide to Altmetrics:
http://impactstory.demo.libguides.com/c.php?g=211074&p=1392895
• ImpactStory blog (http://blog.impactstory.org/)
• Piwowar, H. & Priem, J. (2013). The power of altmetrics on a CV. Bulletin of
the Association for Information Science & Technology.
https://www.asis.org/Bulletin/Apr-13/AprMay13_Piwowar_Priem.html
• Roemer, R. C. & Borchardt, R. (2012). From bibliometrics to altmetrics: A
changing scholarly landscape. College and Research Libraries News, 73(10),
596-600. http://crln.acrl.org/content/73/10/596.full
• Dinsmore, A., Allen, L., & Dolby, K. (2014). Alternative perspectives on impact:
The potential of ALMs and altmetrics to inform funders about research
impact. PLoS Biology, 12(11), e1002003. doi: 10.1371/journal.pbio.1002003
• Steven Roberts website & P&T dossier, both incorporating altmetrics:
http://faculty.washington.edu/sr320/?p=2806
• Altmetric webinar: Applied Altmetrics:
http://godigitalscience.com/view/mail?iID=Y9PXAVDJMH5JVAUPJU79
March 5, 2015

46. References
• Beacham, B., Kalucy, L., McIntyre, E. (2005). Focus on Understanding
and Measuring Research Impact. Retrieved from
http://www.phcris.org.au/phplib/filedownload.php?file=/elib/lib/dow
nloaded_files/publications/pdfs/phcris_pub_3236.pdf
• Garfield, E. (2000). The use of JCR and JPI in measuring short and long
term journal impact. Presented at the Council of Scientific Editors
Annual Meeting: San Diego, CA.
• Leyesdorff, L. (2009). How are new citation-based journal indicators
adding to the bibliometric toolbox? Journal of the American Society
for Information Science and Technology, 60(7), 1327-1336.
• Priem et al. (2010). Altmetrics Manifesto:
http://altmetrics.org/manifesto/
• Priem, J., Piwowar, H., Hemminger, B. M. (2012). Altmetrics in the
wild: Using social media to explore scholarly impact.
http://arxiv.org/abs/1203.4745
March 5, 2015

47. QUESTIONS

48. Concluding Remarks
For additional P&T resources visit
http://academicaffairs.iupui.edu/
P&T Guidelines
Dossier Samples
Online Programs