included in the NISO Update ALA Midwinter, Atlanta, January 21, 2017 http://www.niso.org/news/events/2017/ala_mid/

1. NISO Recommended Practices
to Support Adoption of
Altmetrics
ALA Midwinter
January 21, 2017

2. 1

3. 2

4. Definitions and Use Cases
Code of Conduct
Output Types for Assessment
Data Metrics
Persistent Identifiers and
Assessment
3

5. Definitions & Use Cases
• Caveats:
• Citations, usage data, and altmetrics are ALL
potentially important and potentially imperfect
• Please don’t use altmetrics (or any metrics) as an
uncritical proxy for scholarly impact – consider
quantitative and qualitative information too
• data quality and indicator construction are key
factors in the evaluation of specific altmetrics
(read as: garbage in, garbage out!)
4

6. What is Altmetrics? Definition
Altmetrics is a broad term that encapsulates the digital collection, creation, and
use of multiple forms of assessment that are derived from activity and
engagement among diverse stakeholders and scholarly outputs in the research
ecosystem.
The inclusion in the definition of altmetrics of many different outputs and forms of
engagement helps distinguish it from traditional citation-based metrics, while at
the same time, leaving open the possibility of their complementary use, including
for purposes of measuring scholarly impact.
However, the development of altmetrics in the context of alternative assessment
sets its measurements apart from traditional citation-based scholarly metrics.
5

7. Use Cases
Developed eight personas, three themes:
Showcase achievement: Indicates stakeholder
interest in highlighting the positive achievements
garnered by one or more scholarly outputs.
Research evaluation: Indicates stakeholder
interest in assessing the impact or reach of
research.
Discovery: Indicates stakeholder interest in
discovering or increasing the discoverability of
scholarly outputs and/or researchers.
6

8. Personas: academic/researcher
7

9. Code of Conduct
• Why a Code of Conduct?
• Scope
• Altmetric Data Providers vs.
Aggregators
8

10. Code of Conduct Key Elements
• Transparency
• Replicability
• Accuracy
9

11. Code of Conduct: Reporting
List all available data and metrics (providers & aggregators) and altmetrics data providers from which data are collected (aggregators).
Provide a clear definition of each metric provided.
Describe the method(s) by which data is generated or collected and how this is maintained over time.
Describe any and all known limitations of the data provided.
Provide a documented audit trail of how and when data generation and collection methods change over time with any and all known effects of these changes,
including whether changes were applied historically or only from change date forward.
Describe how data is aggregated.
Detail how often data is updated.
Provide the process of how data can be accessed.
Confirm that data provided to different data aggregators and users at the same time is identical and, if not, how and why they differ.
Confirm that all retrieval methods lead to the same data and, if not, how and why they differ.
Describe the data quality monitoring process.
Provide process by which data can be independently verified (aggregators only).
Provide a process for reporting and correcting suspected inaccurate data or metrics.
10

12. Non-traditional
Outputs
11

13. Alternative outputs
12

14. Persistent Identifiers
13

15. Recommendations re Data Metrics
• Metrics on research data should be made
available as widely as possible
• Data citations should be implemented following
the Force11 Joint Declaration of Data Citation
Principles, in particular:
– Use machine-actionable persistent identifiers
– Provide metadata required for a citation
– Provide a landing page
– Data citations should go into the reference list or
similar metadata.
14

16. Where to next?
15

17. Key Original Ideas Not Yet Done
• What is the role of alternative assessment
metrics in research evaluation and identify
and what gaps exist in data collection around
evaluation scenarios.
• Identify best practices for grouping and
aggregating multiple data sources
• Identify best practices for grouping and
aggregating by journal, author, institution and
funder.
16

18. Questions?
17