Download to read offline
Kyrillidou Future Directions Measuring Impact
The document discusses the impact of scholarly research assessed through usage metrics, presented during a NISO webinar on December 14, 2017. Key speakers from various institutions present ideas regarding altmetrics and the value of research usage. The document emphasizes the importance of recognizing and evaluating scholarly impact in both traditional and alternative formats.
More Related Content
Similar to Kyrillidou Future Directions Measuring Impact
More from National Information Standards Organization (NISO)
Kyrillidou Future Directions Measuring Impact
- 1. FUTURE DIRECTIONS MEASURING THEIMPACT OF SCHOLARLY RESEARCH THROUGH USAGE Martha Kyrillidou, Ph.D. QUALITYMETRICS, LLC Helping libraries achieve success www.qualitymetricsllc.com NISO WEBINAR: Advancing Altmetrics: Best Practices and Emerging Ideas DECEMBER 14, 2017
- 2. Today’s speakers • MikeTaylor, Head of Metric Development, Digital Science • Stacy Konkiel, Director of Research and Education, Altmetric • William Gunn, Director of Academic Outreach, Mendeley • Andy Herzog, Faculty Services and Online Engagement, Department Head, University of Texas at Arlington • Patty Smith, Impact and Dissemination Librarian, Galter Health Sciences Library, Northwestern University • Daniella Lowenberg, Research Data Specialist & Product Manager, California Digital Library • Darby Orcutt, Assistant Head, Collections & Research Strategy North Carolina State University • Martha Kyrillidou, PhD., Principal, QualityMetrics, LLC
- 3.
- 4.
- 5.
- 7. C. Colleen Cook,Trenholme Dean of Libraries, McGill University Illustration by Gregory Houston
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21. It ain’t whatyou don’t know that gets you into trouble It’s what you know for sure that just ain’t so -- Mark Twain
- 22.
- 23. 'First, there wasa usage. Second, there was a part of usage that was valued and a part of it that was misjudged. And third, we are unconditionally and irrevocably in love with the idea of having past work used or re-used … and therefore usage is here to stay with us in mainstream and alternative terms'.
- 24.
Editor's Notes
- #4 What is usage? An indicator of some sort of impact It elludes us because the clicks and bytes we gather to indicate usage are not always accurate but as the data accumulates and forms ‘big data’ it is easier to see patterns while error can be minimized Even though it is not foolproof Patrick OBrien, Kenning Arlitsch, Jeff Mixter, Jonathan Wheeler, Leila Belle Sterman, (2017) "RAMP – the Repository Analytics and Metrics Portal: A prototype web service that accurately counts item downloads from institutional repositories", Library Hi Tech, Vol. 35 Issue: 1, pp.144-158, https://doi.org/10.1108/LHT-11-2016-0122 Terry Plum and Brinley Franklin – developed the MINES for Libraries Method – Measuring the Impact of Networked Electronic Services which is part of the method for calculating indirect costs for universities There is a continuum of impact – one usage does not mean the same thing in all contexts – typology of usages ranging from the mainstream to the alternative
- #5 Writer James Chapman created a list of the most read books in the world based on the number of copies each book sold over the last 50 years.
- #7 He found that the Bible far outsold any other book, with a whopping 3.9 billion copies sold over the last 50 years. "Quotations from the Works of Mao Tse-tung" came in second with 820 million copies sold, and "Harry Potter" came in third with 400 million copies sold. Designer Jared Fanning created an infographic (below) on the most read books in the world, based on Chapman's research.
- #8 In a recent keynote at the summer conference on Performance Measurement and Metrics in Oxford, UK – CC gave a keynote and featured this ilustration by Gregory Houston from McGill. The Guerilla Assessment Librarian Tries to make sense of the data through books, computers, observations And most important – ACT in a positive and forward looking way in shaping the future
- #9 This is a reflection of how librarians are engaging assessment And also a reflection of how they struggle in assessing that engagement
- #10 Altmetrics is a tool in our arsenal And I want to place altmetrics and their future in the context of some foundational ideas that have been around for a while
- #11 The concept of Evaluation – our desire to know where we stand so that we inform our next action
- #12 Placing altmetrics in the context of Assessment – our desire to offer a summative picture of where we stand at a point in time
- #13 Placing altmetrics in the context of Research – our desire to know the answer to new questions we have not asked before
- #14 And I want to borrow from Cresswell’s Research design book his four perspectives to place altmetrics in the context of the four world views Cresswell identifies Postpositivist -- … altmetrics is about numbers in many ways (influenced by postpositivist) Constructivist -- … altemetrics is about increased our understanding by creating new social and historical understanding and … theories eventually Transformative -- … altmetrics is about highlighting the marginalized … yet clearly important influence our schoarly works have in the context of change-oriented society, with new modes of communication manifested in the social media and new emerging platforms and technologies Pragrmatism – altmetrics is real-world oriented and pluralistic Our speakers today spoke to all of these different worldviews that are characterizing altmetrics … as we look into the future it is important to note these key philosophical orientations.
- #15 The future of altmetrics will be defined by the purpose of deploying these new approaches
- #16 The audiece – governments, institutions, paying more attention
- #17 The methods we use – technology, mining, searching alternatives, imposing controls or taking away some of them to experiment with new ways of capturing alternative metrics
- #18 And not least of the types of analytical approaches we deploy
- #19 And not least of the types of analytical approaches we deploy
- #20 And the representation of our findings – is the representation of our findings trustworthy and reliable – have we given credit to where credit is due
- #21 Have we given credit in trustworthy and reliable ways After all usage can take make forms – having bought a book may not be an indication that I read it, having read a book may not be an indicatio that I remember it, having remembered a reference may not be an indication that I recalled it correctly?
- #22 Mark Twain gave an answer in his own unique diction many years ago -- … This quote is on the opening scene of a recent movie – ‘The Big and Short
- #23 So how can we ensure that scholarly impact is represented in its fullest and appropriate level What is usage? An indicator of some sort of impact with gradations – levels of impact Michael Keeble Buckland, (2017) "Library technology in the next 20 years", Library Hi Tech, Vol. 35 Issue: 1, pp.5-10, https://doi.org/10.1108/LHT-11-2016-0131 Purpose The purpose of this paper is to examine the agenda for library technology for the next 20 years. Design/methodology/approach A long-term historically based analysis of the evolving roles of librarians and library technology, especially the catalog. Findings The rise of standardized cataloging codes, communications formats, bibliographical utilities, and software for online searching constitutes a great triumph in universal bibliographical access for everybody. But each reader is unique and no-one is “everybody” so a uniform service is not ideal for all. The ideal librarian knows both the collection and the readers. The catalog is a guide to the collection and a surrogate for the librarian. The librarian understands the readers. The development of library technology will remain significantly incomplete until the uniqueness of each reader is accommodated. Some ways to do that are noted. Research limitations/implications Research and development should focus on relating the uniqueness of individuals to the uniformity of services provided. GENETIC MAKE UP of scholarly impact … DNA studies … altmetrics is part of this family of DNA studies regarding scholarly impact and it will continue to do so.
- #24 In concluding, I would like to To paraphrase Stephenie Meyer in the opening lines of Twilight … 'First, there was a usage. Second, there was a part of usage that was valued [and trustworthy] and a part of it that was random and misappropriated (bots or misappropriated reference/citation). And third, I am unconditionally and irrevocably in love with the idea of building upon earlier ideas, building upon the shoulders of giants … having past work used or re-used … and therefore usage is here to stay with us in mainstream and alternative terms'.