Alternative Metrics

461 views
383 views

Published on

What are Altmetrics and what can they do for me?

Alternative metrics, or altmetrics, are being rapidly applied in a variety of contexts for the STEM community. They are providing a number of new assessment tools for the research community. However, their application isn't just for academic publishers. There are a variety of ways in which trade, commercial and other publishes can use data to make better decisions, improve their operations, enhance discovery and hopefully improve their bottom line. During this session, we will explore what alt metrics are, how they are derived, and and how they are applied. We will also consider some ways in which data analysis can be applied to the publishing business.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
461
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Alternative Metrics

  1. 1. Alternative MetricsHow the future of assessment canimprove discovery and delivery of content Todd Carpenter, Executive Director, NISO Tools of Change Conference, Franfurt Book Fair October 9, 2012
  2. 2. @TAC_NISO Twitter Highlights• So excited to present on #altmetrics today during #tocfbf at #fbf12• Again, I’m pre-tweeting my presentation. Slides to come! #tocfbf #fbf12• #NISO = 1/3 Publishers, 1/3 Libraries, 1/3 automation suppliers. Communities converge to develop standards #tocfbf #fbf12• Standards are everywhere in book publishing, we just don’t notice them anymore. #tocfbf #fbf12• In 1955 Eugene Garfield published “Citation Indexes for Science” in Science http://bit.ly/xfylwh #tocfbf #fbf12• Quote: “If I have seen further it is by standing on ye shoulders of Giants." Sir Isaac Newton #tocfbf #fbf12• Sounds like beginning of a joke: 2 guys at Stanford, they wrote this paper, then started a company in a garage ... #tocfbf #fbf12• Citations are awesome, aren’t they? A perfect solution to all the world’s problems, no? #tocfbf #fbf12• Citations aren’t always a measure of quality. They are also a very lagging indicator of impact #tocfbf #fbf12• #altmetrics a suite of assessment criteria & measures being developed to assess importance of a particular scholarly output #tocfbf #fbf12• #altmetrics examples: usage analysis, metrics of non-traditional content, observational or behavioral analysis, social media #tocfbf #fbf12• How 1 can apply #altmetrics Sell related content, spot trends in near-real-time, ID new fields or new services, marketing #tocfbf #fbf12• Real world #altmetrics examples: http://bit.ly/OzGnGt www.tynt.com http://bit.ly/TcNIh0 http://bit.ly/ PknZb4 #tocfbf #fbf12• Issues needing consensus regarding #altmetrics - Authorship, provenance, definitions of components, privacy, openness #tocfbf #fbf12
  3. 3. About• Non-profit industry trade association accredited by ANSI with ~130 members• Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media• Volunteer driven organization: 400+ spread out across the world• Responsible for standards like ISSN, DOI, Dublin Core metadata, DAISY digial talking books, OpenURL, MARC records, and ISBN (indirectly)
  4. 4. Standards are familiar, even if you don’t notice Image: DanTaylor Image: : Joel Washing
  5. 5. Who is reading published content?Someone we all hope!In our world, people still read content.There are editors, hopefully professors, perhaps students, perhaps college students, perhaps even thegeneral public or even puppies. What we tend to forget though is that in our digitally mediated environment, there is one class of readers,we don’t often think of…. Machines.
  6. 6. Who is reading published content?Someone we all hope!In our world, people still read content.There are editors, hopefully professors, perhaps students, perhaps college students, perhaps even thegeneral public or even puppies. What we tend to forget though is that in our digitally mediated environment, there is one class of readers,we don’t often think of…. Machines.
  7. 7. We stand on the shoulders of giants...In responding to a letter from a scientific rival Robert Hooke,  Sir Isaac Newton wrote;“What Des-Cartes did was a good step. You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen furtherit is by standing on ye shoulders of Giants."
  8. 8. The Mighty Citation
  9. 9. Impact factorFifty-seven years ago, on July 15, 1955, Eugene Garfield, Ph.D published his groundbreaking paper on citation indexing, “Citation Indexes for Science: A New Dimension in Documentationthrough Association of Ideas.” in Science magazine. This innovative paper envisioned information tools that allow researchers to expedite their research process, evaluate the impact of theirwork, spot scientific trends, and trace the history of modern scientific thoughts. With that paper, essentially he launched the field of bibliometrics.Three years later, in July 1958, Eugene Garfield laid the foundations for ISI (Institute for Scientific Information) by borrowing $500 from Household Finance. He hired his first full-timeemployee and began to build an organization that included more than 500 people when it was acquired by The Thomson Corporation in 1992.
  10. 10. Thomson-Reuters IMPACT FACTORThe impact factor and its compilation the Journal Citation Reports have become over the pastfive-plus decades since it was launched, “the” metric for assessing journal quality. Journalslive and die by this metric. In some developing countries, authors are awarded bonuses ifyou’ve been published in a highly-ranked title. Many in the STM community regularly touttheir publication’s performance on charts like this one for physical chemistry. At the end ofJune when the JCR is released, it is often accompanied by a stream of press releasesannouncing this or that title’s Impact Factor. For all it’s importance and value, the IF is animperfect measure and the community has been arguing about its imperfections for years.These include, the time delay of citation data, the inability to compare different domains, thelack of granularity and the figure’s over-use and misapplication. If there was one metric, thatthe scholarly community was interested in finding an alternate to, it probably is this one.That said, it is an ingenious and valuable metric that really has stood the test of time.
  11. 11. Two Stanford StudentsLet’s fast forward about 50 years from Mr Garfield’s publication and the launch of the impact factor. Beyond the impact factor, citation-based assessment metrics do play animportant role in our community, though not in the way you might think, and certainly well beyond the domain of STM publishing.So these two students were working on the Stanford Digital Library Project (SDLP). The SDLPs goal was “to develop the enabling technologies for a single, integrated anduniversal digital library" and was funded through the National Science Foundation among other federal agencies. They were focused on the problem of finding out which webpages link to a given page, considering the number and nature of such backlinks to be valuable information about that page (with the role of citations in academic publishing inmind). This research project was nicknamed "BackRub".
  12. 12. They wrote a paper
  13. 13. They started a company in someone’s garage Photo: AP
  14. 14. whenever you use google, you are using a variant of a bibliometric citation analysis that is acombination of reference linking and usage data to provide your search results. Basically,this is an “alt-metric” by a different name.
  15. 15. A simple conclusion can be drawnCitations are awesome, aren’t they?A perfect solution to all the world’sproblems, no?
  16. 16. Back to Sir Isaac Newton “What Des-Cartes did was a good step.You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants."Reflecting on how I began this talk about citations, let’s return to the famous Newtonianquote. “What Des-Cartes did was a good step. You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If Ihave seen further it is by standing on ye shoulders of Giants." The quote came from an exchange of letters between RobertHooke and Newton about a paper that Newton had written on the properties of light. Hookehad taken umbridge over the paper as it was something he had explored some ten yearsprior in his seminal work “Micrographia”. Newton was not a man to dole out praise,particularly to men with whom he disdained or had scientific disagreements with. It is notlikely that Newton viewed Hooke as an intellectual giant since it was in part Hooke’scriticisms that led to Newton’s self-initiated withdrawal from the Royal Society in 1874. It isvery likely as some scholars have argued that the phrase "on the shoulders of giants" as aveiled insult of Hooke, who was quite short, and also the butt of a successful theater farce atthe time entitled, The Virtuoso.While many believe that was the sentiment being expressed by Newton in his letter to Hooke, some researchers have suggested that he was actually using the phrase "on theshoulders of giants" as a veiled insult of Robert Hooke, who was a rather short man. Newton had a reputation as somewhat of a petty and vindictive man whose ego clashedwith those of his rivals in the scientific and mathematical communities. One of these rivals was Robert Hooke, who had been involved in a long-running fued with Newton overwhich one had discovered the inverse square law. Although Newtons letter to Hooke appeared courteous on the surface, some historians have concluded that he cleverlyemployed the phrase "on the shoulders of giants" to ridicule Hookes lack of physical stature and imply that he lacked intellectual stature as well.
  17. 17. Caveat emptor Citations aren’t always what we think they areThe quote came from an exchange of letters between Robert Hooke and Newton about apaper that Newton had written on the properties of light. Hooke had taken umbridge over thepaper as it was something he had explored some ten years prior in his seminal work“Micrographia”. Newton was not a man to dole out praise, particularly to men with whom hedisdained or had scientific disagreements with. It is not likely that Newton viewed Hooke asan intellectual giant since it was in part Hooke’s criticisms that led to Newton’s self-initiatedwithdrawal from the Royal Society in 1874. It is very likely as some scholars have argued thatthe phrase "on the shoulders of giants" as a veiled insult of Hooke, who was quite short, andalso the butt of a successful theater farce at the time entitled, The Virtuoso.While many believe that was the sentiment being expressed by Newton in his letter to Hooke, some researchers have suggested that he was actually using the phrase "on theshoulders of giants" as a veiled insult of Robert Hooke, who was a rather short man. Newton had a reputation as somewhat of a petty and vindictive man whose ego clashedwith those of his rivals in the scientific and mathematical communities. One of these rivals was Robert Hooke, who had been involved in a long-running fued with Newton overwhich one had discovered the inverse square law. Although Newtons letter to Hooke appeared courteous on the surface, some historians have concluded that he cleverlyemployed the phrase "on the shoulders of giants" to ridicule Hookes lack of physical stature and imply that he lacked intellectual stature as well.
  18. 18. Another problem with citationsCitation data reflects the very last stages associated with the publication process.Usage data can reflect earlier stages and reflect a wide range of scholarlycommunication activities; it serves as an early, and potentially more comprehensiveindicator.
  19. 19. Mapping the World of Science Clickstream data yields high- resolution maps of scienceJohan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009The comprehensiveness of other forms of data collection and analysis can prove much more robust. For comparison, the JCR is a curatedlist of only a fraction of the overall scholarly communications community. This image was published a few years back by Johan Bollen andHerbert Von de Sompel based on large-scale data analysis of clickstream data. It shows the connections between publications based onusage data and user session logs.
  20. 20. Mapping the World of Science Clickstream data yields high- resolution maps of scienceJohan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009The comprehensiveness of other forms of data collection and analysis can prove much more robust. For comparison, the JCR is a curatedlist of only a fraction of the overall scholarly communications community. This image was published a few years back by Johan Bollen andHerbert Von de Sompel based on large-scale data analysis of clickstream data. It shows the connections between publications based onusage data and user session logs.
  21. 21. Define Altmetrics, Please
  22. 22. Usage Based Metrics
  23. 23. Citation of non-traditional contentCitations to newforms ofcommunciationaren’t as simpleas one mightthink.Define a use ofthis? Image: Domenico, Caron, Davis, et al.
  24. 24. Clickstream Analysis
  25. 25. Social Media MetricsImage: Danny Brown, The State of Social Media Marketing 2012
  26. 26. Behavioral Metrics
  27. 27. Multivariate analysis
  28. 28. OK, it’s important, butwhat does it mean for MY BUSINESS
  29. 29. Have you boughtsomething from Amazon?
  30. 30. Did you buy something that“other people bought”?
  31. 31. Realtime Trend Spotting
  32. 32. Innovation often takes place at theintersection of different fields
  33. 33. Image: http://www.crispian.net/CrispiansScienceMap.htmlThe names you are probably most familiar with on this model of science are the names thatare household names, Kepler, Galileo, Newton, Darwin, Einstien, Crick, Leeuwenhoek orWolfram. All worked at the space between (or creating new domains).
  34. 34. Mapping the World of Science Clickstream data yields high- resolution maps of scienceJohan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
  35. 35. Mapping the World of Science Clickstream data yields high- resolution maps of scienceJohan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
  36. 36. Improve products and customer services
  37. 37. Editorial applications
  38. 38. Author services(“How is my book doing?”)
  39. 39. Marketing applications
  40. 40. Is anyone really doing any of this?(Beyond Amazon & Google?)
  41. 41. There are lots of things that make a photo interesting (or not) in the Flickr. Where the clickthroughs are coming from; who comments on it and when; who marks it as a favorite; its tags and many morethings which are constantly changing. Interestingness changes over time, as more and more fantastic photos and stories are added to Flickr.
  42. 42. 70% Anyone here work for a company using TYNT? You may not have ever heard of TYNT, butit’s the 9th largest data collecter on the wbe, right behind AOL’s advertising network and isthe second most popular widget on the web behind the Facebook Connect App according tokruxdigital.com. This widget tracks people’s copy/paste behavior through specific referencelinking.
  43. 43. Ex Libris is a provide of library management software and services. In 2009, ExLibrislaunched the BX service, a scholarly recommender service. Recommendations generated bythe bX system are based on an analysis of usage logs derived from OpenURL link resolutiondata. So users who move from article to article during a session leave a trail of usage data,which is gathered and analyzed to present recommendations of other articles to view, similarin some ways to what Amazon does.
  44. 44. Last summer, Dominique Raccah, CEO of Sourcebooks, presented a very interesting sessionat the IDPF Digital Book 2012 meeting prior to BookExpo discussing how Sourcebooks usesdata that includes large-scale sales data, user studies, web traffic and social analytics tomeasure and optimize their business. You can listen to a recording of her presentation fromthe IDPF website.
  45. 45. Issues we face withalternative metrics
  46. 46. Image: Flickr user karindalziel
  47. 47. Authorship ----- DisambiguationContributor Roles
  48. 48. Basic Definitions Needed(So we are all talkingabout the same thing)
  49. 49. More open exchange of component data
  50. 50. All you need is.... Large Amounts of Data Good AnalystsImplementation Strategy Creativity Standards (we can only do so much)
  51. 51. One Very Last Thing… Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/Charles Mingus (22 April 1922 – 5 January 1979) was an American jazz bassist, composer, bandleader, and occasional pianist also knownfor his activism against racial injustice.Participating in this process is a very valuable thing for your organization, for your customers and end-users.
  52. 52. Thank you! Todd Carpenter, Executive Director tcarpenter@niso.orgNational Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org

×