Are the scientists on to
something with altmetrics?
Exploring the best way to assess and
track the scholarly impact of an author
“As the scholarly workflow is
increasingly migrating to the
web, formerly hidden uses like
reading, bookmarking, sharing,
discussing and rating are
leaving traces online and offer
a new ground to measure…”
Benedikt Fecher and Sascha Friesike
Open Science: One Term, Five
Schools of Thought RatSWD Working
Paper Series May 2013
Teaser: Scholarly Impact Story
Steve Pettifer
 citation count for article 80  viewed or downloaded more than
53,000 times PLOS
Experts Demonstrating Scholarly Impact
When Who/Gain Favor Why
historically- science royalty power and money
historically - legal experts letters respect from
colleagues, funds and
power
present - science UK – “impact
assessment” key
“research governance”
prove the impact of
your research beyond
the academia to get
public funds
present –science and
legal
tenure Committees-
citation counts
tenure
present- science refereed/ peer
reviewed journals
tenure and funding
present- legal Publishers and student
law review editors
published - citations
and funds
Abridged History of Science Scholarship
1665 – Henry
Oldenburg creates
the first scientific
journal
After WWII peer
and editorial
reviews
1950s ISI impact
standards developed
Web-
dissemination of
scholarship as it
happens
research or data can
now be cited
Scholarship
leaving digital
footprint
1992 ISI bought
by a vendor
Letter writing is
used to
disseminate
scholarly
knowledge
Abridged History of Legal Scholarship1808- 1870
non-student
legal
periodicals 1875 Albany Law
School published
the Albany Law
School Journal, only
one year.
1886, a group of eight third-year
students formed a new law club, known
as the Langdell Society, and out of this
club 1887 Harvard Law Review was
founded
2008 push for
empirical research in
legal scholarship
Institutional
repositories/scholarship
for faculty became the
norm 2006-2008
SSRN Legal
Scholarship Network
gains popularity
After WWII rise of
the specialty law
reviews or journals
August 2013- Six of
Duke’s law journals
are online only
1920s “Instructions for Editorial Work” was
prepared by student editors and put in the
hands of the new members of the Review
(Bluebook)
Currently How Is Scholarly Impact
Assessed?
Impact Spectrum
 Journal level impact
 Article Level impact
 Individual research impact
Analytics
Citation
analysis
•Standard
decades Bibliometrics
•Quantitative
study of the
scholarly info Webometrics
•Quantitative
study of the
web
Why do we Care?
Timeliness and
Accuracy
Citations
“Invisible
college”
(discussion,
data, blogs,
tweets)
“sleeping
beauty”
Gaming of the
System
(pay to publish,
cite us or else,
Journal Impact
Factor, ranking
Trading up)
Eugene Garfield
Likened impact factor to nuclear energy describing it as a
force that can help society but can unleash mayhem when
it is misused
Not just the scientists
Traditional Assessment Not a Level Playing Field
Complaints from scientists and researchers Complaints from legal scholars
Skewing the course of research All student editing boards are not the same
add citations to previous articles from the same journal EXpresso and Scholastica - too many submissions,
Flipping up, CV submission
Higher ranked journals are cited more – “will it be cited?” Timely and hot get published
Tenure requires publication journals with if above 5 Publish or perish
Cash bonuses for publishing in high impact journals Publish for stipend
System is short term and affecting the research being
conducted
Solicitation of articles
Flipping up – leveraging lower commitments to work up
the ranks
SSRN or other open source options are easier
Nature rejects ½ submissions before evaluation because
unfashionable
Richard Delgado: much of the civil rights scholarships
was being authored by 26 male white authors who
exclusively cited to each other.
Publish review articles – overview topic
Exact details of the JIF are a trade secret
Altmetrics
“invisible metric”
“ the creation and study of new metrics based on the
Social Web for analyzing and informing scholarship””
http://blogs.plos.org/everyone/2011/11/08/altmetrics-
tracking-scholarly-impact-on-the-social-web-plos-one-
collection/
Not just quantitative
metric as current tools
qualitative assessment
Allows for the impact
metric tool to change
Pinned
retweets
Term coined in a Tweet
Tweets – over 58K
scholarly citation in one
month
altmetrics: a manifesto
“No one can read everything. We rely on filters to make sense of the scholarly
literature, but the narrow, traditional filters are being swamped. However, the
growth of new, online scholarly tools allows us to make new filters; these
altmetrics reflect the broad, rapid impact of scholarship in this burgeoning
ecosystem. We call for more tools and research based on altmetrics.”
http://altmetrics.org/manifesto/
http://jasonpriem.org/2012/05/toward-a-second-revolution-altmetrics-total-
impact-and-the-decoupled-journal-video/
(12:00)
Altmetrics the “invisible metric”
Advantages to Altmetrics
 Tangible
 Timely - impact in days
 Impact audience
 scholars
 practitioners
 clinicians
 educators
 general public
 Openness - data coverage is completely transparent to the users
 Altmetrics are diverse
 Product: articles, datasets, software, blogs, videos
 Platform: beyond journals, institutional repositories, online communities
 Audience: beyond the academy, practitioners, clinicians and the general public
Why do we care?
 Steve Pettifer - wrote article in 2008
“it hit the right note at
the right time”
altmetrics = the whole story
Librarian hat
Altmetric Gathering
 (create profiles, data routing, stat collection)
 SSRN pages
 Twitter –
push your work
your views
have us “Follow” experts in the field
http://kevin.lexblog.com/2013/12/19/lawyers-are-a-bigger-deal-on-
twitter-than-they-think/
 Teach to “flip up”
Great for science but we are law…
 Susskind: crowd sourcing legal information as a new way
for providing legal knowledge.
 Justice Kennedy: there are quality academic (and
professional) legal blogs that offer case or issues analysis
sometimes within days or hours of a major decision being
handed down.
 Sunstein: bloggers often exchange ideas and frequently
debate and share their perspectives.
Great for science but we are law…
Again why do we care?
Jim Milles, Law
Professor at Buffalo
Law School
On Dec. 20, 2013 uploaded a draft of an article on SSRN
Legal Education in Crisis, and Why Law Libraries are Doomed
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2370567
February 2014
Law review
Journal citation
count of 0
http://www.scribd.com/doc/19253028/Twitter-in-Higher-Education-Usage-
Habits-and-Trends-of-Todays-College-Faculty
New Timeline for Assessing Scholarly Impact
 Pre-published draft to SSRN
 Post link to article = Twitter, friendfeed, Facebook, LinkedIn, the law library blog so picked up
by our RSS feed readers
 After a month start checking the blogs
 Submissions eXpresso and Scholastica (flip if you want) http://law.bepress.com/expresso/
https://scholasticahq.com/
 Accepted add it to SSRN
 Published - repost the link
 Blogs, trades, media mentions
 A year out formal citations
UK “impact
assessment”
Tools for Altmetric Gathering
 Altmetrics http://www.altmetric.com/
 Tweets, blogs, and news and more
 Zotero http://www.zotero.org/blog/zoteros-next-big-step/
 Mendeley http://www.mendeley.com/
 Delicious https://delicious.com/
 Impact Story http://impactstory.org/
 Tweets, Blogs, delicious, news and more
 CiteUlike http://www.citeulike.org/
 DataCite http://www.datacite.org/
 Figshare http://figshare.com/
 SlideShare http://www.slideshare.net/?ss
 Prezi https://prezi.com/
 Publish or Perish http://www.harzing.com/pop.htm
 Google Scholar and (since release 4.1) Microsoft Academic Search
 Twitter, Facebook, Pinterest, Google scholar
Questions? Comments? Feedback?

Are the scientists on to something altmetrics 6 16

  • 1.
    Are the scientistson to something with altmetrics? Exploring the best way to assess and track the scholarly impact of an author
  • 2.
    “As the scholarlyworkflow is increasingly migrating to the web, formerly hidden uses like reading, bookmarking, sharing, discussing and rating are leaving traces online and offer a new ground to measure…” Benedikt Fecher and Sascha Friesike Open Science: One Term, Five Schools of Thought RatSWD Working Paper Series May 2013
  • 3.
    Teaser: Scholarly ImpactStory Steve Pettifer  citation count for article 80  viewed or downloaded more than 53,000 times PLOS
  • 4.
    Experts Demonstrating ScholarlyImpact When Who/Gain Favor Why historically- science royalty power and money historically - legal experts letters respect from colleagues, funds and power present - science UK – “impact assessment” key “research governance” prove the impact of your research beyond the academia to get public funds present –science and legal tenure Committees- citation counts tenure present- science refereed/ peer reviewed journals tenure and funding present- legal Publishers and student law review editors published - citations and funds
  • 5.
    Abridged History ofScience Scholarship 1665 – Henry Oldenburg creates the first scientific journal After WWII peer and editorial reviews 1950s ISI impact standards developed Web- dissemination of scholarship as it happens research or data can now be cited Scholarship leaving digital footprint 1992 ISI bought by a vendor Letter writing is used to disseminate scholarly knowledge
  • 6.
    Abridged History ofLegal Scholarship1808- 1870 non-student legal periodicals 1875 Albany Law School published the Albany Law School Journal, only one year. 1886, a group of eight third-year students formed a new law club, known as the Langdell Society, and out of this club 1887 Harvard Law Review was founded 2008 push for empirical research in legal scholarship Institutional repositories/scholarship for faculty became the norm 2006-2008 SSRN Legal Scholarship Network gains popularity After WWII rise of the specialty law reviews or journals August 2013- Six of Duke’s law journals are online only 1920s “Instructions for Editorial Work” was prepared by student editors and put in the hands of the new members of the Review (Bluebook)
  • 7.
    Currently How IsScholarly Impact Assessed? Impact Spectrum  Journal level impact  Article Level impact  Individual research impact Analytics Citation analysis •Standard decades Bibliometrics •Quantitative study of the scholarly info Webometrics •Quantitative study of the web
  • 8.
    Why do weCare? Timeliness and Accuracy Citations “Invisible college” (discussion, data, blogs, tweets) “sleeping beauty” Gaming of the System (pay to publish, cite us or else, Journal Impact Factor, ranking Trading up)
  • 9.
    Eugene Garfield Likened impactfactor to nuclear energy describing it as a force that can help society but can unleash mayhem when it is misused
  • 10.
    Not just thescientists
  • 11.
    Traditional Assessment Nota Level Playing Field Complaints from scientists and researchers Complaints from legal scholars Skewing the course of research All student editing boards are not the same add citations to previous articles from the same journal EXpresso and Scholastica - too many submissions, Flipping up, CV submission Higher ranked journals are cited more – “will it be cited?” Timely and hot get published Tenure requires publication journals with if above 5 Publish or perish Cash bonuses for publishing in high impact journals Publish for stipend System is short term and affecting the research being conducted Solicitation of articles Flipping up – leveraging lower commitments to work up the ranks SSRN or other open source options are easier Nature rejects ½ submissions before evaluation because unfashionable Richard Delgado: much of the civil rights scholarships was being authored by 26 male white authors who exclusively cited to each other. Publish review articles – overview topic Exact details of the JIF are a trade secret
  • 12.
    Altmetrics “invisible metric” “ thecreation and study of new metrics based on the Social Web for analyzing and informing scholarship”” http://blogs.plos.org/everyone/2011/11/08/altmetrics- tracking-scholarly-impact-on-the-social-web-plos-one- collection/ Not just quantitative metric as current tools qualitative assessment Allows for the impact metric tool to change Pinned retweets Term coined in a Tweet Tweets – over 58K scholarly citation in one month
  • 13.
    altmetrics: a manifesto “Noone can read everything. We rely on filters to make sense of the scholarly literature, but the narrow, traditional filters are being swamped. However, the growth of new, online scholarly tools allows us to make new filters; these altmetrics reflect the broad, rapid impact of scholarship in this burgeoning ecosystem. We call for more tools and research based on altmetrics.” http://altmetrics.org/manifesto/ http://jasonpriem.org/2012/05/toward-a-second-revolution-altmetrics-total- impact-and-the-decoupled-journal-video/ (12:00)
  • 14.
  • 15.
    Advantages to Altmetrics Tangible  Timely - impact in days  Impact audience  scholars  practitioners  clinicians  educators  general public  Openness - data coverage is completely transparent to the users  Altmetrics are diverse  Product: articles, datasets, software, blogs, videos  Platform: beyond journals, institutional repositories, online communities  Audience: beyond the academy, practitioners, clinicians and the general public
  • 16.
    Why do wecare?  Steve Pettifer - wrote article in 2008 “it hit the right note at the right time” altmetrics = the whole story
  • 17.
    Librarian hat Altmetric Gathering (create profiles, data routing, stat collection)  SSRN pages  Twitter – push your work your views have us “Follow” experts in the field http://kevin.lexblog.com/2013/12/19/lawyers-are-a-bigger-deal-on- twitter-than-they-think/  Teach to “flip up”
  • 18.
    Great for sciencebut we are law…  Susskind: crowd sourcing legal information as a new way for providing legal knowledge.  Justice Kennedy: there are quality academic (and professional) legal blogs that offer case or issues analysis sometimes within days or hours of a major decision being handed down.  Sunstein: bloggers often exchange ideas and frequently debate and share their perspectives.
  • 19.
    Great for sciencebut we are law… Again why do we care? Jim Milles, Law Professor at Buffalo Law School On Dec. 20, 2013 uploaded a draft of an article on SSRN Legal Education in Crisis, and Why Law Libraries are Doomed http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2370567 February 2014 Law review Journal citation count of 0
  • 22.
  • 24.
    New Timeline forAssessing Scholarly Impact  Pre-published draft to SSRN  Post link to article = Twitter, friendfeed, Facebook, LinkedIn, the law library blog so picked up by our RSS feed readers  After a month start checking the blogs  Submissions eXpresso and Scholastica (flip if you want) http://law.bepress.com/expresso/ https://scholasticahq.com/  Accepted add it to SSRN  Published - repost the link  Blogs, trades, media mentions  A year out formal citations UK “impact assessment”
  • 25.
    Tools for AltmetricGathering  Altmetrics http://www.altmetric.com/  Tweets, blogs, and news and more  Zotero http://www.zotero.org/blog/zoteros-next-big-step/  Mendeley http://www.mendeley.com/  Delicious https://delicious.com/  Impact Story http://impactstory.org/  Tweets, Blogs, delicious, news and more  CiteUlike http://www.citeulike.org/  DataCite http://www.datacite.org/  Figshare http://figshare.com/  SlideShare http://www.slideshare.net/?ss  Prezi https://prezi.com/  Publish or Perish http://www.harzing.com/pop.htm  Google Scholar and (since release 4.1) Microsoft Academic Search  Twitter, Facebook, Pinterest, Google scholar
  • 26.