Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Haustein, S. (2017). The evolution of scholarly communication and the reward system of science

756 views

Published on

Haustein, S. (2017, February). The evolution of scholarly communication and the reward system of science. Fourth Annual KnoweScape Conference 2017, 22–24 February 2017, Sofia (Bulgaria). keynote

http://knowescape.org/knowescape2017/

Published in: Science
  • Hi, good presentation... I was pleased by your slides and it helped me a lot to understand some new concept. Reason is I am new to this field of study. But, later I found one problem, it was written communalism, which created me problem. It should have been 'Communism' I corrected myself by reading the related references at link http://mite.edu.hku.hk/f/acadstaff/367/Communism_Universalism_and_Disinterestedness.pdf ... Thank you
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Haustein, S. (2017). The evolution of scholarly communication and the reward system of science

  1. 1. The  evolution of  scholarly communication   and  the  reward system  of  science Stefanie  Haustein   @stefhaustein
  2. 2. Outline Scholarly  communication From  the  16th century  to  Open  Science Bibliometrics From  library  management  to  research  evaluation Altmetrics Opportunities  and  challenges Conclusions  and  Outlook
  3. 3. Invisible  Colleges http://commons.wikimedia.org/wiki/File:Marin_mersenne.jpg#/media/File:Marin_mersenne.jpg http://commons.wikimedia.org/wiki/File:Henry_Oldenburg.jpg Père  Marin  Mersenne (1588-­‐1648) Henry  Oldenburg (1619-­‐1677)
  4. 4. Scientific  Societies http://upload.wikimedia.org/wikipedia/commons/thumb/6/61/Louis_XIV_und_Colbert_in_der_Akademie.jpg/640px-­‐Louis_XIV_und_Colbert_in_der_Akademie.jpg https://www.rcseng.ac.uk/museums/hunterian/images/lost-­‐museums-­‐2011/the-­‐royal-­‐society-­‐repository/Image%201%20-­‐%20GreshamCollege.jpg/image_preview http://en.wikipedia.org/wiki/Nullius_in_verba#/media/File:Bookplate_of_the_Royal_Society_(Great_Britain).jpg L’Académie  royale  des  sciences 22  December 1666 The  Royal  Society 28  November 1660
  5. 5. Scientific  Journals http://de.wikipedia.org/wiki/Journal_des_s%C3%A7avans#/media/File:1665_journal_des_scavans_title.jpg http://en.wikipedia.org/wiki/Philosophical_Transactions_of_the_Royal_Society#/media/File:Philosophical_Transactions_Volume_1_frontispiece.jpg Le  journal  des  sçavans 5  January 1665 Philosophical Transactions 6  March  1665
  6. 6. Scientific  Articles Harmon,  J.E.  &  Gross,  A.G.  (2007).  The  Scientific  Literature.  A  Guided  Tour.  Chicago:  University  of  Chicago  Press. • Experiments  and  descriptions  of  the  natural  world • Avoiding  “fine  speaking” • Various  styles  of  arguing • Qualitative  and  personal  judgements
  7. 7. Proportion  of  IMRaDadoption  in  medicaljournalsNumberof  references1900  to  2004 Scientific  Articles Larivière,  V.,  Archambault,  É.  &  Gingras,  Y.  (2008).  Long-­‐term  variations  in  the  aging  of  scientific  literature:  From  exponential  growth  to  steady-­‐state  science  (1900-­‐2004).  Journal  of  the   American  Society  for  Information  Science  and  Technology,  59(2),  288-­‐296. Sollaci,  L.B.  &  Pereira,  M.G.  (2004).  The  introduction,  methods,  results,  and  discussion  (IMRAD)  structure:  a  fifty-­‐year  survey.  Journal  of  the  Medical  Library  Association,  92(3),  364-­‐371 • Professionalized  and  highly  specialized • Increased  focus  on  data,  graphs,  tables  and  theory • Impersonal,  technical  and  codified • Style  guides  and  gatekeeping • Citations • Introduction,  Methods,  Results  and  Discussion
  8. 8. Digital  Revolution arXiv submission  statistics  from  http://arxiv.org/stats/monthly_submissions Larivière,  V.,  Lozano,  G.A.  &  Gingras,  Y.  (2014).  Are  elite  journals  declining?  Journal  of  the  Association  for  Information  Science  and  Technology,  65(4),  649-­‐655. • Improved access • Acceleration • Collaboration • Peer  review • Distribution  of  preprints • Decreasing importance  of   scientific journal • Journal  functions • Diversification  of   publication  venues • Symbolic capital  of   journals unchanged Submissionsto  arXiv Share  of  top  1%  mostcitedpapers
  9. 9. Academic Publishing Market Larivière,  V.,  Haustein,  S.,  &  Mongeon,  P.  (2015).  The  oligopoly  of  academic  publishers  in  the  digital  era.  PLoS ONE,  10(6),  e0127502.  doi:   10.1371/journal.pone.0127502 • Aggravation  of   serials  crisis • Elsevier:  30%  increase   of  subscription  price   • Profit  margins  of   commercial  publishers   up  to  40% • Decline  of   scientific  societies   as  publishers • >50%  of  papers   owned  by  five  major   publishers
  10. 10. Open  Access Archambault,  É.,  Amyot,  D.,  Deschamps,  P.,  Nicol,  A.,  Rebout,  L.  &  Roberge,  G.  (2013).  Proportion  of  Open  Access  Peer-­‐Reviewed  Papers  at  the  European  and   World  Levels  2004-­‐2011.  Report  for  the  European  Commission.  http://www.science-­‐metrix.com/pdf/SM_EC_OA_Availability_2004-­‐2011.pdf Budapest  Open  Access  Initiative “immediate,  free  availability  on  the  public  internet,  permitting   any  users  to  read,  download,  copy,  distribute,  print,  search  or   link  to  the  full  text  of  these  articles” • Gold  and  Green • Libre  and  Gratis • Hybrid • Elsevier:  $500  to  5,000 • Springer:  $3,000 • Wiley:  $3,000 Freelyavailablejournal  papers2004  to  2011 Budapest  Open  Access  Initiative  (2002)
  11. 11. Open  Science Kraker,  P.,  Leony,  D.,  Reinhardt,  W.  &  Beham,  G.  (2011).  The  case  for  an  open  science  in  technology  enhanced  learning.  International  Journal  of  Technology   Enhanced  Learning,  3(6),  643-­‐654. “opening  up  the  research  process  by  making  all  of  its  outcomes,   and  the  way  in  which  these  outcomes  were  achieved,  publicly   available  on  the  World  Wide  Web” • Open  Data • Open  Source • Open  Methodology • Open  Access • Open  Peer  Review Krakeret    al.  (2011,  p.  645)
  12. 12. Bibliometrics Gross,  P.L.K.  &  Gross,  E.M.  (1927).  College  libraries  and  chemical  education.  Science,  66(1713),  385-­‐389. Library  collection  management   Journalscitedin  the  Journal  of  the  American  Chemical  Society1926  
  13. 13. Bibliometrics Garfield,  E.  (1955).  Citation  indexes  for  science.  A  new  dimension  in  documentation  through  association  of  ideas.  Science,  122,  108-­‐111. Information  retrieval • “It  would  not  be  excessive  to  demand  that  the   thorough  scholar  check  all  papers  that  have   cited  or  criticized  such  papers,  if  they  could  be   located  quickly.  The  citation  index  makes  this   check  practicable.” • Institute  for  Scientific  Information • Science  Citation  Index • Source  Author Index • Citation  Index Garfield  (1955,  p.  108)
  14. 14. Bibliometrics Price,  D.  J.  d.  S.  (1961).  Science  Since  Babylon.  New  Haven  /  London:  Yale  University  Press, Price,  D.  J.  d.  S.  (1963).  Little  Science,  Big  Science.  New  York:  Columbia  University  Press. Sociology  of  science Derek  J.  de  Solla Price Science  since  Babylon  (1961) Little  Science  – Big  Science  (1963)
  15. 15. Bibliometrics Merton,  R.  K.  (1988).  The  Matthew  effect  in  science,  II:  Cumulative  advantage  and  the  symbolism  of  intellectual  property.  Isis,  79,  606–623.   Sociology  of  science Robert  K.  Merton • Social  norms of  science • Communalism • Universalism • Disinterestedness • Organized skepticism • Matthew  effect “symbolically,  [the  reference]  registers  in  the  enduring  archives   the  intellectual  property  of  the  acknowledged  source  by   providing  a  pellet  of  peer  recognition  of  the  knowledge  claim” Merton  (1988,  p.  621)
  16. 16. Bibliometrics Moed,  H.F.,  Burger,  W.J.M.,  Frankfort,  J.G,  van  Raan,  A.F.J.  (1985).  The  use  of  bibliometric  data  for  the  measurement  of  university  research  performance.   Research  Policy,  14(3),  131-­‐149.   Research  evaluation • Performance  measurement and  policy instrument “When  used  properly,  bibliometric  indicators  can  provide  a   ‘monitoring  device’  for  university  research-­‐management  and   science  policy.  They  enable  research  policy-­‐makers  to  ask  relevant   questions  of  researchers  on  their  scientific  performance,  in  order   to  find  explanations  of  the  bibliometric  results  in  terms  of  factors   relevant  to  policy.” • Commercialization Moedet  al.  (1985,  p.  131)
  17. 17. Bibliometrics Research  evaluation • Part  of  hiring,  promotion  and  funding decisions • Dashboard  tools
  18. 18. Bibliometrics Hvistendahl,  M.  (2013).  China’s  publication  bazaar.  Science,  342(6162),  1035-­‐1039. van  Noorden,  R.  (2013).  Brazilian  citation  scheme  outed:  Thomson  Reuters  suspends  journals  from  its  rankings  for  ‘citation  stacking’,  Nature,  500(7464),  510-­‐511.   Research  evaluation • Oversimplification • Publications  =  productivity • Citations  =  impact • Uninformed  use  and  misuse • Impact  factor • h-­‐index • Adverse  effects • “Salami”  publishing • Honorary  authorship • Self-­‐citations • Citation  cartels
  19. 19. Scholarly metrics Björneborn,  L.  &  Ingwersen,  P.  (2004),  Toward  a  basic  framework  for  webometrics.  Journal  of  the  American  Society  for  Information  Science  and  Technology,   55(14),  1216–1227. Definitions informetrics scientometrics bibliometrics cybermetrics webometrics adaptedfrom:  Björneborn&  Ingwersen(2004,  p.  1217)
  20. 20. Scholarly metrics Otlet,  P.  (1934).  Traité  de  documentation:  le  livre  sur  le  livre,  théorie  et  pratique. Pritchard,  P.  (1927).  Statistical bibliography or  bibliometrics?  Journal  of  Documentation,  25,  348-­‐349.. Bibliometrics informetrics scientometrics bibliometrics cybermetrics webometrics “La  «Bibliometrie»  sera   la  partie  définie  de  la   Bibliologie  qui  s'occupe   de  la  mesure  ou   quantité  appliquée  aux   livres.”   “the  application  of   mathematics  and   statistical  methods  to   books  and  other  media   of  communication” Pritchard  (1969,  p.  348) Otlet  (1934,  p.  14)
  21. 21. Scholarly metrics Altmetrics adaptedfrom:  Björneborn&  Ingwersen(2004,  p.  1217) informetrics scientometrics bibliometrics cybermetrics webometrics altmetrics Björneborn,  L.  &  Ingwersen,  P.  (2004),  Toward  a  basic  framework  for  webometrics.  Journal  of  the  American  Society  for  Information  Science  and  Technology,   55(14),  1216–1227.
  22. 22. Scholarly metrics Priem,  J.  (2014).  Altmetrics.  In  B.  Cronin  &  C.  R.  Sugimoto (Eds.),  Beyond  bibliometrics:  harnessing multidimensional indicators of  performance  (pp.  263–287).   Cambridge,  MA:  MIT  Press.   Rousseau,  R.  &  Ye,  F.  (2013).  A  multi-­‐metric approach for  research evaluation.  Chinese Science  Bulletin,  3288–3290.  doi:10.1007/s11434-­‐013-­‐5939-­‐3 Altmetrics informetrics scientometrics bibliometrics cybermetrics webometrics altmetrics “study  and  use  of   scholarly  impact   measures  based  on   activity  in  online  tools   and  environments” “a  good  idea  but  a   bad  name” Rousseau  &  Ye(2013,  p.  3289) Priem(2014,  p.  266)
  23. 23. Scholarly metrics Definition altmetrics informetrics scientometrics bibliometrics cybermetrics webometrics adaptedfrom:  Björneborn&  Ingwersen(2004,  p.  1217) Scholarly  metrics Björneborn,  L.  &  Ingwersen,  P.  (2004),  Toward  a  basic  framework  for  webometrics.  Journal  of  the  American  Society  for  Information  Science  and  Technology,   55(14),  1216–1227.
  24. 24. Scholarly metrics Haustein,  S.,  (2016).  Grand  challenges  in  altmetrics:  heterogeneity,  data  quality  and  dependencies.  Scientometrics,  108(1),  413–423.   altmetrics informetrics scientometrics bibliometrics cybermetrics webometrics Scholarly  metrics Acts:  viewing,  reading,   saving,  diffusing,   mentioning,  citing,   reusing,  modifying,  etc. Scholarly  documents:   papers,  books,  blog  posts,   datasets,  code,  etc. Scholarly  agents:   researchers,  universities,   funders,  journals,  etc. “[S]cholarly metrics  are   thus  defined  as  indicators   based  on  recorded  events   of  acts  […]  related  to   scholarly  documents  […]   or  scholarly  agents  […].” Haustein  (2016,  p.  348)
  25. 25. Altmetrics Priem,  J.,  Taraborelli,  D.,  Groth,  P.,  &  Neylon,  C.  (2010).  Alt-­‐metrics:  a  manifesto.  October.  Retrieved from http://altmetrics.org/manifesto/   • Information  overload “We  rely  on  filters  to  make  sense  of  the  scholarly  literature,  but   the  narrow,  traditional  filters  are  being  swamped.  However,  the   growth  of  new,  online  scholarly  tools  allows  us  to  make  new   filters;  these  altmetrics  reflect  the  broad,  rapid  impact  of   scholarship  in  this  burgeoning  ecosystem.” • Criticism  against  current  form  of  research   evaluation • Alternative  forms  of  research  output • Alternative  use  and  visibility  of  publications Priem et  al.  (2010)
  26. 26. Altmetrics Coverage per  platform Haustein,  S.,  Costas,  R.,  &  Larivière,  V.  (2015).  Characterizing  social  media  metrics  of  scholarly  papers:  The  effect  of  document  properties  and  collaboration   patterns.  PLoS ONE,  10(5),  e0127830.  doi:  10.1371/journal.pone.0120495 Zahedi,  Z.,  &  Haustein,  S.  (in  preparation).  Which  document  features  attract  users  in  Mendeley?  An  analysis  of  bibliographic  characteristics  of  Web  of  Science   publications  and  Mendeley  readership  counts.
  27. 27. Mathematics  & Computer  Science Natural  Sciences &  Engineering Life  & Earth Sciences Biomedical & Health Sciences Social  Sciences &  Humanities 76,4  % 83,7  % 91,4  % 86,5  % 81,7  % Mendeley 7,5  % 12,9  % 21,6  % 31,7  % 26,0  % Twitter Altmetrics Coverage per  discipline Haustein,  S.,  Costas,  R.,  &  Larivière,  V.  (2015).  Characterizing  social  media  metrics  of  scholarly  papers:  The  effect  of  document  properties  and  collaboration   patterns.  PLoS ONE,  10(5),  e0127830.  doi:  10.1371/journal.pone.0120495 Zahedi,  Z.,  &  Haustein,  S.  (in  preparation).  Which  document  features  attract  users  in  Mendeley?  An  analysis  of  bibliographic  characteristics  of  Web  of  Science   publications  and  Mendeley  readership  counts.
  28. 28. Altmetrics Density and  intensity per  platform Intensity Haustein,  S.,  Costas,  R.,  &  Larivière,  V.  (2015).  Characterizing  social  media  metrics  of  scholarly  papers:  The  effect  of  document  properties  and  collaboration   patterns.  PLoS ONE,  10(5),  e0127830.  doi:  10.1371/journal.pone.0120495 Zahedi,  Z.,  &  Haustein,  S.  (in  preparation).  Which  document  features  attract  users  in  Mendeley?  An  analysis  of  bibliographic  characteristics  of  Web  of  Science   publications  and  Mendeley  readership  counts.
  29. 29. Altmetrics Spearman  correlations with citations Perfectnegativecorrelation Perfectpositive  correlation Haustein,  S.,  Larivière,  V.,  Thelwall,  M.,  Amyot,  D.,  &  Peters,  I.  (2014).  Tweets  vs.  Mendeley  readers:  How  do  these two social  media  metrics differ.   Information  Technology,  56(5),  207–215.  doi:  10.1515/itit-­‐2014-­‐1048 Haustein,  S.,  Costas,  R.,  &  Larivière,  V.  (2015).  Characterizing  social  media  metrics  of  scholarly  papers:  The  effect  of  document  properties  and  collaboration   patterns.  PLoS ONE,  10(5),  e0127830.  doi:  10.1371/journal.pone.0120495
  30. 30. Altmetrics Spearman  correlations with citations NSF  Subdiscipline General  Biomedical Research 2011 Size of data points represents number of Mendeley readers in Twitter graph (left) and number of tweets in Mendeley graph (right). Haustein,  S.,  Larivière,  V.,  Thelwall,  M.,  Amyot,  D.,  &  Peters,  I.  (2014).  Tweets  vs.  Mendeley  readers:  How  do  these two social  media  metrics differ.   Information  Technology,  56(5),  207–215.  doi:  10.1515/itit-­‐2014-­‐1048
  31. 31. Altmetrics Document  types Haustein,  S.,  Costas,  R.,  &  Larivière,  V.  (2015).  Characterizing  social  media  metrics  of  scholarly  papers:  The  effect  of  document  properties  and  collaboration   patterns.  PLoS ONE,  10(5),  e0127830.  doi:  10.1371/journal.pone.0120495
  32. 32. Altmetrics Highly tweeted
  33. 33. Altmetrics Highly tweeted
  34. 34. Altmetrics Highly tweeted
  35. 35. Altmetrics Highly tweeted
  36. 36. Altmetrics Altmetrics  in  the  wild
  37. 37. Opportunities and  Challenges • Heterogeneity • Time  and  timing • Audiences  and  user  groups Altmetrics
  38. 38. Opportunities • Different acts • Diverse  motivations Ø Diverse  impact Challenges • Understanding underlying processes Ø Determining the  meaning of  metrics Heterogeneity of  Altmetrics
  39. 39. Heterogeneity Saving to  Mendeley Mentioning in  News
  40. 40. Heterogeneity Recommending on  F1000 Tweeting
  41. 41. Heterogeneity Bertin,  M.,  Atanassova,  I.,  Gingras,  Y.,  &  Larivière,  V.  (2015).  The  invariant  distribution  of  references  in  scientific  articles.  Journal  of  the  Association  for     Information  Science  and  Technology, 67(1),  164-­‐177.  doi:  10.1002/asi.23367 Distribution  of  referencesalongthe  IMRaDstructure Citing in  a journal  article
  42. 42. Heterogeneity Acts referring to  research objects Haustein,  S.,  Bowman,  T.  D.,  &  Costas,  R.  (2016).  Interpreting “altmetrics”:  Viewing acts on  social  media  through the  lens of  citation  and  social  theories.  Dans   C.  R.  Sugimoto (dir.),  Theories of  Informetrics and  Scholarly Communication  (p.  372–405).  Berlin:  De  Gruyter Mouton.  doi:  10.1515/9783110308464-­‐022 RESEARCH  OBJECT
  43. 43. Time  and  Timing Opportunities • Detailed life  cycle  of  scientific output Ø Fine-­‐grained indicators and  adequate benchmarks Challenges • Versions  of  research output • Publication  dates
  44. 44. Time  and  Timing Journal  article • Submitted manuscript • Revised manuscript • Accepted manuscript • Version  of  Record • Online  publication • Journal  issue • Online  date • Issue  month Ø Adjusting indicators 3  March  2014 15  July  2014 21  January2015 February2015
  45. 45. Time  and  Timing
  46. 46. Time  and  Timing Tweets  before publication?
  47. 47. Time  and  Timing Weekday effects on  Twitter weekday of online publication: based on: 8,765 Springer papers with online publication date 19,010 tweets received within one year of online publication date
  48. 48. Audiences  and  User  Groups Opportunities • Differentiating between types  of  users • Measuring societal impact Challenges • Identifying users and  user  groups • Determining engagement
  49. 49. Audiences  and  User  Groups Alperin,  J.  P.  (2015).  Moving  beyond  counts:  A  method  for  surveying  Twitter  users.  In  altmetrics15:  5  years  in,  what  do  we  know?  Amsterdam,   The  Netherlands.  Retrieved  from:  http://altmetrics.org/wp-­‐content/uploads/2015/09/altmetrics15_paper_3.pdf Tsou,  A.,  Bowman,  T.  D.,  Ghazinejad,  A.,  &  Sugimoto,  C.  R.  (2015).  Who tweets  about  science?  In  Proceedings of  the  2015  International  Society  for   Scientometrics and  Informetrics (pp.  95–100).  Istanbul,  Turkey. Identifying Twitter  users • Altmetric.com  classification • Among  a  random  sample  of  2,000  accounts  tweeting   papers,  34%  of  individuals  identified  as  having  PhD • Of  286  users  linking  to  SciELO articles,  24%  employed  at   university,  23%  students,  36%  not  university  affiliated *based on  Altmetric.com  data  06/2015 (Tsou,  Bowman,  Ghazinejad,  &  Sugimoto,  2010) (Alperin,  2015)
  50. 50. 1 2 3 Audiences  and  User  Groups Haustein,  S.,  &  Costas,  R.,  (2015).  Identifying Twitter  audiences:  who is tweeting about  scientific papers?  Communication  présentée  au  SIG/MET  Workshop,   ASIS&T  2015  Annual Meeting,  7  novembre  2015,  Saint-­‐Louis,  MO  (USA). topics  and collectives academic personal Node size number of  accounts associated with term Node color cluster  affiliation Terms  in  Twitter  bio
  51. 51. Audiences  and  User  Groups Engagement  with scientific papers on  Twitter Haustein,  S.,  Bowman,  T.  D.,  Holmberg,  K.,  Tsou,  A.,  Sugimoto,  C.  R.,  &  Larivière,  V.  (2016).  Tweets  as  impact  indicators:  Examining  the  implications  of   automated  bot  accounts  on  Twitter.  Journal  of  the  Association  for  Information  Science  and  Technology,  67(1),  232–238.  doi:  10.1002/asi.23456
  52. 52. Audiences  and  User  Groups Twitter  arXiv bots
  53. 53. Audiences  and  User  Groups Journal  accounts
  54. 54. Conclusions • Scholarly communication  and  the  reward system   of  science  are  changing • Potential to  become more  transparent  and   diverse • Open  Science • Scholarly metrics • Fundamental difference between posting on   social  media  and  academic publishing • More  metrics =  more  complexity
  55. 55. Outlook More  scientometric studies. More  sociological research. Avoid adverse  effects.

×