Author Level Bibliometrics

1,655 views
1,468 views

Published on

Presentation by Henk Moed, 14th ISSI Conference, Vienna, 15-18 July 2013

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,655
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
35
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Author Level Bibliometrics

  1. 1. Author  Level  Bibliometrics   Henk  F.  Moed       A  talk  at  a  Special  Session  on  Author  Level   Bibliometrics  at  ISSI2013   Vienna,  17  July  2013    
  2. 2.   The  MulG-­‐Dimensional  Research   Assessment  Matrix   Expert Group on the Assessment of University-Based Research (AUBR, 2010)
  3. 3. Multi‐dimensional Research Assessment Matrix (Part) Unit  of   assessment   Purpose   Output   dimensions   Bibliometric   indicators   Other   indicators   Individual   Allocate   resources   Research   producGvity   PublicaGons   Peer  review   Research   group   Improve     performance   Quality,   scholarly   impact   Journal   citaGon   impact   Patents,   licences,   spin  offs   Department   Increase   mulG-­‐discipl.   research   InnovaGon   and  social   benefit   Actual   citaGon   impact   InvitaGons   for   conferences   InsGtuGon   Increase   regional   engagement   Sustainabi-­‐ lity  &  Scale   Internat.  co-­‐ authorship   External   research   income   Research   field     PromoGon,   hiring   Research   infrastruct.    citaGon   ‘presGge’     PhD  com-­‐ pleGon  rates  
  4. 4. Multi‐dimensional Research Assessment Matrix (Part) Unit  of   assessment   Purpose   Output   dimensions   Bibliometric   indicators   Other   indicators   Individual   Allocate   resources   Research   producGvity   PublicaGons   Peer  review   Research   group   Improve     performance   Quality,   scholarly   impact   Journal   citaGon   impact   Patents,   licences,   spin  offs   Department   Increase   mulG-­‐discipl.   research   InnovaGon   and  social   benefit   Actual   citaGon   impact   InvitaGons   for   conferences   InsGtuGon   Increase   regional   engagement   Sustainabi-­‐ lity  &  Scale   Internat.  co-­‐ authorship   External   research   income   Research   field     PromoGon,   hiring   Research   infrastruct.    citaGon   ‘presGge’     PhD  com-­‐ pleGon  rates   Read     column-­‐   wise  
  5. 5. Multi‐dimensional Research Assessment Matrix (Part) Unit  of   assessment   Purpose   Output   dimensions   Bibliometric   indicators   Other   indicators   Individual   Allocate   resources   Research   producGvity   PublicaGons   Peer  review   Research   group   Improve     performance   Quality,   scholarly   impact   Journal   citaGon   impact   Patents,   licences,   spin  offs   Department   Increase   mulG-­‐discipl.   research   InnovaGon   and  social   benefit   Actual   citaGon   impact   InvitaGons   for   conferences   InsGtuGon   Increase   regional   engagement   Sustainabi-­‐ lity  &  Scale   Internat.  co-­‐ authorship   External   research   income   Research   field     PromoGon,   hiring   Research   infrastruct.    citaGon   ‘presGge’     PhD  com-­‐ pleGon  rates  
  6. 6. Base  noGon   The  choice  of  indicators  depends  upon:   •  What  unit  is  to  be  assessed   •  Why  is  the  assessment  done?   •  Which  aspect  is  being  assessed?       Indicators  that  are  appropriate  in  one  type  of   assessment  may  be  inappropriate  in  another  
  7. 7. Valuable  noGons   •  Data  accuracy  is  crucial     •  Use  data  verified  by  authors  themselves     •  Combine  metrics  and  expert  knowledge   •  Impact  factors  no  subsGtutes  of  actual  impact   •  Use  mulGple  indicators   •  Take  into  account  phase  of  career   •  Take  into  account  unintended  effects   •  Present  “interpretaGon  manual”,  pros  &  cons   •  Focus  on  top  vs.  bo_om  of  quality  distribuGon   •  RetrospecGve  vs.  prospecGve  use  of  metrics  
  8. 8. A  base  problem   •  How  to  assess  the  performance  of  an   individual  by  analyzing  products  that  are  the   result  of  teamwork?     •  Most  research  arGcles  are  the  result  of   collaboraGon  within  or  between  teams  
  9. 9. More  and  more  individual   researchers  have  to  be  objecGvely   evaluated  for  promoGon  and  hiring   Use  easily  available  metrics       Count  total  publicaGons  or  #   arGcles  in  journals  with  the  highest   impact  factors  in  the  field         Meta  level  :   Policy  issue   Policy  measure     Bibliometric   operaGonali-­‐   zaGon   c   c   c   c   CASE  1    
  10. 10. Research  community    is  not   sufficiently  oriented  toward   internaGonal    networks   SGmulate  publicaGon  in  good   internaGonal  journals   Count  #  arGcles  in  first  impact   quarGle  of  journals  in  subject  field     Meta  level  :   Policy  issue   Policy  measure     Bibliometric   operaGonali-­‐   zaGon   c   c   c   c   CASE  2    
  11. 11. Professors  are  legally  bound  to  do   research  but  many  of  them  don’t     Allow  only  research  acGve   professors  to  decide  on   recruitment  of  new  research  staff   Select  only  professors  with  >=  3   publicaGons  in  7  years  in   recruitment  commi_ees   Meta  level  :   Policy  issue   Policy  measure     Bibliometric   operaGonali-­‐   zaGon   c   c   c   c   CASE  3    
  12. 12. Wider  issues   •  What  is  an  acceptable  “error  rate”  in  the   assesment  process?   •  Wrong  in  individual  cases    benificiary  for   the  system  as  a  whole   •  What  is  a  fair  assessment  process?  

×