5 years ago…
Nearly 50% of respondents saw Web 2.0 playing a key role
in “Providing Quality Indicators” in 5 years (2013)
...
1 year ago… October 2012
Background & approach:
54,442 individuals were randomly selected from
Scopus
3,090 respondents co...
3
Most widely known by researchers
Impact Factor 82%
H-Index 43%
Journal Usage Factor 10%
????????? 1%
Impact Factor is pu...
4
Impact Factor 82%
H-Index 43%
Journal Usage Factor 10%
Altmetrics 1%
Impact Factor is published by Thomson Reuters, Altm...
Awarenes
s
5
Q2 Which of these do you think are most useful at measuring research quality? (Select up to 3)
64%
29%
29%
28...
Generally metrics with the highest awareness are
also considered to be the most useful
6
Impact factor
SNIP
SJR
Eigenfacto...
7
Assessing the usefulness of potential quality
metrics: by age
 Significant difference between
subset and total (subset ...
8
Assessing the usefulness of potential
quality metrics: by region (1 of 2)
 Significant difference between
subset and to...
9
Assessing the usefulness of potential quality
metrics: by region (2 of 2)
 Significant difference between
subset and to...
Upcoming SlideShare
Loading in …5
×

Michael Habib – Lightning talk at NISO Altmetrics Initiative

537 views
444 views

Published on

Michael Habib – Expectations by researchers Lightning talk at NISO Altmetrics Initiative meeting in San Francisco, CA October 9, 2013

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
537
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Michael Habib – Lightning talk at NISO Altmetrics Initiative

  1. 1. 5 years ago… Nearly 50% of respondents saw Web 2.0 playing a key role in “Providing Quality Indicators” in 5 years (2013) Source: (2collab) Social Media survey - May 2008 - 1,824 respondents
  2. 2. 1 year ago… October 2012 Background & approach: 54,442 individuals were randomly selected from Scopus 3,090 respondents completed Representative response by country and discipline. Error margin 1.5%, at 90% confidence levels 2 Adrian Mulligan, Gemma Deakin and Rebekah Dutton Elsevier Research & Academic Relations
  3. 3. 3 Most widely known by researchers Impact Factor 82% H-Index 43% Journal Usage Factor 10% ????????? 1% Impact Factor is published by Thomson Reuters, Altmetrics were least well known
  4. 4. 4 Impact Factor 82% H-Index 43% Journal Usage Factor 10% Altmetrics 1% Impact Factor is published by Thomson Reuters, Altmetrics were least well known Most widely known by researchers
  5. 5. Awarenes s 5 Q2 Which of these do you think are most useful at measuring research quality? (Select up to 3) 64% 29% 29% 28% 58% 37% 34% 42% 0% 20% 40% 60% 80% 100% Impact factor (n=2,530) SNIP (n=51) SJR (n=126) Eigenfactor (n=285) h-index (n=1,335) Journal Usage Factor (n=309) F1000 (n=155) Altmetrics (n=41) * Only people who said they were aware of a particular metric in Q1 were given the opportunity to select that metric in Q2, *See appendix for background and approach. Research by Elsevier Research & Academic Relations. Impact Factor is published by Thomson Reuters, TOTAL (n=3,090) 82% 2% 4% 9% 43% 10% 5% 1% Researcher perception of most useful % useful
  6. 6. Generally metrics with the highest awareness are also considered to be the most useful 6 Impact factor SNIP SJR Eigenfactor h-index Journal Usage Factor F1000 Altmetrics R² = 0.697 -10% 0% 10% 20% 30% 40% 50% 60% 70% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Percentageofawarerespondentsthatchosethe metricasoneofthemostuseful Percentage of respondents that are aware of the metric The trendline shows the linear trend for the relationship between awareness and usage of metrics Metrics above the line have lower levels of awareness, but are more likely to be rated as useful than the typical awareness-usage relationship Metrics below the line have higher levels of awareness, but are less likely to be rated as useful than the typical awareness-usage relationship *See appendix for background and approach. Research by Elsevier Research & Academic Relations. Impact Factor is published by Thomson Reuters,
  7. 7. 7 Assessing the usefulness of potential quality metrics: by age  Significant difference between subset and total (subset higher)  Significant difference between subset and total (subset lower) Under 36 (n=540) 36-45 (n=920) 46-55 (n=819) 56-65 (n=507) Over 65 (n=242) TOTAL (n=3,090) Article views/downloads (for articles)  43% Citations from materials that are in repositories   43% Share in social network mentions (for articles)    16% Number of readers (for articles)  40% Number of followers (for researchers)   31% Votes or ratings (for articles)   24% A metric that measures the contribution an individual makes to peer review (for researchers)  28% A score based on reviewer assessment (for articles)  28% Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a researcher or a research article?(By age) % Think it would be extremely/very useful 43% 49% 21% 42% 38% 35% 34% 33% 44% 45% 18% 41% 33% 24% 29% 29% 45% 41% 15% 39% 28% 22% 27% 27% 44% 41% 12% 41% 30% 22% 26% 27% 36% 37% 13% 35% 30% 19% 24% 27%
  8. 8. 8 Assessing the usefulness of potential quality metrics: by region (1 of 2)  Significant difference between subset and total (subset higher)  Significant difference between subset and total (subset lower) Africa (n=72) APAC (n=803) Eastern Europe (n=183) Latin America (n=182) TOTAL (n=3,090) Article views/downloads (for articles)     43% Citations from materials that are in repositories    43% Share in social network mentions (for articles)    16% Number of readers (for articles)  40% Number of followers (for researchers)   31% Votes or ratings (for articles)    24% A metric that measures the contribution an individual makes to peer review (for researchers)   28% A score based on reviewer assessment (for articles)    28% Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a researcher or a research article? (By region, slide 1 of 2) % Think it would be extremely/very useful 56% 51% 26% 49% 36% 33% 40% 44% 50% 55% 27% 46% 46% 29% 35% 36% 50% 49% 19% 45% 41% 30% 28% 26% 50% 49% 21% 45% 34% 24% 32% 35%
  9. 9. 9 Assessing the usefulness of potential quality metrics: by region (2 of 2)  Significant difference between subset and total (subset higher)  Significant difference between subset and total (subset lower) Middle East (n=47) North America (n=770) Western Europe (n=1,033) TOTAL (n=3,090) Article views/downloads (for articles)  43% Citations from materials that are in repositories  43% Share in social network mentions (for articles)   16% Number of readers (for articles)   40% Number of followers (for researchers)   31% Votes or ratings (for articles)   24% A metric that measures the contribution an individual makes to peer review (for researchers)  28% A score based on reviewer assessment (for articles)  28% Q3 Thinking about possible new measures of research productivity, how useful do you think the below would be in assessing the quality of a researcher or a research article? (By region, slide 2 of 2) % Think it would be extremely/very useful 40% 40% 19% 43% 32% 28% 32% 34% 41% 42% 10% 36% 23% 19% 26% 26% 36% 32% 11% 36% 23% 22% 23% 22%

×