Gauging Research Output and Influence  Using Traditional Citation Tools and Unique Analytical Solutions.  ALA 2010 Midwint...
Today’s Discussion <ul><li>Web of Science Data </li></ul><ul><ul><li>Foundational strengths and what this provides </li></...
Origins: The Web of Science and Citation Indexing <ul><li>Dr. Eugene Garfield, the concept: A citation index approach to s...
Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development .  Every J...
Web of Science Conference Proceedings <ul><li>Proceedings literature is an essential component of international scholarly ...
Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development .  Every J...
Web of Science –  More than “Science”   Social Sciences Citation Index – 2,700+ Titles Diversity  – Web of Science  Suppor...
Arts & Humanities Citation Index – 1,500 Titles Web of Science –  More than “Science” Diversity  – Web of Science  Support...
<ul><li>One of the truly powerful features of Web of Science is the  Cited Reference Search </li></ul><ul><li>Retrieves in...
<ul><li>A True Citation Index –  Unmatched in Depth and Diversity </li></ul><ul><li>Displaying all cited works, whatever t...
Web of Science Forward Citation Map Psychology, Experimental Computer Science,  Artificial Intelligence Business Religion ...
Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development .  Every J...
Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development .  Every J...
Web of Science – Consistent Indexing
Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development .  Every J...
Faculty and Institutional Publication Output and Evaluation <ul><li>Analyze Results </li></ul><ul><ul><li>View trends and ...
Wright State University Top 10 Cited Papers 1900-2008
Web of Science Citation Report: Ronald Glaser
Citation Report: Influenza Topic Search
Web of Science –  powerful “Analyze Results” features Analyze Results features are present in all ISI Web of Knowledge dat...
Web of Science – Citation Map
Web of Science – Backward Citation Map 2nd generation citations reveal citation pathways
Web of Science ® <ul><ul><li>Web of Science is an extremely valuable “intelligence” resource – a tool for characterizing a...
Web of Science ® <ul><ul><li>Appalachian State University </li></ul></ul><ul><ul><li>A comparison of output and citation o...
Web of Science ® <ul><ul><li>Using Web of Science to compare five faculty researching and publishing within Materials Scie...
Web of Science ® <ul><ul><li>Comparison of “averages” to gauge performance is valuable, but only within the appropriate co...
Web of Science ® <ul><ul><li>Comparison of “averages” to gauge performance is valuable, but only within the appropriate co...
Web of Science ® <ul><ul><li>For serious assessment activities it is essential to take a more sophisticated approach to th...
Evaluation of Research <ul><li>Self-Assessment – understanding where you’ve been, where you are today, and where you want ...
<ul><ul><li>Of Key Importance -- the Establishment of  Baselines …  </li></ul></ul><ul><ul><ul><li>Essential in order to p...
<ul><ul><li>From established Baselines, we can normalize the influence of published works and present  real metrics  to re...
For Your Institution –  Papers Average citations specific to Category, year of article, and document type.
For Your Institution –  Papers Percentile position specific to field and year.  -- The closer to zero, the more highly cit...
Relative Performance At-a-Glance With a  baseline of 1.0  for each, you immediately have a gauge of the level of influence...
For Your Institution –  Authors
For Your Institution –  Disciplines The published intellectual output of a university presented by discipline.  Here too, ...
For Your Institution –  Disciplines The published intellectual output of a university presented by discipline.  Here too, ...
For Your Institution –  Peer Institution Comparisons
For Your Institution –  Peer Institution Comparisons
For Your Institution –  Peer Institution Comparisons
Country  Level Analytics
Citation Analysis is Objective in nature -- a key piece of the overall Research Evaluation puzzle. Citation Metrics Awards...
<ul><ul><li>Eugene Garfield </li></ul></ul><ul><ul><li>Derek J. de Solla Price </li></ul></ul>Thomson Reuters Research Ana...
Gauging Research Output and Influence  Using Traditional Citation Tools  and Unique Analytical Solutions.  ALA 2010 Midwin...
Upcoming SlideShare
Loading in …5
×

Gauging Research Output and Influence

804 views

Published on

This presentation was given at at the Thomson Reuters Breakfast Workshop, at ALA Midwinter 2010.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
804
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • By “characterizing” we mean going beyond simply “how much” to examining output by discipline, or journal, or conference – as well as identifying trends in growth or decline of output.
  • Counting, measuring, comparing quantities, analyzing measurements: quantitative analysis is perhaps the main tool of science. Scientific research itself, and recording and communicating research results through publications, has become enormous and complex. It is so complex and specialized that personal knowledge and experience are no longer sufficient tools for understanding trends or for making decisions. Yet the need to be selective, to highlight significant or promising areas of research, and to manage better investments in science is only increasing. Those in universities, government offices and labs, and boardrooms must decide what research should be supported and what should not, or which research projects and researchers should receive more support than others. Until relatively recently, peer review was the main route by which science policymakers and research funders made policy decisions about science. A library faced with collection decisions, a foundation making funding choices, or a government office weighing national research needs must rely on expert analysis of scientific research performance. Increasingly, universities everywhere must demonstrate their special capabilities to a variety of constituencies. Universities that seek research funding from government agencies and foundations must provide evidence of their accomplishments and capacities. And in many countries, universities, whether public or private, must account for their performance as part of national or professional accountability protocols. Indeed, every university must have a clear, evidence-based understanding of the institution’s performance towards its goals and mission. This understanding is achieved and maintained through ongoing evaluation of all of the institution’s functions. Because research is a central function, the university must evaluate its performance. Data on research performance helps to inform strategic decisions about what areas of research to support or build. It also helps the university leaders understand the institution’s position relative to global and domestic standards of research production: How much research is conducted? What is its impact? How many of the faculty members’ articles are published in first-class journals? Is that number of publications increasing or decreasing? With solid, objective information about production and impact, the university has a strong basis for setting goals, charting progress, making budgetary and hiring decisions, investing in facilities, and working with external agencies.
  • Bibliometrics should always be taken in context
  • The first major award for the scientometric field, the Derek John De Solla Price Award of the journal Scientometrics , was first awarded in 1984 to Eugene Garfield. Little SCiBig Sci 1963 -- This book contains four introductory lectures on quantitative methods in the analysis of historical and modern science: 1. Prologue to the science of science (exponential and logistical growth of scientific publication and manpower); 2. Galton revisited (productivity distributions, laws of l,otka and Zipf); 3. invisible colleges and the affluent scientific commuter (multiple discovery and authorship, Bradford’s law, half-lives of papers); 4. political strategy for big scientists (saturation, language distribution, emergence of Japanese science, big science phenomena).
  • Gauging Research Output and Influence

    1. 1. Gauging Research Output and Influence Using Traditional Citation Tools and Unique Analytical Solutions. ALA 2010 Midwinter Meeting Jan 15-19, Boston
    2. 2. Today’s Discussion <ul><li>Web of Science Data </li></ul><ul><ul><li>Foundational strengths and what this provides </li></ul></ul><ul><ul><li>Unleashed -- unique presentation and utilization in multiple resources </li></ul></ul>
    3. 3. Origins: The Web of Science and Citation Indexing <ul><li>Dr. Eugene Garfield, the concept: A citation index approach to scientific and scholarly discovery – unique methods for identifying associations and connections between published works on a multidisciplinary scale. </li></ul>The Web of Science Foundation
    4. 4. Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development . Every Journal is individually selected for influence within its respective discipline . </li></ul></ul><ul><ul><li>Unique Data - Thomson Reuters captures available author cited references for all published papers back to 1900 </li></ul></ul><ul><li>Diversity -- Truly Multidisciplinary Journal Coverage </li></ul><ul><ul><li>All fields of study are included, 250+ categories – 12,000+ unique titles </li></ul></ul><ul><ul><ul><ul><li>Science Citation Index Expanded – 8,160 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Social Sciences Citation Index – 2,780 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Arts & Humanities Citation Index – 1,500 titles </li></ul></ul></ul></ul><ul><li>Depth -- Great Depth of Coverage is Available </li></ul><ul><ul><ul><ul><li>To 1900 in the Sciences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1900 in the Social Science </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1975 in the Arts & Humanities </li></ul></ul></ul></ul>The Web of Science Foundation
    5. 5. Web of Science Conference Proceedings <ul><li>Proceedings literature is an essential component of international scholarly communication. </li></ul><ul><li>Web of Science Conference Proceedings Citation Indexes : </li></ul><ul><ul><li>Conference Proceedings Citation Index- Science ( CPCI-S ) </li></ul></ul><ul><ul><li>Conference Proceedings Citation Index- Social Science & Humanities ( CPCI-SSH ) 1990-present </li></ul></ul><ul><ul><li>CPCI files are fully integrated into Web of Science, behaving exactly as the SCIE, SSCI and A&HCI files. </li></ul></ul><ul><ul><li>Weekly updates </li></ul></ul><ul><ul><li>Coverage of approximately 12,000 conferences annually </li></ul></ul><ul><ul><li>Proceedings in book or book series form, monographic sources, account for approximately 70% of content. </li></ul></ul>
    6. 6. Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development . Every Journal is individually selected for influence within its respective discipline . </li></ul></ul><ul><ul><li>Unique Data - Thomson Reuters captures available author cited references for all published papers back to 1900 </li></ul></ul><ul><li>Diversity -- Truly Multidisciplinary Journal Coverage </li></ul><ul><ul><li>All fields of study are included, 250+ categories – 11,300 unique titles </li></ul></ul><ul><ul><ul><ul><li>Science Citation Index Expanded – 8,090 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Social Sciences Citation Index – 2,716 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Arts & Humanities Citation Index – 1,500 titles </li></ul></ul></ul></ul><ul><li>Depth -- Great Depth of Coverage is Available </li></ul><ul><ul><ul><ul><li>To 1900 in the Sciences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1900 in the Social Science </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1975 in the Arts & Humanities </li></ul></ul></ul></ul>The Web of Science Foundation
    7. 7. Web of Science – More than “Science” Social Sciences Citation Index – 2,700+ Titles Diversity – Web of Science Supports All Academic Programs Anthropology Gerontology Psychology Area Studies Health Policy & Services Public Administration Business History Public, Environmental & Occupational Health Communication Hospitality, Leisure, Sport & Tourism Rehabilitation Criminology & Penology Industrial Relations & Labor Social Issues Demography Information Science & Library Science Social Sciences Economics International Relations Social Work Education Law Sociology Environmental Studies Linguistics Substance Abuse Ergonomics Management Transportation Ethics Nursing Urban Studies Ethnic Studies Planning & Development Women’s Studies Family Studies Political Science Geography Psychiatry
    8. 8. Arts & Humanities Citation Index – 1,500 Titles Web of Science – More than “Science” Diversity – Web of Science Supports All Academic Programs Archaeology Folklore Medieval & Renaissance Studies Architecture History Music Art Humanities Philosophy Asian Studies Language & linguistics Poetry Classics Literary Reviews Religion Dance Literary Theory Theater Film, Radio, Television Literature
    9. 9. <ul><li>One of the truly powerful features of Web of Science is the Cited Reference Search </li></ul><ul><li>Retrieves information often missed through use of Keywords alone -- essential for truly comprehensive searching. AND… </li></ul><ul><li>The ability to enter a wide range of items in the Cited Work field – not only a Journal but a Film , a notable Book or Report , Novel , or even Work of Art , and retrieve articles that have cited these works – is a way to gather relevant information that may not have been found in any other way. </li></ul>
    10. 10. <ul><li>A True Citation Index – Unmatched in Depth and Diversity </li></ul><ul><li>Displaying all cited works, whatever the form – Journal Articles, Books, Conference Papers, etc. </li></ul><ul><li>Browse – Discover </li></ul><ul><li>Select Precise Works on which to search, or to eliminate from a search. </li></ul>
    11. 11. Web of Science Forward Citation Map Psychology, Experimental Computer Science, Artificial Intelligence Business Religion Business
    12. 12. Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development . Every Journal is individually selected for influence within its respective discipline . </li></ul></ul><ul><ul><li>Unique Data - Thomson Reuters captures available author cited references for all published papers back to 1900 </li></ul></ul><ul><li>Diversity -- Truly Multidisciplinary Journal Coverage </li></ul><ul><ul><li>All fields of study are included, 250+ categories – 11,300 unique titles </li></ul></ul><ul><ul><ul><ul><li>Science Citation Index Expanded – 8,090 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Social Sciences Citation Index – 2,716 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Arts & Humanities Citation Index – 1,500 titles </li></ul></ul></ul></ul><ul><li>Depth -- Great Depth of Coverage is Available </li></ul><ul><ul><ul><ul><li>To 1900 in the Sciences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1900 in the Social Science </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1975 in the Arts & Humanities </li></ul></ul></ul></ul>The Web of Science Foundation
    13. 13. Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development . Every Journal is individually selected for influence within its respective discipline . </li></ul></ul><ul><ul><li>Unique Data - Thomson Reuters captures available author cited references for all published papers back to 1900 </li></ul></ul><ul><ul><li>Consistent indexing </li></ul></ul><ul><ul><ul><li>All author names and all address are indexed for every Web of Science record </li></ul></ul></ul>The Web of Science Foundation
    14. 14. Web of Science – Consistent Indexing
    15. 15. Web of Science Content <ul><li>Quality </li></ul><ul><ul><li>Carefully managed , ongoing Collection Development . Every Journal is individually selected for influence within its respective discipline . </li></ul></ul><ul><ul><li>Unique Data - Thomson Reuters captures available author cited references for all published papers back to 1900 </li></ul></ul><ul><li>Diversity -- Truly Multidisciplinary Journal Coverage </li></ul><ul><ul><li>All fields of study are included, 250+ categories – 11,300 unique titles </li></ul></ul><ul><ul><ul><ul><li>Science Citation Index Expanded – 8,090 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Social Sciences Citation Index – 2,716 titles </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Arts & Humanities Citation Index – 1,500 titles </li></ul></ul></ul></ul><ul><li>Depth -- Great Depth of Coverage is Available </li></ul><ul><ul><ul><ul><li>To 1900 in the Sciences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1900 in the Social Science </li></ul></ul></ul></ul><ul><ul><ul><ul><li>To 1975 in the Arts & Humanities </li></ul></ul></ul></ul>The Web of Science Foundation
    16. 16. Faculty and Institutional Publication Output and Evaluation <ul><li>Analyze Results </li></ul><ul><ul><li>View trends and access subsets of data </li></ul></ul><ul><li>Citation Report </li></ul><ul><ul><li>Quickly produce an overview of a topic or one’s published works, as well as an indication of the collective influence of those works. </li></ul></ul><ul><li>Citation Map </li></ul><ul><ul><li>Visualize two generations of forward and backward citations </li></ul></ul>
    17. 17. Wright State University Top 10 Cited Papers 1900-2008
    18. 18. Web of Science Citation Report: Ronald Glaser
    19. 19. Citation Report: Influenza Topic Search
    20. 20. Web of Science – powerful “Analyze Results” features Analyze Results features are present in all ISI Web of Knowledge databases – as well as All Databases search results.
    21. 21. Web of Science – Citation Map
    22. 22. Web of Science – Backward Citation Map 2nd generation citations reveal citation pathways
    23. 23. Web of Science ® <ul><ul><li>Web of Science is an extremely valuable “intelligence” resource – a tool for characterizing and gauging published output and influence for institutions, individuals, countries, etc. – indispensible for basic comparisons . </li></ul></ul>Journals Papers Authors Institutions Disciplines Web of Science
    24. 24. Web of Science ® <ul><ul><li>Appalachian State University </li></ul></ul><ul><ul><li>A comparison of output and citation of Economics papers. </li></ul></ul><ul><ul><li>Western Illinois University </li></ul></ul><ul><ul><li>Web of Science is an extremely valuable “intelligence” resource – a tool for characterizing and gauging published output and influence for institutions, individuals, countries, etc. – indispensible for basic comparisons . </li></ul></ul>
    25. 25. Web of Science ® <ul><ul><li>Using Web of Science to compare five faculty researching and publishing within Materials Science, we know which produces the more influential work on average. </li></ul></ul><ul><ul><li>But, how “good” is this? </li></ul></ul><ul><ul><li>Avg. Cites Per Paper </li></ul></ul><ul><ul><li>3.2 </li></ul></ul><ul><ul><li>5.7 </li></ul></ul><ul><ul><li>7.8 </li></ul></ul><ul><ul><li>10.5 </li></ul></ul><ul><ul><li>13.3 </li></ul></ul><ul><ul><li>Web of Science is an extremely valuable “intelligence” resource – a tool for characterizing and gauging published output and influence for institutions, individuals, countries, etc. – indispensible for basic comparisons . </li></ul></ul>
    26. 26. Web of Science ® <ul><ul><li>Comparison of “averages” to gauge performance is valuable, but only within the appropriate context, within some definitive or authoritative group. </li></ul></ul>
    27. 27. Web of Science ® <ul><ul><li>Comparison of “averages” to gauge performance is valuable, but only within the appropriate context, within some definitive or authoritative group. </li></ul></ul>
    28. 28. Web of Science ® <ul><ul><li>For serious assessment activities it is essential to take a more sophisticated approach to the data. </li></ul></ul>Bibliometrics Scientometrics Infometrics Citation Metrics
    29. 29. Evaluation of Research <ul><li>Self-Assessment – understanding where you’ve been, where you are today, and where you want to be in the future. </li></ul><ul><li>Use your precious resources more effectively </li></ul><ul><ul><li>Strategic planning </li></ul></ul><ul><ul><li>Assist in measuring ROI </li></ul></ul><ul><ul><li>Researcher-level assessments- tenure review, etc. </li></ul></ul><ul><ul><li>Report progress to government bodies, boards of directors/ trustees, funding agencies, etc. </li></ul></ul><ul><ul><li>Recruiting </li></ul></ul>
    30. 30. <ul><ul><li>Of Key Importance -- the Establishment of Baselines … </li></ul></ul><ul><ul><ul><li>Essential in order to provide comparative capabilities </li></ul></ul></ul>
    31. 31. <ul><ul><li>From established Baselines, we can normalize the influence of published works and present real metrics to reflect research performance. </li></ul></ul><ul><ul><li>Customer defined data sets providing article-level source records and related detail. This is most typically comprised of papers associated with a particular Institution, but can also be “topic” defined. </li></ul></ul>Summary information and metrics on published output and performance at the the institution and country level.
    32. 32. For Your Institution – Papers Average citations specific to Category, year of article, and document type.
    33. 33. For Your Institution – Papers Percentile position specific to field and year. -- The closer to zero, the more highly cited.
    34. 34. Relative Performance At-a-Glance With a baseline of 1.0 for each, you immediately have a gauge of the level of influence of an author’s collective works compared other published work in the field. For Your Institution – Authors
    35. 35. For Your Institution – Authors
    36. 36. For Your Institution – Disciplines The published intellectual output of a university presented by discipline. Here too, gauge relative influence at a glance.
    37. 37. For Your Institution – Disciplines The published intellectual output of a university presented by discipline. Here too, gauge relative influence at a a glance.
    38. 38. For Your Institution – Peer Institution Comparisons
    39. 39. For Your Institution – Peer Institution Comparisons
    40. 40. For Your Institution – Peer Institution Comparisons
    41. 41. Country Level Analytics
    42. 42. Citation Analysis is Objective in nature -- a key piece of the overall Research Evaluation puzzle. Citation Metrics Awards/Honors Funding data Peer review
    43. 43. <ul><ul><li>Eugene Garfield </li></ul></ul><ul><ul><li>Derek J. de Solla Price </li></ul></ul>Thomson Reuters Research Analytics Solutions American Documentation , Vol. 14, No.3 , p. 195-201 1963 Price, DJD Archives Internationale d'histoire Des Sciences No 14, p 85, 1951
    44. 44. Gauging Research Output and Influence Using Traditional Citation Tools and Unique Analytical Solutions. ALA 2010 Midwinter Meeting Jan 15-19, Boston Jeff Dougherty Paul Torpey

    ×