0
Improving Peer  Review   CSR Initiatives toni scarpa  [email_address] 301-435-1109 PRAC  Nov. 3, 2008 National Institutes ...
<ul><li>New Data and the Drivers for Change </li></ul><ul><li>CSR’s Efforts to Enhance Peer Review </li></ul><ul><li>The N...
New Data and the Drivers for Change
CSR Peer Review: 2008 <ul><li>77,000 applications received </li></ul><ul><li>56,000 applications reviewed  </li></ul><ul><...
1st Driver:  The NIH Budget  $13.7 $15.6 $17.8 $20.5 $23.3 $27.1 $28.0 $28.6 $28.6 $29.1 $29.5 $29.5 $0 $5 $10 $15 $20 $25...
2 nd  Driver:  Number of Applications Submitted Historical Growth
SBIR  Applications 7000 6000 5000 4000 3000 2000 1000 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 INST CSR
R01 Applications Reviewed at CSR vs. Other ICs
R21 Applications Reviewed at CSR vs. Other ICs
3 rd  Driver:  Reviewer’s Load Applications Per Reviewer October Council Rounds
4th Driver:  CSR Budget $ Millions
4th Driver:  CSR Budget $ Millions
Annual Savings in Reviewers’ Expenses Budget <ul><li>Non-refundable tickets with one possible change </li></ul><ul><ul><li...
Next Savings? <ul><li>Replace shipping of CDs with online downloading </li></ul><ul><ul><li>Cost of 18,000 CDs  $750,000 <...
CSR’s Efforts to Enhance Peer Review
<ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cy...
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul>
1.  CSR Organization of Peer Review  Divisions Scientific Review Officers Integrated Review Groups
1. CSR Reorganization Rene Etcheberrigaray  Don Schneider Translational and  Clinical Sciences Cardiovascular and  Respira...
Update on CSR Realignments Don Schneider Update on CSR Realignments Don Schneider 1.  CSR Organization of Peer Review
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
2. Recruiting CSR Staff <ul><ul><li>3 Division Directors </li></ul></ul><ul><ul><li>6 Integrated Review Chiefs </li></ul><...
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
3. Shortening the Review Cycle   The Goal <ul><li>To provide applicants a review and score within 3 months of application ...
RO1 A1 Resubmission  Within 4 Months of Original Application
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
4. Holding Out-of-Town Meetings <ul><li>To facilitate participation from colleagues on the West Coast, one meeting out of ...
5. Near-Term Solutions for Recruiting and Retaining the Best Reviewers <ul><ul><li>Move a meeting a year to the West Coast...
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
6.  Second Cycle of IRG Reviews Scheduled 2008-5 IRGs Scheduled 2009-12 IRGs (Continued) Brain Disorders and Clinical Neur...
6.  Performing a Second Cycle of IRG Reviews <ul><li>All pertinent study section chairs were invited </li></ul><ul><li>Adv...
6.  Performing a Second Cycle of IRG Reviews Second Cycle of IRG Reviews  Don Schneider
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
7. Revising Study Section Outlines <ul><li>Cellular Signaling and Regulatory Systems </li></ul><ul><li>[Roster] </li></ul>...
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
8. Training Study Section Chairs <ul><li>Scope and Schedule   </li></ul><ul><li>120 new chairs every year, more if “perman...
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
9. Managing Conflicts of Interest <ul><li>Report of Conflict of Interest Committee </li></ul><ul><li>Noni Byrnes </li></ul>
CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li>...
10. Advancing Additional Review Platforms and Process <ul><li>Video (VED) </li></ul><ul><li>Internet (AED) </li></ul><ul><...
10. Advancing Additional Review Platforms and Processes <ul><ul><li>Editorial Board Review </li></ul></ul><ul><ul><ul><li>...
10. Advancing Additional Review Platforms and Process New Transformative R01 Betsy Wilder
10. Advancing Additional Review Platforms and Process Results of Asynchronous  Electronic Discussion Survey Andrea Kopstein
NIH Enhancement of Peer Review <ul><li>The Charge from Dr. Zerhouni:   </li></ul><ul><li>“ Fund the best science, by the b...
The NIH Director’s Peer Review Initiatives
The NIH Director’s Peer Review Initiatives Corporate NIH Peer Review Enhancement:  Implementation Raynard Kington
This is CSR September 2008
Recognizing Outstanding Reviewers <ul><li>Going above and beyond the call of duty </li></ul><ul><li>Having a significant i...
The First Marcy Speer Award, 2007   <ul><li>Marcy Speer </li></ul>
2008 Marcy Speer  Outstanding CSR Reviewer Award   Dr. David J. Sahn, M.D. Oregon Health & Science University
The 2 nd  Marcy Speer Award, 2008
Upcoming SlideShare
Loading in...5
×

Enhancing Peer Review: CSR Initiatives

528

Published on

Published in: Health & Medicine, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
528
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Enhancing Peer Review: CSR Initiatives"

  1. 1. Improving Peer Review CSR Initiatives toni scarpa [email_address] 301-435-1109 PRAC Nov. 3, 2008 National Institutes of Health U.S. Department of Health and Human Services
  2. 2. <ul><li>New Data and the Drivers for Change </li></ul><ul><li>CSR’s Efforts to Enhance Peer Review </li></ul><ul><li>The NIH Director’s Peer Review Initiatives </li></ul>Improving Peer Review
  3. 3. New Data and the Drivers for Change
  4. 4. CSR Peer Review: 2008 <ul><li>77,000 applications received </li></ul><ul><li>56,000 applications reviewed </li></ul><ul><li>16,000 reviewers </li></ul><ul><li>240 Scientific Review Officers </li></ul><ul><li>1,800 review meetings </li></ul>
  5. 5. 1st Driver: The NIH Budget $13.7 $15.6 $17.8 $20.5 $23.3 $27.1 $28.0 $28.6 $28.6 $29.1 $29.5 $29.5 $0 $5 $10 $15 $20 $25 $30 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Doubling
  6. 6. 2 nd Driver: Number of Applications Submitted Historical Growth
  7. 7. SBIR Applications 7000 6000 5000 4000 3000 2000 1000 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 INST CSR
  8. 8. R01 Applications Reviewed at CSR vs. Other ICs
  9. 9. R21 Applications Reviewed at CSR vs. Other ICs
  10. 10. 3 rd Driver: Reviewer’s Load Applications Per Reviewer October Council Rounds
  11. 11. 4th Driver: CSR Budget $ Millions
  12. 12. 4th Driver: CSR Budget $ Millions
  13. 13. Annual Savings in Reviewers’ Expenses Budget <ul><li>Non-refundable tickets with one possible change </li></ul><ul><ul><li>$15 million </li></ul></ul><ul><li>3,000 fewer reviewers </li></ul><ul><ul><li>$3 million </li></ul></ul><ul><li>15% reviews using electronic platforms </li></ul><ul><ul><li>$5 million </li></ul></ul><ul><li>One meeting a year on the West Coast </li></ul><ul><ul><li>$1.8 million </li></ul></ul>
  14. 14. Next Savings? <ul><li>Replace shipping of CDs with online downloading </li></ul><ul><ul><li>Cost of 18,000 CDs $750,000 </li></ul></ul><ul><ul><li>Cost of shipping $200,000 </li></ul></ul><ul><ul><li>Handling $50,000 </li></ul></ul><ul><li>5% more reviews using electronic platform </li></ul><ul><ul><li>SREA budget $1,500,000 </li></ul></ul><ul><ul><li>CSR budget -$100,000 </li></ul></ul><ul><li>More meetings in Baltimore </li></ul><ul><ul><li>Savings/100 meetings $500,000 </li></ul></ul><ul><li>Review of fellowship applications </li></ul>
  15. 15. CSR’s Efforts to Enhance Peer Review
  16. 16. <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Holding Out-of-Town Meetings </li></ul><ul><li>Recruiting the Best Reviewers </li></ul><ul><li>Performing a Second Cycle of IRG Reviews </li></ul><ul><li>Revising Study Section Guidelines </li></ul><ul><li>Training Chairs </li></ul><ul><li>Managing Conflicts of Interest </li></ul><ul><li>Advancing Additional Review Platforms and Processes </li></ul>CSR’s Efforts to Enhance Peer Review
  17. 17. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul>
  18. 18. 1. CSR Organization of Peer Review Divisions Scientific Review Officers Integrated Review Groups
  19. 19. 1. CSR Reorganization Rene Etcheberrigaray Don Schneider Translational and Clinical Sciences Cardiovascular and Respiratory Sciences Surgical Sciences, Biomedical Imaging and Bioengineering Musculoskeletal, Oral And Skin Sciences Oncology 2 – Translational Clinical Vascular and Hematology Physiological and Pathological Sciences Endocrinology, Metabolism, Nutrition & Reproductive Sciences Immunology Infectious Diseases & Microbiology Digestive, Kidney & Urological Systems Neuroscience, Development and Aging Brain Disorders & Clinical Neuroscience Molecular, Cellular & Developmental Neuroscience Integrative, Functional & Cognitive Neuroscience Emerging Technologies & Training in Neuroscience Biology of Development & Biobehavioral & Behavioral Processes Risk, Prevention& Health Behaviors Epidemiology & Population Sciences Healthcare Delivery & Methodologies AIDS & Related Research AIDS, Behavioral and Population Sciences Basic and Integrative Biological Sciences Biological Chemistry & Macromolecular Biophysics Bioengineering Sciences & Technologies Genes, Genomes &Genetics Oncology 1 – Basic Translational Cell Biology Interdisciplinary Molecular & Training
  20. 20. Update on CSR Realignments Don Schneider Update on CSR Realignments Don Schneider 1. CSR Organization of Peer Review
  21. 21. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul>
  22. 22. 2. Recruiting CSR Staff <ul><ul><li>3 Division Directors </li></ul></ul><ul><ul><li>6 Integrated Review Chiefs </li></ul></ul><ul><ul><li>20 Scientific Review Officers </li></ul></ul>
  23. 23. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul>
  24. 24. 3. Shortening the Review Cycle The Goal <ul><li>To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past. </li></ul>
  25. 25. RO1 A1 Resubmission Within 4 Months of Original Application
  26. 26. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Holding Out-of-Town-Meetings </li></ul>
  27. 27. 4. Holding Out-of-Town Meetings <ul><li>To facilitate participation from colleagues on the West Coast, one meeting out of three each year will be held in the metro areas of: </li></ul><ul><ul><li>Los Angeles </li></ul></ul><ul><ul><li>San Francisco </li></ul></ul><ul><ul><li>Seattle </li></ul></ul><ul><ul><li>Chicago </li></ul></ul><ul><ul><li>Benefits </li></ul></ul><ul><ul><li>Great satisfaction </li></ul></ul><ul><ul><li>Significant savings </li></ul></ul><ul><ul><li>Ranking of hotels </li></ul></ul>
  28. 28. 5. Near-Term Solutions for Recruiting and Retaining the Best Reviewers <ul><ul><li>Move a meeting a year to the West Coast </li></ul></ul><ul><ul><li>Develop a national registry of volunteer reviewers </li></ul></ul><ul><ul><ul><li>Searchable database with 3,500 reviewers </li></ul></ul></ul><ul><ul><li>Provide tangible rewards for reviewers </li></ul></ul><ul><ul><ul><li>No submission deadlines for chartered members </li></ul></ul></ul><ul><ul><ul><li>of study sections (effective February 2008). </li></ul></ul></ul><ul><ul><ul><li>1,574 chartered members used flexible deadlines </li></ul></ul></ul><ul><ul><ul><li>during the last 6 months </li></ul></ul></ul><ul><ul><li>Provide flexible time for reviewers </li></ul></ul><ul><ul><ul><li>Choice of 3 times/year for 4 years or </li></ul></ul></ul><ul><ul><ul><li>2 times/year for 6 years </li></ul></ul></ul>
  29. 29. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Out-of-Town Meetings </li></ul><ul><li>Recruiting the Best Reviewers </li></ul><ul><li>Performing a Second Cycle of IRG Reviews </li></ul>
  30. 30. 6. Second Cycle of IRG Reviews Scheduled 2008-5 IRGs Scheduled 2009-12 IRGs (Continued) Brain Disorders and Clinical Neuroscience (BDCN) and Molecular, Cellular and Developmental Neuroscience (MDCN) Bioengineering Sciences and Technologies (BST) andSurgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Biology of Development and Aging (BDA), Emerging Technologies and Training in Neuroscience (ETTN) andIntegrative, Functional and Cognitive Neuroscience (IFCN) AIDS and Related Research (AARR) and Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) Scheduled 2009-12 IRGs Scheduled 2010-8 IRGs Biobehavioral and Behavioral Processes (BBB) and Risk, Prevention and Health Behavior (RPHB) Vascular and Hematology (VH) and Cardiovascular and Respiratory Sciences (CVR) Healthcare Delivery and Methodologies (HDM) and Epidemiology and Population Sciences (EPS) Musculoskeletal, Oral and Skin Sciences (MOSS) and Digestive, Kidney and Urological Systems (DKUS) Immunology (IMM) and Infectious Diseases and Microbiology (IDM) Genes, Genomes and Genetics (GGG) and Biological, Chemical and Macromolecular Biophysics (BCMB) Oncology 1 — Basic and Translational (OBT) and Oncology 2 — Translational Clinical (OTC) Interdisciplinary Molecular Sciences and Training (IMST) and Cell Biology (CB)
  31. 31. 6. Performing a Second Cycle of IRG Reviews <ul><li>All pertinent study section chairs were invited </li></ul><ul><li>Advice on study section and IRG </li></ul><ul><li>Revision of study sections description/outline </li></ul><ul><li>Training of the chair </li></ul>
  32. 32. 6. Performing a Second Cycle of IRG Reviews Second Cycle of IRG Reviews Don Schneider
  33. 33. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Holding Out-of-Town Meetings </li></ul><ul><li>Recruiting the Best Reviewers </li></ul><ul><li>Performing a Second Cycle of IRG Reviews </li></ul><ul><li>Revising Study Section Guidelines </li></ul>
  34. 34. 7. Revising Study Section Outlines <ul><li>Cellular Signaling and Regulatory Systems </li></ul><ul><li>[Roster] </li></ul><ul><li>The Cellular Signaling and Regulatory Systems (CSRS) study section reviews applications that focus on the initiation and execution of programs that control cellular homeostasis and physiology. A distinguishing characteristic of these applications is an emphasis on signaling networks and the coordination of processes related to cell proliferation, survival, and growth. </li></ul><ul><li>Cell cycle regulation, mitosis, meiosis, checkpoint controls and regulation by ubiquitination </li></ul><ul><li>Proteolytic mechanisms associated with cell cycle, senescence and death </li></ul><ul><li>Programmed cell death and apoptosis, particularly their regulation in the context of stress, growth, and transformation.  </li></ul><ul><li>Proliferation and growth control by the nucleus; signaling pathways regulating transcription </li></ul><ul><li>Integrative cell physiology, e.g., stress, clocks, cellular modeling; cell differentiation and transformation </li></ul><ul><li>Basic studies of cytokine signaling </li></ul><ul><li>Application of state-of-the-art technologies such as imaging and computational modeling of cellular signaling networks </li></ul><ul><li>Study sections with most closely related areas of similar science listed in rank order are: </li></ul><ul><li>Molecular and Integrative Signal Transduction </li></ul><ul><li>Intercellular Interactions </li></ul><ul><li>Membrane Biology and Protein Processing </li></ul><ul><li>Molecular Genetics A </li></ul><ul><li>Molecular Genetics B </li></ul>
  35. 35. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Holding Out-of-Town Meetings </li></ul><ul><li>Recruiting the Best Reviewers </li></ul><ul><li>Performing a Second Cycle of IRG Reviews </li></ul><ul><li>Revising Study Section Guidelines </li></ul><ul><li>Training Chairs </li></ul>
  36. 36. 8. Training Study Section Chairs <ul><li>Scope and Schedule </li></ul><ul><li>120 new chairs every year, more if “permanent” SEPs chairs are included </li></ul><ul><li>5 meetings in January 2009 for chairs recently appointed </li></ul><ul><li>5 meetings in July 2009 for chairs appointed next June </li></ul><ul><li>Training Program </li></ul><ul><li>Share data </li></ul><ul><li>Explain the new changes and the significance </li></ul><ul><li>Share the best practice (ours and theirs) </li></ul><ul><li>Answer questions and address concerns </li></ul><ul><li>Make Chairs more effective stakeholders </li></ul>
  37. 37. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Holding Out-of-Town Meetings </li></ul><ul><li>Recruiting the Best Reviewers </li></ul><ul><li>Performing a Second Cycle of IRG Reviews </li></ul><ul><li>Revised Study Section Guidelines </li></ul><ul><li>Training of Chairs </li></ul><ul><li>Managing Conflicts of Interest </li></ul>
  38. 38. 9. Managing Conflicts of Interest <ul><li>Report of Conflict of Interest Committee </li></ul><ul><li>Noni Byrnes </li></ul>
  39. 39. CSR’s Efforts to Enhance Peer Review <ul><li>Reorganizing CSR’s Review Groups </li></ul><ul><li>Recruiting CSR Staff </li></ul><ul><li>Shortening the Review Cycle </li></ul><ul><li>Holding Out-of-Town Meetings </li></ul><ul><li>Recruiting Reviewers </li></ul><ul><li>Performing a Second Cycle of IRG Review </li></ul><ul><li>Revising Study Section Guidelines </li></ul><ul><li>Training of Chairs </li></ul><ul><li>Managing Conflicts of interest </li></ul><ul><li>Advancing Additional Review Platforms and Processes </li></ul>
  40. 40. 10. Advancing Additional Review Platforms and Process <ul><li>Video (VED) </li></ul><ul><li>Internet (AED) </li></ul><ul><li>Editorial Board </li></ul><ul><li>Order of Discussion </li></ul>
  41. 41. 10. Advancing Additional Review Platforms and Processes <ul><ul><li>Editorial Board Review </li></ul></ul><ul><ul><ul><li>Heavy triage based on innovation and potential science transformation by a small study section of distinguished, broad-science reviewers ( the editors ) </li></ul></ul></ul><ul><ul><ul><li>Specific science reviewed by appropriate reviewers ( the editorial board ) </li></ul></ul></ul><ul><ul><ul><li>Final ranking by the editors </li></ul></ul></ul>
  42. 42. 10. Advancing Additional Review Platforms and Process New Transformative R01 Betsy Wilder
  43. 43. 10. Advancing Additional Review Platforms and Process Results of Asynchronous Electronic Discussion Survey Andrea Kopstein
  44. 44. NIH Enhancement of Peer Review <ul><li>The Charge from Dr. Zerhouni: </li></ul><ul><li>“ Fund the best science, by the best scientists, </li></ul><ul><li> with the least administrative burden…” </li></ul>http://enhancing-peer-review.nih.gov Two advisory committees to the NIH Director
  45. 45. The NIH Director’s Peer Review Initiatives
  46. 46. The NIH Director’s Peer Review Initiatives Corporate NIH Peer Review Enhancement: Implementation Raynard Kington
  47. 47. This is CSR September 2008
  48. 48. Recognizing Outstanding Reviewers <ul><li>Going above and beyond the call of duty </li></ul><ul><li>Having a significant impact on the quality of peer review </li></ul><ul><li>Educating and inspiring other reviewers </li></ul><ul><li>Setting the highest standard of peer review excellence </li></ul>
  49. 49. The First Marcy Speer Award, 2007 <ul><li>Marcy Speer </li></ul>
  50. 50. 2008 Marcy Speer Outstanding CSR Reviewer Award Dr. David J. Sahn, M.D. Oregon Health & Science University
  51. 51. The 2 nd Marcy Speer Award, 2008
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×