William Parks, PhD Professor of Medicine at Cedars-Sinai Medical Center and UCLA Associate Dean for Graduate Research Education Scientific Director, Women’s Guild Lung Institute

1. Significance and Innovation
William Parks, PhD

2. Scored Review Criteria
Investigator Initiated
R-series Grants
• Significance
• Investigator
• Approach
• Innovation
• Environment
Overall
Impact
Overall Impact or
Criterion Strength
Score Descriptor
High
1 Exceptional
2 Outstanding
3 Excellent
Moderate
4 Very Good
5 Good
6 Satisfactory
Low
7 Fair
8 Marginal
9 Poor
!
• Criterion Score
• Whole numbers: 1-9
• 1 (exceptional); 9 (well let’s just hope you never get a 9)
• Given by reviewers but not discussed at study section
• Provided in Summary Statement of all applications
• Overall Impact Score
• Not the mean of the criteria scores
• Different criteria are weighted by each reviewer
• Final Impact Score, Percentile
• Mean of all scores x 10 ➤ 10 – 90
• Percentiled against R01s applications across 3 meetings

3. R01 Grant Sections
• Face Page
• Table of Contents
• Performance Site
• Project Information
• Project Description: Abstract
• Project Narrative: 2 sentences
• Facilities and Other Resources
• Equipment
• Key Personnel
• Biosketches
• Budget (all years)
• Budget Justification
• Cover Page Supplement
• Introduction
(resubmission only)
• Specific Aims
• Research Strategy
• Significance
• Innovation
• Approach
• Preliminary Data
• Research Plan
• Protection of Human Subjects
• Women & Minorities
• Planned Enrollment Table
• Children
• Vertebrate Animals
• References Cited
• Multiple PI Plan
• Letters of Support
• Resource Sharing
• Authentication of Key
Biological and/or Chemical
Resources
• Checklist

4. Innovation

5. Innovation
• Two flavors
1. New concepts or challenges to existing paradigms or dogma
2. New reagents, assays, technologies, etc.
• However, proposals do not need to be innovative
• Thus, not a major review criteria – but can be a big plus

6. Innovation
Applicant’s
Assessment
of their
Innovative
Strengths
Reviewers’
Assessment
of the
Innovative
Strengths

7. Innovation
• Keep it short
• 1 paragraph
• Be realistic

8. Significance

9. What is Significance and How is it Evaluated?
• Not related to the disease or cellular process you are studying
• After all, all diseases are significant
• Basic science research can have an impact
• Rather, if the aims are achieved, will scientific knowledge, technical
capability, and/or clinical practice be improved?
• Hence, Research Approach impacts Significance
• Does the project address an important problem or critical barrier to
progress in the field?
• A wet-lab proposal that is descriptive or derivative or will gather
correlative information will not be significant
- Epidemiology or clinical studies may seek associative findings
• Evaluation of and attention to rigor

10. Common Misconception
• Significance only means translational science, clinical importance, and/or
disease related
• Not true: basic research can have a great an impact
• “NIH’s mission is to seek fundamental knowledge about the nature and
behavior of living systems and the application of that knowledge to
enhance health, lengthen life, and reduce illness and disability.”
• An application does not need to show the potential for clinical impact

11. • Describe the strengths and weaknesses/gaps in the rigor of the prior research (published
and unpublished) that serves as the key support for the proposed project and plans to
address weaknesses/gaps.
• Rigor: Strict application of the scientific method to ensure robust and unbiased
experimental design, methodology, analysis, interpretation, and reporting of results.
• Variables: sample size, sex, age, weight, health condition, etc.
• Must address sex (approach)
• Authentication of key biological and/or chemical resources (1 page)
• Rigor: must be addressed by reviewers under both Significance (previous) and Approach
(planned)
• https://grants.nih.gov/grants/peer/guidelines_general/Reviewer_Guidance_on_Rigor_and_Transparency.pdf
• https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-228.html
Significance – Rigor of Previous Work, Yours and Others

12. WhERE IN THE APPLICATION?
RESEARCH STRATEGY
1 The research strategy
• Describe the strengths and weaknesses in the rigor of
.
• Describe plans to address weaknesses in the rigor of
the prior research.
are factored into research designs and analyses.
Introduction to
Applications
Research
Strategy
Vertebrate
Progress
Report
Publication
List
REVIEW GUIDELINES
3
also be asked to
(see Update 2)!
• Is the prior research rigorous?
address weaknesses in the rigor of prior
strategies to ensure a robust and unbiased
approach
relevant biological
variables, such as sex,
ATTACHMENT FOR AUTHENTICATION OF KEY BIOLOGICAL AND/OR CHEMICAL RESOURCES
2 resources used in the proposed studies.
CELL LINES SPECIALTY CHEMICALS
ANTIBODIES OTHER BIOLOGICS
Standard laboratory reagents that are not
expected to vary do not need to be included in
the plan. Examples are buffers and other common
biologicals or chemicals.
DO NOT put experimental methods
or preliminary data in this section
DO focus on authentication and
validation of key resources
What are the four
elements of rigor?
RIGOR OF
THE PRIOR
RESEARCH
2
RIGOR
OF THE
PROPOSED
RESEARCH
3
BIOLOGICAL
VARIABLES
4
AUTHENTICATION
what you need to know
Send inquiries to
reproducibility@nih.gov
See also NIH Notice NOT-OD-18-228
NIH ENHANCING
REPRODUCIBILITY
GUIDELINES

13. Significance (Background)
• 1-1.5 pages
• Critically review the literature and provide a clear
premise
• No limit on number of citations
• Original, timely papers over reviews
• Rigor: Point out gaps and flaws; strengths, too
• Do not be afraid to say you disagree with something
(but explain why and how you will correct this travesty)
• Be diplomatic
• Limit discussion to things (pathways, diseases, molecules, etc.)
you will study
• Show (tempered) enthusiasm
• Know your audience
• Get the reviewers interested
Pet Peeves
• Needlessly long
• Not focused
• Not timely nor scholarly
• Reliance on reviews
• Uses the word “exciting” more than
once
• Poorly developed premise
• Does not address rigor

14. Impact vs. Significance
Impact
• Likelihood of making a sustained,
powerful influence on the field
• Integrates the 5 scored criteria
• Not the mean of the 5 criteria
Significance
• Focus on relevance and likelihood
of making a meaningful advance if
the aims are achieved.
• Addresses an important problem or
critical barrier to progress
• Topic ≠ Significance