P A T IENT -CENTERED O U T C OM ES R ES EA R CH IN S T ITUTEPatient-Centered Outcomes Research Institute (PCORI)PCORI Online Instructional Guide forReviewers
About PCORI Online What is PCORI Online? • PCORI Online is the tool used to manage all aspects of PCORI’s grants process • PCORI Online is used not only by applicants, but by Reviewers and SROs • From an applicant’s letter of intent (LOI) to scoring for Reviewers - all of this is done via PCORI Online • PCORI Online is accessible anywhere there is Internet access with a web browser • PCORI Online will be the tool you use to download applications for review, score applications, and submit scores and comments All scoring is to be done using the PCORI Online tool. Do not submit scores via email or other means
Accessing PCORI OnlineHow do I Access PCORI Online?• Open your browser and go to the following URL: contracts.pcori.org• Enter your login credentials - this is your username and password combination - and click the ‘Submit’ button• If you are a Reviewer, you should Enter your email address have received your login credentials. If you did not, please contact email@example.com. If you forgot your password, click on ‘Forgot Your Password’• If you continue to have issues logging in, please send an email to firstname.lastname@example.org The time out is set to 1 hour. If there is no activity on the page (e.g. user has not saved the page), then there will be a pop-up 10 min before the timeout.
My Tasks Home Page • After login, PCORI Online takes you to the ‘Tasks’ tab. Here you can see tasks that are assigned to you. • Notice that the task to review applications is listed under ‘My Tasks’ • Under task, click on ‘Complete Phase 1 Reviews’ • On the left is the navigation menu. The green bar indicates which section you are in. If you are idle for 10 minutes in PCORI Online, the system will log you out
ConfidentialityRead the Confidentiality and COI Agreements Select Checkbox Read Agreements • After opening the task for the first time, you are taken to the confidentiality page. Please read the confidentiality agreement and the conflicts of interest agreement in the documents section. • You must agree to the confidentiality agreement and confirm that you viewed/attended the training sessions by checking the ‘I Agree’ check box • If you do not select "I agree", you will not be allowed to retrieve your assignments. If you do not agree with the agreement, contact email@example.com • Click ‘Save and Continue’
Assigned ApplicationsReview Applications Assigned to You Instructions List of assigned Conflicts section applications • After agreeing to the confidentiality agreement, you are taken to the ‘Review Assignments’ page where you will see all the applications that your SRO has assigned to you
Assigned Applications: ColumnsUnder the ‘Assigned Applications’ section youwill see a list of columns. They are defined as follows: EGID- This is a unique ID assigned to the application in the system. No two EGIDs are the same Primary Contact/PI- The main contact for the application. This may also be the PI Organization- The organization from which the application is coming from Role- Your role as the reviewer: Primary, Secondary or Tertiary (For Phase 1, all reviewers are the same, these roles are for internal management) Overall Application Score- This is displayed after you have completed scoring the application Conflict/Decline- This is used to indicate whether you have a conflict with the application or not. Also, if you choose to decline the review, you must use this dropdown Conflict/Decline Type- This a dropdown menu used to indicate the conflict type if you have one and reasons for choosing to Decline. PDF Icon- You may view the application in PDF format at anytime by clicking the PDF icon
Application Conflicts/Declining ApplicationsBefore scoring an application, youmust indicate whether or not youhave a conflict. To do so: No Conflict- If you have no conflict, select ‘No Conflict’ from the ‘Conflict/Decline’ menu Conflict- If you have a conflict, select Conflict from the ‘Conflict/Decline’ menu You must also indicate the type of conflict. Select the type of conflict from the ‘Conflict/Decline Type’ dropdown menu. Decline- If for some reason (apart from a conflict) you do not wish to review the application, e.g. insufficient expertise or time, please select ‘Decline’ from the ‘Conflict/Decline’ dropdown menu and then select the appropriate reason from the ‘Conflict/Decline Type’ dropdown menu Click on the PDF icon to view the application and check out sections of the application such as ‘Personnel’ and ‘Organization’ for potential conflicts
Assigned ApplicationsAccessing the Critique Form Conflicts/Decline Applications No Critique Form longer appear After indicating conflicts or no conflicts, click on the ‘Save’ button Those applications for which you have a conflict or decline to review no longer show up You will then need to click on the ‘Critique Form’ Icon to begin scoring the application Note: If you accidentally decline an application or indicate as having a conflict when there is none, please contact your SRO
Scoring About the application scoring screen: Criteria description From your assigned applications listing, after you have indicated whether you have conflicts or not, the ‘Critique Form’ icon appears. Click on this icon. Upon clicking the icon, you are taken to the scoring page for that particular application PCORI’s eight criteria are listed, along with a section on Human Subjects The eight criteria are each broken out separately with the criterion described, a dropdown for the 1-9 scoring scale, and a comment box Enter bulleted list of three strengths and three weaknesses
Scoring Cont’dFor the application score and individual criterionscores, the far right column in thescoring chart below provides a descriptive guide of how strengths and weaknessesare considered in a rating: Impact Score Descriptor Additional Guidance on Strengths/Weaknesses 1 Exceptional Exceptionally strong with essentially no weaknesses High 2 Outstanding Extremely strong with negligible weaknesses 3 Excellent Very strong with only some minor weaknesses 4 Very Good Strong but with numerous minor weaknesses Medium 5 Good Strong but with at least one moderate weakness 6 Satisfactory Some strengths but also some moderate weakness 7 Fair Some strengths but with at least one major weakness Low 8 Marginal A few strengths and a few major weaknesses 9 Poor Very few strengths and numerous major weaknesses
Scoring Cont’dEntering Scores and Comments Scoring Scale From the dropdown menu select the 1-9 score listed for the specific criterion A narrative is required for each criterion in addition to the 3 strengths and 3 weaknesses Enter your narrative in the comments field. Use a bulleted list describing three strengths and three weaknesses of the application. There is a 5000 character limit in all of the comments fields If you find a conflict of interest at any point during your review of the application, notify your SRO immediately
Human SubjectsEnter Human Subjects Comments Indicate if acceptable or not You must fill out the section on Protections for Human Subjects From the dropdown, indicate ‘Yes’ or ‘No’ whether an application has acceptable risks and/or adequate protections in place for human subjects If ‘No’ in the comments is filed, please provide comments on relevant Human Subjects concerns/flaws in the application If ‘Yes’, in the comments field, please describe the safeguards in place
Overall Application ScoreEnter an Overall Application Score and Comments Finally, you must enter an overall application score for the application Enter the score from 1-9 using the ‘Overall Application Score’ dropdown menu In the comments section, enter your narrative and indicate three strengths and three weaknesses with regards to the overall impact of the application. There is a 5000 character limit for the comments field.
Submitting your Scored ApplicationReview and Submit ‘Submit’ appears when all applications are completed Will indicate complete/incomplete items You must complete all assigned applications before you can submit. The system will not allow you to submit individual applications one at a time. After completing the scoring of each application, click on ‘Save’ Once you have completed scoring all applications, the ‘Submit’ button now appears on the Review and Submit page Click on ‘Submit’ to submit all your applications Once you submit your scored applications, you cannot go back and revise your scores. Make sure your scores are correct before submitting
PCORI Online Tips The time out is set to 1 hour. If there is no activity on the page (i.e., user has not saved the page), there will be a pop-up 10 minutes before the timeout You must complete scoring all of your assigned applications before you can submit. Example: If you are assigned 3 applications, you must complete scoring for all 3 in order to submit. If you need help or have a question, please submit your question to firstname.lastname@example.org. Your request will be responded to within 2 business days.