1. Minnesota State Arts Board
Operating Support
Panelist training session
Program officer: Bonnie Schock
bonnie.schock@arts.state.mn.us
(651) 215-1612 or (800) 866-2787
1
2. Training Agenda
• Program overview
• Advisory review process and panelist role
• Review training guide
• Using the WebGrants system
2
3. Arts Tour Minnesota purpose
To provide general operating support to high quality,
established arts organizations that produce, present, or
exhibit works of art; to organizations that provide a
broad range of services to artists; and to community
arts schools and conservatories that make arts learning
available to Minnesotans of all ages and abilities.
3
Program Purpose
4. Program outcomes
• The arts are interwoven into every facet of community life.
• Minnesotans believe the arts are vital to who we are.
• People of all ages, ethnicities, and abilities participate in the arts.
• The arts thrive in Minnesota.
4
5. Arts Tour Minnesota purpose
• 165+ applications
• 51 full applications reviewed by panel
• 110+ interim applications reviewed by staff
• mix of applicant geographic location, type, and budget size
5
Applicants
6. Funding components
• Formula funds award (must meet all five criteria)
• Merit funds award (for applicants that exceed
expectations in one or more of the review criteria)
• $15,000 minimum - $1 million maximum
6
7. Review Criteria
• Artistic excellence and leadership (0-20 points)
• Management and fiscal responsibility (0-20 points)
• Engagement (0-15 points)
• Public service and public benefit (0-30 points)
• Evaluation and assessment (0-15 points)
7
8. Important Aspects of Scoring
• Each application scored on its own merits
• Always in relationship to evidence
• Use full range
• Consistency
• Formula and merit rounds
• SAVE PRELIMINARY SCORES
8
9. Proposal Walk Through
• Applicant outcomes
• Application narrative
• Financial support materials
– CDP funder report
– Board approved financials
– Audits/990s
• Financial narrative and Financial review
• Organizational leadership (resumes)
• Other support materials
– Program attachment(s)
– Evaluation attachment(s)
– Board approved planning document
– ADA plan
– Background attachment(s)
• ADA Review form
• Artistic visit form(s)
9
10. C
Panelist Role
• Read the application materials carefully and use the
review criteria to evaluate
• Be prepared to participate fully at the panel meeting
• Complete primary and secondary reader
assignments, including administrative visits and
reports to the panel
10
11. Panel Meeting – What to Expect
• WebGrants system will be used through out the two days
• The Arts Board will provide laptops
8:00 a.m. Arrival, computer log in, and
continental breakfast
8:30 a.m. Orientation and training
10:30 a.m. Public Meeting begins
Lunch To be determined
5:00 p.m. Adjourn for the day
11
12. Conflict of Interest
• Financial relationship (2 years)
• On board/serve as a consultant
• Familial relationship
• Bias (person/subject/art form)
• Appearance of a conflict of interest
12
13. Review Procedure
• Applicant name
• Conflict of interest
• Primary reader report
• Secondary reader comments
• General discussion
• Vote and scoring
13
14. Minnesota State Arts Board
FY 2014 Panelist
Training Part 2
WebGrants Overview
6/2013
14
15. LOGIN to WEBGRANTS
15
Access WebGrants by going to:
apply.mnartsboard.com/
•Enter your USER ID
•Enter your PASSWORD
Access WebGrants by going to:
apply.mnartsboard.com/
•Enter your USER ID
•Enter your PASSWORD
6/2013
16. 6/2013
16
My Reviews – Main menu
Within the review section, panelists are able to view each application, take notes and score the applications.
REVIEWER INSTRUCTIONS:
Contains general help and
navigation tips.
MY REVIEWS: Select this for
access to all applications assigned
to you for review.
REVIEWER INSTRUCTIONS:
Contains general help and
navigation tips.
MY REVIEWS: Select this for
access to all applications assigned
to you for review.
18. 6/2013 18
My Reviews Screen
Application Title: title given by the applicantApplication Title: title given by the applicant
ID: WebGrants
number
used for panel
review
ID: WebGrants
number
used for panel
review
Organization: The name of the
organization or individual
Organization: The name of the
organization or individual
19. Tips
• Give yourself plenty of time to read applications
• Keep good notes, write down your questions and comments
• Come ready to discuss
• Look for evidence, not assertions
• Use only the materials provided in the application
19
20. Review Notes – WebGrants
• Keep notes based on review criteria to use during panel
review discussion. Save your notes regularly.
• You may edit notes at any time. DO NOT SUBMIT or MARK
AS COMPLETE.
20
6/2013
21. Conflict of Interest – WebGrants
• Follow the instructions on page 19 of the
WebGrants overview to enter a conflict.
• Following this process will remove the application
from your list to review
NOTE: You will have the opportunity to declare
conflict of interest at the panel review meeting.
21
6/2013
22. DO NOT SUBMIT REVIEWS
DO NOT MARK AS COMPLETE
With the exception of any applications for which you have a
conflict, DO NOT SUBMIT any notes or scores. Ignore the online
instructions to Mark as Complete and Submit Reviews.
22
6/2013
23. After completing the training
presentation, please send an e-mail to
the panel coordinator:
Beth Richardson
beth.richardson@arts.state.mn.us
Contact Beth with any questions about
panel meeting logistics, travel, or
WebGrants.
If you have any questions about
program intent, content of
proposals, budgets, contact:
Bonnie Schock, program officer
(651) 215-1612
(800) 866-2787
bonnie.schock@arts.state.mn.us
Minnesota Relay 7-1-1
www.arts.state.mn.us
23
Thank you for completing this panelist training presentation.
Editor's Notes
Hello, I’m Bonnie Schock, program officer for the Minnesota State Arts Board’s Operating Support program.
Thank you for agreeing to serve as an advisory review panel member. We are so grateful that you are contributing your time and expertise to this important work. It’s not an overstatement to say that the Arts Board could not function without the vital service that you and your fellow panelists provide.
I’m very much looking forward to working with each of you over the next two months.
NEXT SLIDE
This training session offers a brief introduction to the program’s purpose, goals, and structure. It is designed to acquaint you with the advisory review process, and includes essential information on the framework utilized to evaluate and score applications. It is important to complete the training prior to beginning your at-home review of applications, to ensure that you know what to look for. At the conclusion of the training, you will have the information you need to effectively review application materials at home, and prepare for administrative visits, the panel meeting discussion, and scoring.
Please have the Operating Support panelist training guide which you received by mail available as you go through this PowerPoint presentation. These materials are intended to help as you read the applications and again as reference during the panel meeting. We will address them in more detail as we continue through this training.
If you haven’t received your training guide, it is also available as a PDF in the Reviewer Instructions section of WebGrants.
NEXT SLIDE
Turn to the Operating Support program information section beginning on page 3 of your training guide.
As its name implies, this program provides general, unrestricted operating support. We know this type of funding is crucial to the health of arts organizations, but it also means that our job of reviewing applicants must be more comprehensive.
Arts organizations and distinct arts affiliates housed within non-arts nonprofits or public agencies are eligible to apply. In addition, to be eligible all applicants must meet minimum financial requirements (including two year average annual expenses of $160,000 or more, and at least 10% of revenue in charitable arts support), and must fit into one or more of four applicant categories:
Producer,
Presenter,
Artist service, and
Community arts schools and conservatories.
These four applicant categories shape the funding emphasis of the Operating Support program.
I encourage you to read through the Arts Board’s definitions on pages 3 – 4 of the training guide to better understand the range of organizational types the program seeks to fund.
NEXT SLIDE
In November 2008, Minnesotans passed the clean water, land, and legacy amendment to the Minnesota Constitution. As a result, over twenty-five years, three-eighths of one percent of the state’s sales tax will be dedicated as follows:
33 percent to a clean water fund,
33 percent to an outdoor heritage fund,
14.25 percent to a parks and trails fund, and
19.75 percent to an arts and cultural heritage fund
The Operating Support program is funded, in part, through the arts and cultural heritage fund.
Proceeds from the arts and cultural heritage fund “may be spent only on arts, arts education and arts access and to preserve Minnesota’s history and cultural heritage.”
Activities supported by the arts and cultural heritage fund must have measurable outcomes. On this slide are the Arts Board’s long-term strategic outcomes for the Operating Support program. Grantees are expected to evaluate their activities and be able to demonstrate measurable achievements related to outcomes.
In the Applicant outcomes section of each application, applicants have selected one or more of the above program outcomes that they feel their work during the grant period will address. In addition, each has stated one or more applicant outcomes that they intend to achieve with support from the Operating Support program. Applicants have also been asked to discuss these proposed outcomes in greater detail in the first section of the application narrative.
NEXT SLIDE
This is the Arts Board’s largest grant program, now awarding in excess of $13.8 million annually to grantees throughout the state.
We received over 165 applications this year.
Operating Support operates on a four-year funding cycle. Organizations submit full applications once every four years. Only full applications are reviewed by advisory panels. This year two panels will review and provide recommendations on a total of 51 full applications. The remaining applicants submitted abbreviated interim applications which are reviewed by staff.
Staff have done an initial screening for completeness and eligibility. Panelists should consider all applications eligible. If we learn something in the weeks leading up to the panel meetings that affects an applicant’s eligibility, we will alert you.
Each panel reviews a mix of applications representing organizations
located in the Twin Cities Metropolitan area and Greater Minnesota
structured as arts organizations and arts affiliates
providing programming in each of the four applicant categories discussed earlier
and, with budget sizes ranging from a minimum two-year average expenses of $160,000 up to in excess of $25 million
It is important to understand the different types of organizations that you will be reviewing so that you are able to consider each in its appropriate context. For example, the artistic programs and organizational structure of an arts organization producer will naturally differ from that of an affiliated community arts school. The role in the community for a Greater Minnesota organization will vary from that of a Metro organization. And, the organizational capacity and volume of programming for a $200,000 organization will be different than that of a multi-million dollar operation.
NEXT SLIDE
Operating Support grants consist of two parts – Formula Funds and Merit Funds
For Formula Funds: applicants must meet all five Operating Support review criteria to receive a formula funds award. (I’ll go over the review criteria and voting/scoring in a moment.) The amount of the formula funds award is based on the applicant’s qualifying expenses, averaged over fiscal years 2011 and 2012; and also varies depending on the overall number of grantees and the total dollars available to the program.
Applicants who meet all five criteria, and whose performance in one or more of the five criteria exceeds program expectations, will be eligible to receive a merit funds award. Like formula funds, the amount of the merit fund award will vary. It varies depending on the advisory panel score and the number of grantees within the organization’s budget group that are recommended for merit funding.
FY 2015 grants will be no less than $15,000 and no more than $1 million.
NEXT SLIDE
The review criteria section on pages 5 – 6 of your training guide lists the prompts designed to help the applicant address the review criteria and, in turn, to help you evaluate the application.
As you read the application materials, and when you come together in the panel meeting to discuss them, your consideration must be focused on the five review criteria for this program. All of your notes, questions, and comments need to relate to these. The degree to which the applicant addresses as many of the items listed under each review criterion will determine the quality of the application and your score. Please familiarize yourself with the items under each review criterion so that you know what the Arts Board means by “Engagement” or “Fiscal responsibility” and you know what to focus on. I recommend that you keep this list close at hand whenever you are reviewing application materials.
You can see on the slide how the five criterion are weighted in merit scoring. Because the Arts Board is a public agency, investing tax payer dollars, the Public Service and Public Benefit criterion is weighted most heavily. However, all five are important, and applicants must meet minimum expectations in all.
The criteria are, of course, related to one another; however, they are scored individually. An application which demonstrates exceptional evidence of artistic excellence and leadership may appear disconnected and uninformed about its community in the engagement criterion; or vice versa. An applicant which has a robust, innovative culture of evaluation may be lacking in demonstrated public service, etc.
We encourage you to use the notes section of WebGrants and/or another method of careful note-taking to record 1) strengths, 2) questions, and 3) weaknesses for each application in each of the five review criteria.
During the panel meeting discussion, you are encouraged to share your thoughts about the application as they relate to the criteria. When you notice something about an application, ask yourself: which one or more of the criteria does this information speak to?
For example: as you review financial documents, you may find errors in calculation, combined with incomplete or unclear information about the current financial position; this might also be combined with a lack of clarity about financial questions at the administrative visit. These factors should raise questions for you about several evidence items listed under management and fiscal responsibility including “organization’s capacity to effectively manage its operations and resources;” and “staff qualifications.”
Because we are looking at overall organizational performance, the Operating Support program looks at history and evidence of accomplishment in the review criteria. We ask that panelists look for evidence, not assertions, of an applicant’s abilities, track record, and commitment.
I’d like to call your specific attention to one evidence item which is listed under each of the first four review criteria as: “ongoing potential to achieve results consistent with the Operating Support program outcome(s) selected in the application” and in the last criterion, evaluation, as: “plan will effectively measure the Operating Support program outcomes and the proposed organizational outcomes selected in the application.” Throughout the panel process, we ask that you use proposed outcomes to help determine possible impact, always considering the applicant’s demonstrated potential to achieve intended future results, the demonstrated potential to effectively evaluate achievement of results, and the fit between the intended applicant outcomes and the Operating Support program’s aims.
NEXT SLIDE
The scoring rubric on page 7 of your training guide is provided as a reference tool to help you determine how to rank an application, and to support you in remaining consistent from one application to the next.
Each application is considered on its own merit and is not compared to any other application.
You should plan to use the full range of scoring – there may be some proposals that get all 10s or all 20s in each of the five criteria, but the majority will most likely be a mix.
Consistency is important from one application to the next. You may be a tough scorer; or, you may be a generous scorer. Either way, be sure to be consistent and apply the same standards across all applications to ensure fairness.
Please take a moment to read through the rubric. You’ll see that there are six categories: does not meet basic expectations, meets basic expectations, moderately exceeds, well exceeds, strongly exceeds, and exceptionally exceeds. As discussed earlier, the Operating Support program has two funding components. To be recommended for formula funding, by majority vote, the panel must determine that the applicant meets basic expectations in all five review criteria. If you feel that the evidence shows significant gaps or weaknesses in processes or outcomes in a criterion, where plans to address those weaknesses are insufficient, nonexistent, or not being given high enough priority, you should give a ZERO score in that criterion (regardless of scores in other criteria). A zero vote in any one criterion means you should vote “no” in formula funding.
You’ll see on the rubric that the “meets basic expectations” row is unshaded. Scores within this range indicate that you feel that the applicant is meeting standards such that you would recommend them for formula funds; however, they are not exceeding in the criterion, and you would not recommend them for merit funding. Please be aware that any score you give in the moderately exceeds range and above warrants a nomination from you for merit funding in that criterion. We’ll discuss the logistical process for merit nominations in detail during training on the first day of panel meetings.
Prior to the panel meeting, we ask that ALL PANELISTS enter and save preliminary scores in all review criteria into the WebGrants system. DO NOT MARK THEM AS COMPLETE OR SUBMIT THEM, but please do enter and save them in all criteria. For returning panelists, this is an important change from previous years. We are actively analyzing our merit scoring system, and the impact that the panel process has on final scores. Your cooperation on this will help us gather essential data to continually improve our evaluation system. Having preliminary scores in the system will also aid you during the final scoring process.
To enter preliminary scores, under the review forms section in WebGrants, click on Application scoring. Enter whole numbers into each review criterion. If you have entered a zero into any one or more criteria, be sure to check “no” in response to the question “Does this applicant meet basic expectations in all five review criteria?” If you’ve entered a score of 1 or above in all five criterion, respond “yes” to the question.
Preliminary scoring is just that: preliminary. It does not lock you into any decision. Final scores will fluctuate based on the panel discussion, and you will submit final scores following the discussion.
It is the board of directors that makes the awards; the recommendations for formula and merit funding which you develop through the voting and scoring process will go to the board in July, 2014.
NEXT SLIDE
Turn to page 8 of the training guide. This document tells you what components of the application you need to review if you are assigned as primary reader, secondary reader, or general reader. You’ll see that primary and secondary readers must review all application components (except the federal tax exempt letter). There are many optional items for general readers. The reading guide also includes a quick reference regarding the type of information contained in each component.
I’d like to call your attention to a few things about the application materials.
The application narrative consists of several parts. First, the mission, followed by roughly one page of background on the applicant which should include a summary of arts programming as well as discussion of the role the applicant plays in their communities. This is important contextual information. Next is roughly one page of discussion regarding proposed outcomes. This is where we ask the applicant to look ahead, describing both why and how they intend to achieve their goals. Finally, you’ll find roughly 8 pages in which applicants address the review criteria. In this section, applicants were asked to give evidence of how their organization meets each review criterion. They should provide you with evidence – not assertions – including documented past performance and achievements. For example: an application may say something like “XYZ Theater Company provides exceptional performances to the entire region.” This is an assertion, that is not backed up with sufficient detail or evidence for you to evaluate it. However, if the application were to go on to discuss several awards and strong local press quotes, a track record of drawing returning artists year after year, and statistics showing substantial, sustained subscription activity from surrounding zipcodes – now you have some solid evidence to substantiate the claim and relate back to review criteria.
The first document in the financial support materials section is the Cultural Data Project funder report. The Cultural Data Project is a collaborative effort of public and private funders throughout Minnesota (and nationally) and consists of an online system for collecting and standardizing historical financial and organizational data. The CDP enables organizations to view trends in their data, benchmark themselves against peer organizations, and enhance financial management capacity. Its use in Operating Support, for the first time this year, is intended to streamline the overall application, and provide panelists with more consistent, reliable and comprehensive data. The report includes a range of numerical data on staffing, programs, contributors, audiences, and finances. Please see pages 18 – 25 of your training guide for an annotated sample report that points out key features of the report. Please familiarize yourself with that sample report before reviewing CDP funder reports as part of application materials. Be certain to pay attention to the financial health ratios which appear on page 7 of the report (page 24 of the training guide). These, along with the entire financial picture, should contribute to your assessment of overall financial health in relationship to the Management and Fiscal Responsibility criterion. Definitions of the ratios are in a box on page 17 of your training guide.
Applicant Web sites may be accessed via a direct link in the CDP funder report. Consider the site as another support material, looking at how it represents and communicates the applicant’s work appropriate to mission, scope and community, as well as how it invites public engagement with programs and services. As with all support materials, you may find additional evidence of artistic quality, engagement opportunities, etc. on the Web site.
In the financial narrative, applicants are asked to provide information explaining variances from year to year, reasons for deficits or other financial issues. There is also a section that gives them the opportunity to explain anything else about their CDP funder report that might not be readily apparent. This is important information to view in relationship to the financials. The questions themselves offer some perspective on the kinds of things to be watching for when reviewing financials: significant fluctuations from year to year, surpluses and deficits, liabilities, negative net assets, cash on hand, and program expense ratios.
The financial review section has been prepared by the Arts Board’s director of finance and grants administration, who reviews all financial information ahead of time. It provides yes/no answers to simple financial management best practices and very brief notes about how an applicant is fairing in key financial health indicators.
If you are not familiar with the financial terms and concepts that appear in the CDP funder report, and/or the financial reports, please refer to the Financial terms glossary on pages 16 – 17 of your training guide. You also can always call or email me with any questions.
In the organizational leadership form, you’ll see a board list which includes general information about board membership. On the far left, the column titled “list order” indicates the order in which the list should be ordered; i.e., #1 is the first board member on the list, etc. Due to a software glitch, when there are more than 9 members on a board, the list will appear on the screen as #1, #10, #11, #12, etc. and then circle back to #2, #3, #4. We apologize for this, and invite you to view the list understanding that the applicants intended for it to appear in the numerical order indicated at left.
Page 11 of your training guide describes what you’ll find in the ADA review form. This evaluative summary report has been prepared by accessibility expert, Julee Quarvee-Peterson, to aid panelists in assessing ADA efforts as one key component of the Engagement criterion. We ask primary and secondary readers to view both the applicant’s ADA plan (found in other support materials section) as well as this evaluative summary. General readers only need to view this summary.
Artistic visits will be conducted for all applicants. A panelist attends an event and has completed an evaluation of the experience which has been attached to the application. We are still in process of completing artistic visits, so you may come across some applications that do not have a visit report for FY 15. When these have been completed, panel coordinator Beth Richardson will alert you to ensure that you are able to incorporate this important artistic evaluation into your thinking. Please note, many returning applicants will have two artistic reports – one conducted last year, and one conducted this year. This gives you more than one perspective on the applicants’ recent artistic programming, and both should be considered. However, it is normal for new applicants and some returning applicants to have just one report.
Please be aware, sometimes applicants have not provided us with exactly what was requested. When this does not impact the completeness of the application or staff’s ability to verify eligibility, we forward applications to the panel as is. For example you may see some organizations that have not included both a statement of financial activities and a statement of financial position in their board approved financials, or you may find that a resume is missing for one of the primary leadership personnel. This is a quality issue which you may incorporate into your assessment as you believe it applies to review criteria.
NEXT SLIDE
Your most important responsibility, as a panelist, is to understand the application as well as you possibly can.
We know that this role requires a heavy commitment of time and attention, but it is a very important task and we hope you will be as thorough as you can be.
When you come to the panel meeting, we ask that you participate fully, bringing your unique expertise to bear on the application, pointing out the strengths and weaknesses you saw in the materials, and sharing your questions or concerns with other members of the panel.
Your job is to help the Arts Board make the best decisions we can about how to invest these public funds.
Pages 12 – 13 of your training guide detail the additional duties of primary and secondary readers, provide information on how to prepare for and the format of the administrative visit, and an outline for the primary reader report.
Beth Richardson has already shared your primary reader assignments with you; for your reference, all of your assignments are listed in the application review order and reader assignments sections on pages 14 and 15.
To prepare for administrative visits, be sure to read all the application materials in advance. I recommend that you try to complete this review 1-2 days prior to the visit whenever possible. I will contact you ahead of time to arrange a phone conversation or brief in-person meeting prior to the visit so that we can go in jointly prepared and working as a team. Keep in mind that you will be asking the bulk of the questions. As the program officer, I will facilitate, support you as necessary, and participate in the discussion when needed.
In your primary reader review, try to anticipate the questions other panelists may have about the materials. What isn’t clear? Is there anything that appears inconsistent, contradictory, or otherwise confusing? Does the application contain assertions without evidence, where clarification would be beneficial? Pay attention to the financials, and the questions that the Arts Board poses in the Financial Narrative section: are there any significant fluctuations that are unexplained? What is the overall financial position? If there are vulnerabilities in the balance sheet, recent operating deficits, and/or financial health ratios that are below standard, what is the organization doing about it? Is there anything in the audits that you are wondering about? Do you have questions about the nature of the board, its terms of service, the relationship with staff, how decisions are made? Is there anything in the artistic visit form that you’d like to understand better about the nature of their artistic work?
Take careful notes at the visit, so that you’re able to recall information during the panel process. Finally, primary readers prepare and give an oral report which begins the panel deliberations for each applicant. Please follow the outline on page 13 of your training guide and be sure to keep your report, including your initial comments on the review criteria, to five minutes or less.
Remember: during the visit and the primary reader report, the primary reader’s job is to be objective. You are not advocating for or against the applicant, but serving as a resource with a deeper understanding of this particular applicant. This does not mean that you cannot have opinions as a panelist; primary readers also evaluate and score applications. It merely means that your opinion should not carry undo influence on other panel members or dominate the deliberations.
NEXT SLIDE
Let’s fast forward to what will happen during the panel meetings in May.
You will be using the WebGrants system and will be referring to the application materials online, as well as voting for formula funds and inputing merit scores online. The Arts Board will have laptops set up for all panelists.
You’ll have a little time to meet your fellow panelists, test that your online access is working properly, and have some refreshments before the training begins.
The purpose of this day-of training session is to make sure that you fully understand what you need to do during the review, and make sure that you can access and use the WebGrants system. During this time, you will walk through the schedule for the days, you will declare conflicts of interest, and I will review how discussion and scoring will be done.
You’ll also complete a practice review with the sample application that’s included in your applications to review. We encourage you to treat this application as you would all others when preparing for the panel. Take notes, and come ready to discuss.
At 10:30, the public meeting begins, and discussion and scoring of the actual applications gets underway.
The application review order on pages 14 – 15 of the training guide gives you the preliminary review order. Though it is subject to change, we offer it as a resource to help you plan your time.
NEXT SLIDE
At several points in this process, you will be asked if you have a conflict of interest with any application. Please review the conflict of interest policy on page 2 of your training guide. Panelists must declare any and all conflicts. Declarations of conflict of interest happen at the panel meeting. A conflict of interest is not a bad thing. It does not affect the applicant’s score. We calculate scores as an average, based on the number of people who scored that applicant.
You may have a conflict of interest if any of the following are true about your relationship with the applicant organization:
YOU….
Are in receipt of direct financial benefit from the proposal being reviewed
Serve as an employee or governing board member of the applicant organization
Serve with or without payment as a consultant to an applicant being reviewed
Have a familial relationship with the applicant, or a staff, or board member of the applicant organization
Apparent or appearance of conflict of interest occurs when a panelist feels biased either for or against an applicant, medium, style, etc.; or he has a personal affiliation with an applicant or applicant organization.
NEXT SLIDE
We follow the same review procedure for every applicant.
First, the staff facilitator announces the applicant and the applicant ID number and asks if any of you have a conflict of interest to declare.
If someone has a conflict he or she will be excused to leave the review room while the application is discussed.
The review begins with the primary reader report, followed by comments from the secondary reader on the review criteria, and then will be opened up for the panel to discuss the application, using the review criteria as a guide. Each application will be discussed for approximately 20 minutes.
This is why your notes are very important – you will have very little time to recall the key questions or comments you wanted to raise about the proposal. Your notes will remind you of the strengths and weaknesses you identified when you read the application.
The purpose of the discussion is to help you understand the application as fully as possible. The panel members are NOT expected to reach consensus, or convince one another to agree with their point of view.
Panelists are expected to speak respectfully about each application, but also are expected to be honest. Both the positive and negative aspects of the proposal need to be mentioned during the discussion, so that the applicant will have a clear sense of why its application may or may not have been recommended for funding.
Panelists will only discuss information provided by the applicant. No outside information is allowed.
Following the discussion of each application voting on formula funding, nominations for merit funding, and final merit scoring will take place.
NEXT SLIDE
Let’s turn to an introduction to WebGrants, our online grants management and review system.
The applications have been released to you and are available online by using your USER ID and password.
As mentioned above, there is also one SAMPLE application that will be used during the first day’s training. Its title is “AAA sample.”
NEXT SLIDE
The first step to reviewing applications is to log into WebGrants.
You can access WebGrants by typing the URL “apply.mnartsboard.com” into your Internet browser.
A link to WebGrants is also on the front page of the Arts Board Web site.
To log in to WebGrants, enter your assigned User ID and password, then click “Log in.”
<<NEXT SLIDE>>
This slide shows the main menu with two options displayed.
REVIEWER INSTRUCTIONS contain general help and navigation tips, a copy of the training guide which you received by mail as a PDF, and another training booklet called FY 2014 WebGrants Overview. The WebGrants Overview is a step-by-step guide for you to use at home as you learn how to use the WebGrants system.
You will also find the full Program Overview and Application Instructions here, if you want to look at how the applicants were guided.
-The second option, MY REVIEWS, is how you’ll access the applications assigned for your review.
NEXT SLIDE
After clicking on MY REVIEWS, you’ll see this screen which shows you all the applications you’ll be reviewing.
NEXT SLIDE
The ID number is the WebGrants assigned ID and may be helpful to use to find a specific application. By clicking on the column header ID line you can immediately display the applications in numerical order.
The Application Title is the title given to the application by the applicant.
The Organization column lists the name of the organization applying. By clicking on Organization in the column header, you can display the organization names alphabetically. This can be very helpful to be able to quickly find a specific application.
The Applicant column is the name of the contact person who submitted the application.
NEXT SLIDE
We hope the following tips will help you in effectively reviewing application materials and preparing for the panel review meeting.
Give yourself plenty of time to read applications and to prepare for administrative visits.
Pacing your reading is very important; be as thorough with these applications as you would want a panel to be with your own. If you have any problems accessing application materials, contact panelist Beth Richardson by phone or email at any time.
Keep good notes, write down your questions and comments. Feel free to write in the WebGrants notes field or use whatever note taking system works best for you. Your notes will not be accessible to the public.
Come ready to discuss, and tie your comments to review criteria.
Panel discussion is recorded and made available to applicants as audio minutes. Keep your comments constructive. We will only end up funding some of these application but the review process is an opportunity to give all applicants feedback about the strengths you saw as well as the aspects where you were confused or needed more information.
Look for evidence, not assertions
Use only the materials provided in the application to evaluate the project proposal. Outside knowledge should not enter into your considerations. Please do not do other research on the applicant; this is a matter of fairness to all applicants
NEXT SLIDE
Taking good notes is essential to your preparation for panel service.
Follow the instructions in the step-by-step overview to edit and save your notes in WebGrants. Remember to save your notes frequently – a loss of Internet connection may cause you to lose your notes.
You may edit your notes at any time, but do not submit your notes or mark them as complete. You’ll understand this once you’re reading the applications.
Some panelists have found it easier to take notes in a separate word document. Choose whatever system works for you to provide you with the information you’ll need for a quality discussion.
<<NEXT SLIDE>>
There is a process in WebGrants where you can declare a conflict of interest.
You may declare a conflict of interest at any time during the application review process; during your at-home reading, upon arriving to the panel meeting, or during the panel discussion.
If you are certain that you have a conflict with the applicant, follow the instructions in the WebGrants overview.
Following these steps will remove the application from your WebGrants list of applications for review.
<<NEXT SLIDE>>
Here’s one more reminder about reviewing applications at home.
Remember – with the exception of any applications for which you have a conflict, do not submit any notes or scores.
Ignore the online instructions to Mark as Complete and Submit Reviews.
And of course, remember to save your notes often.
<<NEXT SLIDE>>
Thank you for completing this training presentation.
Please send an email to panel coordinator Beth Richardson to let her know you are off and running! Her email address is listed on the slide.
For any questions about panel materials, logistics, WebGrants access, or hospitality, contact
Beth Richardson
(651) 215-1608
If you have any questions about program intent, what to look for when reviewing applications, content of applications, or financials, contact me:
Bonnie Schock
(651) 215-1612
We look forward to our work with you. Thank you.
