Picking the right programs to Start, Stop, Sustain or Grow requires a combination of data, judgment, and process. In the final episode of our webcast series, we shared a process that leads to high-quality program decisions and ensures that these choices have the support of faculty and institutional leaders.
In 2017, our team facilitated this process for colleges and universities that serve over 950,000 students. The approach describes the best available data on markets for academic programs with a decision-making workshop.
Why has this process been successful?
Data-Enabled: Provides relevant market data on student demand, employer demand, and competition
Inclusive: Engages key stakeholders from the institution's administration and faculty
Consistent: Applies consistent scoring techniques to all programs
Transparent: Enables an open, frank discussion of data, scores, and issues
Judgment: Encourages participants to use the data to inform their judgment, not to replace it
2. Confidential www.GrayAssociates.com 2
Goal for Today’s Session
§ Share an overall approach, and best practices, for
an integrated program portfolio evaluation
process
– Participants
– Process
– Analysis
– Integration
5. Confidential www.GrayAssociates.com 5
Make sure there is nothing major blocking the path to a decision-making
process.
Before You Begin…
Photo source: http://i.imgur.com/wsRm02x.jpg
6. Confidential www.GrayAssociates.com 6
Before You Begin…
It is important to understand:
§ Programs have life-cycles
§ You are ready to identify where each of your programs are
§ Doing this once is helpful, doing this systematically creates strength
§ This is not something to be done ‘behind closed doors’
Check out the organization’s assumptions surrounding program evaluation.
No process will overcome unidentified
assumptions.
10. Confidential www.GrayAssociates.com 10
Integrating Data and Knowledge
A program portfolio occupies two dimensions, and both are important to its
evaluation.
Inside the Four Walls
§ Standards
§ Faculty
§ Facilities
§ Cost
Outside the Four Walls
§ Student Interest
§ Employment
Demand
§ Competitive Intensity
§ Your market area
Evaluating a portfolio requires more than just
data. It requires a process.
15. Confidential www.GrayAssociates.com 15
Fourth: Walk Through The Data
Comparisons to other programs help you interpret the data, find
opportunities, see problem areas.
100,000Program X 8,690 9.4% 39,721 -664
17. Confidential www.GrayAssociates.com 17
From Data to Evaluation: Process
This is a well-tested and successful process for looking at a portfolio and
deciding what programs to “Start, Stop, Sustain, or Grow.”
1. Data and
Scoring
2. Workshop
3. Follow-up
and
Management
§ Uses facts and data
effectively
§ Incorporates judgment of
key stakeholders
§ Identifies the best new
programs, not just “good
enough” programs
§ Earns the understanding
and buy-in of key
stakeholders
§ Positions the organization
for next steps
– Creating an action plan for
teaching out, sustaining,
fixing, or growing existing
programs
18. Confidential www.GrayAssociates.com 18
From Data to Evaluation: Process
Why does it work?
Inside the Four Walls Outside the Four Walls
Image Source: http://applesofgold.com/Merchant2/wedding-bands/POL52-24-Set1C.jpg
§ The process is transparent
§ It is cross-discipline, cross-functional
§ Uses quantitative data to inform, Academic
and Operational knowledge to decide
§ It creates alignment
20. Confidential www.GrayAssociates.com 20
Cooperative and Transparent Process
Good program decisions take more than numbers…they take intellectual capital and
teaming.
Workshop participants should
include:
• Academic Leaders and Partners
• Enrollment/Admissions
• Finance
• Marketing
• Operations
• Institutional Effectives
• Student Services
• Career Services
• ….and others with a stake and
knowledge
21. Confidential www.GrayAssociates.com 21
The agenda will vary depending on the number of campuses, the number of award
levels, and whether the focus is on existing programs, new programs, or both.
From Data to Evaluation: Workshop Agenda
Sample Workshop Agenda
Day 1 Review Workshop Objectives
Explain Approach to Program Selection
Decide how to Decide
Refine Program Scoring
Select New Programs
Day 2 Match New Programs to Campuses/Online
Discuss Current Programs: Grow/Sustain/Fix/Stop
Wrap Up
1. Data and
Scoring
2. Workshop
3. Follow-up
and
Management
22. Confidential www.GrayAssociates.com 22
Collaboration and Decision Approach
Collaboration foundations:
The degree to which people feel a process, decision or outcome is theirs
The degree to which people see and understand the problem in the same way
The trait of believing in the honesty and reliability of the other
Decision Models: A Sample
Person-in-Charge
Decides Without
Group Discussion
Person-in-Charge
Decides After
Group Discussion
Majority
Vote
Unanimous
Agreement
Red
Yellow
Green
§ Ownership:
§ Alignment:
§ Trust:
23. Confidential www.GrayAssociates.com 23
The workshop will produce an agreed upon view of your current portfolio, and
identified new program opportunities.
From Data to Evaluation: The Output
The Master Scorecard
24. Confidential www.GrayAssociates.com 24
The Workshop is Just the Start….
There will be plenty of work after the workshop.
1. Data and
Scoring 2. Workshop
3. Follow-up and
Management
§ Deep dives on specific potential new programs
– Employment opportunities: Who are the likely employers? What skills and credentials are
most important in their hiring processes?
– Competitor programs: What content do they cover? How do they position the program in
their marketing materials? Do they indicate what they expect their students to do with their
new credential? How long are those programs, and how are they priced?
– Financial analysis
– Regulatory analysis
– Accreditation analysis
– Identification of clinical/externship sites (if needed)
§ Planning, development, and launch of new programs
§ Gap analysis for existing programs: For existing programs that are underperforming
in attracting or placing students, what are the reasons, and are they fixable?
§ Ongoing analysis of sustainability: Performance tracking for existing programs
26. Confidential www.GrayAssociates.com 26
Track and refresh market data – it can change over time.
Follow a Program’s Market Health
Student Demand
(-5 to +22)
33%
Competition
(-14 to +22)
33%
Employment
(-18 to +20)
27%
Degree Fit
(-50 to +10)
15%
27. Confidential www.GrayAssociates.com 27
Know and Track Your Competitive Environment
Competitive Analysis
($10,000)
$10,000
$30,000
$50,000
$70,000
$90,000
$110,000
$130,000
$150,000
University A
(non
resident)
College B College C University D
(non
resident)
My School University D
(resident)
University A
(resident)
Computer Science
AY 17-18 Tuition and Fees AY 16-17 Tuition and Fees
School Total SH AY 17-18 Tuition and Fees AY 16-17 Tuition and Fees
University A (non resident) 128 $147,033.44 $143,480.00
College B 128 $144,873.00 $145,688.00
College C 128 $138,125.00 $120,220.00
University D (non resident) 128 $127,945.06 $122,074.37
My School 128 $62,770.00 $60,210.00
University D (resident) 128 $57,033.06 $52,314.37
University A (resident) 128 $41,773.44 $31,248.00
28. Confidential www.GrayAssociates.com 28
Use or Create a Program Dashboard
Example
Identify measures that can be tracked and give you insight into program performance.
Metric
Campus
A, B, C
Online
New Starts
Program Headcount
Program Actual Credit Hours
Program Budget Credit
Hours
Program Revenue
Contribution Margin
6-Year Graduation Rate
Retention Rate
Program Completions
Program Share
31. Confidential www.GrayAssociates.com 31
Key Take-Aways
§ Don’t ignore any of the big market elements when you’re evaluating programs.
– Student demand
– Job opportunities
– Competitive intensity
– Degree fit
§ Each data element has some flaws or limitations. That’s why we have people. J
§ Don’t ignore institutional knowledge, but don’t allow it to dictate either.
§ Analyzing programs requires a process that respects and engages key stakeholders.
§ Data can inform, and should, but it is management and academic judgement that is
required to reach informed choices.
Leverage institutional knowledge and expertise – Engage it!
The most important take-away is the simplest: you have the ability to consistently,
comprehensively and cooperatively evaluate programs based on market conditions.
32. Confidential www.GrayAssociates.com 32
For More Information…
To contact Gray Associates:
Mary Upchurch
Senior Partner
Gray Associates, Inc.
Mary.Upchurch@GrayAssociates.com
480-390-7230
Gray Associates
@Gray_Associates