Here we present a case-based talk that applies both systems thinking and the concept of isomorphism to a complex, mulch-institutional college bridge program. For more information please contact Patrick Barlow at Pbarlow1@utk.edu
Retrospective application of systems thinking and isomorphism to a complex multi institutional evaluation project-barlow & heidel 2012
1. RETROSPECTIVE APPLICATION
OF SYSTEMS THINKING AND
ISOMORPHISM TO A COMPLEX
MULTI-INSTITUTIONAL
EVALUATION PROJECT
Patrick B. Barlow
PhD. Student in Evaluation, Statistics, & Measurement, the University of
Tennessee
Eric Heidel, PhD
Assistant Professor of Biostatistics, the University of Tennessee Graduate
School of Medicine.
2. What’s on the Agenda?
• Case presentation
• How can systems thinking be applied to
the Bridge Program?
• How can isomorphism and boundaries
be applied to the bridge program?
• Isomorphism, boundaries, and
evaluator competency
• Questions
4. Program Description
The UT-PSCC Bridge Program was conceived as part of a statewide
initiative to provide additional transfer pathways and strengthen
ties between two-year and four-year state institutions.
Summer Session (UT)
Wait-listed UT students who enroll in the program move into UT housing
Enroll in multiple UT summer courses
Fall Session (PSCC)
Students continue to live on the UT campus
Take full course load at PSCC
Must maintain 2.0GPA & 15 transferable credits
Spring Session (PSCC)
Continue to live at UT
Courses at PSCC
Must have 2.5 GPS & 30 credits
If benchmarks are met, then the student is guaranteed enrollment at UT the
following Fall Semester.
5. Evaluation and Evaluator Role
Student evaluator
– Internal (UT student)
– External (not employed by the program)
Evaluation Design
– Mixed methods design with multiple planned
data sources including survey, interview, and
focus group data.
Program “Schematic”
6. The Problem
Initial lack of…
– Organization
– Communication
Multiple university systems
– Shared responsibilities/offices
– Number of stakeholders
– Physical distance between systems
Constantly evolving program
7. What are systems?
How did we view system boundaries in the Bridge Program?
HOW CAN SYSTEMS BE
APPLIED TO THE BRIDGE
PROGRAM?
8. Systems & Boundaries
In general, a system is a, “complex whole of
related parts” (Cabrera)
In evaluation:
• What is the program? What is it not?
• What are the parts of the program?
• How do these parts relate to one another &
the program as a whole?
• Conscious recognition of these elements is key
9. The Focus
Acknowledging & understanding the
interrelations among program elements as
well as between the program & external
forces.
10. Key Systems in the Bridge Program
UT-PSCC Bridge
Program
TN
Legislature
UT PSCC
11. A Closer Look
UT-PSCC Bridge Program
Pellissippi
The
State
University of
Program Community
Tennessee Admin. College
Student
Admissions Academic
Success
Affairs
Center
Billing/
1stYr. Student Financial Campus Campus
Studies Activities Aid A B
Marketing/
College College Comm.
A B
Advising/ Campus Campus
Orientation A B
Faculty Faculty
Housing/
Dept Dept
Student
A B Services
12. One Step Deeper
• Within the program are The University of Tennessee
two university systems
• Within each university
there are multiple
administrative divisions Division of Student
Life
• Within each division
there are individual
offices University Housing
• Finally, within each
office there are
numerous stakeholders
who are responsible for Executive
providing services. Director
13. What is isomorphism?
How did we apply isomorphism to the Bridge Program?
HOW CAN ISOMORPHISM AND
BOUNDARIES BE APPLIED TO
THE BRIDGE PROGRAM?
14. Isomorphism
• Etymology
– Iso = “equal” and Morphosis = “to form”
• Mathematics
– 1:1 correspondence between two mathematical
sets so that operations cause the same result
for each set
• Biology
– Similarity in form, despite different ancestry
15. Isomorphism
• “Recursive replication” of processes and content (Liddle &
Saba, 1983)
• “Overlay of overlays” with philosophy and theory (Liddle,
1988)
• Categories with different content, but similar form, have
corresponding parts and processes (Hofstadter, 1979)
• Repetitive or similar patterns, translation of models and
principles, identical structure and process, interventions,
behaviors (White & Russell, 1997)
• Unconscious phenomenon that plays an integral role in
understanding relationships between systems (Bernard &
Goodyear, 2004; Gentry, 1986; Williams, 1997)
16. Awareness of Isomorphism
Evaluators can better understand:
– The similarities and overlap between systems
– The interdependency between system
– Corresponding parts of different systems
– How changes and occurrences in one system
exact change in another system
17. Isomorphism & the Bridge Program
UT-PSCC Bridge Program
Pellissippi
The
State
University of
Program Community
Tennessee Admin. College
Student
Admissions Academic
Success
Affairs
Center
Billing/
1stYr. Student Financial Campus Campus
Studies Activities Aid A B
Marketing/
College College Comm.
A B
Advising/ Campus Campus
Orientation A B
Faculty Faculty
Housing/
Dept Dept
Student
A B Services
19. Implications for Future Evaluations
Incorporating both Isomorphism and
systems thinking enhances the evaluator’s
ability to…
1. Visualize and understand program structure
2. Anticipate conflict, difficulties, and/or
ethical dilemmas
3. Engage in reflective practice using past
evaluation experiences as part of promoting
professional development
Will emphasize that we created this presentation to be very visual because systems and isomorphism are both concepts that are better understood by “drawing pictures” instead of reading.
Notes: In this slide I will describe the program as a cooperative effort with the University of Tennessee & Pellissippi State Community College, and that it was A) inspired by a similar program at Clemson University, and B) put in place following a TN legislative initiative (will find to cite it). I will then describe the basic timeline for the program, and will include (not in the slide): Invite only program that is sent to students on the UT wait list, and the average cohort size has been between 60 and 75 students. 2010 Complete College Act of TennesseeSummer Session (UT)Live on the UT campusEnroll in two summer courses at the UT campus.Fall Semester (PSCC)Take full course load at PSCCMust maintain GPA above 2.0 & 15 credit hrs.Continue to live on UT campusSpring Semester (PSCC)Take full course load at PSCCGPA above 2.5 & 30 credit hrs.Receive “Bridge Back to UT” orientationFall Semester Year 2 (UT)Enroll at UT as a full time student
Full notes:Student evaluator: the point here is to outline that I was only a student evaluator, and I was in a dangerous position of being both an internal evaluator (the program reflects UT who employs and educates me) and an external (I am not employed by or have a stake in the program’s success)Internal (UT student)External (not employed by the program)Evaluation Design: Briefly plan to mention that my perspective on evaluation, although I am a stats person by passion and training, is that no whole story can be told with numbers alone. Therefore…Mixed methods design with multiple planned data sources including survey, interview, and focus group data.Program “Schematic”: My advisor told me it may be a good idea to use a program schematic to visually represent the program and various stakeholders. This schematic is actually the backbone for today’s presentation.
Full Notes: This slide outlines the major “problem” with the case I am presenting, and what makes it so complicated. Initial lack of…Organization: No one knew what the program was, and one knew what to do with these students. (Example of students/parents calling in). What was the role of each school? Who led the program? The dual enrollment issue, and how were the various university systems equipped to handle it? (i.e. exceptions and overrides had to be made)Communication: Students/Parents had no idea what was going on (their words). Summer course instructors were given NO information about the program other than the FAQs, were not involved with the planning process, and did not know how to answer student questions. The evaluator was given NO information, limited information, or the wrong information from various stakeholders, emails went unanswered/unreturned, and follow-ups did not happen.Multiple university systemsShared responsibilities/offices: Examples of financial aid, advising, admissions. Number of stakeholders: self explanatoryPhysical distance between systems: Other sides of town from one another, PSCC has multiple campuses and the students could use any of them, student issues with getting to campus while living at UTConstantly evolving program: The turnover in leadership (Evaluator was bounced from at least three supposed “leaders” of the program before finally meeting with the right person). Doing something for the first time is always a learning process.