The document outlines City University of Seattle's approach to program assessment using e-portfolios, emphasizing the integration of course and program learning outcomes to enhance student achievement. It discusses the benefits and challenges of utilizing systemized data collection for formative and summative assessments and provides templates and frameworks for implementation. Presenters Sabrina Crawford and Dr. Laura Williamson highlight the importance of aligning curriculum with learning outcomes and fostering faculty engagement in the assessment process.

1. Systemizing Formative and
Summative Program Data Capture in
E-Portfolios
City University of Seattle
AIR Forum
May 2013
Sabrina Crawford and Laura Williamson, PhD

2. Learning Outcomes
• Participants in this session will walk through City
University of Seattle’s organization process for program
assessment and obtain usable templates to recreate
the process at their own institutions.
• Participants in this session will view how Folio180’s e-
portfolio system can be configured to obtain
systemized formative and summative data on student
achievement of programmatic learning outcomes.
• Participants in this session will discuss the benefits and
challenges of using systemized formative feedback
from faculty and students as a way of achieving a more
thorough look at programmatic assessment.

3. Presenters
• Sabrina Crawford is the Director of Institutional
Effectiveness at City University of Seattle. For the first 16
years of her career, Sabrina worked in the P-12 educational
arena as teacher, program coordinator, and Head of School.
Her passion for assessment and curriculum has most
recently brought her into the field of higher education
assessment and research, where for the past 4 years she
has been fortunate to work on creating systems that enable
data-driven decision-making.
• Dr. Laura Williamson is the Director of the MBA program at
City University of Seattle. For 20 years, Laura has worked in
higher education and high technology and has been
responsible for a variety of instructional technology
projects and program improvement initiatives with the US
Armed Forces and L-3 Corporation.

4. The Basics
• Founded in 1973
• WA Campuses located from Vancouver to
Vancouver
• 11 foreign locations
• In the US, traditionally focused on working adults
– Degree completion and graduate degrees
• In foreign locations – More traditional student
population
• Practitioner faculty deliver real-world relevance
to students around the world who may not
otherwise have access.

5. Strengths of Existing
Program Assessment
• Curriculum development system (CDS) ensures
connection between course learning outcomes
and CityU Learning Goals (CULGs)
• All course assessments linked to CULGs
• Program outcomes existed for all programs
• Courses had identified common assessments
• Existing faculty assessment committee process
• Rubrics embedded in CDS for course assignments
• Discussions had begun on using e-portfolios

6. Challenges to Program Assessment
• Large percentage of adjunct faculty
• No direct connection between course learning
outcomes and program learning outcomes
• Existing process was too big to be
implemented
• Focus on capstone course assessment only
• Rubrics focused on course or skill outcomes
• Wide variety of report output between
programs

7. Framework for Consistency &
Flexibility
Critical to be consistent
• CityU Learning Goals
• Program learning
outcomes
• Course learning
outcomes
• Assessments that
produce direct and
indirect evidence of
student learning
• Program level
assessment of student
learning – PLO rubrics
Important to be
consistent
• Methods of assessing
student learning
• Collection and analysis
of evidence of student
learning
• Example: A case study
with a prescribed
format, adapted to
reflect a local
company or
organization
Critical to be flexible
• Teaching strategies
that respect academic
freedom for faculty
• Learning activities that
respond to different
needs and populations
of students
• Readings, course
materials, and
activities that reflect
local context and
issues

8. ALIGNING OUTCOMES AND
ASSESSMENTS IN CURRICULUM DESIGN
Student Learning Outcomes Assessment

9. CityU’s Outcomes
Assessment Process
The diagram to the right
shows how CityU’s outcomes
assessment process starts
with its mission and
values, and culminates in
plans to achieve higher levels
of student learning.

10. CityU Learning Goals
** Program Outcomes **
Course Outcomes
Assessments
Goal #1 – Connect the Dots

11. CityU Learning
Goals
The CityU Learning Goals
describe exit competencies
for graduates of all degree
programs.
Professional
Competency
and Identity
Communication
and
Interpersonal
Skills
Critical Thinking
and Information
Literacy
Ethical Practice
and Service
Diverse and
Global
Perspectives
Lifelong
Learning

12. The Importance of Program Outcomes
• What specific actions should students be able to do
upon completion of the program?
• This is your PROMISE to your graduates
• They should be current and relevant to the profession
they lead to
• Every faculty member should know how their course is
helping students achieve these
• Every student should know that their courses are
supposed to be helping them develop these skills
• This is an opportunity to show how your program is
unique

13. Begin with Curriculum Mapping
• What courses and assessments within a program are
aligned to each program outcome
• Identify the course work’s level of student learning:
Introduction, Practice, Mastery
• From all the possibilities, identify the assessments best
demonstrate the three levels of indirect and direct
evidence (introduction, practice, and mastery) and
showcase student growth towards mastery of the
outcomes?
• Every course should connect to at least one outcome
and every outcome should have a minimum of one
course identified for each evidence level I,P, or M

14. Mapping Program Outcomes to Courses:
Identifying the best location for each level of student learning
MBA 500:
Essentials of
Business
Mgmt
MBA 501: Global
Business
Communication &
Research
MBA 515:
Project Mgmt
&
Prioritization
MBA 520:
Managing
with
Technology
MBA 550:
Business
Operations
MBA 555:
Business
Strategy
Program
Outcome
3
I – Capsim
Report
P – Final
Project Plan
&
Presentation
P –
Business
Plan
M – Capsim
Program
Outcome
4 I – Porter’s
I – Individual
Persuasive
Paper
Presentation
P – Business
Plan
M – Value
Chain Design
MBA Curriculum Matrix

15. Populate the Program Design Guide
• A one page spreadsheet that shows the alignments
• Populate the Program Learning Outcomes (PLOs) first
and connect to CityU Learning Goals (or Gen Ed) – one
or two max
• Populate the best assignments/courses from the matrix
that show student growth towards each PLO
• Identify the course learning outcome that align to the
identified assessment – the set of 3 should read as a
progression of student learning towards the PLO
• This becomes the template for the e-portfolio

16. CityU
Learning
Goals
Program
Learning
Outcomes
Course Learning Outcomes Courses Required
Assessments
Level
Which CityU
Learning
Goals are
supported by
program
outcomes?
What abilities
must the
learner
successfully
demonstrate
as a result of
this program?
Which course learning outcome
is tied to the required
assessment and supports the
programmatic outcome?
List the
courses
being used
for
assessment
What graded
assessment(s)
provide
evidence that
the learner can
demonstrate
proficiency in
this program
outcome?
I=Introduce
P=Practice
M=Mastery
Professional
Competency
and
Professional
Identity
Critical
Thinking Skills
9. Apply
socially
responsible
and
sustainable
business
practices to
an
organization
Document strategic decision
making.
MBA500 Porter Paper I
Conduct a SWOT analysis MBA525 Sustainability
Case Analysis
P
Utilize business simulations to
make executive management
decisions for every aspect of
leading and managing a
complex, multidimensional
business.
MBA555 Business Plan M
Program Design Guide

17. Create a PLO Rubric
• Mastery assignments to be assessed against
program learning outcomes
• PLO rubric designed to describe each student
learning level:
Introductory, Practice, Mastery
• Ensure student awareness of these levels and
their connection to their assignments
• Mastery assignments to be evaluated by
external evaluators within e-portfolio

18. Annual Outcomes Report Rubric
Introductory Standard Practice Standard Mastery Standard
1. Leverage
managerial
effectiveness
through
recognition of
individual
strengths, values
and leadership
strategies.
Student has a good
understanding of his/her
personal strengths, a real
appreciation of the difference
in others, and is beginning to
understand how these
characteristics influence team
development. Recognizes the
value of working within
diverse, cross-functional
teams but not quite sure on
how to leverage good team
management strategies or
practices. Commits to
achievement of common
goals when working on a
team.
Student starts to leverage
thier own strengths and
those of team members to
better develop team
communication, high ethical
working standards, trust, and
a climate of cooperation and
collaborative problem solving
amongst team members in
order to acheive objectives of
the project. Interacts and
cooperates productively and
maturely with others.
Accomodates the protocols
and expectations of teams.
Selects appropriate media fo
dissemination or
accumulation of information.
Places information in
appropriate context when
listening, reading, writing,
and speaking. Accepts
suggestions and guidance of
team leaders and other
members.
Student leverages their own
strengths and turns their
weaknesses into
opportunities for team
development to manage
projects, co-located teams,
and/or virtual teams.
Student also leverages their
teammates' strengths and
weaknesses into
opportunities for coaching
and mentoring others in
appropriate circumstances.
Accepts both leadership and
participation roles gracefully.
Addresses challenges with
the team as they occur and
works to an acceptable and
ethical resolve with input
from each team member.
Realizes and communicates
the value each member
brings to the team.

19. WAIT!!
That’s a lot of assignments!
How is it Sustainable?

20. Conceptual Framework
Emphasis on Indirect Evidence
of Student Learning
• Evidence at the first two levels – Introductory and Practice –
tracks student growth towards program learning outcomes
• Course-level assessments and their aligned course outcomes
build toward the Mastery summative assessments and
achievement of program learning outcomes
• Assessed formatively, receiving direct feedback from
students and faculty within e-portfolio

21. Conceptual Framework
Emphasis on Direct Evidence
of Student Learning
• Each program identifies major summative assessments
that provide direct evidence of students Mastering
program learning outcomes
• Mastery assessments can be used to assess multiple
program learning outcomes
• Assessed by review team using program learning outcome
rubrics embedded in e-portfolio

22. FOLIO180 – CITYU’S E-PORTFOLIO
SOLUTION
Student Learning Outcomes Assessment

23. Analysis of CityU’s Outcomes Assessment
Approach
• Strengths:
– Emphasis on authentic assessment and relevance for
students
– Integration of CityU, program, and course outcomes
– Widespread faculty support and engagement
– Embeds student feedback and focus on outcomes
• Challenges:
– Increased complexity with new e-portfolio system
– Multiple changes to procedure within a short time frame
– Embed what is learned during pilot year to enhance
process for all programs