Presentation given by Dr. Mary Beth Hanner, provost and chief academic officer of Excelsior College, and Dr. Patrick Jones, vice provost, at the 2012 Association of Test Publishers Conference in Berlin, Germany. Hanner, and Jones’ presentation, “Using Technology to Close the Loop on Student Learning Assessment” focused on the variety of emerging tools on the market, best practices, and future trends in assessments.

1. Using Technology to Close the Loop on Student
Learning Assessment
Mary Beth Hanner, Ph.D, RN, ANEF Patrick Jones, Ph.D.
Provost and Chief Academic Officer Vice Provost
Excelsior College, Albany, NY - USA

2. Purpose
o Describe context and framework for student
learning assessment at a post-traditional degree
completion institution.
o Provide examples of key assessments
conducted at institutional, program and course
levels.
o Show how technology is used to improve the
rigor and effectiveness of student learning
assessments.

3. Background
o Excelsior College founded in 1971 (previously
Regents College).
o In 1998, chartered as a private, independent
college.
o Large adult student body (approximately 33,000
students).
o Distance education, flexible learning format.
Philosophy:
“What you know is more important that where or
how you learned it”

4. Background
o Valuing prior learning is a central premise for adult learners
(Green, 1998).
o Acceptance of a variety of credit sources:
Examples:
• Regionally accredited institutions
• ACE/PONSI approved courses
• Exams – CLEP, DANTES,EC
• Portfolio assessment for prior learning
• Online courses from Excelsior and preferred providers
o Residency requirement (presently minimal, historically none).

5. Background
o Accreditation
• Middle States (all programs)
• NLNAC (specialized accreditation - Nursing)
• ABET (specialized accreditation - Technology)
• IACBE (specialized accreditation - Business)

6. Challenge:
Who is a typical student?
o Career oriented working adult; no 1st time, full-
time students.
o Transfer credits: 25-75% of degree program.
College 2 College 1
Excelsior
Average age: 39
At a distance College
year
College 3 College 5
College 4

7. Accountability and Continuous Improvement
OAIR Steering Committee
School of
School of
Business &
Liberal Arts
Tech
OAIR
Office
School of School of
Health Nursing
Sciences
Institutional Assessment Plan for Student Learning (IAPSL)

8. Institutional Assessment Plan for
Student Learning (IAPSL) & Institutional
Effectiveness Plan (IEP)
o Provide frameworks for assessment of learning
outcomes at course, program and institutional
level and institutional effectiveness, respectively.
o Address the key question “How effectively do we
deliver on our mission and make a distinctive
impact relative to our resources?” (Collins, 2011).
o IAPSL and IEP links:
www.excelsior.edu/IAPSL
www.excelsior.edu/IEP

9. Assessment Process at Schools
Establish clear
learning outcomes
Provide learning
Use the Results
opportunities
Assess Student
Learning

10. Institutional Assessments
o ETS Proficiency Profile™: internet-based test to
measure general education outcomes
o Internet-based alumni & supervisor surveys to
assess student satisfaction & career progression
o National Survey of Student Engagement:
internet-based measure of baccalaureate
student participation

11. Institutional Assessments
o Priorities Surveys for Online Learners: web-
based measure of student satisfaction
o Service Quality Survey: internet-based
assessment of student perceptions of Excelsior
College service quality

12. Program-level Assessments
o Licensure and certification exams for
professional degree programs (e.g.,
NCLEX® for nurses, Uniform CPA
Examination for accountants)
o Capstone courses and assessments for all
graduates
o Alumni and supervisor surveys as indirect
measures of program outcome mastery

13. Course-level Assessments
o Discussion posts, assignments, quizzes,
exams, simulations and essays
administered in BlackBoard 9.1 Learning
Management System
o Credit-by-exams administered at Pearson
VUE testing centers

14. Course-level Assessments
o Credit for prior learning (i.e., portfolio
assessment) to recognize college-level
learning achieved through independent
study, workplace training and other means

15. Compiling & Reporting Assessment
Results
o Challenge: frequency and scope of
assessment activities yield large amounts of
data
o Need technology tool to organize and report
assessment results for decision makers
o TracDat™ selected as software tool for data
storage and reporting

16. Example of TracDatTM Report
School of Health Sciences
Outcomes Means of Assessment and Results Action and Follow-
Criteria up
Analyze legal, HSC 431 Project Analysis of 57% (28/49) of Area of concern -
ethical and Oregon Health Plan students received problem area was
policy issues a grade of B or student’s inability
within health Criterion: 75% of the higher – Needs to propose new
care delivery students will receive a improvement approaches to
systems. grade of B or better on this effectively control
project. health care costs.
Assessment
modified to
compare and
contrast Oregon
plan with another
selected state.

17. Technology Tools to Increase
Student Persistence
o Challenge: faculty need mechanisms to
track student progress in courses and
intervene with students who are not
persisting
o Starfish Early Alert™ system integrated in
Excelsior courses

18. Technology Tools to Increase
Student Persistence
o System identifies students who are not
persisting based on login activity, failing to
submit assignments and/or low grades
o Faculty receive notifications of students
who exhibit these behaviors and intervene
to improve persistence

19. Technology Tools to Verify Student
Identity
o BlackBoard 9.1 requires student user
name and password to access courses
o Excelsior has piloted use of Acxiom
challenge question system to verify
student identity

20. Technology Tools to Verify Student
Identity
o Acxiom system uses student’s name, birth
date and address to select and administer
multiple-choice questions (e.g., Which of
the following was your address in 2008?)
o Access to course content and
assessments limited to students who
answer challenge questions correctly

21. Technology Tools to Detect
Plagiarism
o Excelsior uses Turnitin.com to perform an
originality check of student essays and
papers to detect plagiarism
o Turnitin.com compares student work to
databases of papers and publications to
determine if plagiarism is evident

22. Future Directions
o Increased use of biometrics for student
identity verification
o Integration of adaptive models for
instruction and assessment
o Reliance on open educational resources to
improve access

23. Questions?
Thank you for attending!

24. Thank You!
Please Complete Your Evaluation.