Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012


Published on

Presentation given by Dr. Mary Beth Hanner, provost and chief academic officer of Excelsior College, and Dr. Patrick Jones, vice provost, at the 2012 Association of Test Publishers Conference in Berlin, Germany. Hanner, and Jones’ presentation, “Using Technology to Close the Loop on Student Learning Assessment” focused on the variety of emerging tools on the market, best practices, and future trends in assessments.

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Excelsior mission consistent withfocus of the Grundtvigprogramme for adult learners established as part of the Lifelong Learning Programme 2007–2013 developed by the European Union. The Grundtvigprogramme encompasses all types of learning, whether these take place in the 'formal' or 'non-formal' system of education for adults, or in more 'informal' ways, such as autonomous learning, community learning or experiential learning.
  • Very similar to Grundtvigprogramme of EU initiative. Also knowledge/skills recognized via professional certification in nursing, business, technology.
  • Council for Higher Education Accreditation similar to International Education Accreditation Commission, except IEAC focuses exclusively on distance education programs for working adults. CHEA recognizes 7 regional accrediting bodies in the US along with specialized accrediting organizations such as NLNAC, ABET and IACBE.
  • Reference massively online courses (MOOC), challenges in verifying student identity and authentication of student work in MOOC/distance learning,
  • Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012

    1. 1. Using Technology to Close the Loop on StudentLearning AssessmentMary Beth Hanner, Ph.D, RN, ANEF Patrick Jones, Ph.D.Provost and Chief Academic Officer Vice ProvostExcelsior College, Albany, NY - USA
    2. 2. Purposeo Describe context and framework for student learning assessment at a post-traditional degree completion institution.o Provide examples of key assessments conducted at institutional, program and course levels.o Show how technology is used to improve the rigor and effectiveness of student learning assessments.
    3. 3. Backgroundo Excelsior College founded in 1971 (previously Regents College).o In 1998, chartered as a private, independent college.o Large adult student body (approximately 33,000 students).o Distance education, flexible learning format. Philosophy: “What you know is more important that where or how you learned it”
    4. 4. Backgroundo Valuing prior learning is a central premise for adult learners (Green, 1998).o Acceptance of a variety of credit sources: Examples: • Regionally accredited institutions • ACE/PONSI approved courses • Exams – CLEP, DANTES,EC • Portfolio assessment for prior learning • Online courses from Excelsior and preferred providerso Residency requirement (presently minimal, historically none).
    5. 5. Backgroundo Accreditation • Middle States (all programs) • NLNAC (specialized accreditation - Nursing) • ABET (specialized accreditation - Technology) • IACBE (specialized accreditation - Business)
    6. 6. Challenge: Who is a typical student? o Career oriented working adult; no 1st time, full- time students. o Transfer credits: 25-75% of degree program. College 2 College 1 Excelsior Average age: 39 At a distance College yearCollege 3 College 5 College 4
    7. 7. Accountability and Continuous Improvement OAIR Steering Committee School of School of Business & Liberal Arts Tech OAIR Office School of School of Health Nursing Sciences Institutional Assessment Plan for Student Learning (IAPSL)
    8. 8. Institutional Assessment Plan for Student Learning (IAPSL) & Institutional Effectiveness Plan (IEP)o Provide frameworks for assessment of learning outcomes at course, program and institutional level and institutional effectiveness, respectively.o Address the key question “How effectively do we deliver on our mission and make a distinctive impact relative to our resources?” (Collins, 2011).o IAPSL and IEP
    9. 9. Assessment Process at Schools Establish clear learning outcomes Provide learningUse the Results opportunities Assess Student Learning
    10. 10. Institutional Assessmentso ETS Proficiency Profile™: internet-based test to measure general education outcomeso Internet-based alumni & supervisor surveys to assess student satisfaction & career progressiono National Survey of Student Engagement: internet-based measure of baccalaureate student participation
    11. 11. Institutional Assessmentso Priorities Surveys for Online Learners: web- based measure of student satisfactiono Service Quality Survey: internet-based assessment of student perceptions of Excelsior College service quality
    12. 12. Program-level Assessmentso Licensure and certification exams for professional degree programs (e.g., NCLEX® for nurses, Uniform CPA Examination for accountants)o Capstone courses and assessments for all graduateso Alumni and supervisor surveys as indirect measures of program outcome mastery
    13. 13. Course-level Assessmentso Discussion posts, assignments, quizzes, exams, simulations and essays administered in BlackBoard 9.1 Learning Management Systemo Credit-by-exams administered at Pearson VUE testing centers
    14. 14. Course-level Assessmentso Credit for prior learning (i.e., portfolio assessment) to recognize college-level learning achieved through independent study, workplace training and other means
    15. 15. Compiling & Reporting Assessment Resultso Challenge: frequency and scope of assessment activities yield large amounts of datao Need technology tool to organize and report assessment results for decision makerso TracDat™ selected as software tool for data storage and reporting
    16. 16. Example of TracDatTM Report School of Health Sciences Outcomes Means of Assessment and Results Action and Follow- Criteria upAnalyze legal, HSC 431 Project Analysis of 57% (28/49) of Area of concern -ethical and Oregon Health Plan students received problem area waspolicy issues a grade of B or student’s inabilitywithin health Criterion: 75% of the higher – Needs to propose newcare delivery students will receive a improvement approaches tosystems. grade of B or better on this effectively control project. health care costs. Assessment modified to compare and contrast Oregon plan with another selected state.
    17. 17. Technology Tools to Increase Student Persistenceo Challenge: faculty need mechanisms to track student progress in courses and intervene with students who are not persistingo Starfish Early Alert™ system integrated in Excelsior courses
    18. 18. Technology Tools to Increase Student Persistenceo System identifies students who are not persisting based on login activity, failing to submit assignments and/or low gradeso Faculty receive notifications of students who exhibit these behaviors and intervene to improve persistence
    19. 19. Technology Tools to Verify Student Identityo BlackBoard 9.1 requires student user name and password to access courseso Excelsior has piloted use of Acxiom challenge question system to verify student identity
    20. 20. Technology Tools to Verify Student Identityo Acxiom system uses student’s name, birth date and address to select and administer multiple-choice questions (e.g., Which of the following was your address in 2008?)o Access to course content and assessments limited to students who answer challenge questions correctly
    21. 21. Technology Tools to Detect Plagiarismo Excelsior uses to perform an originality check of student essays and papers to detect plagiarismo compares student work to databases of papers and publications to determine if plagiarism is evident
    22. 22. Future Directionso Increased use of biometrics for student identity verificationo Integration of adaptive models for instruction and assessmento Reliance on open educational resources to improve access
    23. 23. Questions?Thank you for attending!
    24. 24. Thank You!Please Complete Your Evaluation.