We Are All Accountable: StudentLearning Outcomes for Student Services NCSD National Conference Denver, CO Oct. 22-24, 2009 Elise Davis-McFarland, Vice President for Student Services Sharon Kearns, M.Ed., Director of Testing
Student Affairs professionals have the responsibility for ensuring that institutions of higher education become true learning communities committed to providing transformative educational experiences for all students. Learning Reconsidered – ACPA, NASPA
The institution provides student supportprograms, services, and activities consistent with its mission that promote studentlearning and enhance the development of its students. (Student Support Services)
Norht North Central Association Commission on Accreditation and SchoolImprovement Learning goals are designed to press all students to excellence and focus on enhancing the intellectual, personal, physical, social, and career development of students. PS 1 The learning goals are challenging, are appropriate in terms of rigor and diversity, meet student needs, and are consistent with the stated goals of the program. PS 2 Objectives are: identified for the specific subject areas and the various programs such as the activity programs, and
Different Outcomes Student service Satisfaction, needs, program quality Student development Values, attitudes, beliefs, psychosocial development Learning Knowledge and skills
Why Student Learning Outcomes? The business of colleges is teaching and learning. If you’re not a player, you’re just sitting on the sidelines.
You need a frame of reference1. Learning Reconsidered - ACPA/NASPA Cognitive complexity Knowledge, acquisition, integration, application Humanitarianism Civic engagement Interpersonal and intrapersonal competence Practical competence Persistence and academic achievement
You need a frame of reference2. Principles of Good Practice for Student Affairs- ACPA/NASPA Active learning Values & ethical standards High expectations for student learning Systematic inquiry Effective resource use Educational partnerships Community
Assessment should show thatStudent Services Plays a significant role in student learning Makes a difference in academic success Promotes retention and persistence Enhances cognitive development Enhances affective development
Brainstorming & Consensus How about…? How about? What if we…? Hm m.. Hmm..
What if a student asks… What should I be able to do as a result of my interaction with the______ office in Student Services? What will I learn from meetings with my financial aid officer? What would you say?What do you expect students to be able to do/know as a result of their interactions or participation
What should students gain fromStudent Services? Essential life skills Personal responsibility Perseverance Decision making/critical thinking Financial responsibility Technology use
What should students gainfrom Student Services? Essential social skills Effective communication Interpersonalskills Civic mindedness Ethical behavior Adherence to legal and ethical requirements
Planning Pyramid Dept Plans Student Affairs Divisional Plan College Strategic Plan Institutional Mission Davis-McFarland, E.
Mission Pyramid Dept. Mission ) Unit Mission (EOP, EM, SD) Division Mission Institutional Mission Adapted from Bentrim-Tapio, 2008
Mission Statements: Before As part of the Student Services Division of Trident Technical College, the staff members of the Financial Aid/Veterans Assistance Office are committed……. Trident Technical College Registrar’s office mission is to provide the best possible service….. The Educational Talent Search program will sustain a supportive and…… The mission of Career and Employment Services is to prepare students for the transition from…… Student Activities’ mission is to complement the academic program and enhance the students’ college…. What’s missing?
BEFOREMission Statement Example Orientation Services Our Purpose... The mission of Orientation Services at Trident Technical College is to provide a welcoming environment in which new students and their families obtain information necessary for a successful transition into college life. Through our orientation program, we encourage campus involvement, academic success, and student retention. Our aim is to integrate new students and families into campus life while easing the transitions and stresses associated with starting college. In order to achieve these objectives, students and families who attend an orientation will….. View a video to familiarize them with the campus environment and physical facilities Explore Trident Technical College’s Website Talk with an Orientation Leader to Verify the student’s major Learn about important processes, like registration Be assigned an academic advisor Receive important printed materials about the College Find out about student services and organizations Have questions answered Take a walking tour of the campus, if desired Register for courses during the registration period in the Orientation Services Davis-McFarland, E. Centers if they are non-degree students
AFTER Orientation ServicesIn support of the Student Services Divisionmission of encouraging student success,Orientation Services provides acomprehensive orientation program toencourage campus involvement, academicsuccess, and student retention.
A good learning outcome Indicates what the student should know or be able to do Measurable Related to experiences you provide Realistic Aligns with vision, mission and values Adapted from Bentrim-Tapio, 2008
Learning Outcome Statements Specify what students will be able to do, know, and/or demonstrate as a result of participating in or completing an activity/service/program. Should be expressed as knowledge, skills, attitudes or behaviors. Adapted from Bentrim-Tapio, 2008
We always use verbs Bloom’s Taxonomy Describe Demonstrate Develop Identify Difficult to Measure Appreciate Aware of Familiar with
The wording…. Active Verb EXPECTATION As a result of • Demonstrate • 50% improvement • Average performance • Identify • Pre-test post-test • Explain improvementBEHAVIOR INTENTION• Meeting Will be able to Mock• Participating interview• Working with • Time managementACTIVITY Who• Program • Students• Service • Athletes• Activity • Officers Adapted from Bentrim-Tapio
Learning Outcome Examples As a result of students attending financial aid workshops and receiving financial aid education, the student default rate will not exceed 5%. As a result of instruction from the test proctor, students will manage their time in order to complete their tests within the allotted time. As a result of attending officer training sessions and holding office, organization officers will demonstrate improvement in their leadership skills from the beginning of their tenure through completion.
The Assessment Cycle GroundworkDetermine findings Consensususe to enhance &improve Form SLO Committee Mission Objectives Revise mission statementsAnalyze & report Goalsfindings Outcomes Develop surveys, forms, rubrics, etc. Collect data (develop schedule)
Satisfaction vs. Learning is much like the relationship between Reliability and Validity.Reliability means nothingwhen the problem is validity,satisfaction isn’t tooimportant if students don’tlearn.
Satisfaction surveys don’t help you understand all of the contributions of your programs & services don’ttell you how your programs and services contribute to student development & learning don’t tell you everything you need to know about programs & service improvements Bresciani, 2002
Satisfaction 85% of students will agree or strongly agree that mock interviews helped them feel more confident in their interviewing skills. Adapted from Bresciani, 2002
Sir, we’ve had a paradigm shift since you left.
Learning 85% of students who attend the How to Interview workshop will score average or above average on a mock interview evaluation matrix.
Student Services LearningOutcomes Shows Academics that Student Services makes significant contributions to learning Forms the basis for collaborations that Promote measurable student learning Promote student engagement and socio- academic integration Provides models for indirect learning Bresciani, 2002
The Student Services Division encourages student success by providing programs and services that support students’ accomplishment of their academic, student leadership, and social development goals. All student services programs, initiatives, and services are proved in keeping with the principles of good practice for student affairs and services.2006-2008 Student Services Program Assessment The Enrollment Management Unit provides an organizational framework and leadership that facilitates cohesion and coordination between and among its component admissions, financial aid, orientation, registrar, and testingDepartment: Orientation Services services. We intend to be reliable and effective partners in helping our facultyDirector: Brandon Branham and fellow staff members accomplish their work with our students from entryDivision: Student Services to exit. We will accomplish our work in a manner that causes other state and regional two-year colleges to recognize us as an enrollment management unitDean: John Jamrogowicz worthy of benchmarking. Orientation Services provides a comprehensive orientation program that assists students in achieving both their academic and developmental goals through a comprehensive orientation program. We encourage campus involvement, academic success, and student retention. Our aim is to integrate new students into campus life while easing the transitions and stresses associated with starting college. Assessment Summary Key Findings Plan for Improvement Implementation The assessment provided information on The department will work on improving Orientation Leaders are trained to instruct student learning outcomes in the areas of students’ website knowledge and on gathering students on the importance of website advising session preparation, TTC website accurate results for information retention in knowledge. knowledge, and information retention. orientation sessions. Areas of strength for the department are in advising session preparation. Information retention and website knowledge require follow- up. Davis-McFarland, E.
STUDENT LEARNING OUTCOMES FOR PROGRAM REVIEWDepartment: Orientation Services Division: Student Services Year: 2006- 2007A. Student B. Where C. Methods for D. Expected Level E. Data Collection F. Results G. Plan For ImprovementLearning Outcomes are Outcomes of ProgramOutcomes Assessed Assessment PerformanceWhat should Where do you see How does your What is the expected How and when will What are the results How will you use this informationstudents who evidence that the program evaluate level of student you collect the data of the evaluation? to improve the programinteract with your student can do student/graduate performance for the needed to evaluateservice area be these things? skills/abilities? program? the performance ofable to do? (1 the program?outcome perblock)Students Created new Matrix > 80% of the Review of pre/post- 100% of students Actions:demonstrate an student matrix to administered to students will test scores from correctly answeredability to be used by students prior to correctly answer student matrix post-test questions 1. During orientation sessions,adequately Orientation and after individual post-test questions. administered emphasis will be placedprepare for an Leaders for orientation Post-test scores will October 2007 - more heavily upon theacademic advising pre/post-test data sessions. improve ≥20%. November 2007. importance of preparingsession. collection (pre/post-test) adequately for an advising regarding advisor session. visits. Advisor Feedback Advisee evaluation ≥60% of students Review of advisee 75% of advisees Eliminated student matrix for matrix have acceptable evaluation matrices called or e-mailed to next cycle. Will use original evaluations completed by 10 set up an matrix but will speak directly to faculty advisors for appointment. Advisors and not to students. Priority Registration for Fall 2006 term. 69% of advisees Percentages were met or brought necessary exceeded for Fall 2006. No documents to their change is indicated. advising sessions. 95% of advisees communicated course requirements or preferences. Davis-McFarland, E. 66% of advisees arrived on time for advising sessions.
PROGRAM VITAL STATISTICS Indicator Trend Analysis Orientation Numbers by Calendar Year30002500 2562 24162000 2204 2149 20391500 Mean of 2274 and standard deviation of 2111000 Trend analysis indicates a very strong decreasing500 linear trend in the number of Orientations per year. 0 The reported p-value is 0.003 2003 2004 2005 2006 2007 Davis-McFarland, E.
The Student Services Division encourages student success by providing programs and services that support students’ accomplishment of their academic, student leadership, and social development goals. All student services programs, initiatives, and services are proved in keeping with the principles of good practice for student affairs and services.2007-2008 Student Services Program Assessment The Enrollment Management Unit provides an organizational framework andDepartment: Financial Aid leadership that facilitates cohesion and coordination between and among itsDirector: Ellen Green component admissions, financial aid, orientation, registrar, and testing services. WeDivision: Student Services intend to be reliable and effective partners in helping our faculty and fellow staffDean: John Jamrogowicz members accomplish their work with our students from entry to exit. We willNumber of awards FY 2008: 37,189 accomplish our work in a manner that causes other state and regional two-yearTotal amount awarded for FY 2008: $43,522,825 colleges to recognize us as an enrollment management unit worthy of benchmarking.Default Rate: 5.5% Assessment Summary Key Findings Plan for Improvement Implementation The assessment provided information on Investigate the hiring of a loan default Requested and received approval for the student learning outcomes in the areas of coordinator to work with data from the National position and hired a Loan Default Coordinator. student’s financial responsibility and Student Loan Data System (NSLDS) to educate The coordinator is using resources from NSLDS, demonstrates an understanding of the students on their financial rights and ELM, and national data to provide students with importance of academic progress. responsibilities regarding loan borrowing. information regarding Financial Aid rights and responsibilities in addition to how to borrow Areas of strength are in the financial Review the data to determine if the expected responsibly. responsibility of students who are affected by level of performance should be adjusted in light Return to Title IV (R2T4). The data indicates the of the expanded loan limits from the The department is offering workshops both in need for additional resources regarding loan Department of Education. classes and during day/evening hours to explain eligibility and borrowing responsibilities. both the financial responsibilities of students and the importance of academic progress for continued eligibility for financial aid. Davis-McFarland, E.
STUDENT LEARNING OUTCOMES FOR PROGRAM REVIEWDepartment: Financial Aid Division: Student Services Year: 2007 - 2008A. Student B. Where C. Methods for D. Expected Level E. Data Collection F. Results G. Plan For ImprovementLearning Outcomes are Outcomes of ProgramOutcomes Assessed Assessment PerformanceWhat should Where do you see How does your What is the expected How and when will What are the How will you use this information tostudents who evidence that the program evaluate level of student you collect the data results of the improve the programinteract with your student can do student/graduate performance for the needed to evaluate evaluation?service area be these things? skills/abilities? program? the performance ofable to do? the program?Students Default Rates Review default rates The loan default Review of 5.5% of TTC Actions:receiving rate for the college Department of borrowers are 1. More aggressively contactfinancial aid is ≤5%. Education default in default. students regarding theirdemonstrate rate, March 2007. statusfinancial 2. Utilize national data to trackresponsibility. students 3. Update data on website to help with understanding borrowing rights and responsibilities Loan Exit Students ≥85% of students Students who 3.5% of Actions: Counseling successfully successfully received loan exit students 1. Increase access to national complete loan exit complete loan exit counseling completed data. counseling. counseling within 2 notifications during required 2. Notify students earlier of exit months of the Fall 2007 term counseling requirements notification. completed the within 2 3. Provide direct contact with counseling by months students when they apply for 2/28/08. graduation Borrowing Rates Review average Average loan debt Review Datatel Average Actions: student loan debt of for the graduating reports which borrowing 1. Review and adjust average the graduating class. class does not identify the average per graduate loan borrowing rates due to exceed $5250. loan debt for the is $6898 changes in maximum limits. (Note: The $5250 Spring 2008 2. Review rules regarding was based on graduates packaging of student loans. average four-year 3. Conduct individualized eligibility for loan) counseling to discourage Davis-McFarland, E. excessive borrowing
VITAL STATISTICS Indicator Trend Analysis Total Financial Aid Disbursed by Award Type Loans: Trend analysis indicates a strong increasing (by fiscal year ending June 30) linear trend with a p-value of 0.015. The total loan$20,000,000 amount has increased by 86.6% since 2004.$15,000,000 Grants: Trend analysis indicates a rebounding cubic trend with a p-value of 0.10. The total grant amount has decreased by 4.7% since 2004.$10,000,000 Lottery: Trend analysis indicates a weak decreasing $5,000,000 quadratic trend with a p-value of 0.22. The total lottery amount has increased by 5.6% since 2004. $- Scholarship: Trend analysis indicates a strong 2004 2005 2006 2007 2008 increasing linear trend with a p-value of 0.01. The Loan $11,662,833 $13,454,167 $19,584,212 $19,153,743 $21,774,304 total scholarship amount has increased by 59.4%. Grant $13,618,677 $13,467,035 $12,770,029 $12,100,065 $12,977,851 Lottery $6,900,605 $7,426,861 $7,553,574 $8,182,811 $7,292,341 Work-Study: Trend analysis indicates a weak Scholarship $677,768 $654,815 $802,125 $953,854 $1,080,503 increasing linear trend with a p-value of 0.16. The Work-Study $348,031 $376,733 $400,213 $449,859 $397,826 total work-study amount has increased by 14.3%.Source: TTC Office of Finance and Administration: Comprehensive Annual Financial Report June 30, 2008Updated: March 3, 2009 Davis-McFarland, E.