The document reviews the Wisconsin Technical College System (WTCS) and its compliance with Perkins IV legislation, emphasizing the importance of student attainment of industry-aligned technical skills through performance assessments. It outlines the accountability measures implemented to ensure technical skill attainment, including both direct and indirect assessments, and the integration of industry standards into curricula. Additionally, it discusses the role of third-party assessments in measuring educational effectiveness and guiding program improvements.

1. Perkins IV Assessment of Technical Skill Attainment Wisconsin Technical College System Sandy Schmit WTCS Foundation – WIDS Judy Neill

2. Overview of the Wisconsin Technical College System What does Carl Perkins 2006 (Perkins IV) require? How does curriculum fit in? How is WTCS using performance assessment? What will WTCS require for 3 rd Party Assessments? How can WIDS help? Overview

3. The Wisconsin Technical College System (WTCS)

4. 16 Technical Colleges 349 unduplicated Career and Technical Education Programs 1179 Duplicated Career and Technical Education Programs 390,272 enrolled students in 2008, 69,632 full time equivalent 23,617 Degrees granted in 2008 WTCS

5. How do you know when your students “get it”?

6. Assessment?

7. Summative Assessment?

8. Carl Perkins IV?

9. What does Carl Perkins 2006 (Perkins IV) require?

10. “ Student attainment of challenging career and technical skill proficiencies, including student achievement on technical assessments that are aligned with industry-recognized standards, if available and appropriate.” (U.S. Department of Education, 2006). The Law

11. Focuses on industry aligned skill sets. Students know they are acquiring skills that have value & portability. Increases odds that students will be successful in their employment. Ensures that WTCS is educating students to meet the needs of employers. Big Picture Perspective

12. Increased accountability to improve teaching and learning Accreditation (NCA, AQIP, industry) Carl D. Perkins IV legislation and funding Drivers to Process

13. Students Feedback about skills needed on the job Added value for education Colleges Improvement of teaching and learning Supports overall mission Employers Assurance that WTCS graduates have attained technical skills needed on the job Benefits

14. Carl D. Perkins 2006 Newly revised Career and Technical Education (CTE) Law Key Concept - Demonstrate that program graduates have attained the knowledge and skills that are industry validated career and technical skills

15. Preparation for postsecondary education and employment Preparation not only to get a ‘job,’ but ‘academic and technical preparation’ Increased emphasis on achievement of a degree, certificate or credential Perkins IV Areas of Emphasis

16. Increased Accountability New accountability provisions are a reminder that: Funds are not an entitlement Use of funds is flexible & responsive to the accountability data

17. Separate secondary and postsecondary measures Performance levels at state and local levels Negotiations and redirection of funds Increased Accountability

18. Indirect Measures Data that support inferences about the quality of teaching and learning and give insight into why achievement is high or low: Enrollment Retention rates Course completion Student/graduate/employer satisfaction surveys Placement statistics Direct Measures Data that provide evidence that students have achieved the intended outcomes: Performance assessments with rubrics Portfolios Artifacts Performances Outcome referenced tests Increased Accountability for Direct Measurement of Learning

19. Denominator: number of CTE concentrators who took technical courses during the three year reporting period Prior accountability required only indirect measurement: Numerator: Number of CTE concentrators who passed 80% of the technical courses they attempted during the three year reporting period Measure = Increased Accountability for Direct Measurement of Learning

20. Denominator: Number of CTE concentrators who took technical skill assessments during the reporting year New OVAE Guidance for Perkins IV Measurement requires direct measures of skill attainment: Numerator: Number of CTE concentrators who passed technical skill assessments that are aligned with industry-recognized standards , if available and appropriate, during the reporting year Measure = Increased Accountability for Direct Measurement of Learning

21. Department of Education goal is to have end of program outcomes assessments that are: Direct measures of outcomes Independent 3rd-Party Employer validated By 2015 most WTCS assessments should be external or approved assessments. Increased Accountability for Direct Measurement of Learning

22. WTCS TSA Assessments External TSA Assessment (ex. NCLEX, Barber/Cosmetology, etc.) Indirect Assessments Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning

23. WTCS Assessment Standards WTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation. May be developed and/or implemented by the state system. May be developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval. Must be valid and reliable. External Assessment Standards Indirect Assessment Standards Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning

24. WTCS Assessment Standards External Assessment Standards (ex. NCLEX, ASE, Barber/Cosmetology) External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation. Indirect Assessment Standards Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning

25. WTCS Assessment Standards External Assessment Standards Indirect Assessment Standards Non-assessment related indicators such as indirect measures. GPA and/or Course Completion Program Completion – Graduation Teacher-developed exams that do not meet WTCS standard (not WTCS approved) Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning

26. WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments New Accountability for Direct Measurement of Learning WTCS Assessment External Assessment Indirect 2012-2013

27. How does curriculum fit in?

28. Plan Establish outcomes Build assessments Design learning DO Deliver Assess Document CHECK Analyze assessment data Propose improvements ACT Adjust design Adjust delivery Adjust assessment Assessment Process and Curriculum Curriculum

29. WIDS Model What do they need to be able to achieve? Who are the learners? How will we know and show when they’ve achieved it? How will they get there? 3 of 4 components answer assessment questions WHO WHAT HOW

30. Program Outcomes Measurable, observable, and field-specific skills – major outcomes Number 5-7 per program (guideline not rule) Originate from: Current DACUMs Accrediting Agencies National (or other) Skill Standards Advisory Committees Threaded through courses Performance verified with summative assessment of skill performance

31. Integrating Outcomes Linked Competencies (performance assessments) Exit Learning Outcomes Program Outcomes (summative assessment) Core Abilities (summative assessment) Gen Ed Outcomes (summative assessment) Job Task Analysis (DACUM) External Standards (Research-based) Linked Competencies (performance assessments) Linked Competencies (performance assessments) Formative assessment of program outcome Summative assessment of learning outcomes External Program-Level Course-Level Drives Learning Design Assesses Learning Results

32. External Program-Level Course-Level Dental Hygiene Example Graduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services. Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist . ADA Dental Hygiene Education Standard Dental Hygiene Program Outcome Dental Hygiene Competency Drives Learning Design Assesses Learning Results Formative assessment of program outcome Summative assessment of Learning outcomes

33. Summative/Formative Assessment

34. Sound Assessment Valid Outcomes based on standards (industry) Measures intended outcomes Measures application and critical thinking Reliable Performance assessment based on consistent rubrics, scoring guides, and rating scales Consistent process - each learner is assessed in same way as other learners Fair Valid Reliable Learners informed of expectations up front! Feedback to learners

35. Standards for WTCS Approved TSA Assessments

36. WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments WTCS Assessment External Assessment Indirect 2012-2013 New Accountability for Direct Measurement of Learning

37. is valid and reliable; meets WTCS standards/guidelines; is approved by industry (advisory committee); is developed and/or implemented collaboratively by the WTCS colleges; or developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval. WTCS Assessment Standards WTCS approved assessment objectively measures student attainment of industry recognized skills upon graduation and:

38. Available in your handout packet

39. WTCS Standards for 3 rd -Party External Assessments

40. 3 rd Party TSA Also referred to as: External Direct Method Credential License Generally controls entrance into an occupation

41. Validity / Reliability Alignment to Standards Availability of data Value to the Student Development / Administrative Costs Pooling / Sharing Resources Considerations for selecting 3 rd Party External Assessments

42. Can we obtain the results/data? Are cost and administration feasible? Does it add value or control entry to the student/occupation? Is it valid, reliable, reputable, recognized? Is it summative, cumulative or a partial measurement of skills? Considerations for selecting 3 rd Party External Assessments

43. Does the assessment align with program outcomes? Does the assessment measure what needs to be measured? Does the data help improve and strengthen our programs? Considerations for selecting 3 rd Party External Assessments

44. WTCS 3 rd Party Assessments Currently in Place Occupations regulated by the Department of Regulation and Licensing Barber/Cosmetology Funeral Director LPN Registered Nurse Dental Hygienist Massage Therapist Occupational Therapy Assistant Physical Therapy Assistant Respiratory Care Practitioner Veterinary Technician Wisconsin Registered Interior Designer

45. Guidelines for Selecting 3 rd Party Assessments This is a guiding document to assist districts in determining direct technical skill assessment appropriateness. Districts will work with the state office staff in the determination. Keep in mind this is for a small number of programs.

46. Guidelines for Selecting 3 rd Party Assessments

47. Concerns: Only measures part of the program outcomes-”All Code-No Application” Does not indicate job performance Does it help employers? Focused only on technical skills, what about academic skills and soft skills? Guidelines for Selecting 3 rd Party Assessments

48. 3 rd Party Assessment - Example Automotive Service Excellence (ASE) Long-standing credential of automotive technicians A series of 8 certification areas/assessments Would it work for Perkins?

49. Logistics Professional CPL/DL-International Society of Logistics CPIM/CSCP-Association for Operations Management PLS/CTL/DLP conferred by the American Society of Transportation & Logistics 3 rd Party Assessment - Example

50. Federal Aviation Administration (FAA) Airframe and Powerplant Mechanic Certificate Administered by the FAA Written, practical and oral examination Required for entry 3 rd Party Assessment - Example

51. How Can WIDS Help?

52. The Learning & Assessment Design Technology Tool Kit TracDat Blackboard iWebfolio eFolio Questionmark People Soft Banner Datatel

53. Create/maintain Program Outcome Summaries for all new and existing programs. Develop/maintain Course Outcome Summaries for all courses. Design Performance Assessment Tasks for course level (formative) and summative assessments. Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses. Present WIDS documentation as a central feature in accreditation. How can WIDS help? WIDS Model and Software Tools:

54. Facilitate collaborative development of: DACUM or Industry Review Focus Groups Program Outcomes Program Outcome Summaries Summative Performance Assessments Produce Industry Approval Reports Analyzer Reports Updated WIDS files and reports How can WIDS help? WIDS Team

55. How do we communicate WTCS long-standing commitment to industry input and standards? How do we ensure that we use quality assessments? (That we don’t regress to testing regurgitation of information?) How do we ensure that we actually are doing what our model and assessment plans imply? How do we collect and report data? Challenges

56. Current Projects

57. Automotive Technology/Technician NATEF end of program evaluation Accounting WIDS Early Childhood WIDS Criminal Justice WIDS Welding WIDS Current Projects