Chair Academy Perkins Iv Sandy

937 views

Published on

This is the powerpoint from the presentation I gave at the Chair Academy in Nashville on Technical Skill Attainment in Wisconsin

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
937
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Roe – Expand with welcome.
  • Chair Academy Perkins Iv Sandy

    1. 1. Perkins IV Assessment of Technical Skill Attainment Wisconsin Technical College System Sandy Schmit WTCS Foundation – WIDS Judy Neill
    2. 2. <ul><li>Overview of the Wisconsin Technical College System </li></ul><ul><li>What does Carl Perkins 2006 (Perkins IV) require? </li></ul><ul><li>How does curriculum fit in? </li></ul><ul><li>How is WTCS using performance assessment? </li></ul><ul><li>What will WTCS require for 3 rd Party Assessments? </li></ul><ul><li>How can WIDS help? </li></ul>Overview
    3. 3. The Wisconsin Technical College System (WTCS)
    4. 4. <ul><ul><li>16 Technical Colleges </li></ul></ul><ul><ul><li>349 unduplicated Career and Technical Education Programs </li></ul></ul><ul><ul><li>1179 Duplicated Career and Technical Education Programs </li></ul></ul><ul><ul><li>390,272 enrolled students in 2008, 69,632 full time equivalent </li></ul></ul><ul><ul><li>23,617 Degrees granted in 2008 </li></ul></ul>WTCS
    5. 5. How do you know when your students “get it”?
    6. 6. Assessment?
    7. 7. Summative Assessment?
    8. 8. Carl Perkins IV?
    9. 9. What does Carl Perkins 2006 (Perkins IV) require?
    10. 10. <ul><ul><li>“ Student attainment of challenging career and technical skill proficiencies, including student achievement on technical assessments that are aligned with industry-recognized standards, if available and appropriate.” (U.S. Department of Education, 2006). </li></ul></ul>The Law
    11. 11. <ul><ul><li>Focuses on industry aligned skill sets. </li></ul></ul><ul><ul><li>Students know they are acquiring skills that have value & portability. </li></ul></ul><ul><ul><li>Increases odds that students will be successful in their employment. </li></ul></ul><ul><ul><li>Ensures that WTCS is educating students to meet the needs of employers. </li></ul></ul>Big Picture Perspective
    12. 12. <ul><li>Increased accountability to improve teaching and learning </li></ul><ul><li>Accreditation (NCA, AQIP, industry) </li></ul><ul><li>Carl D. Perkins IV legislation and funding </li></ul>Drivers to Process
    13. 13. <ul><li>Students </li></ul><ul><ul><li>Feedback about skills needed on the job </li></ul></ul><ul><ul><li>Added value for education </li></ul></ul><ul><li>  Colleges </li></ul><ul><ul><li>Improvement of teaching and learning </li></ul></ul><ul><ul><li>Supports overall mission </li></ul></ul><ul><li>  Employers </li></ul><ul><ul><li>Assurance that WTCS graduates have attained technical skills needed on the job </li></ul></ul>Benefits
    14. 14. Carl D. Perkins 2006 <ul><li>Newly revised Career and Technical Education (CTE) Law </li></ul><ul><li>Key Concept - Demonstrate that program graduates have attained the knowledge and skills that are industry validated career and technical skills </li></ul>
    15. 15. <ul><ul><li>Preparation for postsecondary education and employment </li></ul></ul><ul><ul><li>Preparation not only to get a ‘job,’ but ‘academic and technical preparation’ </li></ul></ul><ul><ul><li>Increased emphasis on achievement of a degree, certificate or credential </li></ul></ul>Perkins IV Areas of  Emphasis
    16. 16. Increased Accountability <ul><li>New accountability provisions are a reminder that: </li></ul><ul><ul><li>Funds are not an entitlement </li></ul></ul><ul><ul><li>Use of funds is flexible & responsive to the accountability data </li></ul></ul>
    17. 17. <ul><ul><li>Separate secondary and postsecondary measures </li></ul></ul><ul><ul><li>Performance levels at state and local levels </li></ul></ul><ul><ul><li>Negotiations and redirection of funds </li></ul></ul>Increased Accountability
    18. 18. <ul><li>Indirect Measures </li></ul><ul><li>Data that support inferences about the quality of teaching and learning and give insight into why achievement is high or low: </li></ul><ul><ul><li>Enrollment </li></ul></ul><ul><ul><li>Retention rates </li></ul></ul><ul><ul><li>Course completion </li></ul></ul><ul><ul><li>Student/graduate/employer satisfaction surveys </li></ul></ul><ul><ul><li>Placement statistics </li></ul></ul><ul><li>Direct Measures </li></ul><ul><li>Data that provide evidence that students have achieved the intended outcomes: </li></ul><ul><ul><li>Performance assessments with rubrics </li></ul></ul><ul><ul><li>Portfolios </li></ul></ul><ul><ul><li>Artifacts </li></ul></ul><ul><ul><li>Performances </li></ul></ul><ul><ul><li>Outcome referenced tests </li></ul></ul>Increased Accountability for Direct Measurement of Learning
    19. 19. <ul><ul><li>Denominator: number of CTE concentrators who took technical courses during the three year reporting period </li></ul></ul>Prior accountability required only indirect measurement: <ul><ul><li>Numerator: Number of CTE concentrators who passed 80% of the technical courses they attempted during the three year reporting period </li></ul></ul>Measure = Increased Accountability for Direct Measurement of Learning
    20. 20. <ul><ul><li>Denominator: Number of CTE concentrators who took technical skill assessments during the reporting year </li></ul></ul>New OVAE Guidance for Perkins IV Measurement requires direct measures of skill attainment: <ul><ul><li>Numerator: Number of CTE concentrators who passed technical skill assessments that are aligned with industry-recognized standards , if available and appropriate, during the reporting year </li></ul></ul>Measure = Increased Accountability for Direct Measurement of Learning
    21. 21. <ul><li>Department of Education goal is to have end of program outcomes assessments that are: </li></ul><ul><ul><li>Direct measures of outcomes </li></ul></ul><ul><ul><li>Independent </li></ul></ul><ul><ul><li>3rd-Party </li></ul></ul><ul><ul><li>Employer validated </li></ul></ul><ul><ul><ul><li>By 2015 most WTCS assessments should be external or approved assessments. </li></ul></ul></ul>Increased Accountability for Direct Measurement of Learning
    22. 22. <ul><li>WTCS TSA Assessments </li></ul><ul><li>External TSA Assessment </li></ul><ul><ul><li>(ex. NCLEX, Barber/Cosmetology, etc.) </li></ul></ul><ul><li>Indirect Assessments </li></ul>Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning
    23. 23. <ul><li>WTCS Assessment Standards </li></ul><ul><ul><ul><li>WTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation. </li></ul></ul></ul><ul><ul><ul><ul><li>May be developed and/or implemented by the state system. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>May be developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Must be valid and reliable. </li></ul></ul></ul></ul><ul><li>External Assessment Standards </li></ul><ul><li>Indirect Assessment Standards </li></ul>Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning
    24. 24. <ul><li>WTCS Assessment Standards </li></ul><ul><li>External Assessment Standards (ex. NCLEX, ASE, Barber/Cosmetology) </li></ul><ul><ul><li>External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation. </li></ul></ul><ul><li>Indirect Assessment Standards </li></ul>Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning
    25. 25. <ul><li>WTCS Assessment Standards </li></ul><ul><li>External Assessment Standards </li></ul><ul><li>Indirect Assessment Standards </li></ul><ul><ul><li>Non-assessment related indicators such as indirect measures. </li></ul></ul><ul><ul><ul><ul><li>GPA and/or Course Completion </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Program Completion – Graduation </li></ul></ul></ul></ul><ul><ul><ul><li>Teacher-developed exams that do not meet WTCS standard (not WTCS approved) </li></ul></ul></ul>Wisconsin Standards for Assessment - WTCS Increased Accountability for Direct Measurement of Learning
    26. 26. WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments New Accountability for Direct Measurement of Learning WTCS Assessment External Assessment Indirect 2012-2013
    27. 27. How does curriculum fit in?
    28. 28. <ul><li>Plan </li></ul><ul><li>Establish outcomes </li></ul><ul><li>Build assessments </li></ul><ul><li>Design learning </li></ul><ul><li>DO </li></ul><ul><li>Deliver </li></ul><ul><li>Assess </li></ul><ul><li>Document </li></ul><ul><li>CHECK </li></ul><ul><li>Analyze assessment data </li></ul><ul><li>Propose improvements </li></ul><ul><li>ACT </li></ul><ul><li>Adjust design </li></ul><ul><li>Adjust delivery </li></ul><ul><li>Adjust assessment </li></ul>Assessment Process and Curriculum Curriculum
    29. 29. WIDS Model <ul><li>What do they need to be able to achieve? </li></ul><ul><li>Who are the learners? </li></ul><ul><li>How will we know and show when they’ve achieved it? </li></ul><ul><li>How will they get there? </li></ul>3 of 4 components answer assessment questions WHO WHAT HOW
    30. 30. Program Outcomes <ul><li>Measurable, observable, and field-specific skills – major outcomes </li></ul><ul><ul><li>Number 5-7 per program (guideline not rule) </li></ul></ul><ul><li>Originate from: </li></ul><ul><ul><li>Current DACUMs </li></ul></ul><ul><ul><li>Accrediting Agencies </li></ul></ul><ul><ul><li>National (or other) Skill Standards </li></ul></ul><ul><ul><li>Advisory Committees </li></ul></ul><ul><li>Threaded through courses </li></ul><ul><li>Performance verified with summative assessment of skill performance </li></ul>
    31. 31. Integrating Outcomes Linked Competencies (performance assessments) Exit Learning Outcomes Program Outcomes (summative assessment) Core Abilities (summative assessment) Gen Ed Outcomes (summative assessment) Job Task Analysis (DACUM) External Standards (Research-based) Linked Competencies (performance assessments) Linked Competencies (performance assessments) Formative assessment of program outcome Summative assessment of learning outcomes External Program-Level Course-Level Drives Learning Design Assesses Learning Results
    32. 32. External Program-Level Course-Level Dental Hygiene Example Graduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services. Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist . ADA Dental Hygiene Education Standard Dental Hygiene Program Outcome Dental Hygiene Competency Drives Learning Design Assesses Learning Results Formative assessment of program outcome Summative assessment of Learning outcomes
    33. 33. Summative/Formative Assessment
    34. 34. Sound Assessment <ul><li>Valid </li></ul><ul><ul><li>Outcomes based on standards (industry) </li></ul></ul><ul><ul><li>Measures intended outcomes </li></ul></ul><ul><ul><li>Measures application and critical thinking </li></ul></ul><ul><li>Reliable </li></ul><ul><ul><li>Performance assessment based on consistent rubrics, scoring guides, and rating scales </li></ul></ul><ul><ul><li>Consistent process - each learner is assessed in same way as other learners </li></ul></ul><ul><li>Fair </li></ul><ul><ul><li>Valid </li></ul></ul><ul><ul><li>Reliable </li></ul></ul><ul><ul><li>Learners informed of expectations up front! </li></ul></ul><ul><ul><li>Feedback to learners </li></ul></ul>
    35. 35. Standards for WTCS Approved TSA Assessments
    36. 36. WTCS will move toward a balance of WTCS TSA Assessments and External TSA Assessments WTCS Assessment External Assessment Indirect 2012-2013 New Accountability for Direct Measurement of Learning
    37. 37. <ul><ul><ul><ul><li>is valid and reliable; </li></ul></ul></ul></ul><ul><ul><ul><ul><li>meets WTCS standards/guidelines; </li></ul></ul></ul></ul><ul><ul><ul><ul><li>is approved by industry (advisory committee); </li></ul></ul></ul></ul><ul><ul><ul><ul><li>is developed and/or implemented collaboratively by the WTCS colleges; or </li></ul></ul></ul></ul><ul><ul><ul><ul><li>developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval. </li></ul></ul></ul></ul>WTCS Assessment Standards <ul><ul><ul><li>WTCS approved assessment objectively measures student attainment of industry recognized skills upon graduation and: </li></ul></ul></ul>
    38. 38. Available in your handout packet
    39. 39. WTCS Standards for 3 rd -Party External Assessments
    40. 40. 3 rd Party TSA <ul><li>Also referred to as: </li></ul><ul><ul><li>External </li></ul></ul><ul><ul><li>Direct Method </li></ul></ul><ul><ul><li>Credential </li></ul></ul><ul><ul><li>License </li></ul></ul><ul><li>Generally controls entrance into an occupation </li></ul>
    41. 41. <ul><ul><li>Validity / Reliability </li></ul></ul><ul><ul><li>Alignment to Standards </li></ul></ul><ul><ul><li>Availability of data </li></ul></ul><ul><ul><li>Value to the Student </li></ul></ul><ul><ul><li>Development / Administrative Costs </li></ul></ul><ul><ul><li>Pooling / Sharing Resources </li></ul></ul>Considerations for selecting 3 rd Party External Assessments
    42. 42. <ul><li>Can we obtain the results/data? </li></ul><ul><li>Are cost and administration feasible? </li></ul><ul><li>Does it add value or control entry to the student/occupation? </li></ul><ul><li>Is it valid, reliable, reputable, recognized? </li></ul><ul><li>Is it summative, cumulative or a partial measurement of skills? </li></ul>Considerations for selecting 3 rd Party External Assessments
    43. 43. <ul><li>Does the assessment align with program outcomes? </li></ul><ul><li>Does the assessment measure what needs to be measured? </li></ul><ul><li>Does the data help improve and strengthen our programs? </li></ul>Considerations for selecting 3 rd Party External Assessments
    44. 44. WTCS 3 rd Party Assessments Currently in Place <ul><li>Occupations regulated by the Department of Regulation and Licensing </li></ul><ul><ul><li>Barber/Cosmetology </li></ul></ul><ul><ul><li>Funeral Director </li></ul></ul><ul><ul><li>LPN </li></ul></ul><ul><ul><li>Registered Nurse </li></ul></ul><ul><ul><li>Dental Hygienist </li></ul></ul><ul><ul><li>Massage Therapist </li></ul></ul><ul><ul><li>Occupational Therapy Assistant </li></ul></ul><ul><ul><li>Physical Therapy Assistant </li></ul></ul><ul><ul><li>Respiratory Care Practitioner </li></ul></ul><ul><ul><li>Veterinary Technician </li></ul></ul><ul><ul><li>Wisconsin Registered Interior Designer </li></ul></ul>
    45. 45. Guidelines for Selecting 3 rd Party Assessments <ul><li>This is a guiding document to assist districts in determining direct technical skill assessment appropriateness. </li></ul><ul><li>Districts will work with the state office staff in the determination. </li></ul><ul><li>Keep in mind this is for a small number of programs. </li></ul>
    46. 46. Guidelines for Selecting 3 rd Party Assessments
    47. 47. <ul><li>Concerns: </li></ul><ul><li>Only measures part of the program outcomes-”All Code-No Application” </li></ul><ul><li>Does not indicate job performance </li></ul><ul><li>Does it help employers? </li></ul><ul><li>Focused only on technical skills, what about academic skills and soft skills? </li></ul>Guidelines for Selecting 3 rd Party Assessments
    48. 48. 3 rd Party Assessment - Example <ul><li>Automotive Service Excellence (ASE) </li></ul><ul><ul><li>Long-standing credential of automotive technicians </li></ul></ul><ul><ul><li>A series of 8 certification areas/assessments </li></ul></ul><ul><ul><li>Would it work for Perkins? </li></ul></ul>
    49. 49. <ul><li>Logistics Professional </li></ul><ul><ul><li>CPL/DL-International Society of Logistics </li></ul></ul><ul><ul><li>CPIM/CSCP-Association for Operations Management </li></ul></ul><ul><ul><li>PLS/CTL/DLP conferred by the American Society of Transportation & Logistics </li></ul></ul>3 rd Party Assessment - Example
    50. 50. <ul><li>Federal Aviation Administration (FAA) Airframe and Powerplant Mechanic Certificate </li></ul><ul><li>Administered by the FAA </li></ul><ul><ul><li>Written, practical and oral examination </li></ul></ul><ul><ul><li>Required for entry </li></ul></ul>3 rd Party Assessment - Example
    51. 51. How Can WIDS Help?
    52. 52. The Learning & Assessment Design Technology Tool Kit TracDat Blackboard iWebfolio eFolio Questionmark People Soft Banner Datatel
    53. 53. <ul><ul><li>Create/maintain Program Outcome Summaries for all new and existing programs. </li></ul></ul><ul><ul><li>Develop/maintain Course Outcome Summaries for all courses. </li></ul></ul><ul><ul><li>Design Performance Assessment Tasks for course level (formative) and summative assessments. </li></ul></ul><ul><ul><li>Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses. </li></ul></ul><ul><ul><li>Present WIDS documentation as a central feature in accreditation. </li></ul></ul>How can WIDS help? WIDS Model and Software Tools:
    54. 54. <ul><ul><li>Facilitate collaborative development of: </li></ul></ul><ul><ul><ul><li>DACUM or Industry Review Focus Groups </li></ul></ul></ul><ul><ul><ul><li>Program Outcomes </li></ul></ul></ul><ul><ul><ul><li>Program Outcome Summaries </li></ul></ul></ul><ul><ul><ul><li>Summative Performance Assessments </li></ul></ul></ul><ul><ul><li>Produce </li></ul></ul><ul><ul><ul><li>Industry Approval Reports </li></ul></ul></ul><ul><ul><ul><li>Analyzer Reports </li></ul></ul></ul><ul><ul><ul><li>Updated WIDS files and reports </li></ul></ul></ul>How can WIDS help? WIDS Team
    55. 55. <ul><li>How do we communicate WTCS long-standing commitment to industry input and standards? </li></ul><ul><li>How do we ensure that we use quality assessments? (That we don’t regress to testing regurgitation of information?) </li></ul><ul><li>How do we ensure that we actually are doing what our model and assessment plans imply? </li></ul><ul><li>How do we collect and report data? </li></ul>Challenges
    56. 56. Current Projects
    57. 57. <ul><ul><li>Automotive Technology/Technician </li></ul></ul><ul><ul><ul><li>NATEF end of program evaluation </li></ul></ul></ul><ul><ul><li>Accounting </li></ul></ul><ul><ul><ul><li>WIDS </li></ul></ul></ul><ul><ul><li>Early Childhood </li></ul></ul><ul><ul><ul><li>WIDS </li></ul></ul></ul><ul><ul><li>Criminal Justice </li></ul></ul><ul><ul><ul><li>WIDS </li></ul></ul></ul><ul><ul><li>Welding </li></ul></ul><ul><ul><ul><li>WIDS </li></ul></ul></ul>Current Projects

    ×