TESOL Presentation 2010: Assessing Academic Bridge Courses Handout

Assessing Academic Bridge Courses March 25, 2010 Presentation Outline Assessing Academic Bridge  Our Context Courses  Our Mandate & Timeline  Marquette University’s General Assessment Process  Steps for Designing Our Assessment Plan Jean M. Czaja & Brooke Haley  Examples of How We are Carrying Out Assessment in 2 of Our Courses ◦ How they are embedded into the courses Marquette University ◦ sample assessment measures and rubrics  Data & Conclusions from Our First Assessment Cycle TESOL 2010—Boston, MA  Actions We are Taking to Improve Students’ Learning March 25, 2010  Advice for Assessing Academic Bridge Courses 162B Boston Convention Center 3-3:45pm Our Context Our Mandate & Timeline • Marquette University is a private Jesuit university located in Milwaukee,WI. • Marquette is accredited through the North Central Association of Colleges and Schools. • We have a student body of approximately 8,000 undergraduate students and 3,500 post-graduate students. • As part of its accreditation, since Fall 2004 Marquette has been engaged in an initiative to improve: • Of this total, about 500 (4.35%) are international. Approximately 150 (1.88%) are (1) its overall assessment of students’ learning undergraduates, coming from 43 total countries. (2) the “culture of assessment” at Marquette • To support these undergraduates, we offer 4 advanced ESL academic bridge courses • The ESL Program first entered the assessment process in Fall 2006 when it created and an ESL section of Freshman English, which students take for academic credit: and listed specific course learning objectives on course syllabuses. • ESLP 82: Composition • ESLP 83: Reading • Then in Spring 2008, the ESL Program wrote an assessment proposal & devised • ESLP 84: Listening Comprehension initial learning outcomes, performance indicators, and measures. • ESLP 182: Rhetoric and Composition for Non-Native Speakers • ENGL 1 (ESLP section): Rhetoric and Composition • In AY 2008-2009, the ESL Program carried out its first year of assessment, which we • In AY 2009-2010, about 75% of the new international undergraduate students (54) will be reporting on. took our English Placement Test and enrolled in our ESL academic bridge courses. Three Levels of Marquette’s Assessment Process Marquette’s Assessment Process At Marquette, student learning consists of 4 steps: DEFINE: IMPROVE: Articulating goals Taking actions to and performance enhance students’ indicators for learning student learning Feedback for continuous improvement ASSESS: REFLECT: Gathering direct & Reflecting on and indirect evidence interpreting the about how students evidence are meeting the goals Information adapted from Marquette assessment literature ESL Program Information adapted from the Marquette assessment website 1

Assessing Academic Bridge Courses March 25, 2010 Steps for Designing Our Assessment Plan Our Assessment Plan We devised our assessment proposal and plan for the university by: MARQUETTE UNIVERSITY 1. familiarizing ourselves with the assessment terms & process used at UNDERGRADUATE CO-CURRICULAR Marquette LEARNING OUTCOMES 2. meeting with Foreign Language and Study Abroad representatives and examining their learning outcomes, indicators, and measures English as a Second Language Program (ESLP) 3. speaking with ESL professionals, reviewing ESL assessment literature, and attending sessions at TESOL  Assessment information is available for K-12, adult education, and IEPs At the completion of his/her ESL course(s), the student is able to:  No guidelines for learning objectives & assessments of university-level ESL courses 1. Write using academic English 4. devising ESL Program learning outcomes, performance indicators, and 2. Read academic English texts measures for the university assessment committee 3. Listen to and speak using academic English 5. improving existing and creating new assessment specifications and course assessments, which are mapped to performance indicators 4. Use grammar, punctuation, and mechanics in academic writing 6. improving and creating rubrics for the course assessments Program Learning Performance Indicators Measures Use of the Information Outcomes How We are Carrying Out Assessment in 1. Write using  Produce a research paper  Final research paper in ESLP 82 ESLP faculty will collect, academic  Employ components of the writing process  Pre and post diagnostic tests analyze, aggregate data, and English  Use strategies to address writing problems  Pre and post self-assessment of present a report to the Director skills and use of learning strategies and staff of the Office of International Education. The information will be discussed ESLP 82: Writing and commented upon by the staff and faculty for the purpose of advancement of students’ proficiency and Program Learning Outcome 1: Writing using academic English program evaluation. 2. Read  Comprehend literal and implied meanings  Comprehensive final exam in Same as above academic of academic texts ESLP 83, which directly Performance Indicators Measures English texts  Determine meaning of new vocabulary measures learning outcomes  Evaluate and respond to ideas in academic  Pre and post diagnostic tests   texts Employ components of the reading process Pre and post self-assessment of skills and use of learning  Produce a research paper  Final research paper in ESLP 82  Read under time constraints strategies  Use strategies to address reading problems  Employ components of the writing  Pre and post diagnostic tests process 3. Listen to and speak using  Employ note-taking techniques to take notes on and understand academic spoken  Comprehensive final exam in ESLP 84 which directly Same as above  Pre and post self-assessment of academic texts measures learning outcomes  Use strategies to address writing skills and use of learning strategies   English Participate in everyday interactions role- plays Final (oral) presentation from last unit problems  Participate in small group discussions  Pre and post diagnostic tests  Deliver oral presentation on news broadcast  Pre and post self-assessment of  Use strategies to address listening and skills and use of learning speaking problems strategies Rubrics: • Final Research Paper Rubric 4. Use grammar, punctuation,  Understand conventions of standard U.S. English grammar, punctuation, and  Revision of ENGL 1 Paper #1 at the end of ESLP 182 Same as above • Diagnostic Test Rubric and mechanics  Pre and post diagnostic tests mechanics in  Produce a research paper using conventions  Pre and post self-assessment of academic of standard U.S. English grammar, skills and use of learning writing punctuation, and mechanics strategies  Use strategies to address grammar, punctuation, and mechanics problems How We are Carrying Out Assessment in Data from First Assessment Cycle ESLP 182: Rhet/Com for Non-Native Speakers Program Learning Outcome 1: Write using academic English Program Learning Outcome 4: Use grammar, punctuation, and Measures Used to Assess the Outcome: Final Research Paper in ESLP 82 mechanics in academic writing Total Number of Students Assessed: 12 Performance Indicators Measures Program Performance Developing Competent Proficient Learning Indicators (CD, D, F) (B, BC, C) (A, AB)  Understand conventions of standard U.S.  Revision of ENGL 1 Paper #1 at the Outcomes English grammar, punctuation, and end of ESLP 182 1. Write using  Produce a research 0 2 10 mechanics academic paper  Pre and post diagnostic tests English  Produce a research paper using  Employ components of 2 7 3  Pre and post self-assessment of the writing process conventions of standard U.S. English skills and use of learning strategies grammar, punctuation, and mechanics  Use strategies to 0 7 5 address writing  Use strategies to address grammar, problems punctuation, and mechanics problems Rubrics: • Revision of Paper #1 Rubric • Diagnostic Test Rubric 2

Assessing Academic Bridge Courses March 25, 2010 Data from First Assessment Cycle (cntd.) Conclusions from First Assessment Cycle Program Learning Outcome 4: Use grammar, punctuation, and mechanics in academic writing When we discussed the academic year 2008-2009 assessment results, we Measures Used to Assess the Outcome: Revision of ENGL 1 Paper #1 at the end of ESLP 182 reached the following conclusions regarding student learning: and 100-point diagnostic test given at the end of the semester Total Number of Students Assessed: 23 and 22 respectively • The majority of the assessed students were ranked as competent or proficient in most categories. Program Performance Developing Competent Proficient Learning Indicators (CD, D, F) (B, BC, C) (A, AB) • Some students did not meet the expectations, which makes us wonder why they did Outcomes not make adequate progress and what actions we could have taken to help them to 4. Use grammar,  Understand conventions 5 13 5 become competent or proficient. punctuation, and of standard U.S. English mechanics in grammar, punctuation, academic and mechanics • We feel that in general that most results represent an accurate assessment of writing students' learning. This was reinforced by students' performance in class and on  Produce a research 1 6 15 paper using conventions other major assessments. of standard U.S. English grammar, punctuation, and mechanics  Use strategies to No data collected No data collected No data collected address grammar, punctuation, and mechanics problems Actions We are Taking to Improve Advice for Assessing Academic Students’ Learning Bridge Courses  Action #1: We are strengthening course content in areas where we want more students to move from developing to competent on the performance indicators. 1. Give yourself enough time to plan and implement your assessment initiative.  Action #2: We are determining what measures are really needed for the university 2. Understand how assessment is carried out within your university and be sure that assessment. We are fairly certain that in the future that diagnostic tests and your plan will fit with the university’s plan (if applicable). questionnaires will NOT be reported to the university but rather used for:  giving students feedback on their skills 3. Create standardized course learning objectives and make sure that your staff  encouraging students to reflect on their learning & how they can learn better actually emphasizes these areas when they teach these courses.  improving our course planning—helping us to decide areas to emphasize 4. Devise assessment specifications and assessments that directly measure the course  Action #3: We are reexamining and revising our assessment specifications, rubrics, and learning outcomes. assessments to ensure that they measure students' actual proficiency. 5. Create rubrics for all assessments to provide a standard way of assessing students’  #Action 4: We have made some measures carry more value towards students’ final performance. grades so that students are motivated to do well on them and take them seriously. 6. Devise a standard procedure for collecting the evidence from students.  Action #5: We have worked with the International Student and Scholar staff to adjust admissions guidelines so we do not admit students with too low of English proficiency to adequately progress in our courses and to succeed at Marquette. 7. Set aside a specific time for your staff to analyze and honestly reflect on the evidence. Keep in mind the limitations of standardized assessments—they don’t tell the entire story.  Action #6: We are meeting with students who show problem behaviors and/or who receive low grades on their first assessments to give them advice and refer them to additional university resources. Advice for Assessing Academic Thank You & Electronic Copies Bridge Courses (cntd.) 8. As a staff, decide on specific actions you will take to help improve students’ learning. Thank you for coming to our presentation. 9. Embed the learning objectives, assessments, and rubrics into your courses so teachers and students know clearly what they are learning and expected to be able to do. Please contact us for more information about assessment at 10. Try to make the assessment initiative as simple, direct, and manageable as possible, Marquette. Visit http://www.marquette.edu/oie/eslp.shtml for a and only gather as much evidence as needed. link to electronic copies of the handouts or other resources. Click on the “TESOL Presentations” link. 3

TESOL Presentation 2010: Assessing Academic Bridge Courses Handout

by Jean Richie

Accessibility

Categories

Upload Details

Usage Rights

Statistics