Assessment Institute August 21 2008


Published on

Published in: Economy & Finance, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Assessment Institute August 21 2008

  1. 1. Assessment Middlesex County College Christine Harrington Ph.D. August 2008
  2. 2. Assessment- What is it? <ul><li>Assessment is gathering and </li></ul><ul><li>using data to improve </li></ul><ul><li>student learning. </li></ul><ul><li>Integrated and ongoing </li></ul><ul><li>Driven by faculty in collaboration with administration </li></ul>
  3. 3. Assessment: It is NOT…. <ul><li>Faculty evaluation </li></ul><ul><li>About “proving” what we do is effective </li></ul><ul><li>Up to one person to decide </li></ul><ul><li>Designed to be done and put on a shelf until Middle States arrives </li></ul>
  4. 4. The Assessment Process <ul><li>Identify learning outcomes for programs, courses and services. </li></ul><ul><li>A learning outcome is a specific statement that clearly identifies what knowledge, skills and attitudes you expect from graduates of your program, students completing your course, or participating in your service. </li></ul>
  5. 5. The Assessment Process <ul><li>Identify assessment techniques to investigate whether you are accomplishing your learning outcomes. </li></ul><ul><li>Whenever possible, use current techniques such as assignments, exams, papers, presentations (these are all direct measures). Rubrics can be very helpful. You can also use indirect methods such as evaluations, surveys, interviews and focus groups. </li></ul>
  6. 6. The Assessment Process <ul><li>Gather and use the data! </li></ul><ul><li>What were the results? How can you use the results to improve student learning? This is the most valuable part of the process. Be sure to document this step so that it can be easily shared with others. </li></ul>
  7. 7. Why Assessment? <ul><li>Allows us to look at the big picture. </li></ul><ul><li>Keeps us focused on what is important to us- what do we want to accomplish? </li></ul><ul><li>Guides our decision making- how can we improve what we do? </li></ul><ul><li>Useful tool to communicate to others – students, families, accreditation bodies. </li></ul>
  8. 8. Academic Major Learning Outcomes <ul><li>Goal: Identify approximately 3-5 learning outcomes for each academic major. Some of you have already accomplished this! The goal is for these outcomes to be published in the college catalog. </li></ul>
  9. 9. Course Learning Outcomes <ul><li>Goal: Identify approximately 3-5 learning outcomes for each active course. This will help set the stage for assessment plans. </li></ul>
  10. 10. Learning Outcomes: <ul><li>Think Product or End Result! </li></ul><ul><li>As a result of being in your program or course, what will the student know, think or do? </li></ul><ul><li>Use Action Verbs- Consider Bloom’s Taxonomy </li></ul>
  11. 11. Creating Learning Outcomes <ul><li>Although learning outcomes for programs and courses need to be individually created for our college, it can be very helpful to look to discipline specific professional organizations or other colleges for some models to get you started! </li></ul>
  12. 12. Bloom’s taxonomy (adapted from <ul><li>Cognitive Domain- Knowledge: </li></ul><ul><li>Cite Label </li></ul><ul><li>List Enumerate </li></ul><ul><li>Identify Imitate </li></ul><ul><li>Match Name </li></ul><ul><li>Quote Recall </li></ul><ul><li>Reproduce State </li></ul><ul><li>Write </li></ul>
  13. 13. Bloom’s taxonomy (adapted from <ul><li>Cognitive – Comprehension </li></ul><ul><li>Convert Define </li></ul><ul><li>Describe Discuss </li></ul><ul><li>Estimate Explain </li></ul><ul><li>Generalize Identify </li></ul><ul><li>Illustrate Locate </li></ul><ul><li>Paraphrase Restate </li></ul><ul><li>Summarize </li></ul>
  14. 14. Bloom’s taxonomy (adapted from <ul><li>Cognitive – Application </li></ul><ul><li>Apply Chart </li></ul><ul><li>Compute Demonstrate </li></ul><ul><li>Determine Dramatize </li></ul><ul><li>Establish Make </li></ul><ul><li>Manipulate Prepare </li></ul><ul><li>Project Solve </li></ul><ul><li>Use </li></ul>
  15. 15. Bloom’s taxonomy (adapted from <ul><li>Cognitive – Analysis </li></ul><ul><li>Analyze Compare </li></ul><ul><li>Contrast Correlate </li></ul><ul><li>Diagram Dissect </li></ul><ul><li>Differentiate Distinguish </li></ul><ul><li>Infer Investigate </li></ul><ul><li>Limit Outline </li></ul><ul><li>Separate </li></ul>
  16. 16. Bloom’s taxonomy (adapted from <ul><li>Cognitive- Synthesis </li></ul><ul><li>Assemble Create </li></ul><ul><li>Construct Design </li></ul><ul><li>Develop Formulate </li></ul><ul><li>Generate Hypothesize </li></ul><ul><li>Initiate Invent </li></ul><ul><li>Modify Reframe </li></ul><ul><li>Synthesize </li></ul>
  17. 17. Bloom’s taxonomy (adapted from <ul><li>Cognitive- Evaluation </li></ul><ul><li>Access Appraise </li></ul><ul><li>Conclude Critique </li></ul><ul><li>Decide Defend </li></ul><ul><li>Diagnose Evaluate </li></ul><ul><li>Judge Justify </li></ul><ul><li>Rank Recommend </li></ul><ul><li>Support </li></ul>
  18. 18. Sample Learning Outcomes <ul><li>Biology </li></ul><ul><li>Psychology </li></ul><ul><li>Dental Hygiene </li></ul><ul><li>Note: The following are examples of current learning outcomes and are still a work in progress. </li></ul>
  19. 19. Biology Option Science Transfer <ul><li>Graduates will demonstrate an understanding of the biology of plants, animals, and microorganisms with an emphasis on their evolutionary connections. </li></ul><ul><li>Graduates will be able to demonstrate use of lab instrumentation to gather and analyze experimental data. </li></ul><ul><li>Graduates will demonstrate the process of scientific inquiry; including developing hypotheses, designing experiments, recording data, forming valid conclusions and writing laboratory reports. </li></ul><ul><li>Graduates will be able to distinguish science from non-science and be able to evaluate biological research. </li></ul><ul><li>Graduates will demonstrate knowledge of the impact of the biological research on society both historically and currently. </li></ul>
  20. 20. BIO 229 Cell Biology <ul><li>Describe the interrelationship between structure and function at the cellular and molecular levels. </li></ul><ul><li>Demonstrate laboratory skills by use of laboratory instrumentation and experimental techniques. </li></ul><ul><li>Demonstrate critical or analytical thinking skills by analyzing and interpreting experimental data. </li></ul><ul><li>Apply concepts of cell biology to diseases. </li></ul><ul><li>Apply the foundations of chemistry, physics mathematics and technology to cell biology. </li></ul>
  21. 21. Psychology Major <ul><li>Students will have a strong foundational knowledge in psychology as evidenced by being able to discuss the key psychological theories, classic research studies, and recent trends in psychology. </li></ul><ul><li>Students will be able to apply this foundational knowledge of psychology to problems of daily living. </li></ul><ul><li>Students will be able to locate empirical research studies, and will be able to identify and describe the various components of a research study. </li></ul><ul><li>Students will engage in critical thinking as evidenced by the ability to evaluate the validity and ethics of research studies and psychological practices. </li></ul><ul><li>Students will be able to create written documents using APA style that are clear, well-organized, and grammatically accurate. </li></ul>
  22. 22. PSY 123 Introductory Psychology <ul><li>Students will be able to discuss key psychological theories and concepts and will be able to summarize the contributions of key psychological figures. </li></ul><ul><li>Students will be able to describe the various research methods and apply this knowledge to evaluate psychological research studies. </li></ul><ul><li>Students will be able to articulate the importance and value of psychology in today’s world. </li></ul><ul><li>Students will be able to create written documents using APA style that are clear, well-organized, and grammatically accurate. </li></ul>
  23. 23. Dental Hygiene <ul><li>Objective: All National Board content areas will be passed equal to or above the national mean. </li></ul><ul><li>Assessment Tool: National Board Exam </li></ul><ul><li>Results and Use of Data: Pharmacology content area was less than national average. </li></ul><ul><li>Moved course earlier in the curriculum sequence so that students have more experience applying this knowledge in the clinical setting. </li></ul><ul><li>Hired more experienced instructor. </li></ul>
  24. 24. Assessment at Middlesex: Website <ul><li>Learn </li></ul><ul><li>Assessment- What is it? </li></ul><ul><li>Why Assess? </li></ul><ul><li>How do we assess? </li></ul><ul><ul><li>Defining Learning Outcomes </li></ul></ul><ul><ul><li>Assessment Methods </li></ul></ul><ul><ul><li>Using Assessment Data </li></ul></ul><ul><li>Helpful Information </li></ul><ul><li>Share </li></ul><ul><li>Assessment Templates </li></ul><ul><ul><li>Learning Outcomes Cover Sheet </li></ul></ul><ul><ul><li>Assessment Plan </li></ul></ul><ul><li>Program, Course and Service Learning Outcomes </li></ul><ul><li>Assessment Plans and Data </li></ul><ul><li>Course Mapping </li></ul>
  25. 25. Learning Outcome Template for Program, Course or Service
  26. 26. Learning Outcomes: Upon successful completion of the program/course/service, students will be able to: <ul><li>1. </li></ul><ul><li>2. </li></ul><ul><li>3. </li></ul><ul><li>4. </li></ul><ul><li>5. </li></ul>
  27. 27. Assessment Plan Template
  28. 28. <ul><li>Learning Outcome 1 (What do you want students to do, think, or know?) </li></ul><ul><ul><li>Simply repeat outcome listed on outcome template </li></ul></ul><ul><li>Assessment Method: How will you find out if you achieved what you wanted to achieve? </li></ul>Data and Implementation: What did you find out and what will you do with this information?
  29. 29. Course Mapping <ul><li>Course mapping is a great visual tool to determine if we are adequately addressing our program learning outcomes in our required courses for the program. </li></ul>
  30. 30. Key: L = Limited, introduced or briefly addressed M = Moderately addressed H = High, significantly addressed, focus of course           Divisional Elective           Diversity Elective           Additional Courses                       Humanities           Computer Class           Language           HIS 122           HIS 121           Social Science           PED/HED           MAT           ENG 122           ENG 121           Required MCC Courses                                   Required Major Courses Learning Outcome 5 Learning Outcome 4 Learning Outcome 3 Learning Outcome 2 Learning Outcome 1   Major:
  31. 31. Completed Templates: An Example
  32. 32. <ul><li>Learning Outcome 1 (What do you want students to do, think, or know?) </li></ul><ul><li>1. Recognize and discuss key psychological theories and concepts and will be able to summarize the contributions of key psychological figures. </li></ul><ul><li>Assessment Method: How will you find out if you achieved what you wanted to achieve? </li></ul><ul><li>1a. Multiple choice questions targeting psychological theories will be embedded into final exams. </li></ul><ul><li>1b. A short answer question on psychological theories will be given during final and will be evaluated by a rubric </li></ul>Data and Implementation: What did you find out and what will you do with this information? Results from selected multiple choice questions (320 students across a variety of sections) indicated that students taking the final exam (it was optional in some classes), were generally not able to identify key components of psychological theories at an acceptable level (48.5% were able to identify a key component of Freud’s psychodynamic theory, 60% Skinner’s behavioral theory, 35% Vygotsky’s developmental sociocultural theory; 16.2% Erikson’s developmental psychosocial theory; and 37.9% were able to identify differences between the key theories). Application of classical and operant conditioning concepts also did not meet expectations (13.7% and 47.2% of students completing the final were able to correctly apply classical conditioning concepts to an example, and 34.4% were able to differentiate between operant conditioning concepts). Students performed best on questions related to memory concepts as evidenced by 64.1% of students responding accurately to a famous study on short term memory and 75.3% of students were able to correctly identify a short term memory strategy. These results will be discussed with faculty members in the department to identify ways to better meet learning outcomes established. Perhaps we will spend time sharing our expertise and effective teaching strategies with one another and adjuncts. We have to meet to finalize the scoring rubric before compiling data from the short answer question.
  33. 33. Local community colleges with learning outcomes listed for programs and/or courses: Mercer County Community College In catalog, you’ll find program learning outcomes and some are also found in course outlines. Brookdale Community College Learning outcomes can be found in the catalog. Raritan Valley Community College Learning outcomes are listed in the college catalog.
  34. 34. Other websites with useful information on assessment Bridgewater State College Simple, straightforward information on creating learning outcomes and course mapping. Bakersfield College Simple step by step process of creating learning outcomes and the assessment process. Fresno City College Student Learning Outcome handbook- Bloom’s taxonomy clearly demonstrated. Modesto Junior College Clear outline of steps in assessment process. http:// University of Washington http:// NC State University Very comprehensive website, even a bit overwhelming. As a start, click on Assessment of specific skills or content. Here you will find some information by discipline that may be helpful. Association of American Colleges and Universities Comprehensive website on assessment in higher education.
  35. 35. Outcomes and Faculty <ul><li>As you know, faculty members are critical in creating outcomes. The program and course outcomes working documents developed here will be brought to the faculty for their consideration and input on Faculty Development Day. They can always be modified as needed. </li></ul>
  36. 36. Importance of Faculty Support: What Can I Do? <ul><li>Model a positive attitude about assessment </li></ul><ul><li>Continually emphasize that assessment is NOT faculty evaluation </li></ul><ul><li>Keep assessment simple, yet meaningful </li></ul><ul><li>Keep assessment visible- keep it on the agenda for department meetings </li></ul><ul><li>Provide support for assessment whenever possible </li></ul>
  37. 37. Thank you for your support and hard work during this assessment institute! Next Steps: Faculty Input and Developing Assessment Plans