• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,746
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
41
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Sylvia Begins Presenting:
  • Faculty Peer Review : To improve faculty teaching through direct faculty interaction (e.g., attending workshops, visiting each other’s classes, creating teaching portfolios). This is particularly useful for younger faculty, who seem more preoccupied with research. It provides interaction with more experienced faculty mentors. Advocates a change in institutional structure that values and rewards teaching to greater degrees. Mathematics initiatives : Involve understanding at several levels… numerically, graphically, algebraically, and verbally. Emphasize verbal skills. Relate calculus to real world situations. Faculty shift to facilitators of group projects and active learning, then listen to students as they try to develop understanding, identify gaps in knowledge, and clarify what they have just learned. Learning Communities : Promote academic success by emphasizing student-student, faculty-student interaction, and interdisciplinary course linkages. Programs vary in format, but typically fall into one of five categories: linked courses, learning clusters, freshman interest groups, federated learning communities, and coordinated studies. Classes focus on improving the quality of learning and not just content or outcomes. Learning communities promote change in university cultures. Emphasize collaboration between academic affairs and student affairs, and among faculty in terms of sharing knowledge about teaching.
  • Rather than replacing assessment techniques, we found that faculty members typically create new tools and link to traditional assessment methods. Faculty consider skills that students will need in the future as they transfer knowledge from one class to another, or to external work environments. Mathematics and chemistry faculty are realizing the need to improve communication skills as part of an overall math or science “literacy.”
  • English faculty, Landgrant Flagship: “I know [faculty] have been collecting work from representative students from across their four years or five years or however long they’re here, but that still provides such a contextualized picture of what’s going on. Did you interview the faculty who taught those courses to figure out what their goals for those classes were to begin with? And did you talk – can the student talk articulately without the connections that they’ve seen between all these different pieces of material? I guess its so text based, it’s just like, well, show us that you pass this chemistry test or that you’ve written this research paper or whatever, that it doesn’t really attend to the process at all.” Exam scores are now used by math faculty in different ways. Some faculty create a spreadsheet using course goals and sections of the exam, to discover what skills they could teach better (and whether they are meeting their teaching goals). Some departments create similar spreadsheets to determine how well faculty are doing. Some may be using this information in more punitive ways.
  • “ My chair has told me again and again . . . ‘You’re doing a lot of wonderful things in the classroom, but nobody knows about it. You’re not sharing.’ And its not that I don’t want to, not at all, its just that mathematicians have kind of a loner streak in them, you know. Teamwork is something we have to acquire. So I am trying to share more, and I felt kind of driven to.” (Math - LF) “ So I’m a student along with my students . . . . Just this past [week] I put a flow chart up on the board, it was kind of biffed up in some respects and we kind of straightened it out. And after class, one of my students came up, and he said “You know, I really wouldn’t do it that way. This doubling factor that you’re using, explain to me why.” He had a different strategy in mind, and you know, he’s right. So, the next time I wrote it up, I mentioned it, that he suggested this. So I’m with the students, and we kind of move forward together.” (Math - LF) “ Teaching should be led by the faculty, rather than by administration . . . . I think that if you’re going to improve anything it has to come from both the administration and the faculty . . . . An administrator’s job is to really resolve conflict that’s created by people’s self preservation instinct.”(Math - U) “ What I told the math department was look, we’re going to do it education’s way for a year or two, as a pilot, as a study, and I don’t want your reasons why its not going to work. Let’s just monitor, let’s document it, and if you’re right and it doesn’t work, we will have the documentation. Let’s go through the motion.” (Math - U)
  • “ Chemistry for the Citizen” Courses: Use of Current Events in Writing Projects Surviving large lecture classes Short spurts of lecturing interspersed with exercises geared toward keeping students engaged Using space, technology and staff to create community: Landgrant Flagship’s Chemistry Resource Center Large, open, sunny workspace: round tables, surrounded by resources and faculty offices, departmental staff and lab supply room there as well. In a separate part of the room is a computer lab with group-work stations and “roll-away” computer stations (so they can be taken to lab). Very popular space -- “buzzing” Use of graphing calculators in class and in lab: Since the calculators can accept data (for instance, it can read temperatures from an attached thermometer; it can plug into an oscilloscope...) students can collect data directly themselves in class or lab demonstrations, and then carry it but to the computing center, plug their calculators back in and then manipulate the data that way: 1) Students get more direct involvement in the material/demonstrations. 2) There are fewer errors in reading and transferring data.
  • Endemic Resistance: Faculty Associate Assessment with Anti-Intellectualism A Sense of Learning’s Intangibility Resistance to Quantification. “ Assessment vs. Instruction” Conversions in Practice General Shift: Faculty are More Willing to Present Student-Learning Goals Overtly in Courses.
  • Resentment of other departments “ Sometimes I’ve been pretty upset about, actually about content. If I think I’ve got a certain bit of stuff I need to do, and I can’t do it because people aren’t prepared, then the question is what do I do? Do I try to cover the stuff that I believe should have been covered or do I simply go on with what I think is important and let the students fall where they may and I’m not inclined to do that. But I don’t know how easily I can insert myself into someone else’s course and say, come on, get with it.” (Chemistry, LF) Some faculty were alienated by the use of a formal term like “Assessment” to label some of the more organic ways they “clue in” to what their students are learning. We encountered a number of reactions from a simple resistance to labeling their practices to intimidation.
  • Ability to risk : This professor was concerned that he didn’t know if his students were learning anything. Rather than rest with that concern, he wrote it down in a teaching journal, shared this journal with colleagues, and ultimately, with students as well. This is a risk that a person earlier in the career might not take. This faculty member has also gone through a post-tenure review voluntarily , including sending his teaching portfolio out for external review in a way similar to the common review of research portfolios .
  • Explicit Goals for Student Learning Enhance Faculty Use of Innovative Teaching 1) Innovative faculty tend to use assessment 2) But even if some innovative teachers didn’t use assessment techniques from the beginning, many who happened to stumble on assessment techniques or even some who engaged in collecting assessment data out of obligation by their administration -- many of these faculty found practical use for the information and therefore liked and implemented the techniques. Makes Assessment of Student Learning More Practical/Attractive to Faculty. An Emphasis on Written Communication of Concepts Examples: term papers; presentations in Math & Chem. courses. student journals and “learning letters” in English Increases Student Exposure to Material Augments the Information Available for Assessment & the Improvement of Teaching. Other Tensions/Constraints: Content vs. Understanding Resistance to Overt Goals and Quantifiable Assessment . Faculty Empowerment to Assess Student Learning.
  • Assessment as information-gathering Assessment as impetus for innovation: uncovers a problem and points to possible remedies Innovation as impetus for assessment: Provides feedback Enhances faculty & student engagement Reinforces motivation for teaching improvement A link between teaching improvement and assessment improvement Traditional markers may overlook emerging dimensions of student learning GRE Example -- use of standardized tests
  • Landgrant Flagship Promotion/Tenure Example: – Changes instituted by new Vice Chancellor for Academic Affairs and Dean of College of Arts & Sciences – Recent (in the last two years) development of Percentage-type weighting in Tenure/Promotion Process: teaching, research, service. • Anecdotally, as much as 70% teaching allowed • Agreements reached between individual faculty members and their respective departments. – Release time for teaching. This system allows departments to recognize the contributions of faculty whose professional priorities favor teaching over research. The history of one professor at Landgrant Flagship, for example, illustrates the concrete and sudden effects of the new policies. I've been an associate professor since 1980… which is fine with me, you know. It's just not a goal of mine to be a full professor, and I've been told year after year by my Chairs, "Well, you know, you do great work in teaching and service. We just wish you'd publish a couple more articles next year." And I would say, "Yes. I know that's what you have to say to me, but that doesn't reflect my goals, so I'll just keep doing what I think is right, and you'll just keep telling me what you have to tell me and that's okay." But after being told that again just last year in May, in October I was actually asked to put myself forward for promotion with the understanding that the winds had shifted and that my teaching could be seen as my greatest strength. I have some publications; it's not that I have none. And most of them are on teaching, so I was actually asked and I understood that [that] came from the Vice Chancellor through the Dean level to the department, and that meant a lot to me, I must say, even though I'd reconciled myself to never being promoted. That meant a lot to me to have people say, "No …we do not only recognize teaching, but we want to show that we recognize it." Several faculty who spoke with us cited this policy as an indication of a changing climate at Landgrant Flagship. To many on campus, the concrete results of the policy certify the University's commitment -- at departmental, college and central administrative levels -- to the scholarship of teaching.

Transcript

  • 1. New Techniques, New Assessment? Assessment Strategies and Innovative Teaching Practices The PowerPoint slides were developed by Mus Khairy (PhD), Stanford University at California. Educational ,Social Psychologists at German University at Cairo (GUC) unless otherwise noted on specific slides.
  • 2. Teaching, learning, and assessment
    • Question: How have institutions responded to calls for improvement?
    • Prevailing views and criticisms
      • Undergraduate education is in a state of decline
      • Faculty are unwilling to improve teaching
      • Increased emphasis on student assessment will lead to improvements in teaching and learning
  • 3. Response to calls for improvement
    • Numerous disciplinary and cross-disciplinary innovations in teaching and learning have emerged
    • Higher education associations, foundations, and consortia of institutions provide support
    • Improvements in teaching, learning, and assessment are in various stages of evolution on campuses
  • 4. Teaching and learning innovations
    • Peer review of teaching
    • Mathematics/Science curriculum reform
    • Learning communities
    • Innovation : Characterizes ground-up, internal processes
    • Reform : Describes top-down, systemic, or throughout several institutions
  • 5. Case study methodology … campus selection criteria
    • Teaching/learning innovations
    • Ability to look at multiple innovations
    • Same accreditation region with variation in state assessment policy
    • Disciplines of mathematics, English, and chemistry
    • Landgrant Flagship, Urban University, National University
  • 6. Comparison of campuses…
    • Similarities
      • Research I
      • Presence of medical, law and graduate schools
      • Tenure (research only or multiple paths)
      • Highly decentralized academic units
      • Approximately 23-24,000 undergraduates
    • Differences
      • Levels of innovation (top-down or grassroots)
      • Approaches to faculty-administration divide
      • Type of admissions (flexible or selective)
  • 7. Interviews and document gathering
    • Interview protocols
    • Document gathering before and after visits
    • Contact with campus
    • Selection of interviewees
      • “ Change agents”
      • Faculty
      • Department chairs
      • Teaching/learning center directors
      • General education leaders
  • 8. Case study materials
    • Interviews with academic affairs administrators, faculty and dept. chairs, undergraduate education coordinators, and teaching and learning centers
    • Web documents on undergraduate education, individual faculty, and campus initiatives
    • Bulletins, guidelines, reports, assessment plans, memos, and faculty course portfolios
  • 9. Landgrant Flagship
    • Combined missions of landgrant university and state flagship – creates issues of identity
    • Leadership focus on plans for undergraduate education; conversations about coursework “rigor” are prevalent
    • Traditional and new assessment techniques simultaneously informing debate
    • Faculty develop an active interdisciplinary community focused on the scholarship of teaching
  • 10. Land-grant Flagship (cont.)
    • Vice Chancellor initiative awards tenure with more flexible teaching/research ratios
    • Accreditation and academic program review drives development of dept. plans for student assessment
    • Improvement initiatives precede the coordination of student assessment
  • 11. Urban University
    • A large urban campus with multiple missions to the local community (both to students and to businesses)
    • “ Top down” initiatives relating to assessment and teaching/learning improvements, but success dependent on faculty ownership
    • Central administration coordinates all levels of assessment activity on campus
    • Institution garners recognition for innovation
    • Improvement initiatives and assessment activity not converging at the individual faculty level
    • Flexible promotion and tenure system
  • 12. National University
    • Institutional prestige motivates innovation in teaching/learning activity
    • Faculty leadership involved with departmental changes regarding teaching/learning
    • Uncoordinated assessment activity, no central oversight or attention
    • No academic program review process
    • Upcoming accreditation visit may provide impetus for more emphasis on assessment
    • Tenure granted on 40-40-20 model
  • 13. Multiple ways of knowing what students are learning
    • English
      • Class discussions
      • Placement tests combined with other assessment techniques
    • Mathematics/Chemistry
      • Exams
      • Added vehicles for communicating
        • Papers and group projects
        • Short presentations
        • “ Front row duty”
        • Emphasis on communication skills
  • 14. Are assessment and grading different?
    • “ [Communication with students] is assessment in the broad sense. It’s not assessment in the sense of a course grade. . . . I’m not talking about enumerating things that go into the course grade. But – and I do some assessment, I mean just talking to the students, you get a sense of where they’re at, who’s more advanced, who’s not, but, so the one minute papers and the background knowledge probe are certainly broader assessment practices…”
  • 15. Are assessment and grading different? (cont.)
    • “… But by and large, I mean, you know, the bulk of the grading, if you want to think of assessment as grading, the bulk of the grading is done still on our exams but I’ve broadened it out, and sort of tempered it somewhat with other things including the writing and the presentations and you know, homework and stuff like that…”
    • Landgrant Flagship, Math
  • 16. Perceptions that assessment unfairly or prematurely judges
    • Does assessment = judging?
    • “ I was not as anxious to put assessment into [learning communities] this year. I think sometimes if people feel you’re judging right away, that it’s not good, and I also know that we want to involve [staff members].”
    • – Landgrant Flagship, General Education
  • 17. Perceptions about assessment (continued)
    • Do grades unfairly label students?
    • “ I don’t believe in grades’, some [faculty] have said. ‘My classes are so process oriented that students have the chance to keep working on whatever it is until they raise their grades high enough. I give them huge amounts of feedback and they will just implement the feedback and its impossible for them not to get good grades.’ ‘The focus of the class is so personal, that how can you grade people down for expressing their opinions about their own lives?”
    • Landgrant Flagship, English
  • 18. Questions about student learning
    • During college
      • Does class performance improve?
      • Is post test score well above pre test score?
      • Does student enroll in subsequent or related classes?
      • How do students perform in subsequent or related classes?
    • After graduation
      • Do students develop technology and communication skills?
      • Do graduates get jobs?
      • Are companies happy with graduates’ skills?
      • Do our students’ scores compare well with other institutions?
  • 19. Gathering evidence of student learning
    • English
      • Portfolios
        • Student portfolios
        • Course portfolios
        • Teaching portfolios
      • Issues
        • Representative or best students
        • One polished product or many drafts
      • Student learning at center of discussions on portfolios and teacher assessment
    • Math/Chemistry
      • Different use of exams
        • Pre and post tests
        • Aggregated results of class performance
        • Dept.-wide finals with comparison across sections
      • Issues
        • Is score comparison across classes used to identify student skill levels or to punish faculty?
  • 20. Evidence of student learning becomes relevant at multiple levels
    • Student Assessment
    • Teaching Assessment
    • Program Assessment
      • Within Major
      • Service Courses
    • Institutional Status
      • Accreditation
      • Reputation
    • Post-Graduation
      • Response to Employer Demand
      • Alumni Satisfaction
  • 21. Math: Different attitudes
    • Innovation as usual
      • Sense of responsibility to other departments
      • Strong sense of departmental cohesion
      • Interest in student opinions
    • Innovations as prestigious
      • Sense of leading a discipline
      • Sense of institutional status tied to new teaching practices
    • Resistance and turf issues
      • Faculty feel dictated to by upper administration and pedagogical experts
      • Lots of departmental turf issues
      • Endurance of underprepared students
  • 22. Chemistry: Making practical connections
    • Practical implications of course material made more explicit
    • Surviving large lecture classes
    • Using space, technology and staff to create community: Landgrant Flagship’s Chemistry Resource Center
  • 23. English: A case in resistance & change
    • Endemic resistance: Faculty equate assessment with anti-intellectualism
    • Conversions in practice
    • General shift: Faculty are more willing to present student-learning goals overtly in courses
  • 24. Overarching Themes … Patterns of resistance
    • Resentment of other departments: Suspicion that service-course faculty innovate at the expense of student preparation
    • “ Assessment” & the Buzzword Effect - a lack of interaction between micro- and macro- levels of assessment?
  • 25. Patterns of community building
    • Develop an appreciation for others’ teaching
    • “ One of the things I enjoyed about the peer review project
    • (is) probably the increased amount of time I’ve spent
    • talking to people outside my field about teaching . . . I’ve
    • come to understand how faculty are different in many
    • ways.”
    • – Urban University, English
  • 26. Patterns of community building (continued)
    • Establish relationships built around teaching
    • “ If you’ve worked with Teaching/Learning centers like that you realize that there soon turns to be a kind of a group of faculty that many of them show up to many other things, and so you end up over the long haul seeing a lot of people, I suppose 50 percent of the people are kind of regulars at this.”
    • – Landgrant Flagship, Math
  • 27. Patterns of recommitment: Lessons from senior faculty
    • Teaching innovation can reinvigorate career
      • “ I had a burnout experience and so that is what I reckon paved the way for my readiness for this experience. There’s nothing like a trauma to shake things up in a hurry.” – Landgrant Flagship, English
    • Increased investment in institution
      • “ I’m much more ready to invest in the institution. Its clear that I’m going to be here, and teaching is something which is a benefit primarily or at least at first glance to the institution.” – Urban University, English
  • 28. Lessons from senior faculty (continued)
    • Ability to risk and try new things
    • “ One of [my colleagues] said ‘why don’t you take part of the teaching journal and give it to the students and ask for a response?’ And I did, and it became part of my teaching portfolio that I made.”
    • – Landgrant Flagship, English
  • 29. Lessons: Classroom assessment & teaching improvement
    • Explicit goals for student learning
    • An emphasis on written communication of concepts
    • Other tensions/constraints:
      • Pace: Content vs. Understanding
      • Resistance to overt goals and assessment
      • Faculty empowerment to assess student learning
  • 30. How is improvement linked with assessment of student learning?
    • Assessment as information-gathering
      • Assessment as impetus for innovation: uncovers a problem and points to possible remedies
      • Innovation as impetus for assessment:
        • Provides feedback
        • Enhances faculty & student engagement
        • Reinforces motivation for teaching improvement
    • A link between teaching improvement and assessment improvement
        • Traditional markers may overlook emerging dimensions of student learning
  • 31. What types of institutional structures encourage improvement?
    • Flexible promotion/tenure processes
      • Separate tracks: research, teaching, service, balanced case
      • Flexible percentage weighting in review process: teaching, research, service
    • Teaching/Learning Centers can facilitate faculty ownership in teaching improvement
  • 32. Institutional structures… (continued)
    • Opportunity for interdisciplinary dialogue about teaching
    • Pressures from accreditation and program review
  • 33. Cross-case comparisons
    • National initiatives link faculty into networks across campuses - (e.g. external evaluation of teaching)
    • Development of faculty expertise when it comes to student learning/assessment - knowledge about practice
    • Highly decentralized environments, a strong central vision and faculty leadership in depts. are important to create change
  • 34. Assessment of student learning occurring at multiple levels
    • Classroom level
      • “ It’s not all high-science!”
    • Departmental level
      • Across sections
    • Interdepartmental expectations
      • Service course dynamics
    • Administrative/Formal levels
      • Faculty performance: Promotion/Tenure
      • Program review
      • Institutional accreditation
  • 35. Recommendation: Link assessment and improve teaching
    • Build on faculty interest in the scholarship of teaching
    • Revise tenure and promotion policies to reward teaching innovations and collection of evidence of student learning
    • Coordinate the multiple levels of assessment activity to create a coherent portrait of how the campus is “making a difference”
    • Ecological model linking assessment and improvement
  • 36. Institutional research implications
    • Participate at the initial stage in assisting innovations to develop useful assessments
    • Keep communication lines open regarding innovative activities on campus
    • Involvement may require evaluation of standard educational practices as well as innovative practice
    • Coordinate the involvement of more individuals in assessment as the results of the innovations appeal to a broader audience
  • 37. Research challenges
    • Entry: who grants it determines what interviewees say
    • Avoiding perception of participating in a specific campus agenda
    • Sense of one campus more “poking and prodding”
    • Getting faculty to open up when they want to know our position on various contentious issues
    • Fitting into the faculty schedule
    • Logistics
    • Cost