We’re often taught about the cognitive or “knowledge” part of learning. Bloom’s Taxonomy of Educational Objectives is the most commonly one used to write instructional objectives since it uses very descriptive verbs to handle varying learning levels. This visualization is the most helpful in showing how faculty often teach at the lower two levels, yet test at the upper two levels, often without helping the learners go through the other layers. If you assess at the top two levels, by what method are you “assessing?” Developing a quiz or test to assess “criticize” is going to be challenging. Read more about it at http://projects.coe.uga.edu/epltt/index.php?title=Bloom%27s_Taxonomy
You're probably familiar with Bloom's Taxonomy for the cognitive domain, but when was the last time you saw Bloom's Taxonomy for the psychomotor domain? This reflects the fact that these verbs are much less commonly used in course objectives to refer to skill attainment, even when our disciplines deal with the behavior or skill part of learning.
But what about the Feeling part? This part deals with the affective domain -- feelings, attitudes, beliefs and values. Bloom's taxonomy which identifies a comparable hierarchy for the Affective Domain is also almost entirely ignored. Yet some would say that this element is the most crucial in affecting permanent change – or impact – in learning.
Here is a simple way to visualize the relationship between these three different domains as identified by Bloom's taxonomies.
What typically happens in higher education is that we often teach the content and then test for a skill - sidetracking the attitudes, feelings and values of the learner. However, internalizing the learning is a function of the Affective Domain; the process of generating and mining insight helps “make meaning” for the learner. Read more about Making Meaning and the Constructivist Theory at http://www.hitl.washington.edu/publications/r-97-47/two.html
The differences between intrinsic and extrinsic motivation can be a bit tricky to sort out since learning can involve an interconnected combination of the two. A better way of understanding how to cultivate impact assessment is to understand its basis in the term locus of control, which refers to an individual's belief about the extent to which they can control events that affect them. An internal locus of control is a belief in one's ability to exert control over events, while an external locus of control is a belief that events are more in control and that things will happen regardless of our efforts. Locus of control has both internal and external frames of reference. So for example, a learner with a strong external locus of control might be more highly dependent on instructor-centered learning delivery. in extreme cases, learners with a strong external locus of control might exhibit what University of Pennsylvania psychologist Martin Seligman called &quot;learned helplessness,&quot; which is characterized by a reiative inability to act independently. Because education and traditional assessment tend to ignore the affective domain, they also tend to ignore the importance of learners' internal frames of reference. Education tends to focus on the intellectual, pays occasional attention to the social, and usually ignores the spiritual, emotional, and personal dimensions of the learning experience. It is this Internal Frame of Reference that “makes meaning” to the learner. Adult Learning Theory also states that “critical reflection is a key to transformational learning and part of andragogy’s self-directed learning. Reflection/think time is another essential principle to creating an effective learning experience for adults. Adult learners need time to contemplate the ramifications of the learning experience to their experience and responsibilities.” If development [impact] is an outcome of transformational learning, then creating an effective adult learning opportunity needs to take personal development into consideration.(For more details, read the Andragogy section found at http://en.wikibooks.org/wiki/Learning_Theories/Adult_Learning_Theories; for some examples, see http://www.utwente.nl/cw/theorieenoverzicht/Theory%20clusters/Health%20Communication/Health_Belief_Model.doc/). As a result, impact assessment shifts authority to learners by opening the possibility of paying attention to all the important dimensions of learners' internal frames of reference. The process invites learners and instructors to develop a greater internal locus of control, which in turn allows them to relate the internal to external frames of reference, for instance &quot;satisfiers&quot; (in Herzberg's theory) such as achievement, recognition, and the work itself.
Impact data can also be used for &quot;external&quot; purposes such as assessing external motivators which are driven by accreditation systems and institutional requirements.
Impact assessment involves collecting data on student learning which expands the usual domain of inquiry to include learners' internal frames of reference and possible motivating factors outside the classroom. The key to the impact assessment method described in this workshop is simple: learning how to ask good questions (which also includes well-formed statements which invite response). In the Writing Across the Disciplines movement, we’ve learned not only to “learn to write” but “write to learn.” It is from the writing that “insights” emerge. The collection of these insights lead to “impact.” We have a plethora of questions to elicit content knowledge in our courses, and we're well practiced at asking them. But do we have a range of questions that we can offer and/or re-work to begin collecting insights? Chances are, the answer is &quot;no&quot; - so let's change that...
For starters, here is a list of questions which can be asked to elicit reflection and insights about feelings as an entree into the affective domain. [From the work of Barbara Bass and Joan McMahon, Maryland Writing Project, Towson University (MD).]
Here is a list of questions dealing with content. If they seem prosaic and unremarkable, that's because they are more the most part. There is a subtle difference between asking these types of questions, which invite learners to reflect on their work and mine or generate insights, and asking other types of questions which don't elicit that response. Of course, there are also less subtle differences in play here, specifically: - the difference between asking these questions and not asking them at all, and - the difference between asking them with the intent of cultivating insight and assessing impact, and asking them without following up on that intent.
The same observations apply to this type of questions.
These questions invite learners to assess the impact of the learning process itself. These are student engagement type questions, which also give instructors the ability to make “course corrections” -- modifications in your course in mid-stream based on student input and reflection.
For example, if you asked these questions, what do you predict would happen? How would that impact your course re-design?
Here is an example of collecting impact data. What question do you think this instructor asked to elicit this response? Marj, [the instructor], Now that the grades are in and you won't think that I'm trying to get one over on you.........I wanted to take a moment to let you know how much I appreciated this class and the leadership and compassion you brought to it. I was talking to my wife about the last night of class and the impact it has had on me. I found myself explaining to her that I felt I could have done a much better job with my projects. But that's not really the point - I learned a great deal about myself from the projects and the process that I used for the projects. I learned a great deal about how I learn, the best way for me to learn, and the best way for me to get the most out of time. What I learned from this course is more valuable than any grade, I believe that I have learned some life-long lessons. Something that you said struck me that night more than anything I've heard since I began the program and that was that you were providing us with a service that we paid for - that we were getting what we paid for, and that reminded me of a quote I once heard from Frederick Douglas, he said &quot;you may not always get what you pay for, but you always pay for what you get.&quot; Those who enroll in a program that has as it's core, as it's center, the focus of human development should expect to work, to achieve new heights in their own development - I believe that for the first time in this program, that's what I received. I can truly say that I will walk away from this semester with something permanent and substantial, and that is worth a tremendous amount to me - that is, in my eyes, the very essence of education. I wanted you to know that I am truly appreciative of your work and effort, and I know that others are too. I just wanted you to know these things. Thank you for everything you provided me and the other students with. Thanks. JD.
Impact Assessment Workshop <ul><li>John Sener </li></ul><ul><li>Joan D. McMahon </li></ul>July 27 - August 5, 2011 -- Module 2
MODULE 2 HOW DO I BUILD IMPACT ASSESSMENT INTO MY COURSE? NOTE: USING THE POWERPOINT “VIEW” OPTION IN THE MENU BAR, SELECT NOTES PAGES FOR EASIER READING.
COGNITIVE DOMAIN Goodhart, F. Verdi P. Kennedy S. Assuring Quality in Health Education . Presented at the Mid-Atlantic College Health Association, October 25, 1991, Baltimore. Reprinted with permission from Dr. Susan Kennedy compute describe discuss explain express identify locate report restate review tell translate apply calculate dramatize employ examine illustrate interpret operate practice schedule sketch solve use cite count define draw list name record relate repeat underline analyze appraise calculate categorize compare contrast debate diagram differentiate examine inventory question test arrange assemble collect compose construct create design formulate integrate manage organize plan prescribe propose appraise assess choose compare criticize estimate evaluate judge measure rank rate revise score select KNOWLEDGE COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION The hierarchical steps in the cognitive domain. Simple Complex
PSYCHOMOTOR DOMAIN The hierarchical steps in the psychomotor domain. Goodhart, F. Verdi P. Kennedy S. Assuring Quality in Health Education . Presented at the Mid-Atlantic College Health Association, October 25, 1991, Baltimore. Reprinted with permission from Dr. Susan Kennedy PERCEPTION ADAPTATION COMPLEX OVERT RESPONSE MECHANISM GUIDED RESPONSE SET distinguish hear see smell taste touch adjust approach locate place position prepare copy determine discover duplicate imitate inject repeat adjust build illustrate indicate manipulate mix set up calibrate coordinate demonstrate maintain operate adapt build change develop supply construct create design produce ORGANIZATION Simple Complex
AFFECTIVE DOMAIN Goodhart, F. Verdi P. Kennedy S. Assuring Quality in Health Education. Presented at the Mid-Atlantic College Health Association, October 25, 1991, Baltimore. Reprinted with permission from Dr. Susan Kennedy The hierarchical steps in the affective domain. Simple Complex RECEIVING CHARACTERIZATION ORGANIZATION VALUING RESPONDING accept attend develop realize receive recognize reply behave complete comply cooperate discuss examine obey observe respond accept balance believe defend devote influence prefer pursue seek value codify discriminate display favor judge order organize relate systematize weigh internalize verify (formal instruction does not address)
Locus of Control as the Basis for Impact Assessment <ul><li>Internal Frame of Reference </li></ul><ul><ul><li>Personal, Emotional, Social, Intellectual, Spiritual </li></ul></ul><ul><ul><li>Insight </li></ul></ul><ul><li>External Frame of Reference </li></ul><ul><ul><li>Achievement, Recognition, Work itself </li></ul></ul><ul><ul><li>Responsibility, Advancement, Growth </li></ul></ul>
Using Impact Data for “External” Purposes <ul><li>Course Evaluations </li></ul><ul><li>Justification for online offerings </li></ul><ul><li>Proof of Student Learning </li></ul><ul><li>Proof of Skill/Competency attainment </li></ul><ul><li>Proof of Compliance to Standards </li></ul>
Re-Configuring Traditional Assessments <ul><li>Learner Focused Discussions: What questions can you ask to get to “impact?” </li></ul>
Questions Dealing with Feelings <ul><li>How can you show, rather than tell what happened? </li></ul><ul><li>How did you feel when this happened? </li></ul><ul><li>How did you react? </li></ul><ul><li>What do you think your essay needs next? </li></ul><ul><li>What points are you worried about? </li></ul><ul><li>What was your reason for including this information? </li></ul><ul><li>How can you more clearly explain what you mean here? </li></ul><ul><li>How can you organize your ideas to illustrate how important this was to you? </li></ul><ul><li>How can you sustain your reader’s interest here? </li></ul>
Questions Dealing with Content <ul><li>Why did you include this information? </li></ul><ul><li>How can you expand this idea? </li></ul><ul><li>Which point do you want to focus on? </li></ul><ul><li>How can you be more specific here? </li></ul><ul><li>How can you eliminate some of this repetition? </li></ul><ul><li>What evidence can you produce for support? </li></ul><ul><li>Where would this detail make more sense? </li></ul>
Questions Dealing with Style/Format <ul><li>Where might this idea make more sense? </li></ul><ul><li>What words can you eliminate here? </li></ul><ul><li>Are all of these sentences/words necessary? </li></ul><ul><li>Where can you create some paragraphs? </li></ul><ul><li>How can you catch the reader’s attention right from the start? </li></ul><ul><li>How can you make some connections for your reader? </li></ul><ul><li>Where are you taking this idea? </li></ul><ul><li>What is the difference between this paragraph and the one before it? </li></ul><ul><li>How can you build up more gradually? </li></ul><ul><li>Does this sentence fit here? </li></ul><ul><li>How can you separate these two ideas? </li></ul><ul><li>How can you rearrange your ideas to build to your point more logically? </li></ul><ul><li>Which transition might be more appropriate here? </li></ul><ul><li>How can you make a clearer connection between these two ideas? </li></ul><ul><li>How can you tie these ideas together? </li></ul>
Mid-term Questions <ul><li>What are you discovering about your own learning-to-learn process? </li></ul><ul><li>How do you anticipate this will affect your learning in the future? </li></ul><ul><li>How are you different as a learner in this medium? </li></ul><ul><li>How are you experiencing this process? </li></ul><ul><li>How did you change as a learner through your involvement with this course? </li></ul><ul><li>How is using this process clarifying your electronic personality? </li></ul>