Slides

Writing Effective Questions The most important thing to keep in mind when you develop test questions is that your job as an educator is to teach people so that they can learn and be successful . The idea is not to trick learners, or fail them, or dazzle them with your brilliance. Within this context, an effective test question is one that assesses: • whether a particular objective was achieved or mastered by the learner • whether the instruction was successful This points to the direct relationship between the learning outcomes and objectives and the questions you develop to test learner progress. It also points to the idea that tests are not just for learners. They also provide feedback to instructors and course developers about the effectiveness of the course content and its presentation. By the time you come to develop test questions, you should have already written and classified each objective in the related learning material. Ideally, a variety of cognitive levels is represented in the objectives (that is, some deal with facts , some with concepts , and some with the application of the information). Most tests rely too heavily on testing recall of facts. If the objectives are clearly defined and well written, however, the result will be test questions that also address a variety of cognitive levels.

When to Use Which Kind of Question Question types include: instructor marked, multiple choice, multiple multiple-choice, short answer, calculation, and true/false. There is always some debate about when it is best to use one type of question or another. Generally, for each objective to be tested, you should create some of each type of question, because the variety will improve the quality of the test question bank. The more questions and types of questions you have in the question bank, the better the chance that you will be able to locate suitable questions for specific tests. To some degree, the language used in the objectives may suggest using of a particular type of test question. For example, an objective that says a learner will be able to “discuss x” suggests an essay question. An objective that says a learner will be able to “identify x” suggests a multiple choice, short answer, or true/false question. Doing the work “up front” when you write learning objectives significantly reduces the amount of work required to develop effective test questions.

There are also practical considerations involved in choosing which types of test questions to develop, for example: • What kinds of tests are going to be delivered? For final exams, you might want to emphasize questions that assess the higher levels of thinking. For formative tests, perhaps questions that require recall and application of facts are appropriate. • How much time is available to create test questions? Certain types of questions require more time to create than others. Depending on the time available, you may tend to create more of one type than another. • How much time is available to mark test questions? Instructor-marked questions require more time to mark than other types, which may be machine-marked. Depending on the marking time have available, you may tend to create more of one type than another. • What skills do you want the learners to use? Certain types of questions encourage the use of different skills. For example, essay questions require learners to use their writing skills and their abilities to analyze, integrate, and organize information. • What is your personal preference? You may have preferences about using different types of questions, based on your experience. • What is the reading level of the learners? Certain types of questions require more reading and writing skills than others. Depending on the reading level of the learners, you may or may not want to create questions that require learners to exercise their reading skills. • What kind of technology is to be used for delivery? Some question features may not be appropriate, depending on the type of technology to be used for delivery. For example, if the course is to be delivered over slow communication lines (perhaps to a remote area), then you should not create questions that include large media files, because downloading will become frustrating to learners.

The table below outlines some of the advantages and disadvantages of each type of question. • hard to use for testing higher levels of knowledge and skills • learners may tend to guess • difficult to create unequivocally true or false statements • easy to mark • easy to collect statistics from • good for assessing mastery of facts True/False • may take some time to develop, because you need to identify any and all synonymous answers • fairly easy to develop • good for assessing mastery of details and specific knowledge Short Answer • time-consuming to develop good questions with suitable distracters • hard to use for testing higher levels of knowledge and skills • learners may tend to guess • easy to mark • easy to collect statistics from • good for assessing mastery of details and specific knowledge • tests all lower levels of learning Multiple Choice (and Multiple Multiple-Choice) • time-consuming to mark • may require you to defend your marking scheme • good for assessing complex learning processes and creativity • relatively easy to write Instructor Marked Disadvantages: Advantages: Question Type:

Tips for Creating Instructor-marked Questions An instructor-marked question is one for which the answer requires instructor evaluation. The most common type of instructor-marked question requires an essay answer. Below are some tips to help you create effective instructor-marked questions. Reserve instructor-marked questions for testing higherlevel thinking skills. The evaluation of instructor-marked questions often looks subjective. In the question header, clearly outline the marking scheme so that you can defend your evaluation if necessary. Indicate specific things that you will be looking for and how each is weighted (for example, grammar, spelling, presentation, clarity of argument, appropriateness of solution, etc.). Indicate to the learner how long a response you expect, and within what time limit. Use key words in the questions to specify the thinking processes you expect the learner to use (for example, compare and contrast , describe , analyze , etc.). Limit the scope of the question or problem so that it directs the learner to a particular type of answer. Tips for Creating Multiple-choice Questions A multiple-choice question requires that the learner choose a single best response from several choices. A multiple multiple-choice question requires that the learner choose all of the correct responses that apply, given several choices. For this type of question, there must be more than one correct response among the choices. For your information, the chance of a learner guessing on a multiple choice exam and scoring 60% or better is about 1 in 100,000,000 if there are at least 60 items, each with 4 answer choices. Below are some tips to help you create effective multiple-choice or multiple multiple-choice questions. NOTE: When you create a multiple multiple-choice question, be sure to indicate in the question how many correct answers there are. For example, “ Which of the following countries grow oranges? There are 3 correct responses.”

If you use a question header to introduce the question, say “Choose the best answer” rather than “Choose the correct answer” because there are often exceptions. Ensure that the stem of the question presents a selfcontained question or problem with enough information that the learner can speculate on a possible answer without looking at the responses. Also ensure that the stem always contains a verb. ; The capital of Canada is ____. A.) Ottawa B.) Toronto : The capital ____. A.) of Canada is Ottawa B.) of Canada is Toronto Make the stem of the question straightforward and clear. Avoid ambiguity and extra reading. ; The Italian Renaissance lasted for A.) a few months B.) a few years : The Italian Renaissance, which represented a rebirth in the arts and sciences, lasted for A.) a few months B.) a few years Avoid using negatives in the stem of the question. If you must use a negative, place it at the end of the sentence and highlight it. ; Which of the following is a Canadian provincial capital? ; All of the following are Canadian provincial capitals EXCEPT : : Which of the following is not the name of a Canadian provincial capital? Avoid providing information in the stem of the question that could be used to answer another question on the test. Watch for and eliminate grammatical cues that may reveal the correct response. ; A / An ____ watch has hands that show the hour, minute, and second. A.) analog B.) digital

: An ____ watch has hands that show the hour, minute, and second. A.) analog B.) digital Ensure that the correct answer completes rather than begins a sentence. ; The capital of Canada is _____. : ______ is the capital of Canada. Create 4 or 5 plausible responses for each question (including the correct answer). Do not include “nonsense” or unreasonable responses. Ensure that there is only one correct (or best) answer for a multiple-choice question. Do not use synonyms or equivalents to trick the learner. ; 4 cubed is equal to A.) 7 B.) 12 C.) 16 D.) 64 : 4 cubed is equal to A.) 6.4 B.) 0.64 C.) 64 D.) 64.0 Avoid the use of absolutes in responses. ; The sun A.) rises in the east B.) sets in the east C.) revolves around the earth D.) rises in the west : The sun A.) always rises in the east B.) sets in the east C.) revolves around the earth D.) rises in the west

Distribute the correct answer evenly among the responses to reduce the chance that a learner will try to guess the correct answer based on the pattern of correct answers. For example, do not always make C the correct response. Make all responses approximately the same length to reduce the chance that a learner will try to guess the correct answer based on the length of the response. Avoid using humour in the responses. It dates quickly and can easily be offensive. Avoid using “All of the above” and “None of the above” as responses. Most learners immediately eliminate them as possibilities, because such generalizations are so often incorrect. Present choices vertically for increased readability. ; How many Canadian provinces are there? A.) 5 B.) 6 C.) 10 : How many Canadian provinces are there? A.) 5 B.) 6 C.) 10 Arrange responses logically, either in alphabetical or chronological order.

Tips for Creating Short-answer Questions A short-answer question is one for which the answer consists of a brief text or numeric entry, such as words, phrases, equations, or numbers. Below are some tips to help you create effective shortanswer questions. • Word the questions to provide precise guidance about how the learner should respond. ; The distance from the earth to the sun is ____ kilometers. : The distance from the earth to the sun is ____. • Have only one blank in the question, and place it at the end of the question, not at the beginning or in the middle. ; The sum of 12 and 34 is ____. : The ____ of 12 ____ 34 is 46. • If you are looking for a key word in the response, provide a suitable prompt for that word in the question. ; The animal with the highest blood pressure is the ____. : The animal with the highest ____ is the giraffe. • Word the questions to ensure that there is only one correct answer for each question. ; The country with the greatest population is ____. (Do not abbreviate. Spell out the full name.) : The biggest country is ____. (Do not abbreviate. Spell out the full name.) • Indicate to the learner whether you expect him/her to use a particular method to arrive at the answer. • Indicate to the learner what degree of precision you expect (for example, number of decimal places), and whether you expect units to be included in the response. • Establish guidelines for what answers will receive full or partial marks. Where appropriate, indicate the weight for each part of the question within the question itself.

Tips for Creating True/False Questions A true/false question requires that the learner choose an either/or response from two choices. The choices can be true or false , yes or no , agree or disagree . Below are some tips to help you create effective true/false questions. • Avoid the use of negatives in the statement. ; The capital of Canada is Ottawa. : The capital of Canada is not Ottawa. • Avoid the use of absolutes (for example, all , none , never , always ) because they might give away the correct answer. ; The month of February has 29 days. : The month of February always has 29 days. • Make the statements straightforward and clear. Avoid ambiguity. ; The Italian Renaissance lasted only a few months. : The Italian Renaissance, which represented a rebirth in the arts and sciences, lasted only a few months. • Avoid using statements that are too broad, because they generally cannot be answered by an unequivocal true or false. ; Permeability tends to be more important than porosity in the recovery of oil. : Permeability is always more important than porosity in the recovery of oil. • Avoid using two items in one statement unless you are measuring a cause-effect relationship. ; A leap year occurs every four years. ; February has 29 days in a leap year. : A leap year occurs every fourth year, and February has 29 days in a leap year. • Make all statements approximately the same length to reduce the chance that a learner will try to guess the correct answer based on the length of the statement.

• Maintain an approximately equal number of true and false statements on a test, and distribute them randomly to reduce the chance that a learner will try to guess the correct answer based on the quantity or pattern of true or false answers. Writing Useful Answer Feedback Statements Although it can be time-consuming to prepare useful feedback statements for each correct and incorrect answer to a question, such feedback is an important aspect of interactive learning on-line. Answer feedback statements are related to the four possible types of responses from learners: correct, partially correct, expected wrong, and unexpected wrong. Possible appropriate feedback for each type of answer is shown in the following table. Acknowledges error. May show the correct answer, hints for arriving at the correct answer, or direction to suitable learning resources. Unexpected Wrong (that is, one that you did not anticipate) Acknowledges error. May show the correct answer, hints for arriving at the correct answer, diagnosis of error, or direction to suitable learning resources. Expected Wrong (that is, one you have seen learners make in the past) Acknowledges error. May show the correct answer, hints for arriving at the correct answer, diagnosis of error, or direction to suitable learning resources. Partially Correct Acknowledges correct response. May be congratulatory, or give additional information. Correct Possible Appropriate Feedback: Answer:

In order for answer feedback to be useful to the learner, it is important that, for each wrong response, you diagnose how a learner would arrive there and include a tip about how to find the correct answer. Here are some characteristics of effective feedback: descriptive, not evaluative – This helps to prevent a defensive reaction from learners. specific, not general – It should be directed toward behaviour that the learner can change. addresses learner needs as well as instructional goals – The feedback information should relate back to the learning objective being tested. timed to appear immediately after the action that resulted in a wrong response clear and straightforward – Learners should be able to understand immediately “what went wrong.” It is a good idea to ask learners to re-phrase the feedback messages from time to time as a quality-control check.

Using Taxonomies to Classify Questions A taxonomy is a title by which to classify or organize questions into categories based on common characteristics. Taxonomies generally have multiple levels or sub-categories associated with them. For example, if you decided to use Bloom’s “cognitive domain” as a taxonomy, then the levels or sub-categories associated with that taxonomy would include “knowledge, comprehension, application, analysis, synthesis, and evaluation.” You could classify a question using a combination of taxonomies. For example, one question can be classified against the following taxonomies: 1. Knowledge (a level from Bloom’s taxonomy) 2. Laboratory (a level from a hypothetical “Location” taxonomy) 3. Technology Kit (a level from a hypothetical “Resources Required” taxonomy) 4. Visual (a level from a hypothetical “Special Needs” taxonomy) Taxonomically classified questions are especially useful when you create assessment definitions to make up exams, because you can search the question bank and include only those questions that emphasize (for example) certain knowledge or skills. For a final exam, you might want to test higher levels of thinking, so you might only include questions from Bloom’s taxonomy levels analysis , synthesis , and evaluation . In another situation, you might select questions based on a “location” taxonomy to create an exam that contains only questions to be answered in a laboratory because of the need to have access to certain equipment. Some ideas for taxonomies that you might use or create include: cognitive or thinking skills (with the levels defined in Bloom’s Taxonomy of Learning: knowledge, comprehension, application, analysis, synthesis, evaluation ) research skills (with such levels as computer searching , interviewing , evaluating information ) locations or environments (for situations in which the question must be answered in a particular location, with levels such as laboratory , job site , in the presence of an instructor , etc.) particular topics or content areas (for example, job-site safety with levels such as first aid , CPR , companyregulations , etc.)

Different Types of OBJECTIVES What are Goals? Goals are broad, generalized statements about what is to be learned. Think of them as a target to be reached, or "hit." What are Instructional Objectives? Instructional objectives are specific, measurable, short-term, observable student behaviors. Objectives are the foundation upon which you can build lessons and assessments that you can prove meet your overall course or lesson goals. Think of objectives as tools you use to make sure you reach your goals. They are the arrows you shoot towards your target (goal). The purpose of objectives is not to restrict spontaneity or constrain the vision of education in the discipline; but to ensure that learning is focused clearly enough that both students and teacher know what is going on, and so learning can be objectively measured. Different archers have different styles, so do different teachers. Thus, you can shoot your arrows (objectives) many ways. The important thing is that they reach your target (goals) and score that bullseye!

Common Types of Objectives Psychomotor : Physical skills (e.g., "The student will be able to ride a two-wheel bicycle without assistance and without pause as demonstrated in gym class."); actions which demonstrate the fine motor skills such as use of precision instruments or tools, or actions which evidence gross motor skills such as the use of the body in dance or athletic performance. See also a detailed description of the psychomotor domain . Cognitive : understandings, awarenesses, insights (e.g., "Given a description of a planet, the student will be able to identify that planet, as demonstrated verbally or in writing." or "The student will be able to evaluate the different theories of the origin of the solar system as demonstrated by his/her ability to compare and discuss verbally or in writing the strengths and weaknesses of each theory."). This includes knowledge or information recall, comprehension or conceptual understanding, the ability to apply knowledge, the ability to analyze a situation, the ability to synthesize information from a given situation, and the ability to evaluate a given situation. See also Blooms' Taxonomy . Affective : attitudes, appreciations, relationships (e.g., "Given the opportunity to work in a team with several people of different races, the student will demonstrate an positive increase in attitude towards non-discrimination of race, as measured by a checklist utilized/completed by non-team members."). See also a detailed description of the affective domain.

Tips for Writing Objectives Objectives should specify four main things: Audience - Who? Who is this aimed at? Behavior - What? What do you expect them to be able to do? This should be an overt, observable behavior, even if the actual behavior is covert or mental in nature. If you can't see it, hear it, touch it, taste it, or smell it, you can't be sure your audience really learned it. Condition - How? Under what circumstances will the learning occur? What will the student be given or already be expected to know to accomplish the learning? Degree - How much? Must a specific set of criteria be met? Do you want total mastery (100%), do you want them to respond correctly 80% of the time, etc. A common (and totally non-scientific) setting is 80% of the time. This is often called the ABCD's of objectives, a nice mnemonic aid!

Examples of Well-written Objectives Audience - Green Behavior - Red Condition - Blue Degree - Pink Psychomotor - "Given a standard balance beam raised to a standard height, the student (attired in standard balance beam usage attire) will be able to walk the entire length of the balance beam (from one end to the other) steadily, without falling off, and within a six second time span. " Cognitive (comprehension level) - "Given examples and non-examples of constructivist activities in a college classroom, the student will be able to accurately identify the constructivist examples and explain why each example is or isn't a constructivist activity in 20 words or less." Cognitive (application level) - "Given a sentence written in the past or present tense, the student will be able to re-write the sentence in future tense with no errors in tense or tense contradiction (i.e., I will see her yesterday.)." Cognitive (problem solving/synthesis level) - "Given two cartoon characters of the student's choice, the student will be able to list five major personality traits of each of the two characters, combine these traits (either by melding traits together, multiplying together complimentary traits, or negating opposing traits) into a composite character, and develop a short (no more than 20 frames) storyboard for a cartoon that illustrates three to five of the major personality traits of the composite character." Affective - "Given the opportunity to work in a team with several people of different races, the student will demonstrate an positive increase in attitude towards non-discrimination of race, as measured by a checklist utilized/completed by non-team members." If you're paying attention here, you'll notice two things: As you move up the "cognitive ladder," it become increasingly difficult to precisely specify the degree. Affective objectives are the hardest objectives for most people to write and assess. They deal almost exclusively with internal feelings and conditions that can only be artifically observed externally. The verbs you use to describe the overt, measurable activity can be tricky to write. Fortunately, a page on psychomotor objectives , a page on cognitive objectives ( Blooms' Taxonomy ), and a page on affective objectives exists to assist you.

Typical Problems Encountered When Writing Objectives Describe what behavior you must observe. No true overt, observable performance listed. False performance Simplify, include ONLY ABCDs. Describes instruction, not conditions False givens Be more specific, make sure the behavior, condition, and degree is included. The objective does not list the correct behavior, condition, and/or degree, or they are missing. False/missing behavior, condition, or degree Simplify/break apart. The objective is too broad in scope or is actually more than one objective. Too vast/complex Solutions Error Types Problems

Writing Different Types of Questions And Objectives Prepared by: Irene Rose C. Canoy Carrie Faith T. Dalman Maria Buema V. Lingating Submitted to: Marionito Hinacay

Slides

by paxxx

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Slides Presentation Transcript