The evaluation test_descriptionDocument Transcript
The Evaluation Test EDTECH 505Who likes tests? Right – almost no one. I’ve found that graduate students in particular (even if they areteachers who gives tests frequently) get quite nervous about actually *taking* tests. Its normal foranyone to be concerned about not knowing how to answer questions, receiving poor grades, etc. All thisresults in a relatively high level of test anxiety (although this has a certain usefulness despite thediscomfort it causes).For the “evaluation test,” for EDTECH 505, my goal is not to trick you or to see how many minute detailsfrom the text you can recall. I am not particularly concerned, either, with your ability to correctlyidentify dozens of definitions of glossary words. The assessment for your reading of the course text, TheABCs of Evaluation (Boulmetis & Dutwin, 2005), could take many forms – ranging from multiple choiceitems, to short-answer questions, to essay-length questions. While each of these types of assessmentshas its place in the learning process, I have developed a more compelling way to assess your knowledge.If you have taken a class with me previously, you will recognize the format. This is what I call a “visualmetaphor” exam. Let me explain.By “visual metaphor,” I mean that you have to take certain concepts (identified by me) and find, for eachconcept, some kind of visual representation of it (in this case, I mean a still picture or even somethingyou create). To be a true metaphor, the picture is not an exact representation of the topic, but instead arepresentation of it that reminds you of the concept. For example, I recently gave a presentation and Iwanted to talk about trustworthiness of the study on a slide. Rather than fill the slide with text (blah!), Iused a picture of a wild bird eating out of a person’s hand (I love Flickr.com!!!)The pictures, though, are not the only part of the test. There will be some actual words that you have towrite. These words are limited, though. I ask for the limited text for two reasons: 1) I do not want toread essays and few, if any, want to write them, and 2) (more importantly) a student who writes acompact, precise statement in her or his own words is much more likely to remember it.The Nitty GrittyYour visual exam has 33 precisely slides. These are listed in tabular format (below). Each one is based ona things you have read or defined over the past few weeks.On each “text” slide, you will have three areas of text: 1. Across the top will be the slide title (these are provided on the table below). 2. In the middle is your ~ 40 word (36-44 is the acceptable range) statement that explains the concept. 3. At the bottom (in a slightly smaller font) is the URL that points to the picture you’ve used OR the words “Created by author” if it is a picture from your own collection.NO text will appear on the visual slide unless the text is somehow part of the graphic (as it might be ifyou draw an evaluation model), and no pictures or distracting graphics should appear on the text slides.
Submission LogisticsYou can create your visual exam in any way you would like, but I will only evaluate it by usingGoogleDocs. Whatever you create, then, must be something I can see by ONLY using GoogleDocs. If yousimply upload a presentation, which causes me to have to open PowerPoint or some other program,then you will fail the exam on a technicality (which counts heavily toward any ‘do-over’).If you are unsure how to use the presentation tool in GoogleDocs, then there’s no better time to learnit! Actually, I am assuming you already know how to use one presentation software or another, and theskills readily transfer. If you are still lost, then I encourage you to use either A) the Help documentation,B) Tutorials on YouTube or on another site, or C) Your peer’s knowledge. Notice that I did not say,“Contact me,” as I do not give tutorials on productivity software or websites (Why not? Because I figurethat our graduate students, who are studying educational technology, should be able to figure it out ontheir own. The encouragement of these kind of skills is part of my ‘hidden curriculum”).Some people create a presentation directly from within the GoogleDocs’s presentation tool, whileothers use PowerPoint or the OpenOffice version of it, and then upload it to GoogleDocs. If you upload,you have to allow GoogleDocs to convert the presentation into GoogleDoc format. If you choose thisoption, be careful. If you make something too fancy on your laptop/desktop, as sometimes there is adisconnect between advanced features and the web-based version.After creating the GoogleDoc presentation, you must put it in the EDTECH 505 folder that you created atthe beginning of the course and shared with me. That way, I can simply go to your folder and see thepresentation there. You do NOT need to “publish” the presentation so that it can be viewed by anyoneon the web. Simply put it in the shared folder, and I’ll see it there. There is NOTHING to upload viaMoodle regarding this test.The presentation will be in the folder that already bears your name, so the file name does not need yourown. Instead, it can simply be “505 Reading Test.”
Slide Content TableSlides do not have to be numbered via text that you insert.Slide # Slide Title Text Content 1 Title slide Student name, Semester, and Class Section (a picture can be used here if you’d like – either of yourself or one that is particularly interesting/attractive ). 2 Reflection Slide You can write enough text to fill this slide if you would like, but I want you to respond to this prompt, “If you were to to consider the most important things youve learned about evaluation over the past few weeks, what would those lessons be? Why do you find these aspects to be important? Importantly, how are they related to you job (or future job) as an educational technologist?” There is no wrong or right answer here, but you should write a thoughtful response that has some substance to it. Do not simply repost a blog entry – though you are free to use thoughts from your previous entries as inspiration. Single sentences or incoherent thoughts will have points deducted. PLEASE do not use a font face any smaller than “12”. Merci. 3 (picture slide) A visual representation of (AVRO) the idea, “Why we should evaluate” 4 Rationale for Evaluation Your answer to the question “Why we should evaluate.” 5 (picture slide) AVRO the relationship of evaluation to educational technology 6 Evaluation & EdTech Your explanation of the relationship between evaluation and educational technology. 7 (picture slide) AVRO the difference between research and evaluation 8 Research & Evaluation Your explanation of the difference between evaluation and research. 9 (picture slide) AVRO of efficiency, efficacy, and impact (or just one of these is fine) 10 Efficiency, Efficacy, & Impact Your explanation of the difference between efficiency, efficacy, and impact. 11 (picture slide) AVRO of the impact of evaluation on decision making 12 Impact of Evaluation Your explanation of the impact evaluation has on decision making 13 (picture slide) AVRO one evaluation model (here, you should create a visual representation of the model). 14 (Model #1 name) Your explanation of the purpose of the first selected model 15 (picture slide) AVRO a second evaluation model (here, you should create a
visual representation of the model).16 (Model #2 name) Your explanation of the purpose of the second selected model17 (picture slide) AVRO quantitative vs. qualitative data18 Quantitative vs. Qualitative Your explanation of the difference between qualitative and quantitative data19 (picture slide) AVRO nominal, ordinal, interval & ratio data (so, four pictures).20 Levels of Data Your explanation of the differences between these types of data (you are allowed to use up to 36 - 60 words on this slide only).21 (picture slide) AVRO four different kinds of data collection instruments (again, four pictures)22 Data Instruments Your identification of the four instruments you placed on slide #21. (this will be very simple).23 (picture slide) AVRO formative and summative evaluations (two pictures)24 Formative vs. Summative Your explanation of the difference between formative and summative evaluation.25 (picture slide) AVRO population vs. sample26 Samples and Populations Your explanation of the difference between a population and a sample.27 (picture slide) AVRO validity vs. reliability28 Validity and Reliability Your explanation of the difference between validity and reliability29 (picture slide) AVRO independent and dependent variables30 Independent & Dependent Your explanation of the difference between independent and dependent variables31 (picture slide) AVRO criterion-referenced vs. norm-referenced tests32 Criterion- vs. Norm- Your explanation of the difference between criterion- referenced referenced and norm-referenced tests33 Activity Evaluation Answer this question: Please provide your feedback on this particular kind of “test.” What is your opinion about the use of visual metaphor to reinforce your learning?
Grading RubricCATEGORY OUTSTANDING SATISFACTORY BELOW (9.3-10) (8 – 9.2) EXPECTATION (< 8)Submission of test In GoogleDocs (10) Not in GoogleDocs (0)Slides in correct order Yes (10) No (0)Reflection slide Solid writing; great Good writing, sufficient Contains either poor depth; obvious depth. Does a decent writing, surface-level connection to EdTech. job at reflection but reflection, lack of No more than one slide. perhaps lacks a quality connections, incorrect No “wasted words” (ex., of reflectiveness or formatting, etc. “I think, I believe, I feel, connection that would etc.) . Font is be “outstanding.” Font is appropriate size. appropriate size.Picture Slides Overall incredible Good pictures or Fuzzy pictures, pictures pictures or graphics that graphics … the depth of or graphics that lack make me say, WOW! metaphor may be meaning, overall The visual metaphor is somehow lacking OR it pictures that are just interesting. Not seems to be completely silly or random, etc. necessarily obvious, but unrelated (tantamount connected. to a “random” picture”)Text slides The title appears at the Title and photo credit The slide title is missing top. The 40 word (+/- 4 appear on all slides. The or incorrect; the text words) explanation is 40 word (+/- 4 words) description of the idea succinct and correct. explanation is good – or concept is incorrect The photo credit perhaps something is or missing; the photo appears at the bottom incorrect or phrased credit it missing on a oddly. number of slides (-1 for each slide where the credit is missing)Instructors discretion Up to 5 points overall 55 points total possible
Frequent Questions 1. Can I use more than one picture on my picture slide? a. Yes, but not more than four. 2. I would rather just write an essay. Can I? a. No. 3. I would rather just take a 100 question multiple choice test. Can I? a. If you design it, vet it, and provide the answer key, I’ll think about it. 4. Can my pictures simply be silly things that I find wherever and put into the middle of slide. I will be sure that they are lossy and are of generally poor quality and hard to see. a. NO!! The picture(s) you select for each idea should fill the slide. They should be of good quality and should fill the slide itself (you do not need the highest resolution photo to do this, necessarily). You should try to find pictures that have Creative Commons licensing. 5. If I upload something incorrectly or if you cannot open it, do I automatically get a zero? a. While Id be sorely tempted to do so, you will not get a zero. However, you will not be eligible for full credit once things get corrected. If I find a submission error, I will let you know, and youll have 5 minutes to fix. Ok, longer than that.. but not as long as a week.REFERENCESBoulmetis, J., & Dutwin, P. (2005). The ABCs of evaluation (2nd Ed.). San Francisco, CA: Jossey-Bass