Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

- Imperialcheminc by IndiaMART InterM... 237 views
- HTML Parsing With Hpricot by Subhransu Behera 4374 views
- Dew Refrigeration Sales & Service, ... by IndiaMART InterM... 625 views
- Seraverdad by mifukaeri 133 views
- Jaws2008 Presen12 by umekoumeda 890 views
- 1st Periodical Test by Airel Atendido 2547 views

1,085 views

Published on

(3 trainings)There is an entire field dedicated to the science of instruction. This training will provide the tools to review your existing courses to see if they are following the best practices of planning, delivering, and assessing learning outcomes of your course. This part 2 of 3 focuses on building a table of specifications or test blueprint to measure your desired learning outcomes.

No Downloads

Total views

1,085

On SlideShare

0

From Embeds

0

Number of Embeds

5

Shares

0

Downloads

41

Comments

0

Likes

1

No embeds

No notes for slide

- 1. In April 2011, The CPE PhD, LLC became Course Outcomes, LLC. The training materials are referred to in this packet are owned and copyrighted by Course Outcomes, LLC and can only be used for free trainings. They may not be rebranded nor sold without express written permission from Course Outcomes. <br />Course Outcomes and The CPE PhD<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 2. Thank you for using The CPE PhD’s training services. The content of this training is copyrighted by The CPE PhD and may not be resold, repackaged, or printed for profit. Organization Members may download and repackage materials for their organization but not for profit without express written permission from The CPE PhD.<br />Intro Slide<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 3. Managing Director’s Message<br />An introduction to hour 2: Table of Specifications<br />or test blueprint<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 4. Table of Specifications<br />and Test Blueprints<br />Part II<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 5. Test Construction Philosophy<br />600 possible items from my course.<br />My 60 item test.<br /> Suppose you wrote all possible questions for everything your course has taught a student (600 items or so). If you put all 600 of these items in a box, and pulled out 60 items, this would be a sample taken from your population of 600 items. <br /> A test is a sample of items you could have asked. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 6. Test Construction Philosophy<br /> Most of us do not have time to write 600 items, nor would we leave our certification tests up to a random drawing, so we create a structure to test the more important elements of our course. <br />600 possible items from my course.<br />My 60 item test.<br />I don’t have time to write 600 items so I’m going to write 6 items on history of SOX, 4 items on section 2, etc.<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 7. Test Construction Philosophy<br /> We also use intuitive reasoning to tell us that if a person can answer one item, they could probably answer another item. (e.g. If a person correctly knows the square root of 81 and 36, then how likely is it that they know the square root of 25?) <br /> Exactly how many items we need to write to make a generalization that a student knows the content can be answered intuitively via your experience or verified by a psychometrician. <br />Psychometrician is a fancy way of saying professional test writer.<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 8. The way a student judges if your test is fair is by three criteria:<br /> 1) Is your item sample a fair representation of what they should or could have learned?<br /> Is your test aligned with your instruction?<br />That test had nothing to do with what we learned!<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 9. The way a student judges if your test is fair is by three criteria:<br /> 2) Is a score on the 60 item test generalizable to the larger pool of what the student has learned in the course?<br />Is your test generalizable? <br />This test score doesn’t really show all that I have learned.<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 10. The way a student judges if your test is fair is by three criteria:<br /> 3) Do the 60 items represent (in a fairly weighted fashion) all possible items you could have asked or did you over/under emphasize some points?<br /> Is your test fairly weighted?<br />My tests only focus on my favorite parts of my course.<br />Have you ever had a professor like this?<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 11. Purpose of this Training<br />Construct a fair test that is aligned with your instruction.<br />Once your generalinstructional objectives and specific learning outcomes are established you are now prepared to create what is called a Table of Specifications. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 12. The table of specifications serves as guide for both instruction and assessment. <br />Table of Specifications<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 13. The table of specifications is a table or matrix that has three parts:<br />A list of content areas<br />A list of student abilities<br />Number of Test items<br />Both the content area and student abilities are derived from your SLOs.<br />Table of Specifications<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 14. GIO – Students will understand the background of Sarbanes-Oxley legislation.<br />SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation. <br />As an Example<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 15. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 16. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 17. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 18. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 19. SLO – Students will be able to identifyspecific dates and events surrounding the Sarbanes-Oxley legislation. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 20. Now we are ready to begin planning how many<br />items should be written.<br />Notice that you can write several items in each cell that assess students’ ability to do something with the content.<br />On what date was the Sarbanes-Oxley legislation passed?<br />June 24, 2001<br />October 23, 2003<br />January 3, 2005<br />December 4, 2008<br />What event spurred the passing of the Sarbanes-Oxley legislation?<br />Exxon Valdees Disaster<br />Enron Scandal<br />Other plausible but incorrect option<br />Other plausible but incorrect option<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 21. An example of <br />a table of specifications<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 22. © 2011 Course Outcomes, www.courseoutcomes.com<br />
- 23. As an example, maybe with audit committees and independent auditors students only need to identify certain guidelines but will never need to implement any procedures regarding these topics.<br />Notice also that with the exception of the cells with zeros, no cell has less than 2 items.<br />The numbers represent the number of test items written for each specific learning outcome.<br />Note that some cells have zeros<br />This would be done if you plan on having more than one assessment covering the same content. <br />(e.g. assessments for each day as well as a post-assessment) <br />Otherwise you run the risk of the students seeing the same item on two different exams.<br />This will happen when content does not lend itself to a particular student ability. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 24. When selecting items for a test it is important to ensure that the test items reflect the emphasis of your instruction. <br />Alignment of instruction<br />and assessment<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 25. For example if hypothetically you spend 90% of your time teaching about financial disclosures<br />Alignment of instruction<br />and assessment<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 26. For example if hypothetically you spend 90% of your time teaching about financial disclosures and 10% of your time teaching about internal controls,<br />Alignment of instruction<br />and assessment<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 27. For example, if hypothetically you spend 90% of your time teaching about audit controls and 10% of your time teaching accounting statistics, you would want this ratio to be reflected in the test questions.<br />Alignment of instruction<br />and assessment<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 28. 90% of your instruction teaching “Y”<br />90% of your <br />test assessing “Y”<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 29. With this principle in mind an actual table of specifications may look like this:<br />Alignment of instruction<br />and assessment<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 30. © 2011 Course Outcomes, www.courseoutcomes.com<br />
- 31. Now you will be able to see what percentage of the content and student abilities are sampled by your test. <br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 32. © 2011 Course Outcomes, www.courseoutcomes.com<br />
- 33. At this point you can ask yourself, “Do these percentages reflect the emphasis of my instruction and course assignments?” <br />These percentages are calculated<br />by all of the items in the table; in this case – 24. <br />2/24 = 8%<br />This means that 8% of the test covers specific dates and events.<br />It also means that 38% of the assessment tests student’s ability to identify<br />If not you may need to make adjustments to your - <br />by dividing the number of items in this entire row<br />Test<br />Instruction<br />and/or<br />Specific Learning Outcomes<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 34. Your Assignment<br />Create a table of specifications for your course which reflects:<br />(1) a balance of instruction and assessment (i.e. 25% of my instruction focuses on achieving this SLO so my test will measure 25% on this SLO). <br />(2) the number of items which should be written.<br />Ensure that your table of specifications has rows for the content, columns for student abilities, and numbers in the cells representing the number of items you will write.<br />The content and student abilities will be taken from the specific learning outcomes you prepared for the first assignment.<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 35. CPE PhD test creation tool<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 36. CPE PhD test creation tool<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 37. CPE PhD test creation tool<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 38. CPE PhD test creation tool<br />
- 39. CPE PhD test creation tool<br />
- 40. CPE PhD test creation tool<br />
- 41. CPE PhD test creation tool<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />
- 42. If you have any questions or need assistance check the discussion board or contact Dr. Daniel Winder at the courseoutcomes@gmail.com<br />www.courseoutcomes.com<br />© 2011 Course Outcomes, www.courseoutcomes.com<br />

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment