Course outcomes.com_hour_2_table_of_specifications

  • 837 views
Uploaded on

Aligning Outcomes, Instruction, and Assessment …

Aligning Outcomes, Instruction, and Assessment
(3 trainings)There is an entire field dedicated to the science of instruction. This training will provide the tools to review your existing courses to see if they are following the best practices of planning, delivering, and assessing learning outcomes of your course. This part 2 of 3 focuses on building a table of specifications or test blueprint to measure your desired learning outcomes.

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
837
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
36
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. In April 2011, The CPE PhD, LLC became Course Outcomes, LLC. The training materials are referred to in this packet are owned and copyrighted by Course Outcomes, LLC and can only be used for free trainings. They may not be rebranded nor sold without express written permission from Course Outcomes.
    Course Outcomes and The CPE PhD
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 2. Thank you for using The CPE PhD’s training services. The content of this training is copyrighted by The CPE PhD and may not be resold, repackaged, or printed for profit. Organization Members may download and repackage materials for their organization but not for profit without express written permission from The CPE PhD.
    Intro Slide
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 3. Managing Director’s Message
    An introduction to hour 2: Table of Specifications
    or test blueprint
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 4. Table of Specifications
    and Test Blueprints
    Part II
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 5. Test Construction Philosophy
    600 possible items from my course.
    My 60 item test.
    Suppose you wrote all possible questions for everything your course has taught a student (600 items or so). If you put all 600 of these items in a box, and pulled out 60 items, this would be a sample taken from your population of 600 items.
    A test is a sample of items you could have asked.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 6. Test Construction Philosophy
    Most of us do not have time to write 600 items, nor would we leave our certification tests up to a random drawing, so we create a structure to test the more important elements of our course.
    600 possible items from my course.
    My 60 item test.
    I don’t have time to write 600 items so I’m going to write 6 items on history of SOX, 4 items on section 2, etc.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 7. Test Construction Philosophy
    We also use intuitive reasoning to tell us that if a person can answer one item, they could probably answer another item. (e.g. If a person correctly knows the square root of 81 and 36, then how likely is it that they know the square root of 25?)
    Exactly how many items we need to write to make a generalization that a student knows the content can be answered intuitively via your experience or verified by a psychometrician.
    Psychometrician is a fancy way of saying professional test writer.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 8. The way a student judges if your test is fair is by three criteria:
    1) Is your item sample a fair representation of what they should or could have learned?
    Is your test aligned with your instruction?
    That test had nothing to do with what we learned!
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 9. The way a student judges if your test is fair is by three criteria:
    2) Is a score on the 60 item test generalizable to the larger pool of what the student has learned in the course?
    Is your test generalizable?
    This test score doesn’t really show all that I have learned.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 10. The way a student judges if your test is fair is by three criteria:
    3) Do the 60 items represent (in a fairly weighted fashion) all possible items you could have asked or did you over/under emphasize some points?
    Is your test fairly weighted?
    My tests only focus on my favorite parts of my course.
    Have you ever had a professor like this?
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 11. Purpose of this Training
    Construct a fair test that is aligned with your instruction.
    Once your generalinstructional objectives and specific learning outcomes are established you are now prepared to create what is called a Table of Specifications.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 12. The table of specifications serves as guide for both instruction and assessment.
    Table of Specifications
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 13. The table of specifications is a table or matrix that has three parts:
    A list of content areas
    A list of student abilities
    Number of Test items
    Both the content area and student abilities are derived from your SLOs.
    Table of Specifications
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 14. GIO – Students will understand the background of Sarbanes-Oxley legislation.
    SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
    As an Example
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 15. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 16. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 17. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 18. SLO – Students will be able to identify specific dates and events surrounding the Sarbanes-Oxley legislation.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 19. SLO – Students will be able to identifyspecific dates and events surrounding the Sarbanes-Oxley legislation.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 20. Now we are ready to begin planning how many
    items should be written.
    Notice that you can write several items in each cell that assess students’ ability to do something with the content.
    On what date was the Sarbanes-Oxley legislation passed?
    June 24, 2001
    October 23, 2003
    January 3, 2005
    December 4, 2008
    What event spurred the passing of the Sarbanes-Oxley legislation?
    Exxon Valdees Disaster
    Enron Scandal
    Other plausible but incorrect option
    Other plausible but incorrect option
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 21. An example of
    a table of specifications
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 22. © 2011 Course Outcomes, www.courseoutcomes.com
  • 23. As an example, maybe with audit committees and independent auditors students only need to identify certain guidelines but will never need to implement any procedures regarding these topics.
    Notice also that with the exception of the cells with zeros, no cell has less than 2 items.
    The numbers represent the number of test items written for each specific learning outcome.
    Note that some cells have zeros
    This would be done if you plan on having more than one assessment covering the same content.
    (e.g. assessments for each day as well as a post-assessment)
    Otherwise you run the risk of the students seeing the same item on two different exams.
    This will happen when content does not lend itself to a particular student ability.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 24. When selecting items for a test it is important to ensure that the test items reflect the emphasis of your instruction.
    Alignment of instruction
    and assessment
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 25. For example if hypothetically you spend 90% of your time teaching about financial disclosures
    Alignment of instruction
    and assessment
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 26. For example if hypothetically you spend 90% of your time teaching about financial disclosures and 10% of your time teaching about internal controls,
    Alignment of instruction
    and assessment
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 27. For example, if hypothetically you spend 90% of your time teaching about audit controls and 10% of your time teaching accounting statistics, you would want this ratio to be reflected in the test questions.
    Alignment of instruction
    and assessment
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 28. 90% of your instruction teaching “Y”
    90% of your
    test assessing “Y”
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 29. With this principle in mind an actual table of specifications may look like this:
    Alignment of instruction
    and assessment
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 30. © 2011 Course Outcomes, www.courseoutcomes.com
  • 31. Now you will be able to see what percentage of the content and student abilities are sampled by your test.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 32. © 2011 Course Outcomes, www.courseoutcomes.com
  • 33. At this point you can ask yourself, “Do these percentages reflect the emphasis of my instruction and course assignments?”
    These percentages are calculated
    by all of the items in the table; in this case – 24.
    2/24 = 8%
    This means that 8% of the test covers specific dates and events.
    It also means that 38% of the assessment tests student’s ability to identify
    If not you may need to make adjustments to your -
    by dividing the number of items in this entire row
    Test
    Instruction
    and/or
    Specific Learning Outcomes
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 34. Your Assignment
    Create a table of specifications for your course which reflects:
    (1) a balance of instruction and assessment (i.e. 25% of my instruction focuses on achieving this SLO so my test will measure 25% on this SLO).
    (2) the number of items which should be written.
    Ensure that your table of specifications has rows for the content, columns for student abilities, and numbers in the cells representing the number of items you will write.
    The content and student abilities will be taken from the specific learning outcomes you prepared for the first assignment.
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 35. CPE PhD test creation tool
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 36. CPE PhD test creation tool
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 37. CPE PhD test creation tool
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 38. CPE PhD test creation tool
  • 39. CPE PhD test creation tool
  • 40. CPE PhD test creation tool
  • 41. CPE PhD test creation tool
    © 2011 Course Outcomes, www.courseoutcomes.com
  • 42. If you have any questions or need assistance check the discussion board or contact Dr. Daniel Winder at the courseoutcomes@gmail.com
    www.courseoutcomes.com
    © 2011 Course Outcomes, www.courseoutcomes.com