• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Test construction (for content staff) eg feb08 erp

Test construction (for content staff) eg feb08 erp






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Test construction (for content staff) eg feb08 erp Test construction (for content staff) eg feb08 erp Presentation Transcript

    • Test Construction Procedure Zarko Vukmirovic Workshop: Test Construction: 2008 CAPS Forms Assembly
      • Scope and Objectives
      • Scope:
        • To present and discuss the procedure for assembling CAPS operational and field-test forms.
        • The procedure is also designed to be a model for other assessment programs.
      • Objectives:
        • Participants will learn the principles of test construction and will be able to use the technical resources for assembling of test forms.
        • Participants will assemble 2008 CAPS operational forms.
    • Definitions
      • The purpose of the Test Construction
        • Assemble the O perational and F ield-test forms
        • Use the Item Bank
        • Use Test Construction Specifications .
      • The final products
        • Pull Lists (list of items included in the test)
        • Test Map (precise specification of item locations in the test)
    • Responsibilities
      • Test Development Director
        • Works with Psychometrician on Test Construction Specifications ,
        • Coordinates the process and approves pull lists.
      • Content Specialist
        • Selects items using Test Construction guidelines,
        • Checks accuracy of the final pull list.
    • Responsibilities
      • Psychometrician
        • Works with Test Development Director on Test Construction Specifications ,
        • Provides training for Content Specialists and other staff,
        • Assists with item selection,
        • Interprets item and form statistics,
        • Approves pull lists
      • Editorial Team
        • Reviews the items in the final pull list.
      • Quality Assurance Staff
        • Verifies the final pull lists against source documents.
      • IT Staff
        • Provides support for using information technology
    • Inputs & Assumptions
      • Documents:
        • Test Construction Specifications (including domain and cluster specifications).
      • Item Bank:
        • Item Content
        • Item Statistics
      • Assumptions:
        • Item content and statistics databases are populated with sufficient number of items.
        • All items are reviewed for content and psychometric qualities and accepted by review committees.
        • All staff is trained for test construction procedure
    • Outputs
      • Content Specialist will create
        • Pull Lists
        • Test Maps.
      • Psychometrician will create:
        • Curves: TCCs, TIFs, SEMs,
        • Statistical Summary Sheets,
        • Test Construction Report
    • Procedure (1)
      • Content Specialist selects operational items using the Item Bank and following Test Construction Specifications .
        • Match the blueprint and follow other content guidelines
        • Match statistical targets (average p-value and point-biserial correlations)
      • Content Specialist provides the pull list to Psychometrician (Item Bank: an export file is generated).
    • Procedure (2)
      • Psychometrician will use a statistical program to merge the pull list with the statistical data for the items (if using Item bank: provided in the export file).
      • Psychometrician will use a statistical program to create Statistical Summary Sheets and test curves (TCC, TIF, and SEM).
      • Psychometrician will verify that items and forms characteristics meet requirements specified in Test Construction Specifications .
    • Procedure (3)
      • Item Statistics checks
        • Number of Items (overall test and by domain)
        • Item difficulty
        • Item discrimination
        • Options analysis
        • DIF statistics
      • Forms characteristics checks
        • Maximum Points (overall test, by reporting category, by item type)
        • Does obtained TCC approximates target TCC sufficiently
        • Is TIF sufficiently high at al relevant scale points
        • Is SEM low at relevant points
      • Linking items checks
        • Content representation
        • Statistical representation
        • Position maintenance
    • Procedure (4)
      • If all the criteria are met Psychometrician will sign off the form.
      • If modifications are needed, Psychometrician specifies the guidelines for changes .
      • Optionally, Psychometrician will meet with Content Specialist to review and make the necessary changes.
      • When the changes are complete, steps 1 - 3 are repeated.
    • Procedure (5)
      • Once all changes are complete and the criteria are met, Psychometrician, Content Specialist, and TDD will sign off the form.
      • If the customer wants to review the test map, it is at this point that the test map will be sent to the customer by the TDD.
      • The TDD will provide a cover letter along with an explanation of the test map.
      • After customer approval the test map is forwarded to composition staff to prepare the form for printing.
    • Test Construction System
      • Item Bank
        • Search and review items
        • Selecting items to be included in test forms
        • Printing forms preview
        • Exporting item IDs (and statistics)
      • SPSS program
        • Generating TCCs, TIFs, SEMs,
        • Generating statistical summary sheets
    • Test Characteristic Curve
    • Test Information Curve
    • Standard Error of Measurement