Symposium 2020: Mark Reckase - Translation of Policy to Cut Score

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Translation of Policy to Cut-Score:
What Needs to be Considered?
Mark D. Reckase
Michigan State University

Standards Are
Common:
How are they
developed?
 Body mass index
 Impact on crash test dummies
 Nutritional information c...

Some of my History
 My Ph.D. is in psychology with emphasis on statistics and
measurement.
 For years, I avoided learnin...

Some Observations
 Different standard setting methods yield different results.
 This is troubling – it means an informed...

Some Tentative
Conclusions
 Standard setting processes
are not equally good.
 All standard setting processes
are modifie...

These observations
and the need
to defend our work
led me to investigate
the credibility
of standard setting
methods.
But ...

There is no such thing as a
theory of standard setting!

Basic Concepts
for a Theory
of Standard Setting

Standard Setting
Is Not Standard Setting!
It is translation of statement of policy
to the language of the reporting score
...

A Theory of
Standard Setting
 Standard is set by the statement of
policy – the Policy Definition.
 The Policy Definition...

The Policy Definition of the National
Council of State Boards of Nursing
 “Licensure is the process by which boards of nu...

The Implied
Continuum
 A test is developed or selected to
operationally define the continuum.
 Evidence for the validity...

Panelists Are Translators from Policy
to the Reporting Score Scale
The panelists are translating
policy not making policy....

A Problem:
Policy Definitions Tend to Be Vague
 To make the definition more precise and
concrete, they are translated to ...

A Problem:
Few Persons
Are Expert in
the Content
and the
Reporting
Score Scale
Can find persons who are expert
in the lang...

Panelist Selection and Training
•Content of the test
•The examinee population
Panelists need to be expert in
•The details ...

Schematic
for the
Theory
Agency
Policy
Definition
Content Specific
Definition
Hypothetical
Construct
Design and
Developmen...

The Standard Setting Method
 The goal is to help panelists perform the
translation.
 The method needs to provide informa...

A Validity Argument for the Standard
Policy implies
continuum
Policy implies
continuum
Policy implies
level
Policy implies...

Implications
of the Theory
 Policy definition should be carefully
crafted to support continuum and level
 Reporting scor...

Reference
 Reckase, M. D. (2006). A conceptual framework
for a psychometric theory for standard setting
with examples of ...

Questions?

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1035460 views
Storytelling For The Web: Integrate... by Chiara Aliotta 947335 views
Artificial Intelligence, Data and C... by OECD Directorate ... 869118 views
How to Leverage AI to Boost Employe... by SocialHRCamp 364901 views
2024 State of Marketing Report – by... by Marius Sescu 207323 views
Everything You Need To Know About C... by Expeed Software 227648 views

View on Slideshare

1 of 22 Ad

View on Slideshare

1 of 22 Ad