The document outlines the steps in developing a scale for measurement, emphasizing the importance of clearly defining constructs and generating a comprehensive item pool. It discusses the need for expert review, item validation, and appropriate response formats to ensure the scale effectively measures the intended construct while minimizing ambiguity. It also highlights considerations for scale specificity, item quality, and response options, including the use of positively and negatively worded items.

1. Steps in Scale Development

2. Steps
 Determine what you want to measure
 Generate an item pool
 Determine a response format
 Have experts review the item pool
 Inclusion of validation items
 Administer items to a development sample
 Evaluate the items
 Optimize scale length

4. Determine What You Want to Measure
 Theory is key for clarity
– Ground the content of the scale in theories on the
construct of interest
– Limit the bounds of the construct so that it does not
drift into un-intended domains
– Specify a theoretical model to guide the scale’s
development
 Can be as simple as a well-formulated definition of
the construct being measured
 Can be as involved as a description of how the new
construct will relate to existing constructs

5.  Specificity is key to clarity
– Constructs relate better to each other when they
match in levels of specificity
• Do you want your measure to assess very specific
behaviors or be a more global measure of the construct?
• Actively decide the level of specificity that is
appropriate based on the intended use of the scale
– Areas to consider when actively deciding your scale’s
specificity:
• Content domain, setting, population

6.  Be clear about what to include
– Is your construct distinct from others?
– Does the measure match my goals for its use?
– Avoid using items that might “cross over” into a
related construct
– Be cautious of similar items that may assess very
different phenomena
• Know the frame of reference for and intended
purpose of your scale

7. Generate an Item Pool
 Create and select items with the specific
measurement goal in mind
– Use your description of the scale’s purpose to guide
this process
– Each item is a test of the strength of the latent
variable
– Think creatively about the construct of interest

8.  Be over inclusive and redundant
– Theoretical models that guide scale development are
based on redundancy
– Content that is common across many items will
aggregate, canceling out their irrelevant aspects
– Redundancy allows you to compare items and have a
preference for one over the other
– While redundancy is most prevalent in the initial
item pool, some redundancy in the final item pool is
desirable

9.  How many items do you need?
– More than you plan include in the final scale
– Lots of items increases your chances of good internal
consistency
– Initial pool can be three to four times larger than the
final pool

10.  Starting the writing process
– Focus less on quality and more on expressing relevant
ideas
– Write quickly and uncritically
– Be critical after you have 3 to 4 times as many items as
you need
 Identify a variety of ways to state the central concept
the scale is intending to measure
• Paraphrase the construct of interest
• Create additional statements that get at the same idea
somewhat differently
• Seek alternative ways to express important ideas

11.  Good Items
– Unambiguous
– Targets the appropriate reading level for the intended sample
– Instructions should be unnecessary
– Specific
– Avoid jargon
– Avoid asking opinions
– Avoid biased language
 Bad Items
– Exceptionally lengthy
– Unnecessarily wordy
– Multiple negatives
– Double barreled items
– Ambiguous pronoun references
– Non monotonic questions

12.  Positively and negatively worded items
– Positively worded = Items indicating high levels of
the latent variable when endorsed
– Negatively worded = Items indicating low levels of
the latent variable when endorsed
– Purpose of including both in a scale is to avoid
acquiescence, affirmation, or agreement
– Can be confusing to respondents
– Reverse worded items can perform poorly

13. Step 3: Determine a Response Format
 a Response Format • This step should occur at the same time you are
generating items so they are compatible • Example response formats: –
Thurstone Paired Comparison Scale
 Thurstone Paired Comparison Scale • Between each pair of things,
which one is most important to you, personally (when buying food in
general)?
 Example response formats (continued): – Guttman Scaling - “I think the
following contains pornographic materials.” Adult movies rated XXX A
[Yes] B [Yes] C [Yes] Scale Value 4 Playboy magazine [Yes] [Yes] [No] 3
Lingerie ads [Yes] [No] [No] 2 New York Times [No] [No] [No] 1 Subject
 Equally weighted items –All items in the scale are viewed as equivalent
“detectors” of the construct of interest –They are imperfect indicators but
can be aggregated into an acceptably reliable scale –Allows for a variety
of response options

14. Why is this item poor?

15.  Optimum number of response categories – Variability is important • Have
lots of items • Have lots of response options within items – Respondents
must be able to meaningfully discriminate between options • Ability to
discriminate between items may depend on specific wording or physical
placement of the response options – Investigator’s ability and willingness
to record a large number of values for each item
 Optimum number of response categories (cont’d) – Odd or even number
of response options depends on the investigator’s purpose • Odd = implies
a central “neutral” point • Even = forces commitment in one direction •
Neither is superior to the other
 Types of response formats – Likert Scale • Item is presented as a
declarative statement, followed by response options • Response options are
worded so they have roughly equal intervals of agreement • Used most
frequently to measure opinions, attitudes, beliefs • Must consider how
strongly you should word items in the initial item pool