HM404

1. © 2017 American Health Information Management Association
© 2017 American Health Information Management Association
Health Informatics Research Methods:
Principles and Practice, Second Edition
Chapter 11: Selecting the
Research Design and Method
and Collecting Data

2. © 2017 American Health Information Management Association
Learning Objectives
• Select a research design and method appropriate to the
research question.
• Articulate the processes of data collection.
• Identify a data collection instrument appropriate to the
research question.
• Determine standard and suitable tools and techniques to
collect data.
• Select a sampling technique appropriate to the research
question.
• Explain how data collection procedures affect studies’
timelines and the quality of their collected data.
• Use key terms associated with instruments, sampling,
samples, and data collection appropriately.

3. © 2017 American Health Information Management Association
Selecting a Research Design
and Method
• Purpose of the research (most important)
• Internal validity and external validity
– Internal: Extent to which researchers’ design and
processes are likely to have prevented bias and increased
accuracy of results
– External: Extent to which results can be applied to other
settings, populations, or other phenomena
• Other factors in selection
– Skills
– Time
– Money and resources
– Potential subjects

4. © 2017 American Health Information Management Association
Collecting Data
• Planning
• Selecting an instrument
• Determining a collection method
• Deciding upon a collection strategy and
sample
• Performing pre-collection procedures
• Collecting data

5. © 2017 American Health Information Management Association
Plan for Data Collection
• What, how, by whom, and when
• Also includes timelines for obtaining
approvals, training collectors, and performing
pilot study
• Detail necessary for other researchers to
replicate and reproduce
• Important to document plan and its execution
to support validity of studies’ results

6. © 2017 American Health Information Management Association
Plan for Data Collection (cont.)
Quantitative
• Detailed and step-by-step
• Conjunction with statistical
analysis plan
– All necessary data collected
– Sufficient numbers of cases
Qualitative
• Less structured than
quantitative, dependent upon
‒ Time available
‒ Knowledge of phenomenon
‒ Available instruments
‒ Planned analyses
‒ Researcher’s experience

7. © 2017 American Health Information Management Association
Selection of an Instrument
• Instrument: Standardized, uniform way to
collect data
– Using a well-designed instrument minimizes
bias and maximizes the certainty of the
independent variable’s effect on dependent
variable

8. © 2017 American Health Information Management Association
Types of Instruments
• Checklists
• Coding schemes and
manuals
• Clinical screenings
and assessments
• Educational tests
• Index measures
• Interview guides
• Personality tests
• Projective techniques
• Psychological tests
• Questionnaires
• Rating scales
• Scenarios
• Vignettes
• And many others

9. © 2017 American Health Information Management Association
Sources of Instruments
• Electronic databases
– Health IT Survey Compendium
– Human Factors: Workbench Tools
– HaPI (Health and Psychosocial Instruments)
– Mental Measurements Yearbook with Tests in
Print
• Articles found during literature review

10. © 2017 American Health Information Management Association
Validity of Instruments
• Validity (of instruments): Extent to which the
instrument measures what it is intended to measure
– Face validity
• Subject matter experts
– Content validity
• CVR
• Essential, useful but not essential, not necessary
• CVI
– Construct validity
• Construct
• Convergent, discriminant, concurrent
– Criterion validity

11. © 2017 American Health Information Management Association
Reliability of Instruments
• Reliability: Extent to which a procedure or an
instrument yields similar results over repeated
trials, over time, across similar groups, within
individuals, and across raters
– Dependable and consistent in measurement
– Consistency, dependability, and reproducibility
characterize reliable instruments
• Types
– Interrater reliability and intrarater reliability
– Test-retest reliability
– Internal consistency reliability

12. © 2017 American Health Information Management Association
Reliability of Instruments (cont.)
• Interrater reliability and intrarater reliability
– Intraclass correlation coefficient
– Cohen’s kappa coefficient
• Test-retest reliability
– Pearson product-moment correlation coefficient
– Intraclass correlation coefficient
• Internal consistency reliability
– Split-half reliability coefficient (Spearman-Brown
correction)
– Kuder-Richardson formula
– Cronbach’s alpha

13. © 2017 American Health Information Management Association
Factors in Selecting an
Instrument
• Purpose: Match
between researcher’s
purpose and
instrument’s purpose
– Theoretical
underpinnings
– Operational definitions
• Most important to
assure that the data
collected are relevant to
the research question
• Satisfactory ratings for
validity and reliability in
a developed instrument
• Style and format of the
instrument
– Clear and direct
– Formats
• Delivery medium
• Language
• Age groups

14. © 2017 American Health Information Management Association
14
Attributes of Items
• Structured questions
– Closed-ended
– Choice from list of possible responses
– Advantage
• Easier for subject to complete
• Easier for researcher to tally and analyze
– Example: What is your gender?
___Male
___Female

15. © 2017 American Health Information Management Association
Attributes of Items (cont.)
• Unstructured questions
– Open-ended
– Free-form responses
– Advantage:
• Collect in-depth data
• Discover potentially unknown aspects of issue
– Example: What barriers prevent you from
exercising?

16. © 2017 American Health Information Management Association
Attributes of Items (cont.)
• Semi-structured questions
– Begin with structured question and followed
by unstructured to clarify
– Advantages of structured and unstructured
questions
– Would you consider yourself physical fit?
___Yes
___No
• Why or why not?

17. © 2017 American Health Information Management Association
Attributes of Items
• Numerical
• Respondent enters
number
• Metric data
• Specify unit of
measure
• Categorical
• Respondent selects
category or grouping
• Nominal or ordinal
data
– All-inclusive
– Mutually exclusive
– Form meaningful
clusters
– Sufficiently narrow or
broad

18. © 2017 American Health Information Management Association
Attributes of Items (cont.)
• Scales: Form of categorical item that uses progressive
categories, such as size, amount, importance, rank, or
agreement
• Scale
– Points (2, 3, 4, 5-verbal frequency, 7-expanded Likert)
– Likert scale (5-point)
– Reliability improves from 2 to 7, but then improvement trivial
• Semantic differential scale
– Perspectives and images
– Words that are polar opposites on ends of continua

19. © 2017 American Health Information Management Association
Attributes of Items (cont.)
• Standardized categories found during
literature review or obtained from
authoritative sources
– Races, age groups, and other subpopulations
– Allow comparisons to other researchers’
results

20. © 2017 American Health Information Management Association
Attributes of Items (cont.)
• Feasible logistics
– Public domain items can be copied and used
freely
– Proprietary items must be purchased and
cannot be copied
• Hidden costs
– Scoring
– Users manual or scoring guide

21. © 2017 American Health Information Management Association
Examples of Instruments Used
in Health Informatics and HIM
• System Usability Score (SUS)
• Software Usability Measurement Inventory
(SUMI)
• Questionnaire for User Interaction
Satisfaction (QUIS)

22. © 2017 American Health Information Management Association
Techniques and Tools of Data
Collection
• Survey: Systematic collection of self-report data
through interviews or questionnaires
– Census survey: All members of the population
– Sample survey: Representative members of the
population
• Observation: Collection of data by noting and
recording
– Tools developed prior to use
– Transcription prior to coding and analysis
– Rich data
– Saturation

23. © 2017 American Health Information Management Association
Techniques and Tools of Data
Collection (cont.)
• Elicitation: Collecting data by evoking,
bringing out, or drawing out through
interview or review of documents
– Purpose of obtaining unarticulated or tacit
knowledge is what classifies a technique as
elicitation
– Uncover informants’ unarticulated knowledge
– Used to obtain users’ and experts’ views

24. © 2017 American Health Information Management Association
Techniques and Tools of Data
Collection (cont.)
Data sources
• Primary
• Secondary
Data access
• Approval or permission
• Individual or aggregate
data
• Public or proprietary data
• Location of data
Data mining: use of various analytical tools to discover
new facts, valid patterns, and relevant relationships in
large databases

25. © 2017 American Health Information Management Association
Target Population, Sample, and
Sampling
• Target population: Set of individuals (or
objects) of interest to the researchers
• Sample: Set of units, such as portion of a
target population
• Sampling: Process of selecting the units to
represent the target population
– Sampling frame
– Coverage error
– Sampling error

26. © 2017 American Health Information Management Association
Methods of Sampling
• Random sampling
– Quantitative
– Unbiased selection of
subjects from target
population in which all
members have equal
and independent
chance of being
selected
– Underpins many
statistical tests
• Nonrandom sampling
– Qualitative
– No use of statistical
methods of probability
to select samples
– No equal or
independent chance
of selection

27. © 2017 American Health Information Management Association
Types of Samples
RANDOM NONRANDOM
Simple Convenience*
Stratified Purposive
Systematic Snowball
Cluster Quota
Theoretical
*Also sometimes used in quantitative research studies

28. © 2017 American Health Information Management Association
Sample Size and Sample Size
Calculation
• Sample size
– Number of subjects
determined by researcher to
be included in order to
represent the population
– Should be large enough to
support statistical tests
• Sample size calculation
– Quantitative and qualitative
procedures to estimate the
appropriate sample size
– Best guess
– Considerations
• Purpose
• Relationships among level of
significance, power, effect
size, statistical test, and
sample size
• Information about target
population
• Others

29. © 2017 American Health Information Management Association
Response Rate
• Response rate: Number of people who
completed or interviewed divided by total
number of people in the sample
• Adequacy of response rate
– Review literature for typical response rates and
factors affecting response rates
– Mixed mode approach
– Response bias: Systematic difference between
responders and nonresponders

30. © 2017 American Health Information Management Association
Data Collection Procedures
• Data collection procedures common to
both quantitative studies and qualitative
studies
– Approvals of oversight committees
– Training and testing
– Pilot study
– Assembling and storing data

31. © 2017 American Health Information Management Association
Review
• Selecting the appropriate research design and method increase the
likelihood that the data collected are relevant, high quality, and directly
related to the research question
• Factors in selecting a research design and method include purpose, internal
and external validity, and others
• Supporting the quality of the data and the study’s results are the processes
of data collection, planning, selecting an instrument, determining techniques
and tools, deciding upon a sampling strategy and sample, performing pre-
collection procedures, and collecting data
• An instrument is a standardized, uniform way to collect data
• Factors in selecting an instrument are its validity and reliability of which
there are multiple, the researcher’s purpose, and others
• Techniques and tools of data collection vary by the method
• Random sampling and nonrandom sampling are methods of sampling and
several types of samples exist
• Procedures associated with collecting data should be taken into account