Health Informatics Research Methods: Principles and Practice, Second Edition Chapter 11: Selecting the Research Design and Method and Collecting Data © 2017 American Health Information Management Association © 2017 American Health Information Management Association Learning Objectives Select a research design and method appropriate to the research question. Articulate the processes of data collection. Identify a data collection instrument appropriate to the research question. Determine standard and suitable tools and techniques to collect data. Select a sampling technique appropriate to the research question. Explain how data collection procedures affect studies’ timelines and the quality of their collected data. Use key terms associated with instruments, sampling, samples, and data collection appropriately. © 2017 American Health Information Management Association Selecting a Research Design and Method Purpose of the research (most important) Internal validity and external validity Internal: Extent to which researchers’ design and processes are likely to have prevented bias and increased accuracy of results External: Extent to which results can be applied to other settings, populations, or other phenomena Other factors in selection Skills Time Money and resources Potential subjects © 2017 American Health Information Management Association 3 Collecting Data Planning Selecting an instrument Determining a collection method Deciding upon a collection strategy and sample Performing pre-collection procedures Collecting data © 2017 American Health Information Management Association Plan for Data Collection What, how, by whom, and when Also includes timelines for obtaining approvals, training collectors, and performing pilot study Detail necessary for other researchers to replicate and reproduce Important to document plan and its execution to support validity of studies’ results © 2017 American Health Information Management Association Plan for Data Collection (cont.) Quantitative Detailed and step-by-step Conjunction with statistical analysis plan All necessary data collected Sufficient numbers of cases Qualitative Less structured than quantitative, dependent upon Time available Knowledge of phenomenon Available instruments Planned analyses Researcher’s experience © 2017 American Health Information Management Association 6 Selection of an Instrument Instrument: Standardized, uniform way to collect data Using a well-designed instrument minimizes bias and maximizes the certainty of the independent variable’s effect on dependent variable © 2017 American Health Information Management Association 7 Types of Instruments Checklists Coding schemes and manuals Clinical screenings and assessments Educational tests Index measures Interview guides Personality tests Projective techniques Psychological tests Questionnaires Rating scales Scenarios Vignettes And many others © 2017 American Health Information Management Associ ...

1. Health Informatics Research Methods: Principles and Practice,
Second Edition
Chapter 11: Selecting the Research Design and Method and
Collecting Data
© 2017 American Health Information Management Association
© 2017 American Health Information Management Association
Learning Objectives
Select a research design and method appropriate to the research
question.
Articulate the processes of data collection.
Identify a data collection instrument appropriate to the research
question.
Determine standard and suitable tools and techniques to collect
data.
Select a sampling technique appropriate to the research
question.
Explain how data collection procedures affect studies’ timelines
and the quality of their collected data.
Use key terms associated with instruments, sampling, samples,
and data collection appropriately.
© 2017 American Health Information Management Association
Selecting a Research Design and Method
Purpose of the research (most important)
Internal validity and external validity
Internal: Extent to which researchers’ design and processes are
likely to have prevented bias and increased accuracy of results
External: Extent to which results can be applied to other

2. settings, populations, or other phenomena
Other factors in selection
Skills
Time
Money and resources
Potential subjects
© 2017 American Health Information Management Association
3
Collecting Data
Planning
Selecting an instrument
Determining a collection method
Deciding upon a collection strategy and sample
Performing pre-collection procedures
Collecting data
© 2017 American Health Information Management Association
Plan for Data Collection
What, how, by whom, and when
Also includes timelines for obtaining approvals, training
collectors, and performing pilot study
Detail necessary for other researchers to replicate and reproduce
Important to document plan and its execution to support validity
of studies’ results
© 2017 American Health Information Management Association
Plan for Data Collection (cont.)

3. Quantitative
Detailed and step-by-step
Conjunction with statistical analysis plan
All necessary data collected
Sufficient numbers of cases
Qualitative
Less structured than quantitative, dependent upon
Time available
Knowledge of phenomenon
Available instruments
Planned analyses
Researcher’s experience
© 2017 American Health Information Management Association
6
Selection of an Instrument
Instrument: Standardized, uniform way to
collect data
Using a well-designed instrument minimizes bias and
maximizes the certainty of the independent variable’s effect on
dependent variable
© 2017 American Health Information Management Association
7
Types of Instruments

4. Checklists
Coding schemes and manuals
Clinical screenings and assessments
Educational tests
Index measures
Interview guides
Personality tests
Projective techniques
Psychological tests
Questionnaires
Rating scales
Scenarios
Vignettes
And many others
© 2017 American Health Information Management Association
Sources of Instruments
Electronic databases
Health IT Survey Compendium
Human Factors: Workbench Tools
HaPI (Health and Psychosocial Instruments)
Mental Measurements Yearbook with Tests in Print
Articles found during literature review
© 2017 American Health Information Management Association
9
Validity of Instruments
Validity (of instruments): Extent to which the instrument
measures what it is intended to measure

5. Face validity
Subject matter experts
Content validity
CVR
Essential, useful but not essential, not necessary
CVI
Construct validity
Construct
Convergent, discriminant, concurrent
Criterion validity
© 2017 American Health Information Management Association
10
Reliability of Instruments
Reliability: Extent to which a procedure or an instrument yields
similar results over repeated trials, over time, across similar
groups, within individuals, and across raters
Dependable and consistent in measurement
Consistency, dependability, and reproducibility characterize
reliable instruments
Types
Interrater reliability and intrarater reliability
Test-retest reliability
Internal consistency reliability
© 2017 American Health Information Management Association
11
Reliability of Instruments (cont.)
Interrater reliability and intrarater reliability
Intraclass correlation coefficient

6. Cohen’s kappa coefficient
Test-retest reliability
Pearson product-moment correlation coefficient
Intraclass correlation coefficient
Internal consistency reliability
Split-half reliability coefficient (Spearman-Brown correction)
Kuder-Richardson formula
Cronbach’s alpha
© 2017 American Health Information Management Association
12
Factors in Selecting an Instrument
Purpose: Match between researcher’s purpose and instrument’s
purpose
Theoretical underpinnings
Operational definitions
Most important to assure that the data collected are relevant to
the research question
Satisfactory ratings for validity and reliability in a developed
instrument
Style and format of the instrument
Clear and direct
Formats
Delivery medium
Language
Age groups
© 2017 American Health Information Managemen t Association
13

7. 14
Attributes of Items
Structured questions
Closed-ended
Choice from list of possible responses
Advantage
Easier for subject to complete
Easier for researcher to tally and analyze
Example: What is your gender?
___Male
___Female
© 2017 American Health Information Management Association
14
Attributes of Items (cont.)
Unstructured questions
Open-ended
Free-form responses
Advantage:
Collect in-depth data
Discover potentially unknown aspects of issue
Example: What barriers prevent you from exercising?
© 2017 American Health Information Management Association
15

8. Attributes of Items (cont.)
Semi-structured questions
Begin with structured question and followed by unstructured to
clarify
Advantages of structured and unstructured questions
Would you consider yourself physical fit?
___Yes
___No
Why or why not?
© 2017 American Health Information Management Association
16
Attributes of Items
Numerical
Respondent enters number
Metric data
Specify unit of measure
Categorical
Respondent selects category or grouping
Nominal or ordinal data
All-inclusive
Mutually exclusive
Form meaningful clusters
Sufficiently narrow or broad
© 2017 American Health Information Management Association
17

9. Attributes of Items (cont.)
Scales: Form of categorical item that uses progressive
categories, such as size, amount, importance, rank, or agreement
Scale
Points (2, 3, 4, 5-verbal frequency, 7-expanded Likert)
Likert scale (5-point)
Reliability improves from 2 to 7, but then improvement trivial
Semantic differential scale
Perspectives and images
Words that are polar opposites on ends of continua
© 2017 American Health Information Management Association
18
Attributes of Items (cont.)
Standardized categories found during literature review or
obtained from authoritative sources
Races, age groups, and other subpopulations
Allow comparisons to other researchers’ results
© 2017 American Health Information Management Association
19
Attributes of Items (cont.)
Feasible logistics
Public domain items can be copied and used freely
Proprietary items must be purchased and cannot be copied
Hidden costs
Scoring

10. Users manual or scoring guide
© 2017 American Health Information Management Association
20
Examples of Instruments Used in Health Informatics and HIM
System Usability Score (SUS)
Software Usability Measurement Inventory (SUMI)
Questionnaire for User Interaction Satisfaction (QUIS)
© 2017 American Health Information Management Association
21
Techniques and Tools of Data Collection
Survey: Systematic collection of self-report data through
interviews or questionnaires
Census survey: All members of the population
Sample survey: Representative members of the population
Observation: Collection of data by noting and recording
Tools developed prior to use
Transcription prior to coding and analysis
Rich data
Saturation
© 2017 American Health Information Management Association
Techniques and Tools of Data Collection (cont.)
Elicitation: Collecting data by evoking, bringing out, or

11. drawing out through interview or review of documents
Purpose of obtaining unarticulated or tacit knowledge is what
classifies a technique as elicitation
Uncover informants’ unarticulated knowledge
Used to obtain users’ and experts’ views
© 2017 American Health Information Management Association
Techniques and Tools of Data Collection (cont.)
Data sources
Primary
Secondary
Data access
Approval or permission
Individual or aggregate data
Public or proprietary data
Location of data
Data mining: use of various analytical tools to discover new
facts, valid patterns, and relevant relationships in large
databases
© 2017 American Health Information Management Association
24
Target Population, Sample, and Sampling
Target population: Set of individuals (or objects) of interest to
the researchers
Sample: Set of units, such as portion of a target population
Sampling: Process of selecting the units to represent the target
population
Sampling frame
Coverage error

12. Sampling error
© 2017 American Health Information Management Association
Methods of Sampling
Random sampling
Quantitative
Unbiased selection of subjects from target population in which
all members have equal and independent chance of being
selected
Underpins many statistical tests
Nonrandom sampling
Qualitative
No use of statistical methods of probability to select samples
No equal or independent chance of selection
© 2017 American Health Information Management Association
26
Types of
SamplesRANDOMNONRANDOMSimpleConvenience*Stratified
PurposiveSystematicSnowballClusterQuotaTheoretical
*Also sometimes used in quantitative research studies
© 2017 American Health Information Management Association
27
Sample Size and Sample Size Calculation
Sample size
Number of subjects determined by researcher to be included in

13. order to represent the population
Should be large enough to support statistical tests
Sample size calculation
Quantitative and qualitative procedures to estimate the
appropriate sample size
Best guess
Considerations
Purpose
Relationships among level of significance, power, effect size,
statistical test, and sample size
Information about target population
Others
© 2017 American Health Information Management Association
28
Response Rate
Response rate: Number of people who completed or interviewed
divided by total number of people in the sample
Adequacy of response rate
Review literature for typical response rates and factors affecting
response rates
Mixed mode approach
Response bias: Systematic difference between responders and
nonresponders

14. © 2017 American Health Information Management Association
Data Collection Procedures
Data collection procedures common to both quantitative studies
and qualitative studies
Approvals of oversight committees
Training and testing
Pilot study
Assembling and storing data
© 2017 American Health Information Management Association
Review
Selecting the appropriate research design and method increase
the likelihood that the data collected are relevant, high quality,
and directly related to the research question
Factors in selecting a research design and method include
purpose, internal and external validity, and others
Supporting the quality of the data and the study’s results are the
processes of data collection, planning, selecting an instrument,
determining techniques and tools, deciding upon a sampling
strategy and sample, performing pre-collection procedures, and
collecting data
An instrument is a standardized, uniform way to collect data
Factors in selecting an instrument are its validity and reliability
of which there are multiple, the researcher’s purpose, and others
Techniques and tools of data collection vary by the method
Random sampling and nonrandom sampling are methods of
sampling and several types of samples exist
Procedures associated with collecting data should be taken into
account

15. © 2017 American Health Information Management Association
1
7
Presentation on Threats to the Global Environment
Congratulations! The members of the United Nations found
great value in the two analyses you provided. They are now
asking you to develop a PowerPoint presentation that addresses
the four most critical threats to the global environment. They
are listed in the table below.
Energy sources
Civil war
Globalization
Poor health of entire populations
Lack of educational opportunities
Cultural taboos
Inappropriate uses of technology
Climate change
To complete this task, you must do the following:
Step I. Narrow the List from Eight to the Four Most Critical
Threats
To complete this step, complete the following tasks in order:
· Review research on each of the eight threats listed in the
table.
· Determine what you believe to be the current and potential
future impacts of each threat on the global environment.
· Choose the four threats that you see as the most critical by
considering which pose the greatest or most immediate risk to
us.
Step II. Create the PowerPoint Presentation
The completed version of this presentation will include a

16. minimum of 19 slides. Your audience consists of the United
Nations General Assembly.
PPT Content and Structure
1. A Title Slide: Include your name, course title, current date,
and the name of your instructor.
2. An Introduction Slide: List the four threats you chose, and in
the Notes section offer a brief narrative justifying these choices
3. Body Slides: The slide content is listed in the outline below.
For each body slide you develop, please include a paragraph in
the Notes section explaining how the details you have provided
in the slide are pertinent to the United Nations’ discussion on
selecting and prioritizing goals.
I. For your first threat(this is the threat you consider to be the
greatest risk/highest priority)
a. One slide on a brief history and assessment of the threat
b. One slide on the countries most affected by the threat, and
how those countries are affected (please give examples)
c. One slide on the effects of this threat on the world population
as a whole
d. One slide including a chart, graph, or compelling visual that
relates to the content you present in body slides a–c
II. For your second threat (this is the threat you consider to be
the second greatest risk/second highest priority)
a. One slide on a brief history and assessment of the threat
b. One slide on the countries most affected by the threat, and
how those countries are affected (please give examples)
c. One slide on the effects of this threat on the world population
as a whole
d. One slide including a chart, graph, or compelling visual that
relates to the content you present in body slides a–c
III. For your third threat (this is the threat you consider to be
the third greatest threat/highest priority)
a. One slide on a brief history and assessment of the threat
b. One slide on the countries most affected by the threat, and
how those countries are affected (please give examples)

17. c. One slide on the effects of this threat on the world population
as a whole
d. One slide including a chart, graph, or compelling visual that
relates to the content you present in body slides a–c
IV. For your fourth threat (this is the threat you consider to be
the fourth greatest threat/highest priority)
a. One slide on a brief history and assessment of the threat
b. One slide on the countries most affected by the threat, and
how those countries are affected (please give examples)
c. One slide on the effects of this threat on the world population
as a whole
d. One slide including a chart, graph, or compelling visual that
relates to the content you present in body slides a–c
4. A Conclusion Slide: Summarize your findings for the
Assembly.
5. Reference Slide: You can include full-text citations in the
Notes section of each slide or provide a reference slide at the
end of the presentation with the full citations of your sources.
Note: Please discuss the threats in order of priority as described
above, so the threat you consider the greatest should be
discussed first in the presentation and so on.
Note: Please use at least five credible sources to back up your
discussion
Note: The body slides should summarize your key takeaways,
whereas the Notes section of each body slide should discuss the
evidence and the details that support your takeaways. The
content in both the Notes and body sections requires citations
and sources.