HM404 Ab120916 ch12

© 2017 American Health Information Management Association
Health Informatics Research Methods:
Principles and Practice, Second Edition
Chapter 12: Analyzing Data and
Presenting Results

Learning Objectives
• Analyze statistical data for decision making.
• Interpret descriptive and inferential statistics.
• Describe the use of quantitative and qualitative
data in decision making.
• Present research results in formats consistent with
recognized standards.
• Use key terms associated with analyzing data and
presenting results appropriately.

Analyzing Data and Presenting
Results
– Quantitative data analysis
• Examines, probes, and
transforms large amounts of
numerical data into
understandable information
• Application of descriptive
statistics and inferential
statistics
– Qualitative data analysis
• Systematically working with
data to create coherent
descriptions and explanations
of phenomena
• Multiple analytic techniques,
but two common used are
grounded theory and content
analysis
•Researchers transform their data into results through
analysis

Quantitative Data Analysis
• Research data can be subjected to more than
one statistical test or technique
• Descriptive statistics and tabular and
graphical displays explore and describe data
• Inferential statistics are performed to make
predictions and test hypotheses
• Researchers maximize use of their data by
using a variety of statistical tests and
techniques to analyze different aspects of
their data

Statistical Analysis Plan (SAP)
• Statistical analysis plan: Document that contains
technical and detailed descriptions of statistical
analyses that will be performed on a research
study’s variables and other data
• Developed as part of overall plan for research
study
• Purpose is to determine whether data collection
strategies will obtain all data necessary for
planned statistical tests
• Lists each data element required for each planned
statistical test

Statistical Significance vs.
Practical Significance
• Statistical significance
– Statistical significance
based on calculations
– Effect (association or
difference) is not
random chance
– Very large sample can
create statistical
significance just by
sheer size (small can
mask)
• Practical (clinical)
significance
– Significance alone
meaning importance
– Worthy of influencing
decisions, practices, or
policies
– Effect size
– Not guaranteed by
statistical significance
Researchers use the word significance two ways

Null Hypothesis Significance Testing
and Significance Level
Significance level
• Pre-established threshold to
reject null
• Alpha level (α)
• Lower alpha to avoid type I
error
P-value
• Output of statistical test for
significance
• P less than alpha can be
rejected
• P-value not sole determinant
• Purpose of null hypothesis significance testing (NHST):
Determine the likelihood that the research findings are not
result of random chance or biased sample
• Applied to null hypothesis
• Reject the null

Power
• Power: Probability of identifying real differences or relationships
between groups
– Specifically: Likelihood of failing to reject a false null hypothesis (error
that there is no difference or relationship when one really exists)
– Cutoff usually set at 0.80 or higher
• Power at 0.80 means there is a 20 percent chance that the researcher will
wrongly determine that no difference or relationship exists in the results, when in
actuality, there is a difference or relationship
– Cutoff set for power depends on the research
• Less stringent than significance level because failing to assert a difference or
relationship has fewer ramifications for researchers and the general public than
falsely asserting a difference or relationship when none really exists
– Beta (B) designates the probability of making a type II error because
power is 1.0 – B.

Type I Error and Type II Error
• Type I error occurs
when the researcher
erroneously rejects
the null hypothesis
when it is true; in
actuality, there is no
difference or
relationship
• Type II error occurs
when the researcher
erroneously fails to
reject the null
hypothesis when it is
false; in actuality,
there is a difference

Preparation of Data
• Garbage in, garbage out
• Begins with data collection plan
• Procedures in place
– Transcribing
– Data entry
– Scoring
– Quality checks
• Also includes selecting unit of analysis, intention-
to-treat analysis, addressing missing values, and
data cleaning

Selecting the Unit of Analysis
• Unit of analysis: Group, object, or
phenomenon for which researchers have
collected data to analyze
– Study’s focus
– Unit of analysis should match unit of
randomization
– Selecting correct unit of analysis increases
likelihood that study’s results will be accurate
– Explicitly stating unit of analysis in research plan
may help researchers avoid a unit of analysis
error

12
Intention-to-Treat (ITT) Analysis
Purposes
• Minimize bias
• Maintain effects of
randomization
• Reliably indicate the
effects of treatments
Processes
• Count subjects despite
dropping out, etc.
• Accurate results
(potentially conservative)
• Reported in “Results”
often with flow diagram
Intention-to-treat (ITT) analysis: principle in which subjects
of a randomized controlled trial (RCT) are analyzed within the
group to which they were originally allocated with no regard to
noncompliance or deviations from protocol

Addressing Missing Values
• Missing values is common problem
• Data incomplete
• Variables do not contain values for some subjects or cases
• Failure to address missing values jeopardizes study’s results
• Terms
– Missing completely at random (MCAR)
– Missing at random (MAR, misnomer)
– Missing not at random (MNAR)
• Processes
– Case deletion
– Single imputation (substituting values)
– Maximum likelihood estimation
– Multiple imputation

Cleaning Data
• Data cleaning: Process of detecting, diagnosing,
and editing faulty data
– Purposes
• Find and correct errors
• Minimize errors’ impact on study’s results
– Process
• Finding duplications
• Checking internal consistency
• Identifying outliers
– May require many hours of detailed work
– Number and type of errors and how they were
cleaned should be reported

Descriptive Statistics
• Chapter 9
– Frequency distributions
– Tables
– Graphical displays
– Measures of central
tendency
– Measures of dispersion
– Some correlations
• This chapter
– Sensitivity, specificity,
and predictive values
– Receiver operating
characteristic (ROC)
curve analysis
– Measures of effect
• Confidence intervals
• Odds ratios
• Risk reduction statistics
• Others

Sensitivity and Specificity
• Sensitivity: Ability of a
measure to detect a
characteristic when the
characteristic exists
• Specificity: Ability of a
measure to detect the
absence of a
characteristic when it is
absent
• Sensitivity and specificity
are related to type I and
type II errors
• Likelihood ratio:
Merger into one number
of sensitivity and
specificity
• True positive (TP), correct labeling of an
individual as having the disease or
outcome.
• False negative (FN), incorrect labeling
of an individual as not having the
disease or outcome when he or she
does; false negatives are associated
with type II errors.
• True negative (TN), correct labeling of
an individual as not having the disease
or outcome when he or she does not.
• False positive (FP), incorrect labeling of
an individual as having the disease or
outcome when he or she does not; false
positives are associated with type I
errors.

Positive and Negative
Predictive Values
• Predictive values are useful because they put the
results of positive and negative indicators into context
• Positive predictive value (PPV)
– Probability that a person has the characteristic when the
measure is positive
– Also known as precision rate (particularly in information
retrieval) and post-test probability of disease
• Negative predictive value (NPV)
– Probability that a person does not have a characteristic
when the measure is negative

Receiver Operating Characteristic (ROC)
Curve Analysis
• Receiver operating characteristic
(ROC) curves plot sensitivity versus
specificity at different thresholds and
graphically show a measure's ability
to predict an outcome
• Measures performance
– Predictive algorithms
– Diagnostic tests
– Other
• Visualization of trade-off between
sensitivity and specificity
• ROCs graph all possible cut points
• Area under the curve (AUC)
– Perfect performance is 1.0
– Random guessing is 0.05
Perfect
performance
with 100%
AUC
Line is
random
guessing

Measures of Effect
• Measures of effect
– Put a study’s results in context for practitioners by
providing information about magnitude of
association or difference
– Can also be used to determine statistical
significance by showing extent to which null
hypothesis is false and represents degree to
which sample’s results differ from null hypothesis
– Show practical significance
– Examples: Confidence intervals, odds ratios, risk
reduction statistics, and other indxes of effect

Confidence Interval (CI)
• Confidence interval (CI): Range of values for
a sample’s characteristic within which it is
estimated that the population’s characteristic
lies
• Indicate the precision (degree of certainty) of
the estimate
• Confidence limits, an upper limit and a lower
limit, are on each end of the range
• Can be calculated for means, proportions,
risk ratios, odds ratios, and other statistics

Confidence Interval (CI) (cont.)
Narrower CIs
• Greater homogeneity
• Larger sample size
• Lower confidence level
(90% narrower than
95%)
Wider CIs
• Greater heterogeneity
• Smaller sample size
• Higher confidence level
(95% wider than 90%)
• Way to measure precision of estimate
• Width of calculated CIs affected by factors, such as
heterogeneity/homogeneity of sample, sample size, and
selected confidence level

Confidence Level
• Confidence level: Probability that the confidence
interval includes the population’s value
• Set to represent desired level of certainty
• Percentage: Confident that the true results will be
in the CI’s range that percent of the time
• Common percentages
– 90% confidence level (10 percent significance level)

Odds Ratio (OR)
• Odds ratio (OR):
chance of an event
occurring in one group
compared to the chance
of it occurring in another
group
• Ratio of ratios
– Ratio of the odds of the
treatment group to the
odds of the control
group with individual
odds of both groups
being calculated
• OR=(A/C)/(B/D)

Risk Reduction Statistics
• Risk reduction statistics provide the probability of
success for an intervention or exposure and each
outcome’s expected probability
• Analyses are conducted related to exposures to
interventions or to risks
• Absolute risk (AR)
– Probability of an occurrence of an event in an entire
population of subjects
• Absolute risk reduction (ARR)
– Arithmetic difference between the event rate of two
groups

Risk Reduction Statistics (cont.)
• Relative risk (RR)
– Rate of risk of an outcome in exposed subjects to the
rate of risk of an outcome in unexposed subjects
• Relative risk reduction (RRR)
– Percentage that an intervention reduces risk in the
experimental group compared to the control group
• Number needed to treat (NNT)
– Number of people who need to receive an
intervention in order to for one person to benefit from
the intervention (considered user-friendly)

26
Other Indexes of Effect Size
• Effect sizes: Quantify the degree to which a
study’s results should be considered important or
unimportant
• The larger the value of the effect size, the greater
the presence of the phenomenon under study
• Effect sizes are valuable statistical tests because
they are unaffected by sample size, unlike NHST
that was described earlier
• Families of indexes of effect size
• Selective examples: Cohen’s d, Hedges’ g, Glass’
delta (Δ), and R2

Tabular and Graphical Display
• Tables and graphs are generated to record
and examine data, to describe variables, to
compare and contrast variables, and to see
relationships among variables, and to present
results
• Choice depends information being
communicated
• Purpose is to support readers’ understanding
• Present data one way or mode

Tabular and Graphical Display
(cont.)
• Table
– Exact numerical values are being presented
– Reader comparing a few values at a time
– Level of data
• Graph
– Visual comparison needed
– Example: Trend data
– Uses of x-axis and y-axis
28

Graphical Display
• Bar charts visually present data showing comparisons between and
among variables and illustrating major characteristics in the
frequency distribution of data
• Histograms show major characteristics in the distribution of data and
summarize data about variables whose values are numerical and
measured on an interval or ratio scale
• Line graphs show trends for one variable over time with x-axis
representing time and y-axis representing frequency of an event
• Scatter graphs (plots or diagrams) show the association between
two variables by graphing their data points along x and y axes
• Pie charts visually show the proportions (percentages) of a variable
in each value, relationships among the values, and the whole

Inferential Statistics

Inferential Statistics: Other
Factors
• Level of data
• Sample size
• Number of variables
• Independence of samples
• Randomness of sample
• Linearity
• Others (consult with a statistician about other
appropriate statistical tests)

Sensitivity Analysis
• Sensitivity analysis: Investigation of a study’s results to
see whether results differ
– If decisions on handling the data are changed
– For subgroups within the data
• Purposes
– Check robustness of a study’s results. Results are
considered robust when the results remain fairly
consistent, despite variations in the handling of the data
– Put actionable information into the hands of decision
makers
– Note: Sensitivity analysis is not the same as sensitivity and
specificity

33
Data Mining
• Primary analysis: Analysis of original research data by
researchers who collected the data for a specific study
– Primary data are the data that were collected to answer
the researchers’ specific research question
• Secondary analysis is any research in which
researchers use data for purposes not defined nor
predicted in the original study’s design
– Examples: Reusing data to answer a different question,
combining the dataset with another dataset, or applying
different statistical tests
– Secondary data were originally collected for another
specific purpose

34
Data Mining (cont.)
• Data mining: Semiautomated and automated processes for
exploring large databases and for detecting relevant patterns and
relationships
• Analyzes data to generate descriptive or predictive models
• Differs from other quantitative analyses
– Can begin without a precise, preestablished hypothesis, allowing the
data in the database to generate hypotheses
– Data were not collected for this specific analysis
– Must deal with heterogeneous data fields
• Terms
– Algorithm
– Pattern
– Model
• Many methods of data mining that can be used iteratively

Statistical Conclusion Validity
• Statistical conclusion
validity: extent to which the
statistical conclusions about
the relationships in the data
are reasonable
• Judges soundness of
researcher’s conclusions
based on study’s findings
• Threatened by:
– Lack of power
– Lack of reliability
– Extreme heterogeneity of the
subjects
– Use of inappropriate statistical
test
– Fishing
Three Prongs of Statistical
Conclusion Validity: Research
Design, Data Collection, and
Data Analysis

Qualitative Data Analysis
• Qualitative data analysis: Systematic process of working with data to
create coherent descriptions and explanations of phenomena
• Applied to nonnumerical observations, such as gestures, activities,
space, and perceptions
• Over 20 qualitative analytic techniques
• Cyclical and iterative process with three major activities

37
Grounded Theory
• Grounded theory: Refers both to the theories that the
technique generates and to the technique itself
• Purpose is to discover or to generate theories through
the analysis of data
• Data generate theories through coding, categorization,
and comparison (also known as the constant
comparative method)
• Conceptualization is core of grounded theory
– Conceptualization results in the identification of hidden
patterns, enduring relevance and meaning, and
abstractions of time, place, and people
– Conceptualizations form the generalizations of theories

Grounded Theory (cont.)
Process
• Data collection, data analysis, and
generation of hypotheses and theories
are concurrent, intertwined, and
iterative activities
– Phenomena observed
– Observations recorded and coded as
incidents
– Unit of analysis is the incident; each incident
is coded
– Coded incidents are data
– Data are often represented by illustrative or
characteristic quotes
– Develop conceptual categories to fit coded
data
– Revealed gaps and discrepancies are filled
by more observations
– Data collection, coding, and analysis
continue until all data fit or are accounted for
Four stages of
grounded theory
• Comparing incidents
applicable to each
category
• Integrating categories
and their properties
• Delimiting theory
• Writing theory

39
Content Analysis
• Content analysis is the systematic analysis of communication, which
makes replicable and valid inferences from texts or other meaningful
matter to the contexts of their use
• Analysis of written documentation and other modes of
communication, such as speech, body language, images and
photographs, music, television shows, commercials, movies, and
other symbolic matter
• Purposes
– Identify dominant findings and make generalizations
– Study and predict behaviors
• May be quantitative or qualitative
• May be deductive or inductive
• Content analysis is essentially a coding operation in which the
coded text (or communication) is the data

40
Content Analysis: Process
• Researchers iteratively cycle through the following
process:
– Identify a unit of analysis (single, meaningful, undivided
whole)
– Code the unit of analysis by labeling with annotations or
scales
– Assess reliability of coding
– Identify key terms, characteristics, or other attributes from
the coding
– Categorize the data by classifying groups of coded data
with similar meanings (major step in the analysis)
– Abstract the categories into overarching themes

Conclusion Validity
• Conclusion validity: Extent to which observations, patterns,
and inferences are reasonable (qualitative counterpart of
statistical conclusion validity)
– Clarity of the logic in assigning categories
– Exhaustiveness of search for confirming and disconfirming data
– Ability of final interpretation to encompass evidence and patterns
– Inclusion of critical examinations of researchers’ perspectives
and their potential to bias interpretations
– Convincing warrant (justification) for the researchers’ claims
(interpretations)
– Credibility, transferability, dependability, and confirmability
support conclusion validity

Presentation of Results and
Discussion
• Results
– Tabular and graphical display
• Choice of table or graph depends on data to be displayed
• Purpose support readers’ understanding
– Narrative
• Report results (findings) with no commentary, explanation, or interpretation
• Style
– Past tense
– Objective, precise, and factual
– Merely recording in narrative
– Reporting
• For each hypothesis
• Characteristics of the sample and similarity with population
• Main and supplemental statistical tests’ results
• Use of “statistical significance” versus “significance”
• Tables and graphs

Discussion (cont.)
• Results
– One mode—narrative, table, or graph or other
figure—that is the most effective
– Tabular and graphical display
• Choice of table or graph depends on data to be displayed
• Purpose support readers’ understanding
– Narrative
• Report results (findings) with no commentary, explanation, or
interpretation
• Style of language
– Past tense
– Objective, precise, and factual
– Merely recording in narrative

Discussion (cont.)
• Results (cont.)
– Reporting
• For each hypothesis
• Characteristics of the sample and similarity with
population
• Main and supplemental statistical tests’ results
• Use of “statistical significance” versus
“significance”

Discussion (cont.)
• Discussion
– Focus on study’s important results
– State any additions to field’s body of knowledge
– Compare and contrast to existing literature and
explain why results are similar or different
– Discuss
• Achievement of aims
• Expansion of advancement of theory or model
• State assumptions, limitations, recommendations, and
implications for practice
– Interpret

Review
• Researchers transform their data into results through quantitative
data analysis or qualitative data analysis
• Quantitative data can be examined with more than one statistical
technique
• Statistical significance does not guarantee practical significance
• Purpose of null hypothesis significance testing is to determine the
likelihood that the research findings are not result of random chance
or biased sample
• Power is the probability of identifying real differences or
relationships between groups
• Researchers may make type I errors or type II errors
• Concepts associated with data preparation include the unit of
analysis, intention to treat analysis, missing values, and data
cleaning

Review (cont.)
• Descriptive statistics include frequency distributions, tables, graphical
displays, measures of central tendency, and measures of dispersion;
sensitivity and specificity; predictive values; ROC curve analysis; and
measures of effect
• A purpose of tables and graphs is to support readers’ understanding
• Inferential statistics includes parametric and nonparametric tests
• Many methods of data mining exist and they are often used iteratively
• Statistical conclusion validity is the extent to which the statistical
conclusions about the relationships in the data are reasonable
• Grounded theory and content analysis are commonly used qualitative
analytic techniques
• Conclusion validity is the extent to which observations, patterns, and
inferences are reasonable
• Researchers report their findings in the results section of a scholarly paper
and interpret their findings in the discussion section

HM404 Ab120916 ch12

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to HM404 Ab120916 ch12

Similar to HM404 Ab120916 ch12 (20)

More from BealCollegeOnline

More from BealCollegeOnline (20)

Recently uploaded

Recently uploaded (20)

HM404 Ab120916 ch12