This document discusses methods for evaluating research quality and ethics in data analysis. It describes key aspects of good research such as being organized, objective, controlled, and empirical. Good research topics are within a line of research, relevant to the real world, possible to conduct, and meaningful. When analyzing data, analysts should use a consistent method, consider the ethical implications of decisions, and protect data quality and confidentiality. The document also identifies potential risks to ethics like prioritizing professional needs over ethics.

3. How do you know when something is true?

4. Methods of knowing
• Tenacity – It’s true because it has always
been true
• Intuition – It’s true because it is self-
evident
• Authority – It’s true because I trust the
source

5. So what’s the problem?

6. Quality research is
• Organized
• Objective
• Controlled
• Qualitative or quantitative
• Empirical

7. Our class

8. Good research topics are
–In a line of research OR
–From the real world
–Possible
• Scope
• Access
• Data Quality
• Ethical
–Meaningful

9. The ethics of analysts

10. Framework
• Decisions have ethical
implications
• Decisions should be made with
a consistent method

11. • Data – a record (can be tangible or electronic)
that is used as a basis for decision-making,
discussion or calculation that requires
processing and/or analysis to have meaning
• Data Scientist – A professional who uses the
scientific method to answer questions with
data
• Data quality – the truthfulness of data

12. • Signal – a meaningful interpretation of data
that is based on scientific evidence and
knowledge
• Noise – other interpretations of data
• Algorithm – a set of rules used in problem
solving

13. Should she?
• The University of Central Carolinistan has a
pretty good Propaganda department. They do
student evaluations of courses, and the
propaganda department, which taught 2,000
class sections, had an average score of 5.37 on
these evaluations. The entire university
average across 10,000 sections was 5.35. The
head bureaucrat sends out a press release
saying Propaganda is better than the
University as a whole. Should she?

14. • Statistics – collecting and analyzing numbers in
large quantities
• Statistical significance – a statistical assessment
of whether the observed finding is real or caused
by chance
• Causation – a relationship between a first and
second phenomenon in which the second is a
consequence of the first.
• Spurious correlation – a relationship caused by a
hidden or lurking variable

17. • Ludic fallacy – thinking the real world
(complex!) is comparable to the models used
in experiments and modeled with math
• Naïve interventionism – preferring to do
something over nothing when nothing may be
more appropriate
• Naïve rationalism – belief that explanations
will necessarily follow investigations

18. Humans are rational beings
Humans want to maximize utility (get the most
for themselves out of a transaction)

19. Explain charity

20. Risks to ethics
• Hammurabi Risk Management – the builder
knows more than the inspector and can hide
flaws in the foundations
• Ethical inversion – putting the needs of the
profession ahead of the ethics (aka politics)
• Narrative fallacy – the need to fit a story to a
set of facts

21. • Why did Donald Trump get elected president?
– Narrative: disenfranchised working class voters
– Narrative: the Russians did it

22. Recognizing ethical risk spots?
• Could the action be damaging to people or
community?
• Does the action have ramifications beyond
legal or institutional concerns?

23. Ethics and data
• Analysts shouldn’t attempt to provide explanations
beyond their ability
• Analysts should provide their methods, to the ability of
their client to understand, including limitations of the
data and the insights
• Analysts should protect confidential information
• Analysts should avoid conflicts of interest
• Analysts should use the data science method
– Careful observation
– Analysis for potential meaning
– Formation of hypotheses
– Empirical testing of hypotheses