This document provides an overview of research methods taught in an empirical research methods course. It discusses both quantitative and qualitative research methods. For qualitative methods, it describes techniques like ethnography, case studies, and grounded theory research. It explains data collection methods for qualitative research including direct observation, participant observation, in-depth interviews, and analyzing documents. The document also covers qualitative data analysis including coding data and developing models and theories from the analysis.

1. IS 4800 Empirical Research Methods
for Information Science
Class Notes April 4, 2012

2. •quantitative
•Descriptive
•demonstration
•correlational
•experiments
•qualitative
•ethnography
•case studies
•mixed
Overview of Research Methods

3. Steps in the research process
• Identify a phenomenon of interest
• Iterate:
– Investigate current state of knowledge (lit. review ?)
– Narrow down your interest to a research question or
hypothesis
• Identify research method to employ (survey,
experiment, ethnography, case study)

4. Steps in the research process (cont.)
• Operationalize research question or hypothesis
– Define the source of your data
• Sample population and recruitment method (if relevant)
• And/or the location/activities to be observed
– Define variables and/or data collection methods and
instruments
– Identify how the analysis will be carried out
----YOU NOW HAVE A RESEARCH PROPOSAL ------

5. Steps in the research process (cont.)
• Carry out your observations
• Analyze your data
• Draw conclusions, write up the results
Example: Freshman retention – a major concern of
every University !!
A quantitative study: midterm exam question
A qualitative study (we will discuss that)

6. Quantitative vs. Qualitative
 Goals
 Kinds of data
 Data collection methods
 Kinds of analyses
 Kinds of explanations
 When used in system development
lifecycle?

7. Goals of qualitative research
• Identify/compare patterns of behavior
• Analyze beliefs/subjective experience
– How people view a situation or experience
– Distinguish multiple perspectives
• Compare beliefs/self-reports to actual behavior
• DESCRIBE a complex phenomenon
• EXPLAIN why things happen

8. 8
Case studies of failed system development
or deployment
Ethnographic studies of workgroup
practices prior to introduction of new
technology
Examples of qualitative studies

9. 9
• Case study research
•Explore in depth one activity or project
•Limited in time and place
•Data collection by observation, interviews, artifacts . . .
•Goal: tell a coherent “story” with lessons learned
•Most common methodology for IT empirical studies
• Ethnography
•Observation in natural setting
•Observer may become part of the group to experience
directly how its members interact
•Goal: identify patterns of interaction (power structures,
problem-solving/goal achievement)
Some qualitative methods in brief

10. 10
• Grounded theory research
• Data collection from artifacts and/or interviews
• Develop a set of categories and a model telling how they
relate to each other
• Goal: explain the meaning of what is observed
• Involves an bottom-up iterative process of data
collection/theory formation
Some qualitative methods (cont.)

11. + Systematic rules and procedures already worked
out, and can be followed
+ Traditional, accepted as “proof”
- Closed-ended questions may lead to ignoring
important factors and relationships
- Quantitative methods cannot handle phenomena
that are difficult to turn into variables
Advantages/disadvantages of
quantitative studies

12. Lecture 1 - Introduction 12
+ more innovative and creative
+ capable of addressing issues that do not lend themselves
to being described by variables
- conclusions may be less credible
Advantages/disadvantages of
qualitative methods

13. 13
Pragmatic philosophy – find out whatever you can
using whatever methods are possible
Involves both qualitative and quantitative elements
 (at least 2 stages of research)
Advantages/Disadvantages
+ combines structure and flexibility
- requires more time and resources
Mixed Methods

14. 14
Pattern 1: “instrument” data followed by in-depth interview
to get insight on the reasons for the observed
relationships and capture any insights you
overlooked in study design
Examples of Mixed Method Designs
Pattern 2: exploratory study followed by survey or experiment
to generalize the results – representative of a
long-term research program

15. •Studies of computer/supported learning
•Studies of IT impacts in medicine
•Computer-supported collaboration (in general)
•Case study or ethnography for groupware
•Grounded theory study of chat groups
Applications of the Methods
Which Method would you choose?

16. Goals of qualitative research
• Identify/compare patterns of behavior
• Analyze beliefs/subjective experience
– How people view a situation or experience
– Distinguish multiple perspectives
• Compare beliefs/self-reports to actual behavior
• Ideally, propose a model to describe or explain

17. What is a model ??
What counts as evidence ??
Elements/types of qualitative models
• Define a taxonomy – useful aggregation of data
– Rogers’ adopter categories
• Innovators
• Early adopters
• Early majority
• Late majority
• Laggards

18. Elements/types of qualitative models
• Define properties that affect outcomes
– Perceived attributes of innovation that help explain
their different rates of adoption
• Relative advantage
• Compatibility
• Complexity
• Trialability
• Observability
• Explains individual’s likelihood of adopting

19. Elements/types of qualitative models
• Identify a relationship of interest
– Rogers: homophily/heterophily and diffusion
• Homophily is the degree to which two or more
individuals who interact are similar in certain attributes
such as beliefs, education, socioeconomic status, etc.
• Although homophily enables better communication,
innovation requires heterophily, at least regarding
knowledge of and experience with an innovation, and it
is likely that heterophily in that area co-occurs with
other important differences.
• This explains why some groups are slower to
adopt new behaviors

20. Elements/types of qualitative models
• Characterizing a process as a sequence of steps
– Rogers: the innovation/diffusion process
• Knowledge
• Persuasion
• Decision
• Implementation
• Confirmation
• Characterizing the “shape” of a process
– Rogers: the S-shaped curve

21. 21
Identify/define the variables (and their coding)
Design the “instrument” – a measurement process or
technique
Types of instruments:
a survey (paper, phone, Web)
a form for observer to fill in (experiment or field study)
a prototype system (with interaction capture)
Field testing/validating instruments part of quantitative
methods
Collecting data for quantitative studies

22. Qualitative Data and Collection
Methods
• Direct observation
– Participant observation
• In-depth interviews
– Focus groups
• “Artifacts” – usually text or Databases

23. Direct Observation
– May be in person or use audio or videotape,
observe through a 1-way mirror
– Unlike participant observation, often focused on
specific events (how many, how often, by whom,
observe patterns – for example, interruptions at a
meeting)

24. What to observe
• Spatial relations
• Activities
• Communication
– Verbal
– Other
• Tasks
– How work is allocated

25. How to be an effective observer
• Preparation
• Stay in the background
• Be factual and objective in your notes
– (interpretation comes later)
• Taking notes:
– Hand written usually
– Type in to computer later
• EXPANDING NOTES (ASAP)

26. “(Participant) observation”: in
natural setting
• “Participant” observation occurs when you
interact casually and/or form relationships with
informants
• How much you actually “participate” depends
on the goals of the study.

27. Participant observation
• Advantages:
– Offers insights into complex behavior
– Identify the “right questions” for further study
– Verify/correct self-reports
• Disadvantage:
– Time consuming
– Data collection is difficult
– Problem of subjectivity

28. How to operationalize
• Field notes
– Text
– Diagrams, maps
– Can result in numerical data
• Interviews (interviewer more clueful)
• Focus groups (facilitator more clueful)

29. What to observe
• Spatial relations
• Activities
• Communication
– Verbal
– Other
• Tasks
– How work is allocated
– See Table 3 in reading

30. Ethics
• Do not disrupt the activity your are observing
versus
• Do not mislead
• No formal rules about disclosing your role as a
researcher when engaging in casual
conversation – article suggests a point where
you want to ask specific question
• Disclosure includes: right of refusal,
confidentiality

31. Protecting confidentiality when
data is unique
• Separate identify info from field notes entered
into the computer
• People, organizations/companies, should be
given fictitious names

32. How to be an effective observer
• Preparation
• Stay in the background
• Be factual and objective in your notes
– (interpretation comes later)
• Taking notes:
– Hand written usually
– Type in to computer later
• EXPANDING NOTES

33. Tips
• Leave space
• Take notes strategically
• Use abbreviations
• Cover a range of observations:
Body language, etc.

34. Tips
• Leave space
• Take notes strategically
• Use abbreviations
• Cover a range of observations:
Body language, etc.

35. Participant observation: in natural
setting
• “Participant” observation occurs when you
interact and/or form relationships with
informants
• Demanding and time-consuming
• How much you actually “participate” depends
on the goals of the study.
• Subjects may “forget” you are a researcher

36. How to operationalize
direct/participant observation
• Field notes
– Text
– Diagrams, maps
– Can result in numerical data
• Interviews (interviewer more clueful in P.O.)
• Focus groups (facilitator more clueful in P.O.)
“Water cooler” effect

37. Participant observation
• Advantages:
– Offers insights into complex behavior
– Identify the “right questions” for further study
– Verify/correct self-reports
• Disadvantage:
– Time consuming
– Data collection is difficult
– Problem of subjectivity

38. Ethics of direct observation/
participant observation
• Do not disrupt the activity your are observing
versus
• Do not mislead
• No formal rules about disclosing your role as a
researcher when engaging in casual
conversation – some authors suggest a point
where you want to ask specific question
• Disclosure includes: right of refusal,
confidentiality

39. Protecting confidentiality when
data is unique
• Separate identify info from field notes entered
into the computer
• People, organizations/companies, should be
given fictitious names

40. In-depth interview/focus group
• Probes the interviewee(s) views of the
phenomenon of interest
• Interviewer/facilitator should be neutral
• Data collected: transcript, audio/video
recording, notes

41. In-depth interview/focus group
Interviewer should:
• Start with some open-ended questions
• Follow up by asking “how” and “why”
• Keep the discussion on track

42. Documents
• Memos and meeting notes
• Transcripts of conversations or speeches
• Manuals and policy handbooks
• Newspapers and magazines
• Internet-based research
– Email
– Web sites
– Blogs
• Especially important in case studies

43. Qualitative Data Analysis
by John V. Seidel
• Description of how to go about analyzing transcripts
of interviews, documents, and/or field notes.
• Focus on “coding”
– First identify “events”
– Assign terms that represent concepts of interest
– Organizing codes into a scheme
– Building qualitative models using the coding scheme as the
model vocabulary
• Focus on iterative nature of QDA

44. Two perspectives on coding
• Objectivist perspective
– Condensed representation of facts
– Can be subjected to hypothesis testing
– Strong burden of consistency/completeness
• Heuristic perspective
– Signposts pointing to things you care about
– Foundation for further analysis

46. Elements/types of qualitative models
• Examples from Rogers’ theory of innovation
diffusion
– VCR’s
– Cell phones
– Metric system
– Seat belts in cars
– Dvorak keyboard

48. Three analogies to explain this
• Jigsaw puzzle analogy
• A little data and a lot of right brain
• Multi-threaded DNA (patterns among the
patterns)