SlideShare a Scribd company logo
1 of 50
Download to read offline
Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research

October 16, 2013
Covert Naturalistic Observation

Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research

October 16, 2013
User research is unnatural.

Covert Naturalistic Observation - Jim Ross

4
User research is unnatural.

Source: Flickr – Thomas Link

Covert Naturalistic Observation - Jim Ross

Source: Flickr – Thomas Link

5
User research is unnatural.

Covert Naturalistic Observation - Jim Ross

6
We get informed consent from participants.

Covert Naturalistic Observation - Jim Ross

7
But knowing that you’re being observed affects behavior.

Source: Flickr – Blue Oxen

Covert Naturalistic Observation - Jim Ross

8
Covert naturalistic observation

Source: Flickr – Sean Hobson
Covert Naturalistic Observation - Jim Ross

10
We usually use overt naturalistic observation.

Covert Naturalistic Observation - Jim Ross

11
It’s rare that we use covert naturalistic observation.

Source: Flickr – James Emery
Covert Naturalistic Observation - Jim Ross

12
Who uses covert naturalistic observation?

Covert Naturalistic Observation - Jim Ross

13
Who uses covert naturalistic observation?

Covert Naturalistic Observation - Jim Ross

14
Who uses covert naturalistic observation?

Covert Naturalistic Observation - Jim Ross

15
Covert naturalistic observation
It allows you to observe realistic, natural behavior.

Source: Flickr – David Hillowitz
Covert Naturalistic Observation - Jim Ross

17
Trying to remain covert makes observation more difficult.

Source: Flickr – Lars Plougmann
Covert Naturalistic Observation - Jim Ross

18
You can’t study interfaces or close up interactions.

Source: Flickr – Walter Lim
Covert Naturalistic Observation - Jim Ross

19
You can’t ask questions, so you rely on observation only.

Source: Flickr – Danielle Scott
Covert Naturalistic Observation - Jim Ross

20
Ethical concerns about consent, privacy, and deception

Source: Flickr – dustpuppy
Covert Naturalistic Observation - Jim Ross

21
Covert naturalistic observation
Studying behavior and interactions in a public place

Source: Flickr – Drew XXX
Source: Flickr – dustpuppy
Covert Naturalistic Observation - Jim Ross

23
Covert naturalistic observation
1. Plan what you want to observe.

Covert Naturalistic Observation - Jim Ross

25
2. Gather intelligence – do some research on:
•
•
•
•

The domain
User characteristics
Tasks and behaviors
Existing research

Covert Naturalistic Observation - Jim Ross

26
3. Scope out the location ahead of time: layout, people,
activity level.

Source: Flickr – Ben Salter
Covert Naturalistic Observation - Jim Ross

27
4. Consider notifying management and security.

Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross

28
5. Observe with partners.

Source: Flickr – Carey Akin
Covert Naturalistic Observation - Jim Ross

29
6. Observe over several shorter sessions to see:
• More people
• Different situations
• Different time periods

Covert Naturalistic Observation - Jim Ross

30
6. Observe over several shorter sessions to avoid:
• Fatigue
• Information overload
• “Getting caught”

Covert Naturalistic Observation - Jim Ross

31
7. Blend in – dress like everyone else.

Source: Flickr – Michael Ocampo
Covert Naturalistic Observation - Jim Ross

32
7. Blend in – do what others are doing.

Source: Flickr – James Emery
Covert Naturalistic Observation - Jim Ross

33
7. Blend in – use a mobile device.

Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross

Source: Flickr – Elvert Barnes
34
8. Don’t take obvious notes.

Source: Flickr – Geek Calendar
Covert Naturalistic Observation - Jim Ross

35
8. Don’t take obvious notes – use a phone or tablet.

Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross

36
9. Treat recording as secondary – take photos or video
last.

Covert Naturalistic Observation - Jim Ross

37
9. Treat recording as secondary – use a phone or tablet.

Source: Flickr – Jesus Leon
Covert Naturalistic Observation - Jim Ross

38
9. Treat recording as secondary – don’t record audio.

Covert Naturalistic Observation - Jim Ross

39
10. Debrief after each observation session.

Source: Flickr – Sean MacEntee
Covert Naturalistic Observation - Jim Ross

40
10. Debrief after each observation session.

Covert Naturalistic Observation - Jim Ross

41
11. Do it ethically – observe in public places.

Source: Flickr – Dominic Alves
Covert Naturalistic Observation - Jim Ross

42
11. Do it ethically – focus on groups not individuals.

Source: Flickr – Alfonso Pierantonio
Covert Naturalistic Observation - Jim Ross

43
11. Do it ethically – protect privacy.

Source: Flickr – David Goehring
Covert Naturalistic Observation - Jim Ross

44
11. Do it ethically – don’t deceive.

Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross

45
12. Combine it with other methods.

Covert Naturalistic Observation - Jim Ross

46
Covert naturalistic observation


Overt and covert methods each have
advantages and disadvantages.

Covert Naturalistic Observation - Jim Ross

48


With understanding of the advantages and
disadvantages, we can use overt and covert
methods as appropriate to best understand
the user experience.

Covert Naturalistic Observation - Jim Ross

49
Covert naturalistic observation

Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research

October 16, 2013

More Related Content

Viewers also liked

Merriam ch 9 5.25.10
Merriam ch 9 5.25.10Merriam ch 9 5.25.10
Merriam ch 9 5.25.10
Daberkow
 
sampling in research methodology. qualitative and quantitative approach
sampling in research methodology. qualitative and quantitative approach sampling in research methodology. qualitative and quantitative approach
sampling in research methodology. qualitative and quantitative approach
Samantha Jayasundara
 
Non probability sampling 2
Non probability sampling 2Non probability sampling 2
Non probability sampling 2
Sanket Gaikwad
 
Sampling1[1]
Sampling1[1]Sampling1[1]
Sampling1[1]
jilly17
 
Sampling in qualitative research
Sampling in qualitative researchSampling in qualitative research
Sampling in qualitative research
Dr. V Vorvoreanu
 
Quantitative And Qualitative Research
Quantitative And Qualitative ResearchQuantitative And Qualitative Research
Quantitative And Qualitative Research
doha07
 
Qualitative and quantitative methods of research
Qualitative and quantitative methods of researchQualitative and quantitative methods of research
Qualitative and quantitative methods of research
Jordan Cruz
 

Viewers also liked (20)

Quantitative reasearch.
Quantitative reasearch.Quantitative reasearch.
Quantitative reasearch.
 
Types of research
Types of researchTypes of research
Types of research
 
Merriam ch 9 5.25.10
Merriam ch 9 5.25.10Merriam ch 9 5.25.10
Merriam ch 9 5.25.10
 
Qualitiative data analysis: data triangulation
Qualitiative data analysis: data triangulationQualitiative data analysis: data triangulation
Qualitiative data analysis: data triangulation
 
sampling in research methodology. qualitative and quantitative approach
sampling in research methodology. qualitative and quantitative approach sampling in research methodology. qualitative and quantitative approach
sampling in research methodology. qualitative and quantitative approach
 
Qualitative data collection
Qualitative data collectionQualitative data collection
Qualitative data collection
 
Ethnographic Research for Design Innovation
Ethnographic Research for Design InnovationEthnographic Research for Design Innovation
Ethnographic Research for Design Innovation
 
Observation in Qualitative Research
Observation in Qualitative ResearchObservation in Qualitative Research
Observation in Qualitative Research
 
Non probability sampling 2
Non probability sampling 2Non probability sampling 2
Non probability sampling 2
 
Design of qualitative research
Design of qualitative researchDesign of qualitative research
Design of qualitative research
 
Triangulation
Triangulation Triangulation
Triangulation
 
Sampling1[1]
Sampling1[1]Sampling1[1]
Sampling1[1]
 
Nonprobability Sampling
Nonprobability SamplingNonprobability Sampling
Nonprobability Sampling
 
Sampling in qualitative research
Sampling in qualitative researchSampling in qualitative research
Sampling in qualitative research
 
Quantitative And Qualitative Research
Quantitative And Qualitative ResearchQuantitative And Qualitative Research
Quantitative And Qualitative Research
 
Research Methodology
Research MethodologyResearch Methodology
Research Methodology
 
Sampling and Sample Types
Sampling  and Sample TypesSampling  and Sample Types
Sampling and Sample Types
 
RESEARCH METHOD - SAMPLING
RESEARCH METHOD - SAMPLINGRESEARCH METHOD - SAMPLING
RESEARCH METHOD - SAMPLING
 
Qualitative and quantitative methods of research
Qualitative and quantitative methods of researchQualitative and quantitative methods of research
Qualitative and quantitative methods of research
 
Types of Research
Types of ResearchTypes of Research
Types of Research
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Recently uploaded (20)

Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 

Covert Naturalistic Observation

  • 1. Jim Ross Design Philadelphia: The Bleeding Edge of Design Research October 16, 2013
  • 2. Covert Naturalistic Observation Jim Ross Design Philadelphia: The Bleeding Edge of Design Research October 16, 2013
  • 3.
  • 4. User research is unnatural. Covert Naturalistic Observation - Jim Ross 4
  • 5. User research is unnatural. Source: Flickr – Thomas Link Covert Naturalistic Observation - Jim Ross Source: Flickr – Thomas Link 5
  • 6. User research is unnatural. Covert Naturalistic Observation - Jim Ross 6
  • 7. We get informed consent from participants. Covert Naturalistic Observation - Jim Ross 7
  • 8. But knowing that you’re being observed affects behavior. Source: Flickr – Blue Oxen Covert Naturalistic Observation - Jim Ross 8
  • 9.
  • 10. Covert naturalistic observation Source: Flickr – Sean Hobson Covert Naturalistic Observation - Jim Ross 10
  • 11. We usually use overt naturalistic observation. Covert Naturalistic Observation - Jim Ross 11
  • 12. It’s rare that we use covert naturalistic observation. Source: Flickr – James Emery Covert Naturalistic Observation - Jim Ross 12
  • 13. Who uses covert naturalistic observation? Covert Naturalistic Observation - Jim Ross 13
  • 14. Who uses covert naturalistic observation? Covert Naturalistic Observation - Jim Ross 14
  • 15. Who uses covert naturalistic observation? Covert Naturalistic Observation - Jim Ross 15
  • 17. It allows you to observe realistic, natural behavior. Source: Flickr – David Hillowitz Covert Naturalistic Observation - Jim Ross 17
  • 18. Trying to remain covert makes observation more difficult. Source: Flickr – Lars Plougmann Covert Naturalistic Observation - Jim Ross 18
  • 19. You can’t study interfaces or close up interactions. Source: Flickr – Walter Lim Covert Naturalistic Observation - Jim Ross 19
  • 20. You can’t ask questions, so you rely on observation only. Source: Flickr – Danielle Scott Covert Naturalistic Observation - Jim Ross 20
  • 21. Ethical concerns about consent, privacy, and deception Source: Flickr – dustpuppy Covert Naturalistic Observation - Jim Ross 21
  • 23. Studying behavior and interactions in a public place Source: Flickr – Drew XXX Source: Flickr – dustpuppy Covert Naturalistic Observation - Jim Ross 23
  • 25. 1. Plan what you want to observe. Covert Naturalistic Observation - Jim Ross 25
  • 26. 2. Gather intelligence – do some research on: • • • • The domain User characteristics Tasks and behaviors Existing research Covert Naturalistic Observation - Jim Ross 26
  • 27. 3. Scope out the location ahead of time: layout, people, activity level. Source: Flickr – Ben Salter Covert Naturalistic Observation - Jim Ross 27
  • 28. 4. Consider notifying management and security. Source: Flickr – Elvert Barnes Covert Naturalistic Observation - Jim Ross 28
  • 29. 5. Observe with partners. Source: Flickr – Carey Akin Covert Naturalistic Observation - Jim Ross 29
  • 30. 6. Observe over several shorter sessions to see: • More people • Different situations • Different time periods Covert Naturalistic Observation - Jim Ross 30
  • 31. 6. Observe over several shorter sessions to avoid: • Fatigue • Information overload • “Getting caught” Covert Naturalistic Observation - Jim Ross 31
  • 32. 7. Blend in – dress like everyone else. Source: Flickr – Michael Ocampo Covert Naturalistic Observation - Jim Ross 32
  • 33. 7. Blend in – do what others are doing. Source: Flickr – James Emery Covert Naturalistic Observation - Jim Ross 33
  • 34. 7. Blend in – use a mobile device. Source: Flickr – Elvert Barnes Covert Naturalistic Observation - Jim Ross Source: Flickr – Elvert Barnes 34
  • 35. 8. Don’t take obvious notes. Source: Flickr – Geek Calendar Covert Naturalistic Observation - Jim Ross 35
  • 36. 8. Don’t take obvious notes – use a phone or tablet. Source: Flickr – Elvert Barnes Covert Naturalistic Observation - Jim Ross 36
  • 37. 9. Treat recording as secondary – take photos or video last. Covert Naturalistic Observation - Jim Ross 37
  • 38. 9. Treat recording as secondary – use a phone or tablet. Source: Flickr – Jesus Leon Covert Naturalistic Observation - Jim Ross 38
  • 39. 9. Treat recording as secondary – don’t record audio. Covert Naturalistic Observation - Jim Ross 39
  • 40. 10. Debrief after each observation session. Source: Flickr – Sean MacEntee Covert Naturalistic Observation - Jim Ross 40
  • 41. 10. Debrief after each observation session. Covert Naturalistic Observation - Jim Ross 41
  • 42. 11. Do it ethically – observe in public places. Source: Flickr – Dominic Alves Covert Naturalistic Observation - Jim Ross 42
  • 43. 11. Do it ethically – focus on groups not individuals. Source: Flickr – Alfonso Pierantonio Covert Naturalistic Observation - Jim Ross 43
  • 44. 11. Do it ethically – protect privacy. Source: Flickr – David Goehring Covert Naturalistic Observation - Jim Ross 44
  • 45. 11. Do it ethically – don’t deceive. Source: Flickr – Elvert Barnes Covert Naturalistic Observation - Jim Ross 45
  • 46. 12. Combine it with other methods. Covert Naturalistic Observation - Jim Ross 46
  • 48.  Overt and covert methods each have advantages and disadvantages. Covert Naturalistic Observation - Jim Ross 48
  • 49.  With understanding of the advantages and disadvantages, we can use overt and covert methods as appropriate to best understand the user experience. Covert Naturalistic Observation - Jim Ross 49
  • 50. Covert naturalistic observation Jim Ross Design Philadelphia: The Bleeding Edge of Design Research October 16, 2013