The document discusses covert naturalistic observation in user research, highlighting its challenges and the ethical implications involved. It emphasizes the importance of planning, blending in, and observing behaviors without disrupting the environment. The author argues that both overt and covert methods have their advantages and can be used effectively to enhance user experience understanding.

1. Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research
October 16, 2013

2. Covert Naturalistic Observation
Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research
October 16, 2013

4. User research is unnatural.
Covert Naturalistic Observation - Jim Ross
4

5. User research is unnatural.
Source: Flickr – Thomas Link
Covert Naturalistic Observation - Jim Ross
Source: Flickr – Thomas Link
5

6. User research is unnatural.
Covert Naturalistic Observation - Jim Ross
6

7. We get informed consent from participants.
Covert Naturalistic Observation - Jim Ross
7

8. But knowing that you’re being observed affects behavior.
Source: Flickr – Blue Oxen
Covert Naturalistic Observation - Jim Ross
8

10. Covert naturalistic observation
Source: Flickr – Sean Hobson
Covert Naturalistic Observation - Jim Ross
10

11. We usually use overt naturalistic observation.
Covert Naturalistic Observation - Jim Ross
11

12. It’s rare that we use covert naturalistic observation.
Source: Flickr – James Emery
Covert Naturalistic Observation - Jim Ross
12

13. Who uses covert naturalistic observation?
Covert Naturalistic Observation - Jim Ross
13

14. Who uses covert naturalistic observation?
Covert Naturalistic Observation - Jim Ross
14

15. Who uses covert naturalistic observation?
Covert Naturalistic Observation - Jim Ross
15

16. Covert naturalistic observation

17. It allows you to observe realistic, natural behavior.
Source: Flickr – David Hillowitz
Covert Naturalistic Observation - Jim Ross
17

18. Trying to remain covert makes observation more difficult.
Source: Flickr – Lars Plougmann
Covert Naturalistic Observation - Jim Ross
18

19. You can’t study interfaces or close up interactions.
Source: Flickr – Walter Lim
Covert Naturalistic Observation - Jim Ross
19

20. You can’t ask questions, so you rely on observation only.
Source: Flickr – Danielle Scott
Covert Naturalistic Observation - Jim Ross
20

21. Ethical concerns about consent, privacy, and deception
Source: Flickr – dustpuppy
Covert Naturalistic Observation - Jim Ross
21

22. Covert naturalistic observation

23. Studying behavior and interactions in a public place
Source: Flickr – Drew XXX
Source: Flickr – dustpuppy
Covert Naturalistic Observation - Jim Ross
23

24. Covert naturalistic observation

25. 1. Plan what you want to observe.
Covert Naturalistic Observation - Jim Ross
25

26. 2. Gather intelligence – do some research on:
•
•
•
•
The domain
User characteristics
Tasks and behaviors
Existing research
Covert Naturalistic Observation - Jim Ross
26

27. 3. Scope out the location ahead of time: layout, people,
activity level.
Source: Flickr – Ben Salter
Covert Naturalistic Observation - Jim Ross
27

28. 4. Consider notifying management and security.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross
28

29. 5. Observe with partners.
Source: Flickr – Carey Akin
Covert Naturalistic Observation - Jim Ross
29

30. 6. Observe over several shorter sessions to see:
• More people
• Different situations
• Different time periods
Covert Naturalistic Observation - Jim Ross
30

31. 6. Observe over several shorter sessions to avoid:
• Fatigue
• Information overload
• “Getting caught”
Covert Naturalistic Observation - Jim Ross
31

32. 7. Blend in – dress like everyone else.
Source: Flickr – Michael Ocampo
Covert Naturalistic Observation - Jim Ross
32

33. 7. Blend in – do what others are doing.
Source: Flickr – James Emery
Covert Naturalistic Observation - Jim Ross
33

34. 7. Blend in – use a mobile device.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross
Source: Flickr – Elvert Barnes
34

35. 8. Don’t take obvious notes.
Source: Flickr – Geek Calendar
Covert Naturalistic Observation - Jim Ross
35

36. 8. Don’t take obvious notes – use a phone or tablet.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross
36

37. 9. Treat recording as secondary – take photos or video
last.
Covert Naturalistic Observation - Jim Ross
37

38. 9. Treat recording as secondary – use a phone or tablet.
Source: Flickr – Jesus Leon
Covert Naturalistic Observation - Jim Ross
38

39. 9. Treat recording as secondary – don’t record audio.
Covert Naturalistic Observation - Jim Ross
39

40. 10. Debrief after each observation session.
Source: Flickr – Sean MacEntee
Covert Naturalistic Observation - Jim Ross
40

41. 10. Debrief after each observation session.
Covert Naturalistic Observation - Jim Ross
41

42. 11. Do it ethically – observe in public places.
Source: Flickr – Dominic Alves
Covert Naturalistic Observation - Jim Ross
42

43. 11. Do it ethically – focus on groups not individuals.
Source: Flickr – Alfonso Pierantonio
Covert Naturalistic Observation - Jim Ross
43

44. 11. Do it ethically – protect privacy.
Source: Flickr – David Goehring
Covert Naturalistic Observation - Jim Ross
44

45. 11. Do it ethically – don’t deceive.
Source: Flickr – Elvert Barnes
Covert Naturalistic Observation - Jim Ross
45

46. 12. Combine it with other methods.
Covert Naturalistic Observation - Jim Ross
46

47. Covert naturalistic observation

48. 
Overt and covert methods each have
advantages and disadvantages.
Covert Naturalistic Observation - Jim Ross
48

49. 
With understanding of the advantages and
disadvantages, we can use overt and covert
methods as appropriate to best understand
the user experience.
Covert Naturalistic Observation - Jim Ross
49

50. Covert naturalistic observation
Jim Ross
Design Philadelphia: The Bleeding Edge of Design Research
October 16, 2013