• Save
Diary studies for UX
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Diary studies for UX

on

  • 2,707 views

A detailed overview of the pros and cons of the diary method for user research

A detailed overview of the pros and cons of the diary method for user research

Statistics

Views

Total Views
2,707
Views on SlideShare
2,697
Embed Views
10

Actions

Likes
9
Downloads
0
Comments
1

3 Embeds 10

https://twitter.com 5
http://www.linkedin.com 3
http://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • This kind of research is generally more qualitative than quantitative, but you can collect data that is quantifiable too.\n\nQual research is about understanding reasons for action, not just recording the action. Understanding motivation and also barriers that prevent action.\n
  • Isn’t just an academic exercise, it can help us find new customer needs.\n\nLearn new ways we can more effectively sell our products.\n\nPeople live in complex environments and varying cultural contexts that affect how we behave. Language, relationships, how we spend our spare time - cultural context.\n
  • M-pesa (M for mobile, pesa = swahili for cash or money) - started in 2007 after period of ethnographic research. Completely new service based on existing technology and unmet needs.\n\nMost of Kenya is rural, and many migrants work in Nairobi. Need to send money home.\n\nCan be done through bus network, but theft was rife.\n\nFew Kenyans have bank accounts, and traditional money transfer is expensive.\n\nHowever, mobile ‘phone adoption is widespread, and ‘phones can be rented on the street.\n\nSo researchers worked with safaricom & developed way of transferring via SMS.\n\nNow tens of millions of subscribers, spreading across africa.\n
  • Traditional anthropological methods involve long-term observation in the context of the group studied.\n\nM-pesa research involved several researchers spending months on the ground in Kenya. Though in this case the commercial benefit easily justified the cost.\n\nWe don’t have the time or the money to constantly take holidays for the next 3 years, so we need faster ways of observing and understanding that context.\n\nIn the case of travel we can visit airports, holiday resorts, and of course interview customers.\n
  • Vast majority of commercial customer research is based on interviews - in office or lab.\n\nRelies on participants accurately and honestly recalling what they’ve done over an extended period of time. This will inevitably contain mistakes.\n\nThe very act of trying to recall memories that are out of context is difficult. It’s a cognitive limitation of all human beings.\n\nFor example, most people find learning a foreign language easier when in an environment where that language is spoken rather than in the classroom. This is not proven fact, but part of the theory of situated cognition.\n\nThere are many areas where memory may be affected by natural bias though: hindsight bias, choice-supportive bias, crypotmnesia (remembering old memories and thinking their new, or other people’s ideas / memories and thinking their your own).\n\n
  • This is where diary studies become useful. Depending on set-up, can reduce impact of cognitive bias and recall.\n\nYou can track participants over time, for intermittent activities that might be hard to observe.\n\nYou can get the participant to collect the data for you, and you see it from their perspective, in context, at the right time.\n\nCommonly used in recent years to monitor mobile ‘phone usage where it would be unreasonable to observe.\n
  • Learn how the specific activity fits into people’s daily lives.\n\nWhen do they take time out to think about holidays?\n\nWho do they talk to?\n\nHow do they make decisions?\n
  • There are some situations where you really can’t follow people.\n\nFor example: Difficult to observe mobile use over time, because it can be used anywhere.\n\n
  • Your participants can be anywhere in the world. You don’t need to see them face-to-face.\n\nThey can interact in different ways too, including by using their mobiles.\n\n
  • Depending on your research plan, you can collect both.\n\nAlso able to correlate the two because based on the same observed event.\n\nIf monitoring holiday planning:\n\na.) How many visits to a shop before booking?\nb.) How long from first web search / holiday initiation and booking?\nc.) How does website effectiveness correlate with shop visits / contact-centre calls?\n\nOpen-ended data capture allows for broad qual. data too.\n\n
  • You can monitor almost any kind of activity because it’s so unobtrusive.\n\nDarwin used it in the 1840s to monitor speech development of his children over a long period (a year?).\n\nUS psychologist Ericsson used to study what makes people expert violinists - practice hours or natural ability?\n\nThis study gathered extensive quantitative data about practice time and how it was broken down to analyse the problem.\n
  • Participants record activities at the time of a given event, or at least that’s what we’re after.\n\nAcademic studies of the effectiveness of this have found around 25-30% of participants in these kinds of studies will in reality not complete their entries at the required time, but instead delay either deliberately or due to external factors. \n\nUseful when you need precise quantitative data about what is happening and when, but less effective at gathering qualitative data because it requires immediate feedback. Depends though.\n\nGenerally called “experience sampling”. Can refer to a good article by Intel on this method.\n
  • More common for diaries.\n\nParticipants record activities on a regular or researcher-led basis, which may require some recall of events prior to that time.\n\nTypically a participant might be required to complete a diary entry on a daily or weekly basis, and may need to recall activities for that day or week.\nEffective for understanding ecology and reasoning behind activities / behaviour.\n
  • Can be entirely closed and structured. Much easier to analyse (can be done in a spreadsheet) and easy for the participant to complete.\n\nLimited to quantifiable data, no qualitative input so lose that richness.\n
  • Can use some form of questioning to guide feedback, whilst still leaving room for qualitative feedback.\nThis approach works well by combining data types gathered, and ensuring participants stick close to the relevant subject matter.\n
  • \n
  • \n
  • If you have the time and budget, you can really broaden the types of data you get back. This is really useful, and when you use multiple methods of data collection other than written prose, it’s often called a “cultural probe”.\n
  • Traditionally you would send out a paper diary or probe kit, and wait for them to be posted back, but there are other ways for people to participate.\n\n\n
  • You can collect photographs, either by participants using their own devices or by the researcher sending out cameras.\n\nNow that most people have mobiles with cameras, this is even easier.\n\nThis can be a great way of communicating visually what might be hard to describe in written form.\n\nA study I worked on with BBC had people send us photos. We were researching media use (TV, internet, music) and how they were mixed together. Photos of how people’s living rooms were setup, how they all sat together doing different things in the same space, were all really enlightening.\n\n
  • There are now so many different ways participants can communicate.\n\nFacebook, Twitter, email, blogs. All are good ways to communicate.\n\nMeans less recall. Participants can share a photo, an email, a message on Twitter or Facebook, and do it at the time of the event.\n\nMobile devices make this really simple now, and a common activity for most people.*\n\n* Obviously this depends on where and with whom you’re researching. Might not be so effective for coal miners, airline pilots or nursery workers!\n
  • However you gather the data, assuming there is a qualitative element, you’ll have a lot of data.\n\nThe best way to analyse diary study data is using grounded theory, or a simplified version of it.\n\nThere are several steps to this - and I’m by no means an expert on grounded theory - but I’ll give you an overview.\n
  • This is a fairly typical example of the kind of data you might get back from a diary or other qualitative research.\n\nNarrative in form and varying writing styles.\n
  • First step is identifying critical instances. Broad, general patterns you can identify in the text. We’ll refine these later, so it’s fine to start with what’s relatively obvious.\n\nDon’t worry what these mean or signify, or whether they match other patterns, it’s just elements of text that are informative and relevant to the study.\n\nTo some extent the way you analyse your data is going to be subjective, because it’s interpretive. You’re not noting down simple facts, you’re looking for patterns, indications of emotion, motivations etc.\n\nAlthough to be fair there’s an element of this in quantitative too - you choose what to measure in the first place!\n\n
  • Once you’ve done this for all passages of text (for a large study it’s easier to do this on a participant by participant basis) you can then start to assign categories - open coding (abstract conceptual labels).\n\nSo I might assign the first comment to a label called “Visiting the shop”.\nI might assign the second comment to “Fitting into my day”.\n\n
  • Once you’ve done this for all passages of text (for a large study it’s easier to do this on a participant by participant basis) you can then start to assign categories - open coding (abstract conceptual labels).\n\nSo I might assign the first comment to a label called “Visiting the shop”.\nI might assign the second comment to “Fitting into my day”.\n\n
  • Then we move onto a deeper analysis of the initial codes. Axial coding involves finding patterns where we can potentially merge labels together, and also often find ways in which labels have pre-conditions or specific contexts.\n\nFor example we might find that browsing brochures happens almost exclusively at home, or of course that browsing a brochure always requires a shop visit to have taken place beforehand in order to get the brochure.\n
  • This phase is where we review our labels and try to unify them into our central grounded theory. This is our overall understanding and interpretation of the meaning in a study.\n\nThis gives us a framework of understanding about our research question. So in the case of our planned diary study it might be a story if you like, about how people plan holidays, which we can document and diagram in different ways.\n
  • We can create written narratives describing the theory, or elements of it, like the scenarios we created.\n
  • We can describe it visually, and map elements of the theory, a bit like our user journeys.\n\n
  • And of course we use personas as a way of summarising key behaviour patterns and the contexts and pre-conditions that we’ve found.\n\nBut you can document your theory in many different ways, as long as it suits the data you’re representing.\n\nThat is ultimately how it helps us to design new features, functionality or even an entire website.\n
  • I’m not going to detail it now, but there is a related method from psychology called experience sampling.\n\nEssentially it’s similar, but works by alerting you at specific times - through a PDA or mobile - and asks you specific questions.\n\nIt helps to minimise cognitive bias by asking you “in the moment” rather than recalling, but by asking specific questions it also limits the qualitative aspect of the feedback and steers it towards your specific questions. Might be a good alternative where you want to study very specific behaviours.\n\nI can refer anyone interested to a good article on it by Intel researchers.\n
  • ...to all of you for staying awake.\n

Diary studies for UX Presentation Transcript

  • 1. Musicians, birdwatchers & speech therapistsWhat diary studies can tell us about human behaviour July 2011
  • 2. Qualitative researchWe strive to understand not only what people do, but why they do it
  • 3. Commercial qualitative researchDiscover customer needs, and how products & services fit into a person’s life
  • 4. M-pesaCombining technology with cultural context
  • 5. Commercial realityWe need faster, cheaper methods to obtain the same data
  • 6. Limitations & weaknessesData quality is affected by cognitive bias
  • 7. Diary studiesBecause you can’t always be there...
  • 8. Benefits of diary studiesDevelop a rich understanding of context and environment
  • 9. Benefits of diary studiesGain insight into intermittent or difficult to observe situations
  • 10. Benefits of diary studiesIt can be run remotely
  • 11. Benefits of diary studiesCombine qualitative and quantitative data
  • 12. Benefits of diary studiesFlexible methodology suits a range of projects
  • 13. Types of diary studyReal-time feedback or “experience sampling”
  • 14. Types of diary studyElicitation
  • 15. How we collect the dataStructured
  • 16. How we collect the dataSemi-structured
  • 17. How we collect the dataUnstructured
  • 18. They can be simplePaper diaries
  • 19. Or complexDigital camera, voice recorder, paper diary, stickers, cards, pages of notes...
  • 20. Improving diaries with technologyIt doesn’t need to be just paper
  • 21. PhotographsA picture tells a thousand words
  • 22. Social mediaMultiple channels of communication
  • 23. AnalysisMaking sense of the data
  • 24. I grabbed some brochures from First Choice today. I had a few minutes to spare at lunchtime, and the shop was only down the road from my office. I spent an hour or so looking over them tonight with a coffee. I’ll get a few ideas before I speak to Joe, he’s not interested until I narrow it down a bit.Example data
  • 25. I grabbed some brochures from First Choice today. I had a few minutes to spare at lunchtime, and the shop was only down the road from my office. I spent an hour or so looking over them tonight with a coffee. I’ll get a few ideas before I speak to Joe, he’s not interested until I narrow it down a bit.Critical instances
  • 26. I grabbed some brochures from First Choice today. I had a few minutes to spare at lunchtime, and the shop was only down the road from my office. Visiting the shop I spent an hour or so looking over them tonight with a coffee. I’ll get a few ideas before I speak to Joe, he’s not interested until I narrow it down a bit.Open coding
  • 27. I grabbed some brochures from First Choice today. I had a few minutes to spare at lunchtime, and the shop was only down the road from my office. Fitting into my day I spent an hour or so looking over them tonight with a coffee. I’ll get a few ideas before I speak to Joe, he’s not interested until I narrow it down a bit.Open coding
  • 28. I grabbed some brochures from First Choice today. I had a few minutes to spare at lunchtime, and the shop was only down the road from my office. I spent an hour or so looking over them tonight with a coffee. I’ll get a few ideas before I speak to Joe, he’s not interested until I narrow it down a bit.Axial coding
  • 29. Selective coding
  • 30. Persona 1 Scenario 1: Get inspired Whilst out doing her Christmas shopping, Helen walks past her local travel agent and decides to drop in to get some brochures. She always takes a family holiday during the school sumer holidays, and she likes to plan - and book - early. She’s always been to Spain in the past, but friends keep recommending Turkey, so she picks up some brochures for both countries. At home later that evening, she starts to skim through the brochures over a coffee. At this point she just likes to have a glance through the photographs, to get an idea of what areas are worth looking at in more detail. She knows the prices aren’t completely accurate, but she uses them to make sure she doesn’t waste time looking at places that are out of her price range. When she finds a hotel that she’s interested in, she notices that there’s a web link advertised on the page offering more detail. She opens her laptop and visits the link from the brochure. This takes her straight to a page with more detail on the same hotel. This one particularly appeals now because it shows her photographs of other families with children playing around the pool and, in others, families enjoying the evening entertainment. She can browse through a massive range of photographs too - showing everything from the inside of each type of room, to the pool, bar and restaurants. She can see some indicative prices on the same page too, including prices for families like hers - 2 adults and 2 children. It’s not necessarily the dates she wants to travel, but it gives her an idea what it might cost. There’s a calendar too so that she can get the right prices for when she might really want to travel. She’ll definitely take a look at that another time, but not right now - she’s just getting ideas at this stage.Selective coding
  • 31. Persona 1 Scenario 1: Get inspired Whilst out doing her Christmas shopping, Helen walks past her local travel agent and decides to drop in to get some brochures. She always takes a family holiday during the school sumer holidays, and she likes to plan - and book - early. She’s always been to Spain in the past, but friends keep recommending Turkey, so she picks up some brochures for both countries. At home later that evening, she starts to skim through the brochures over a coffee. At this point she just likes to have a glance through the photographs, to get an idea of what areas are worth looking at in more detail. She knows the prices aren’t completely accurate, but she uses them to make sure she doesn’t waste time looking at places that are out of her price range. When she finds a hotel that she’s interested in, she notices that there’s a web link advertised on the page offering more detail. Persona 1 Browser http://www.google.co.uk 3.) Unit page - Price Browser http://www.google.co.uk S e a r c 5.) Itinerary h Looking for holiday ideas Thomson Thomson From what shes read so far, Helen Helen notices that theres an extra Unit page likes the look of this holiday, but is it Itinerary page that allows her put together her really in her price range if she can only own itinerary of events and activities. She opens her laptop and visits the Visits the Thomson website after reading a brochure her straight to a page 1.) link from the brochure. This takes Browser http://www.google.co.uk Thomson travel in the summer holidays? Thats really useful for her because it starts to bring the holiday to life a bit with more detail on the same hotel. This one particularly appeals now because it shows her Package holidays Helen visits the site to see more information on the this is a hand writing font it looks nice this is a hand this is a hand writing font it looks nice this She can see an option straight away more. holidays shes seen in the brochures. When she gets to Find a holiday writing font it looks is a hand writing font it looks nice this is a nice this is a hand hand writing font it looks nice writing font it looks this is a hand writing font it looks nice this that will allow her to see how much it the site, the first thing shes looking for is a place to type photographs of other families with in the codes shes picked out from the brochures. pool and, in others, families children playing around the nice is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice this is a would cost, using the dates that suit She can get an idea of how much it hand writing font it looks nice her. This will save her wasting time would cost over the course of the enjoying the evening entertainment. She can browse through a massive range of looking at holidays out of her price week too, which is really handy for range. budgeting. photographs too - showing everything from the inside of each type of room, to the pool, bar 4.) Unit page - Activities Browser http://www.google.co.uk 6.) Next option Browser and restaurants. http://www.google.co.uk Thomson One of the key things Helen needs to Thomson 2.) Thomson unit page Browser Now that she has the key facts about http://www.google.co.uk Thomson Unit page know is how child-friendly this holiday would be. Not only facilities like a Unit page this holiday, Helen wants to check out After typing in the code, Helen is swimming pool or creche, but are the next option on her list. So she Unit page shown a page about the holiday. She can see some indicative prices on the same page too, including prices for families like there things for the kids to do? Its saves this one in her shortlist and difficult to keep them entertained if moves on. She can see a range of photographs this is a hand writing font it looks nice this is a hand writing font it looks nice this is a hers - 2 adults and 2 children. It’s not necessarily the dates she wants to travel, but it gives and crucially for her, she hand writing font it looks nice there arent events or activities in the this is a hand writing font it looks nice this and video, this is a hand writing font it looks nice this is a hand writing font it looks nice this is a is a hand writing font it looks nice this is a hand writing font it looks nice hand writing font it looks nice resort. this is a hand writing font it looks nice this The first thing she notices on the next can see information on what she can this is a hand writing font it looks nice this is a hand writing font it looks nice this is a this is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice this holiday is that its remembered her her an idea what it might cost. There’s a calendar too so that she can get the right pricesthere with the kids, and what do for hand writing font it looks nice is a hand writing font it looks nice this is a this is a hand writing font it looks nice this hand writing font it looks nice details, and given her a price is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice this facilities they have. automatically - great! is a hand writing font it looks nice this is a hand writing font it looks nice when she might really want to travel. She’ll definitely take a look at that another time, but not right now - she’s just getting ideas at this stage.Selective coding
  • 32. Persona 1 Scenario 1: Get inspired Helen Tate 41 years old, married, with 2 daughters aged 12 and 16 Whilst out doing her Christmas shopping, Helen walks past her local travel agent and decides to drop in to get some brochures. She always takes a family holiday during the school sumer holidays, and she likes to plan - and book - early. “The most important things for me are that it’s got to be cheap and have hot She’s always been to Spain in the past, but friends keep recommending Turkey, so she picks weather, with plenty to keep the kids up some brochures for both countries. occupied. Other than that it doesn’t really matter” At home later that evening, she starts to skim through the brochures over a coffee. At this Explorer point she just likes to have a glance through the photographs, to get an idea of what areas are worth looking at in more detail. Advice seeker Deal finder She knows the prices aren’t completely accurate, but she uses them to make sure she doesn’t waste time looking at places that are out of her price range. When she finds a hotel that she’s interested in, she notices that there’s a web link advertised on the page offering Experience Goals Get the reassurance I need to feel confident in my choices more detail. Persona 1 Browser http://www.google.co.uk 3.) Unit page - Price Browser http://www.google.co.uk S e a r c 5.) Itinerary h Looking for holiday ideas Thomson Thomson From what shes read so far, Helen Helen notices that theres an extra Itinerary Independent Unit page likes the look of this holiday, but is it really in her price range if she can only Holiday Goals Itinerary page that allows her put together her own itinerary of events and activities. planner traveller She opens her laptop and visits the Visits the Thomson website after reading a brochure her straight to a page 1.) link from the brochure. This takes Browser http://www.google.co.uk Thomson travel in the summer holidays? Thats really useful for her because it I want good value for money starts to bring the holiday to life a bit with more detail on the same hotel. This one particularly appeals now because it shows her Package holidays Helen visits the site to see more information on the Narrative this is a hand writing font it looks nice this is a hand this is a hand writing font it looks nice this She can see an option straight away It’s more important to get a reliable holiday than an exciting holiday more. holidays shes seen in the brochures. When she gets to Find a holiday writing font it looks is a hand writing font it looks nice this is a nice this is a hand hand writing font it looks nice writing font it looks this is a hand writing font it looks nice this that will allow her to see how much it the site, the first thing shes looking for is a place to type photographs of other families with in the codes shes picked out from the brochures. pool and, in others, families children playing around the nice is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice this would cost, using the dates that suit She can get an idea of how much it is a hand writing font it looks nice this is a hand writing font it looks nice We’re always focussed on getting value for money, as well as having her. This will save her wasting time Holiday Criteria would cost over the course of the a good holiday obviously! I dont mind going back to places Ive enjoying the evening entertainment. She can browse through a massive range of looking at holidays out of her price week too, which is really handy for visited before, as long as we can be pretty sure its got good weather range. budgeting. 1.) Good weather and plenty to entertain the kids. This is our only holiday of the year photographs too - showing everything from the inside of each type of room, to the pool, bar so I cant afford to take the risk of getting it wrong. 4.) Unit page - Activities Cheap holiday 2.) Browser http://www.google.co.uk 6.) Next option Browser and restaurants. http://www.google.co.uk Thomson One of the key things Helen Evening 3.) needs to entertainment We’re not big on foreign food or seeing all the sights anyway, we just Thomson 2.) Thomson unit page Browser Now that she has the key facts about http://www.google.co.uk Thomson Unit page know is how child-friendly this holiday 4.) Activities and entertainment for children want to have a good time, so constantly going to new places doesn’t would be. Not only facilities like a Unit page this holiday, Helen wants to check out After typing in the code, Helen is particularly appeal to me anyway. Unit page shown a page about the holiday. 5.) Clean swimming pool or creche, but are hotel facilities the next option on her list. So she She can see some indicative prices on the same page too, including prices for families like there things for the kids to do? Its saves this one in her shortlist and I usually start planning our summer holiday a long way in advance. difficult to keep them entertained if moves on. She can see a range of photographs this is a hand writing font it looks nice this is a hand writing font it looks nice this is a Even though we can only go in the school summer holidays, I’ll still hers - 2 adults and 2 children. It’s not necessarily the dates she wants to travel, but it gives and crucially for her, she hand writing font it looks nice there arent events or activities in the this is a hand writing font it looks nice this and video, Research & booking touchpoints this is a hand writing font it looks nice this is a hand writing font it looks nice this is a is a hand writing font it looks nice this is a hand writing font it looks nice get some brochures around Christmas, so thaton the next time to get The first thing she notices we have hand writing font it looks nice this is a hand writing font it looks nice this this is a hand writing font it looks nice this can see information on what she can this is a hand writing font it looks nice this is a hand writing font it looks nice this is a resort. is a hand writing font it looks nice this is a hand writing font it looks nice the best deal for the summer. It givesits remembered her look forward holiday is that us something to is a hand writing font it looks nice this is a hand writing font it looks nice this is a hand writing font it looks nice this her an idea what it might cost. There’s a calendar too so that she can get the right pricesthere with the kids, and what do for hand writing font it looks nice is a hand writing font it looks nice this is a this is a hand writing font it looks nice this hand writing font it looks nice Scans brochures for ideas details, and given her a price is a hand writing font it looks nice this is a to as well. hand writing font it looks nice this is a hand writing font it looks nice this facilities they have. automatically - great! is a hand writing font it looks nice this is a hand writing font it looks nice when she might really want to travel. She’ll definitely take a look at that another time, but not Looks online for more in-depth research We usually go to Spain but some friends have mentioned Turkey a Calls travel agent to check prices few times, so I’ve picked up a couple of brochures about it. Looks right now - she’s just getting ideas at this stage. Books through Travel agent or online really nice, and apparently you’re guaranteed hot, sunny weather. We generally book online but I try to call the travel agent first. You What can TUI do for her? never know if they can offer you a better deal, so it’s worth checking. Provide price information well in advance For me holidays are all about getting away and relaxing, but when Allow her to check and compare weather at different destinations you have kids then you can’t relax unless they’re having a good time, so that becomes the focus of your holiday. To be fair though, it’s Provide information on what entertainment is available for her childnre really nice for Dave and I when we see the girls are enjoying themselves too so we don’t really mind.Selective coding
  • 33. Experience samplingAn alternative to diaries?
  • 34. And that’s it!Thanks...