Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sense-making with open-ended text

340 views

Published on

This presentation (the 3rd of four in our series on this topic) looks at sense-making with open-ended text, for example: survey open-ends, social media text, letters and emails, and other forms of text communication. The webinar highlights a range of approaches that enable the researcher to identify the meaning in the data and construct narratives that enable end-users of the research to utilise the sense-making process.

Published in: Education
  • Be the first to comment

Sense-making with open-ended text

  1. 1. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Finding & Communica-ng the Story Lesson 3 of 4 (2017 series) Sense-making with Open-ended Text Ray Poynter September 2017
  2. 2. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Agenda 1.  The context for open-ended text 2.  Finding stories with a frameworks approach 3.  Linked and Independent text 4.  Methods of analysing qual 5.  CAQDA 6.  Top Tips
  3. 3. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Series Schedule – 2017 •  How to see the meaning in numbers 31 May •  How to communicate numerical informaKon 5 July •  Sense-making with open-ended text 7 September •  Think, Feel, Do PresentaKons that change behaviour 9 November
  4. 4. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Open-ended Text Yes –  Survey verbaKms –  Text from online discussions/groups that you did not moderate –  SM & blog posts –  LeSers and emails from customers Not today –  Images –  Video –  Other artefacts –  Speech –  Face/body –  Your own notes/journals
  5. 5. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter The Frameworks Approach 1.  Define and frame the problem 2.  Establish what is already known –  And, what is believed/expected 3.  Organise the data to be analysed 4.  Apply systemaKc analysis processes 5.  Extract and create the story
  6. 6. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Who is the project for? _________________ What is the business issue/problem that is being addressed? __________________________________________________ What does the business want to do, once it has addressed this issue? ______________________________________________________ What do we already know? Item Held by: Descrip-on 1  ______ ______ ______________ 2  ______ ______ ______________ 3  ______ ______ ______________ Assump-ons and predic-ons Who What 1.  ______ ______ 2.  ______ ______ Simplified
  7. 7. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Define the Problem A problem defined is a problem half-solved Talk to the client (ideally including end-client) –  What is the background to the research? –  Why is it happening now? –  What does success look like? –  What acKons depend on the findings? –  What do people believe? What do they expect the results to be?
  8. 8. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Use the business ques-on as a lens •  Insights need to link to outcomes and acKon •  What is happening? •  Why is it happening? •  What changes: – Can I make? – Can I encourage others to make?
  9. 9. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter The Field of Play – Open-ended Text QuanKty of text 1000 survey answers to “Favourite brand of sob drink?” A shopping list 10 answers, “Brand other …” Complexity A poem A lover’s leSer IDI with a denKst 100 pages of user feedback 100s of online discussions 1000s of complaint leSers Millions of posts about Apple
  10. 10. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Standalone or Linked Text? •  Prompted and individual •  Unprompted and individual •  Linked within one person •  Linked between people
  11. 11. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Prompted and Individual What are the most important factors to consider for the future of sampling? [GRIT 2017 Q1 / 2 Survey] •  “Quality of sample. More clients need to understand just how much of a problem this issue is.” •  ”Allowing for diversity among those in the sample. I want a sample that is representaKve of my target audience” •  “Accuracy” •  “Ability to catch bots on online surveys”
  12. 12. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Unprompted and Individual
  13. 13. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Linked Within One Person Which agency is Innova-ve? Why do you say that? “Lucid” “Ability to leverage plahorm to connect buyers and sellers” “System 1 formerly Brain Juicer” “Send out a lot informaKon about what they're doing with webinars and case studies” “Sawtooth” “Cukng edge technology and knowledge in specialized field of market research” “Happy Thinking People” “Screening and integraKng current trends into a holisKc research approach” [GRIT 2017 Q1 / 2 Survey]
  14. 14. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Linked Between People [John] I think Saucony are the best running shoes at the moment. [Sally] Me too! [Pete] Yes, every Cme [Adele] They are nice, but I think they are too expensive. [John] I think they are worth it, they feel more comfortable and supporCve, don’t even have to break them in. [Sally] Yes, definitely worth it 4 people discussing running shoes All 4 like Saucony, but 1 person thinks they are ‘too expensive’ Humans have no problem with ‘Me too!’ and ‘Yes, every Kme’ being about Saucony – if the discourse is present
  15. 15. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter If you can use TripAdvisor you can do qual
  16. 16. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter The TripAdvisor Paradigm •  What are the themes, elements and concepts? •  Why have they wriSen it? •  What sort of person are they? •  What were their expectaKons? •  What ‘role’ are they ‘performing’ •  What elements can I triangulate? •  Where are the strong feelings, beliefs, comments
  17. 17. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Academic versus Commercial Analysis of Qualita-ve Data Many qual techniques are used by both academic & commercial researchers , e.g. conversaKon analysis, grounded theory, etc But! –  Timelines vary, commercial one day to one week, academic can be months (or years) –  Success can vary, commercial = beSer business decision, academic = advancing knowledge –  Purity of methodology, academic more pure, commercial more pragmaKc (oben means using hybrids)
  18. 18. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Common Analy-cal Approaches •  Grounded Theory – created by Glaser & Strauss in the 1960s adopts a formal approach to coding the data, linking the codes into concepts, linking these into categories, and creaKng an overarching structure. Tends to require plenty of Kme. Tries to ignore exisKng theories – increasing sensiKvity to the content of the data. InducKve approach, general theories from specific observaKons. •  Abduc-ve Analysis – compares the data with the theories and expectaKons, idenKfy the non-expected and leap (abduct) from these observaKons to a new theory that is sufficient and probably. •  Content Analysis – is popular both with tradiKonal researchers and those seeking to computerise some or all of qualitaKve analysis. As with other approaches, the data is coded and categorised, but in content analysis the frequency of codes and categories and the frequency of links between them is taken into greater account that with most other methods. The use of ‘’counKng’ increases the importance of sampling when using content analysis.
  19. 19. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Common Analy-cal Approaches •  Narra-ve Analysis – focuses on the enKre text, not subdivided components. Enter the text (coding/memoing), interpreKng, verifying (e.g. alternaKve explanaKons), represenKng (write the plot of the story), illustraKng (e.g. finding quotes, drawing diagrams). •  Conversa-on Analysis – is one form of Discourse Analysis, CA, ConversaKon Analysis, was developed from the work of Harvey Sacks’ work in the 1960s & 1970s. CA looks at how people speak, the paSerns they use, how they create meaning, for example: turn-taking, repairs, dispreferred responses. ConversaKon analysis pays less aSenKon to what people say than the way they say it. •  Thema-c Analysis – the focus is to generate themes from the data. In parKcular paSerns (e.g. codes and categories) are idenKfied in the early data (e.g. the first interviews or focus groups) and then used as tools to analyse subsequent data. One difference between themaKc and grounded theories is that grounded theory seeks to create a broader theory, themaKc analysis tends to be happy to create a narraKve to explain the data.
  20. 20. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Semio-cs Semio-cs was developed from the work of Ferdinand de Saussure from the later 19thCentury onwards. SemioKcs is the study of meaning-making by looking at the use of signs and symbols (which can be any form of data, including worlds, brands, images, sounds etc.) SemioKcs does not require the collecKon of data from research parKcipants; semioKcs if frequently conducted with artefacts that exist in the ‘real world’ rather than in an MR created world. However, semioKcs can be applied to MR data, just as it can be applied to any other data. Sign Signified Signifier Sign Rose Sign Passion Rose Sign Passion
  21. 21. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Overarching Structure No uniform No books Travel costs School fees Worry Mind elsewhere Tired in School Headaches Lack school materials Unable to pay school costs Worry about dependents Feeling exhausted Physically & emoKonally stressed Can’t afford school These children have tangible problems Adapted from www.open.edu/openlearnworks/mod/resource/view.php?id=52658
  22. 22. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Conversa-on Analysis Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either prevenKve materials or restoraKve materials; what do you take in account when you decide which one to adopt? Well, that's a good quesKon. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own liSle research about that, because I don't really trust the research that comes with the product and once again what other denKsts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products. Conversa-on Analysis Pauses/Repairs/Disconnects: Person is portraying that they are not confident. Restructured answer “Well, that’s a good quesKon.” – Indicates the quesKon was not a good quesKon, deals with it by saying ‘Don’t know’ and then proceeds to answer what he/she thinks the quesKoner is hoping to learn. From an example of Grounded Theory www.biomedcentral.com/imedia/4037816045634649/supp3.doc
  23. 23. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Discourse Analysis Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either prevenKve materials or restoraKve materials; what do you take in account when you decide which one to adopt? Well, that's a good quesKon. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own liSle research about that, because I don't really trust the research that comes with the product and once again what other denKsts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products. DA - Foo-ng The role the denKst is filling? Somebody who is not confident, and who is doubhul about the sources available to him/her.
  24. 24. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Discourse Analysis Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either prevenKve materials or restoraKve materials; what do you take in account when you decide which one to adopt? Well, that's a good quesKon. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own liSle research about that, because I don't really trust the research that comes with the product and once again what other denKsts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products. DA – Repe--on Reliability & “Internet sites” No repeKKon of cost. Cost is a ‘preferred response’ – it is used and discarded.
  25. 25. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Discourse Analysis Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either prevenKve materials or restoraKve materials; what do you take in account when you decide which one to adopt? Well, that's a good quesKon. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own liSle research about that, because I don't really trust the research that comes with the product and once again what other denKsts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products. DA – Evalua-ve terms I quite like my own liSle research I don’t really trust the research that comes with the product Some of those internet forums are actually quite good for new products
  26. 26. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter DA Thoughts Q. What did you take into account when you decided to buy this new technology? What did we... we looked at cost, we looked at reliability and we sort of, we compared a few different types, talked to some people that had them. Q. When you say you talked to some people who were they? Some dental colleagues. There's a couple of internet sites that we talked to some people... people had tried out some that didn't work very well. Q. So in terms of materials either prevenKve materials or restoraKve materials; what do you take in account when you decide which one to adopt? Well, that's a good quesKon. I don't know. I suppose we [laughs] look at reliability. I suppose I've been looking at literature involved in it so I quite like my own liSle research about that, because I don't really trust the research that comes with the product and once again what other denKsts are using and what they've been using and they're happy with. I'm finding the internet, some of those internet forums are actually quite good for new products. The story? The denKst lacks confidence, he/she menKons cost, but comes back to the topic of reliability. He/she distrusts the research from the manufacturers, so tries to do his/ her own research, by connecKng with people who have used the new products, via internet forums Sales Recommenda-on Connect this type of denKst with happy users. Encourage reliability tesKmonials and SM posts.
  27. 27. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Common Analy-cal Elements •  Saturated analysis – keep going unKl you stop finding new/useful things •  Structure – find/create an architecture to what you find •  Make notes of what you find, linking back to the data, highlighKng examples •  Look to support AND break hypotheses
  28. 28. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Computers & Qualita-ve Analysis •  Scissors & coloured pens è Word, Excel etc •  CAQDAS – Computer Aided QualitaKve Data Analysis Sobware, e.g. Nvivo •  Text analyKcs, from word clouds to Leximancer •  Social Media analysis, e.g. Brandwatch & Radian 6 •  Coding sobware, e.g. Ascribe •  Photos and Video organising, e.g. Google Photos and Living Lens Your organisaKon’s Framework should specify the tools to be used, storage protocols, and approaches to things like memos, tags, and notes.
  29. 29. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Holis-c Coding HT The Research Business 1990s Wendy Gordon, Colleen Ryan, Rebecca Wynberg et al [GRIT 2017 Q1 / 2 Survey]
  30. 30. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Holis-c Coding
  31. 31. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Holis-c Coding •  Random order of parKcipants •  Sort by who is most innovaKve brand •  Sort by country, and/or client/supplier, and/or number of employees •  Sort by whether samples gekng beSer or worse
  32. 32. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter www.qsrinternaKonal.com Some thoughts from the starter version. There is also the Windows Pro and Plus versions.
  33. 33. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Word Frequency and Stemming [GRIT 2017 Q1 / 2 Survey – most important factor for future of sampling]
  34. 34. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Word Clouds [GRIT 2017 Q1 / 2 Survey – most important factor for future of sampling]
  35. 35. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Word Trees [GRIT 2017 Q1 / 2 Survey – most important factor for future of sampling]
  36. 36. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Real Coding
  37. 37. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Building the Architecture
  38. 38. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Reading More than Once
  39. 39. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter There is More J Especially in Plus and Pro, for example: – Search, not just exact and stemmed, but also synonyms, specializaKons, and generalizaKons. – NCapture to collect social media text – AutomaKc coding by theme and senKment – Visit their website for more detail www.qsrinternaKonal.com
  40. 40. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Leximancer •  Starts with automated opKons •  Looks at words in context •  Creates concepts and themes •  The user can then drill down hSps://info.leximancer.com/
  41. 41. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Most important factor for future of Sampling
  42. 42. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter
  43. 43. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Drilling Down
  44. 44. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Ngrams www.masscogniKon.com Suppliers Clients Bigrams
  45. 45. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Triangula-on Suppliers Clients AdjecKves
  46. 46. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter The Field of Play – Open-ended Text QuanKty of text 1000 survey answers to “Favourite brand of sob drink?” A shopping list 10 answers, “Brand other …” Complexity A poem A lover’s leSer Q&A text re denKstry purchases 100 pages of user feedback 100s of online discussions 1000s of complaint leSers Millions of posts about Apple Read it Read and analyse it 1)  Sample, read, analyse 2)  CAQDA, read, analyse Use sobware
  47. 47. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Finding the Story •  Use the client’s quesKon as the lens •  Tag, code, memo the material as you analyse •  Challenge what is known/believed •  Find the main story •  Find the relevant excepKons/differences •  Create an overall structure, the plot •  Is it good news or bad news?
  48. 48. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Top Tips 1.  Define the problem before starKng the analysis (before starKng the research is best) 2.  Understand what qualitaKve analysis is, use the paradigm of understanding TripAdvisor 3.  Use the business quesKon as a lens to find the main story, the relevant excepKons 4.  Create a story with a clear architecture 5.  Learn how to use at least one of the computer aided tools
  49. 49. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Thank You! Follow me on Twieer @RayPoynter Or sign-up to receive our weekly newsleeer at hep://NewMR.org
  50. 50. Finding and Communica-ng the Story 2017 – Lesson 3 of 4 Sense-making with Open-ended Text - Ray Poynter Q & A Ray Poynter The Future Place

×