Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to search for insights in survey data

106 views

Published on

In this ‘How to’ webinar, NewMR’s Ray Poynter and Infotools’ Geoff Lowe look at strategies for finding the story in survey data, and in particular that part of the process that focuses on searching a data set efficiently to identify the key findings, which will provide the building blocks for the story.

The presentation shows how to systematically use a ‘question and seek’ method to build insights.
The webinar starts by looking at what can be achieved within the data collection tool (QuestionPro), then SPSS, and finally using some of the visual and labour-saving features offered by Infotools.

**Here are some earlier webinars on the same theme, you might find the slides and recordings of interest**

- How to use Correlations to find Insights, May 2019: accessible here https://newmr.org/events/2019/how-to-use-correlations

- How to Use Comparisons to Find & Tell Stories in the Data, Feb 2019: accessible here https://newmr.org/events/2019/how-to-use-comparisons-to-find-and-tell-the-story-in-the-data

- How to make numbers more visual, October 2018: accessible here https://newmr.org/events/2018/how-to-make-numbers-more-visual

Published in: Education
  • Be the first to comment

How to search for insights in survey data

  1. 1. How to search for insights in survey data? Ray Poynter with Geoff Lowe from Infotools Webinar, 25 June 2020 Ray Poynter NewMR Geoff Lowe Infotools
  2. 2. Sponsors Gold Silver
  3. 3. Agenda The NewMR framework The example data During fieldwork Checking and organising Main Story – SPSS Relevant Exceptions – Harmoni Q & A
  4. 4. NewMR Framework 1.  Define the problem 2.  Link to the wider context 3.  Organise the data 4.  Use the business question as a lens 5.  Find the main story, the big picture 6.  Find the relevant exceptions 7.  Create an overall story 8.  Use Think, Feel, Do! to create & communicate the story
  5. 5. Defining the Problem 1.  Are researchers coping with the pandemic crisis? 2.  If not, what are the problems and who is experiencing them? 3.  What should the leaders of the research industry be doing?
  6. 6. Link to the Wider Context WFH became the norm Trade bodies creating COVID-19 resource pages Lots of concern about the plight of freelancers/sole traders Widespread expectation that people will have spare time on their hands (with some arguing this was not so) Many client companies in hibernation Global marketing / advertising budgets slashed
  7. 7. This is the second wave So, review messages from Wave 1 •  Most people are coping with the pandemic crisis, but about one-in-six are not •  Most people are working from home, which is OK for most people, but not for about 20% of us •  The key things that research leaders should be doing are: –  Advocate for research –  Help support the wellbeing of our people
  8. 8. The Research •  A survey conducted with insight and research professionals •  A second wave of a study conducted a month earlier •  Additional research question –  Has anything changed between the end of March and the end of April – if so, are there implications for what research leaders and organisations should be doing?
  9. 9. Organise the Data •  This starts during fieldwork –  Are there any problems? –  What are the emerging messages? –  Do we need to tweak the research? •  Checking the data •  Creating a data set that can be investigated
  10. 10. During Fieldwork - QuestionPro
  11. 11. Read the last question first
  12. 12. Google, Portuguese detected – ‘not working’ Google, Portuguese detected – ‘without research work’
  13. 13. Emerging Messages •  Lots of mentions about sole traders •  Some really sad cases •  About half the people reported drop in income •  The data looks pretty similar to the first wave
  14. 14. Checking the Data Response ID IP Address Timestamp (mm/ dd/yyyy) Duplicate Time Taken to Complete (Seconds) Responden t Email Country Code Region How do you, personally, feel today? Which of the following best describes your current (this week) work situation? Select all that apply Working in an office Working from home 51861253 **RAA Blocked** 04/25/2020 02:37 FALSE 27 **RAA Blocked** **RAA Blocked** **RAA Blocked** 3 51861304 **RAA Blocked** 04/25/2020 02:49 FALSE 85 **RAA Blocked** **RAA Blocked** **RAA Blocked** 3 1 51861325 **RAA Blocked** 04/25/2020 02:53 FALSE 45 **RAA Blocked** **RAA Blocked** **RAA Blocked** 3 1 51861327 **RAA Blocked** 04/25/2020 02:54 FALSE 311 **RAA Blocked** **RAA Blocked** **RAA Blocked** 3 1 51861365 **RAA Blocked** 04/25/2020 03:07 FALSE 345 **RAA Blocked** **RAA Blocked** **RAA Blocked** 4 51861373 **RAA Blocked** 04/25/2020 03:12 FALSE 119 **RAA Blocked** **RAA Blocked** **RAA Blocked** 4 1 51861391 **RAA Blocked** 04/25/2020 03:18 FALSE 183 **RAA Blocked** **RAA Blocked** **RAA Blocked** 3 1 51861407 **RAA Blocked** 04/25/2020 03:27 FALSE 66 **RAA Blocked** **RAA Blocked** **RAA Blocked** 4 1
  15. 15. Checking the open-ended comments Better leads imply better opportunities for investment - https://peerberry.com/. Investment stimulus is required [https://www.mintos.com/en/], we have projects naturally as well. Investment is golden key to positive growth [https:// pensionfund.org/], 2$M looked upon favourably.
  16. 16. Check Which best describes your organisation? How many people work in your organisation (including you)? Age? Is there anything else you would like to tell us about this topic or this survey? Other Other Use No 2 3 3 Yes No 3 1 4 Thank you for this & hope there's something positive coming up for all of us, soon..... Yes 2 1 5 Yes 2 4 5 Yes 2 4 6 No Yes 2 1 3 Yes 2 3 3 No Yes 1 7 2 NA Yes 2 1 6 Yes
  17. 17. Creating a Data Set •  Export to SPSS –  Tip, try to avoid CSV or Excel, they lose the Question and Labels •  Group the countries into regions –  Including all the ‘Other specify’ •  Integrate with previous Wave –  Adding a Wave variable –  Sense check the combined data set
  18. 18. Use the Business Question as a Lens •  We are not looking for everything that is ‘interesting’ •  We are seeking to answer business questions and provide insight into decisions Our Business Questions 1.  Are researchers coping with the pandemic crisis? 2.  If not, what are the problems and who is experiencing them? 3.  What should the leaders of the research industry be doing? Additional Research Question Has anything changed between the end of March and the end of April – if so, are there implications for what research leaders and organisations should be doing?
  19. 19. Searching not Trawling Use our questions to tell us where to look and what to look for
  20. 20. The Main Story Before looking for exceptions, we look for the main story, in the context of our questions Our Business Questions 1.  Are researchers coping with the pandemic crisis? 2.  If not, what are the problems and who is experiencing them? 3.  What should the leaders of the research industry be doing? Additional Research Question Has anything changed between the end of March and the end of April – if so, are there implications for what research leaders and organisations should be doing?
  21. 21. About one-in-six pick a sad face – similar to wave 1 About 5% working in an office – similar to wave 1 Nearly 90% working from home – similar to wave 1
  22. 22. About 10% not working, needs accounting for in the reporting of percentages – similar to wave 1 About 3% picked other, not many, but read open ends when looking for detail – similar to wave 1 Question not asked in Wave 2, useful background information About 80% say working from home suiting them at the moment, base = those working from home – similar to wave 1
  23. 23. Statistical Significance •  With a sample of over 2000 people (more than 1000 per wave) there are lots of differences that are statistically significant •  Most of them are NOT interesting in terms of the main story •  In this sort of study I am looking for differences around 10% points or more – this is story finding not an exercise in measuring the minutiae
  24. 24. About one-third are working fewer hours, one-third the same, and one-third working more hours. About half have seen their income fall, about half have seen no change, and almost nobody (1.6%, 18 people) is reporting an increase in income.
  25. 25. Suggestions for our Leaders Raw Text Notes Give clear updates on how the company is performing. If there are potential redundancies coming, warm people so they can budget appropriately now Company, leadership, transparency No idea NA no NA Treat the employees as human and don't sack them to showing the lack of business growth Company, leadership, welfare Not seeing enough coverage in the media if how people should interpret research More discussion on the limitations of the use of virtual connection in qual research. ... Eg the inability to really understand body language MR specific Just be yourself - you are a leader, heads up, maybe it will not be perfect but don,t stop - lead as much as you can every day. Imagine what is good as a goal today and be the leader. Talk about that being careful about wording, natural language, no hate, compassion - one day up, another neutral/sleepy or pause, just do it, keep walking and they will follow. Your team depends on your strength and direction Moral guidance and leadership Think long term Beyond COVID Stop bugging us with online options!! We know about them. It’s our clients that have to get back to rewarding projects or resuming work. Focus on projects not methods Encourage brands to spend money on research to help them be successful in the future Advocate for more research Keep up the pressure on MPs and the Treasury to help one man bands Advocate for hardship cases, sole traders
  26. 26. Processing the Suggestions •  Focus on things our leaders could be doing to tackle current problems –  Don’t report every topic mentioned •  Look for the main story •  Don’t be swayed by hard cases or articulate responses •  Generalise the specific to facilitate finding the main story
  27. 27. Generalise the specific to facilitate finding the main story “Keep up the pressure on MPs and the Treasury to help one man bands” 1. Advocate with Governments and similar bodies 2. To help MR people in need 3. Specifically sole traders / freelancers / one man bands etc
  28. 28. Relevant Exceptions •  Use your key stories and issues as the dependent variable •  In most cases that means setting the question of interest as the columns –  And setting the other variables as the rows •  Look for big differences, around 10% or more
  29. 29. 90% of people who picked a Happy Face said working from home suits them, this drops to 68% for people who picked a Sad Face – so people who do NOT favour WFH are a potential relevant exception There is an interaction with change in hours worked and face chosen. 41% of Happy Face are working similar hours, but this drops to 28% for Sad Face Strong interaction between Face and income change. 41% of Happy Faces say income lower, but 64% of Sad Faces say income lower.
  30. 30. Relevant Exceptions New Zealand Better and Worse Off Geoff Lowe Infotools Interactive Session See the recording of the presentation
  31. 31. Think, Feel, Do! •  The Think is the data •  The Do is a)  Some people need help, find out who needs help, try to help them b)  Advocate for the importance of market research c)  Several secondary Do’s, such as going digital, being resilient, and thinking about the long term •  The Feel is the way we communicate the Do
  32. 32. The Story? •  Some people suggested the main story is 80% of people are coping fine •  But – what is the implied action? No action? •  The ‘Do’ we want to create is ‘find who has a problem and see if you can help them’ •  So the story has to be ”Most are OK, but some aren’t”
  33. 33. Lots More – In the Report
  34. 34. Q & A Sue York NewMR Ray Poynter NewMR Geoff Lowe Infotools
  35. 35. Sponsors Gold Silver
  36. 36. Thank You
  37. 37. Thank You

×