Surveys for communicatorsGlenn O’Neil
(these slides used for originally forworkshop held on 14 June 2013 forGeneva Communicators Network)
31. Introduction2. Planning3. Creation4. Management5. AnalysisSchedule
4Why a survey?surveys are used to collect information fromor about people ……to describe, compare or explain theirknowledge...
5Other research methods:– Desk studies– Interviews– Focus groups– Content analysis– Case studies– Records/statistics1. Int...
6Recommended: combination of qualitativeand quantitative methods1. Introduction
7Advantages of surveys:– Inexpensive– Obtains current information– Can obtain a great deal of information at onetime– Prov...
8Disadvantages of surveys:– People dont always tell the truth– People make mistakes– Representative samples are difficult ...
91. IntroductionIn communications - surveys are used for:– Assessing opinions/attitudes/behaviours ofaudiences– Measuring ...
10All surveys follow a similar timeline – orshould!2. Planning
112. Planning - timelineReviewbackgroundinformationSurveydesign& objectivesPreparesurvey& final reportoutlineDraft questio...
12Setting objectives (defining the project):– What is the time period – when are the resultsrequired?– What is the audienc...
13Moving from concepts to questions:– Operationalisation – translating abstract concepts intosomething that can be observe...
14Example: Moving from concepts to questions:3. CreationAssess successofcampaignReachUnderstandingInteractionAdhesionConce...
153. Creation– Survey outline can be a guide in creatingstructure of final report (think about it at thisstage!)– This hel...
163. CreationWhat are the variables we are often trying tomeasure in communications?– Awareness / Knowledge– Understanding...
173. CreationWriting questions– Some questions are easy to create (such asgender, length of employment, etc.)– But questio...
183. CreationWriting questions- Different question types are suited to differenttypes of information sought, for example:-...
193. CreationTypes of questions- Open questions- Closed questions:- Likert scale- Rank order scale- Multiple choice (singu...
203. CreationTypes of questions: Open questions- On all surveys its good to have at least oneopen-ended question- Open-end...
213. CreationAre there areas of weakness in your communicationsprogramme that are hindering your performance?Please list...
223. CreationIn your view, what are the two greatest weaknesses ofyour communications programme?In the past 12 months, w...
233. CreationQuestions: Likert scaleMost commonly used scale to measureattitudes and opinions:“The most important aspect o...
243. CreationQuestions: Likert scale– Create points on a scale that cover all dimensions– Consider using scales already de...
253. CreationHow would you rate your agency’s website?oPoor oGood oVery good oExcellentHow often do you use your mobile ...
263. CreationHow would you rate your agency’s website?oTerrible oPoor oGood oExcellentIn an average working day, how oft...
273. CreationLikert scale & attitudes– Questions often go beyond asking about anattitude as a whole– But ask about concret...
283. Creation“Are you satisfied with your job?”o Yes o No o Don’t know“Please indicate your level of satisfaction with t...
293. CreationRank order scale questions– These questions ask respondents to rank orstate their priority on items– Useful f...
303. CreationWhich of the following features would you like for yourorganisation’s website?e-newsletter[] recruitment too...
313. CreationMultiple choice (singular) questions– These questions ask respondents to chooseone item on a list– Are used t...
323. CreationWhat is your current work?o Administrativeo Managemento Staffo TechnicalWhich of the following best describ...
333. CreationMultiple choice (plural)questions– These questions ask respondents to choosemultiple items on a list– Useful ...
343. CreationWhat is your prior major field of working experience?o Politicalo Diplomatico Militaryo Developmento Humanit...
353. CreationBinary (yes/no)questions– These questions ask respondents to chooseone of two responses– Should have only one...
363. CreationShould the UN cut its communications and fundraisingbudget?o Yeso NoShould the UN substantially reduce its ...
373. CreationNumerical questions– These questions ask numerical data fromrespondents– Often used to collect budget and dem...
383. CreationHow may children do you have?O1-3 O3-6 O6-10What is your age?[18] – pull down list with numbers 0-100What ...
393. CreationHow may children do you have?O None O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O More than 10What is your age?ounder 18...
403. CreationFormulating questions – Hints– Only ask for one piece of information per question– Avoid overly complex words...
413. CreationFormulating questions – Hints– Avoid unreasonable recall periode.g. “Please provide budget details for the pa...
423. CreationFormulating questions – Hints– Avoid response options not matching the question– Avoid response options that ...
433. CreationSurvey componentsAside from the questions other components must beprepared:– Instructions– Transition texts (...
443. CreationSurvey components - instructions– Instructions appear on the first page of the survey– Instructions duplicate...
453. CreationSurvey components - instructionsExample (short form):This questionnaire is part of a review of the communicat...
463. CreationSurvey components – transition text– Transition texts are used in longer questionnaires wheresome explanation...
473. CreationSurvey components – thank you text– Thank you text appears at the end of the questionnaire.In online surveys,...
483. CreationSurvey components – invitation text– Invitation texts are important as they play a key role indetermining if ...
493. CreationSurvey components – invitation textsExample (online survey):From: mrose@unod.chTo: glenn.oneil@gmail.comSubje...
503. CreationPutting the survey together– Start with easy to answer questions– Broad to the narrow– Non-sensitive to the s...
513. CreationPutting the survey togetherWhich question should come first?1) How important or not is the intranet to your w...
523. CreationPutting the survey togetherAvoiding “response set”Staffing policies are fair and applied equallyoStrongly dis...
534. ManagementGetting ready to send the survey…– Once you have created the questions and placedthem in an appropriate ord...
544. ManagementTesting survey - Issues– How long did it take to complete it?– Are the questions understandable?– Did the s...
554. ManagementPrint verses online surveysPrint (respondents receive a word document, andrespond by completing it and post...
564. ManagementPrint verses online survey– The choice depends also on the context– For small groups that can be physically...
574. ManagementLaunching & promoting surveysOnce your survey has been tested and you havemade the appropriate adjustments,...
584. ManagementLaunching & promoting surveysThe question:How can we get the necessary number of people tocomplete the surv...
594. ManagementLaunching & promoting surveysSome advice to increase responses:• Ensure your survey is short, logical and a...
604. ManagementLaunching & promoting surveysDuring the period that respondents arecompleting the survey:• Check regularly ...
614. ManagementA word about response rates & sampling– In communications, we often are surveying precisegroups (e.g. staff...
624. ManagementA word about response rates & sampling– When dealing with groups under 100 try and get asmany responses as ...
635. AnalysisBasics of analysing survey data– What do the survey results show?– Looking for totals, averages, trends, patt...
645. AnalysisBasics of analysing survey dataTwo main types of analysis methods for quantitative data:– Descriptive statist...
655. AnalysisBasics of analysing survey dataDescriptive statistics can be displayed in different waysComparisonRanking/pri...
665. AnalysisBasics of analysing survey dataDescriptive statistics can be displayed in different waysTrends / changesRelat...
675. AnalysisBasics of analysing survey dataTallies/frequency counts: how many people fit into acategory – often displayed...
685. AnalysisBasics of analysing survey dataCross tabulation: Basic data split to show whichdemographic groups make up cat...
695. AnalysisBasics of analysing survey dataCross tabulation
705. AnalysisBasics of analysing survey dataPutting responses together: Certain question responses(but not all) can be put...
715. AnalysisBasics of analysing questionnaire dataHints on evaluating quantitative data– Simple counts, proportions and c...
725. AnalysisBasics of analysing survey dataEvaluating qualitative data– For survey, qualitative data is collected through...
735. AnalysisBasics of analysing survey dataSteps for evaluating qualitative data– 1. Read through the responses.– 2. Crea...
745. AnalysisReporting and using results– Now you have made some analysis of the datayou are ready to write a report– How ...
755. AnalysisReporting and using resultsHints on going from data to a report– Return to the original research question ask...
765. AnalysisReporting and using resultsBest practices for report writing– Quantitative data is best represented in graphs...
776. Conclusion– Some say that survey writing is more of an art than ascience– Always look towards existing, well-establis...
78Contact detailsoneil@owlre.comglennoneilwww.owlre.com@glenn_oneilwww.intelligentmeasurement.wordpress.comContacts:
Upcoming SlideShare
Loading in …5
×

Surveys for communicators

4,217 views

Published on

A one day workshop on surveys for communicators. Increasingly communicators need the ability to evaluate their activities and know what their audiences think and desire. Being able to design and set-up online surveys is a key tool for communicators for soliciting feedback and interacting with audiences. These slides from the workshop will take participants from the design to the analysis stage. Workshop originally conducted on 14 June 2013 for the Geneva Communicators Network.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Surveys for communicators

  1. 1. Surveys for communicatorsGlenn O’Neil
  2. 2. (these slides used for originally forworkshop held on 14 June 2013 forGeneva Communicators Network)
  3. 3. 31. Introduction2. Planning3. Creation4. Management5. AnalysisSchedule
  4. 4. 4Why a survey?surveys are used to collect information fromor about people ……to describe, compare or explain theirknowledge, feelings, values and behavioursBut they are not the only research method!1. Introduction
  5. 5. 5Other research methods:– Desk studies– Interviews– Focus groups– Content analysis– Case studies– Records/statistics1. Introduction
  6. 6. 6Recommended: combination of qualitativeand quantitative methods1. Introduction
  7. 7. 7Advantages of surveys:– Inexpensive– Obtains current information– Can obtain a great deal of information at onetime– Provide quantitative data1. Introduction
  8. 8. 8Disadvantages of surveys:– People dont always tell the truth– People make mistakes– Representative samples are difficult to obtain– People refuse to participate– Writing questions is tricky1. Introduction
  9. 9. 91. IntroductionIn communications - surveys are used for:– Assessing opinions/attitudes/behaviours ofaudiences– Measuring reputation of organisations– Assessing staff satisfaction– Evaluating communication products,campaigns, projects or programmesWhat other uses?
  10. 10. 10All surveys follow a similar timeline – orshould!2. Planning
  11. 11. 112. Planning - timelineReviewbackgroundinformationSurveydesign& objectivesPreparesurvey& final reportoutlineDraft questions& othercomponentsPlacequestionsin appropriateorderTestsurveyRevisesurveyfollowing testLaunch &PromotesurveyMonitorresponses& sendremindersas neededCollectresponsesAnalyse dataDraft reportPresent reportStep 1: Planning Step 2: Creation Step 3: Management Step 4: Analysis
  12. 12. 12Setting objectives (defining the project):– What is the time period – when are the resultsrequired?– What is the audience of interest (and are there sub-groups)?– What information is needed?– How will the information be collected?– What form should the results take? How are theresults going to be used?– How accurate should the results be?– Is this project going to be repeated?2. Planning
  13. 13. 13Moving from concepts to questions:– Operationalisation – translating abstract concepts intosomething that can be observed and measured:Concepts -> Variables ->Questions3. Creation
  14. 14. 14Example: Moving from concepts to questions:3. CreationAssess successofcampaignReachUnderstandingInteractionAdhesionConcept -> Variables -> QuestionsDo you recall having seen campaign xy?Which of the following is the bestdescription of what you recall ofthe campaign?Etc.Etc.
  15. 15. 153. Creation– Survey outline can be a guide in creatingstructure of final report (think about it at thisstage!)– This helps to avoid “survey creep”
  16. 16. 163. CreationWhat are the variables we are often trying tomeasure in communications?– Awareness / Knowledge– Understanding– Opinion / attitude– Behavior
  17. 17. 173. CreationWriting questions– Some questions are easy to create (such asgender, length of employment, etc.)– But questions that ask for information ormeasure attitudes or behaviour are moredifficult to create
  18. 18. 183. CreationWriting questions- Different question types are suited to differenttypes of information sought, for example:- Likert scale is suited to measuring attitudes- A yes/no question is suited to measuring if aprecise action has been taken
  19. 19. 193. CreationTypes of questions- Open questions- Closed questions:- Likert scale- Rank order scale- Multiple choice (singular response)- Multiple choice (plural response)- Binary (e.g. yes/no)- Numerical response
  20. 20. 203. CreationTypes of questions: Open questions- On all surveys its good to have at least oneopen-ended question- Open-ended questions are often useful forprobing the “why” and the “how”- Limit the number of open-ended questionsyou include – as analysis of results takestime!
  21. 21. 213. CreationAre there areas of weakness in your communicationsprogramme that are hindering your performance?Please list problems that have arisen in the managementof your programme:Please provide details of projects in your organisation:Do you have comments or feedback?
  22. 22. 223. CreationIn your view, what are the two greatest weaknesses ofyour communications programme?In the past 12 months, what have been the mainchallenges in the management of your programmePlease provide short descriptions (1 paragraphmaximum) of major projects currently underwayin your organisation:This questionnaire has been about campaignmanagement. Do you have any further comments onthis subject?
  23. 23. 233. CreationQuestions: Likert scaleMost commonly used scale to measureattitudes and opinions:“The most important aspect of internal communications is accuracy”o Strongly agreeo Agreeo Neither agree nor disagreeo Disagreeo Strongly disagree
  24. 24. 243. CreationQuestions: Likert scale– Create points on a scale that cover all dimensions– Consider using scales already developed and used inother questionnaires– Scales should have maximum seven points– Labels (“good”, “satisfied”) need to be as clear aspossible– The use of a middle (neutral) point is debatable– Begin with negative points (left) and move to positivepoints (right)
  25. 25. 253. CreationHow would you rate your agency’s website?oPoor oGood oVery good oExcellentHow often do you use your mobile phone?oNever oHardly ever oSometimes, oAlwaysThe current mobility policy is suitable for meoStrongly disagree oDisagree oNeutral oAgree oStrongly agree
  26. 26. 263. CreationHow would you rate your agency’s website?oTerrible oPoor oGood oExcellentIn an average working day, how often do you use yourmobile phone?o0 times ounder 5 times o6-10 times o11-15 times o16-20 times, omore than 20 timesThe current mobility policy is suitable for meoStrongly disagree oDisagree oAgree oStrongly agree No opinion
  27. 27. 273. CreationLikert scale & attitudes– Questions often go beyond asking about anattitude as a whole– But ask about concrete instances of anattitude in a series of questionsExample: job satisfaction (see over)
  28. 28. 283. Creation“Are you satisfied with your job?”o Yes o No o Don’t know“Please indicate your level of satisfaction with thefollowing aspects of your job”:VerydissatisfiedDissatisfied Neithersatisfied nordissatisfiedSatisfied Very satisfiedcurrent salary o o o o ocurrent benefits o o o o orelations withcolleagueso o o o orelations withsupervisorso o o o o
  29. 29. 293. CreationRank order scale questions– These questions ask respondents to rank orstate their priority on items– Useful for learning of what is of value forrespondents and what are their priorities
  30. 30. 303. CreationWhich of the following features would you like for yourorganisation’s website?e-newsletter[] recruitment tool[] flash modules[] video modules[] page print functionOut of the following features, please select the top threeyou would like for your organisation’s website:(select number 1 for this highest and number 3 for the lowest)[0]e-newsletter[0] recruitment tool[0] flash modules[0] video modules[0] page print function
  31. 31. 313. CreationMultiple choice (singular) questions– These questions ask respondents to chooseone item on a list– Are used to check knowledge, collectinformation & demographic data, assesspriorities– In online questionnaires, long lists aretypically put in a pull-down menu (e.g. “what isyour country of residence?”)
  32. 32. 323. CreationWhat is your current work?o Administrativeo Managemento Staffo TechnicalWhich of the following best describes you current work?o Senior management (director level)o Professional staffo Support staffo Other (please specify __________)
  33. 33. 333. CreationMultiple choice (plural)questions– These questions ask respondents to choosemultiple items on a list– Useful in providing a prepared category list forrespondents– Are used to check knowledge, collectinformation & demographic data
  34. 34. 343. CreationWhat is your prior major field of working experience?o Politicalo Diplomatico Militaryo Developmento HumanitarianWhat is your prior major field of working experience?[] Political[] Diplomatic[] Military[] Development[] Humanitarian[] Other, (please specify_________)
  35. 35. 353. CreationBinary (yes/no)questions– These questions ask respondents to chooseone of two responses– Should have only one thought / action inmind. e.g. « Have you graduated fromuniversity? »– Often misused where Likert scale should beused (as yes/no may not represent alldimensions of a response)
  36. 36. 363. CreationShould the UN cut its communications and fundraisingbudget?o Yeso NoShould the UN substantially reduce its communicationsbudget?o Yeso No(consider adding “don’t know”)
  37. 37. 373. CreationNumerical questions– These questions ask numerical data fromrespondents– Often used to collect budget and demographicdata– Must be carefully thought out as they oftenare sensitive issues
  38. 38. 383. CreationHow may children do you have?O1-3 O3-6 O6-10What is your age?[18] – pull down list with numbers 0-100What was your annual income last year?$_________
  39. 39. 393. CreationHow may children do you have?O None O1 O2 O3 O4 O5 O6 O7 O8 O9 O10 O More than 10What is your age?ounder 18 o19- 30 o31-40 o41-50 o51-60 o61-70 oover 70(or use “what year were you born in?)In which category does your annual income of last yearfit best?oBelow $40,000 oBetween $40,001 & $50,000 oBetween $50,001 & $60,000oBetween $70,001 & $80,000 oAbove $80,000
  40. 40. 403. CreationFormulating questions – Hints– Only ask for one piece of information per question– Avoid overly complex words/sentences– Avoid use of undefined terms, e.g. “year”,“technology”, “management”– Be consistent in the use of words and terms– Ensure questions are answerable– Avoid biased questions,e.g. “Do you still have confidence in your topmanagement?”
  41. 41. 413. CreationFormulating questions – Hints– Avoid unreasonable recall periode.g. “Please provide budget details for the past fiveyears”– Make questions as specific and concrete as possible– Don’t ask questions which require information or alevel of skill which respondents may not havee.g. “What is the percentage increase in your ITbudget from 2007 to 2008?”.
  42. 42. 423. CreationFormulating questions – Hints– Avoid response options not matching the question– Avoid response options that are not overlapping ormix “levels”– Avoid response options that are not exhaustive
  43. 43. 433. CreationSurvey componentsAside from the questions other components must beprepared:– Instructions– Transition texts (if required)– Thank you text– Invitation text
  44. 44. 443. CreationSurvey components - instructions– Instructions appear on the first page of the survey– Instructions duplicate some information from theinvitation text as invitation letter/email may getseparated from the survey– Instructions should specify if any special preparationor documents are required before starting the survey
  45. 45. 453. CreationSurvey components - instructionsExample (short form):This questionnaire is part of a review of the communications unitbeing conducted by the Evaluation Department. Your feedbackon this issue would be highly appreciated. This questionnaire willtake some 10 minutes to complete. All feedback provided isanonymous and will be treated confidentially. Please completethe survey by 1 October 2013. If you have any comments orquestions, please contact Ms Marie Rose (mrose@unod.ch)
  46. 46. 463. CreationSurvey components – transition text– Transition texts are used in longer questionnaires wheresome explanation is required for respondents whenmoving from subject to subject:Example:The next ten questions will ask if you agree or disagree withdifferent UN Human Resources policies
  47. 47. 473. CreationSurvey components – thank you text– Thank you text appears at the end of the questionnaire.In online surveys, it appears on a final page.Example:Thank you for your feedback and comments on the ABCmanagement structure. If you have any further comments orquestions on this project, please contact Ms Marie Rose(mrose@unod.ch).
  48. 48. 483. CreationSurvey components – invitation text– Invitation texts are important as they play a key role indetermining if respondents complete the survey– Invitation texts need to include the following information:• Explain why you want an input• Tell respondents who you are• Tell people how long it will take• Reassure respondents about their privacy• Keep it shortFor online surveys:• Take care with “from”, “to” and “subject” fields• Make sure the link works
  49. 49. 493. CreationSurvey components – invitation textsExample (online survey):From: mrose@unod.chTo: glenn.oneil@gmail.comSubject: Communication Unit view - Seeking your feedbackDear colleague,The Evaluation Department is currently undertaking a review of the Communication Unit.We would very much appreciate your feedback on this subject by completing a brief onlinesurvey. This survey will take some 10 minutes to complete. All replies are anonymous and will betreated confidentially. Please complete the survey by 1 October 2013.To complete the survey, please click here >>If this link does not work, please copy and paste the following link into your internet window:http://www.surveymonkey.com/survey123Thank you in advance; your feedback is very valuable to us.Kind regards,Marie RoseResearch OfficerEvaluation Departmentemail: mrose@unod.chtel: ++ 41 22 917 000****
  50. 50. 503. CreationPutting the survey together– Start with easy to answer questions– Broad to the narrow– Non-sensitive to the sensitive– Demographic questions always last– Group subjects together– Questions should be in a logical order– Ensure routing (branching) exists to skip not relevantquestions– Avoid “response set”
  51. 51. 513. CreationPutting the survey togetherWhich question should come first?1) How important or not is the intranet to your work?2) What are the important tools you use daily for yourwork?
  52. 52. 523. CreationPutting the survey togetherAvoiding “response set”Staffing policies are fair and applied equallyoStrongly disagree oDisagree oAgree oStrongly agreePeople are treated differently under Staffing policiesoStrongly disagree oDisagree oAgree oStrongly agree
  53. 53. 534. ManagementGetting ready to send the survey…– Once you have created the questions and placedthem in an appropriate order you are now ready tosend it out…– But wait! Test it first!– You may not have access to the intendedrespondents but test the survey at least withcolleagues– For online surveys, important to test if the surveyfunctions as planned
  54. 54. 544. ManagementTesting survey - Issues– How long did it take to complete it?– Are the questions understandable?– Did the survey flow comfortably?– Do the questions obtain the information needed?– Are new questions needed?– Are their questions that can be removed?– Is the “don’t know” / “no opinion” being used a lot?– Should we try different methods of data collection?
  55. 55. 554. ManagementPrint verses online surveysPrint (respondents receive a word document, andrespond by completing it and posting/emailing back)• Respondents can answer questions in order they like• Respondents can see length of survey• Typically has low response ratesOnline (respondents receive a link by email)• In most cases, respondents must complete survey in onesitting• Response rates can be monitored in real time• Issues of data confidentiality
  56. 56. 564. ManagementPrint verses online survey– The choice depends also on the context– For small groups that can be physically handed thesurvey consider print– For large groups dispersed globally consider onlineOther options: telephone (CATI) surveys,face-to-face surveys, mailed surveys
  57. 57. 574. ManagementLaunching & promoting surveysOnce your survey has been tested and you havemade the appropriate adjustments, you are nowready to launch it. Have you done the following:• Reviewed the timeline of the project?• Identified the correct target audience & arranged access tothem?• Ensured that all relevant stakeholders are briefed on theproject?• Planned some time for follow-up and monitoring?• For an online survey, emptied the database of test results?
  58. 58. 584. ManagementLaunching & promoting surveysThe question:How can we get the necessary number of people tocomplete the survey in order to make the project validand credible?
  59. 59. 594. ManagementLaunching & promoting surveysSome advice to increase responses:• Ensure your survey is short, logical and answerable• Explain incentive for completing survey(e.g. access to results)• Ensure that you have the “buy-in” from key stakeholders• Ensure that your invitation covers all points mentioned above• Give people sufficient time to respond (minimum 3 weeks)• Spend some time encouraging key relays with audiences• Plan to send out at least one reminder – but no more thanthree
  60. 60. 604. ManagementLaunching & promoting surveysDuring the period that respondents arecompleting the survey:• Check regularly the number of responses received• Ensure that all sub-audiences are represented• Send out reminders based on response rate• Be available to respond to any queries• Correct any spelling / numbering errors but do notfundamentally change the survey once launched
  61. 61. 614. ManagementA word about response rates & sampling– In communications, we often are surveying precisegroups (e.g. staff, stakeholders, clients)– With these groups (often small in number) themaximum amount of people are desired to complete asurvey– Although not 100% applicable, sampling theory canoffer some guidanceBut how many responses will make a surveycredible?
  62. 62. 624. ManagementA word about response rates & sampling– When dealing with groups under 100 try and get asmany responses as possible– Divide your audiences into sub-audiences and ensureresults are representative of all sub-audiences– Small response rate will be more credible ifcomplemented by other research (e.g. interviews)
  63. 63. 635. AnalysisBasics of analysing survey data– What do the survey results show?– Looking for totals, averages, trends, patterns,differences, relationships and changes– Different type of analysis required for quantitative(closed questions) and qualitative (open questions)data– Both qualitative and quantitative results provideevidence to support conclusions and recommendations
  64. 64. 645. AnalysisBasics of analysing survey dataTwo main types of analysis methods for quantitative data:– Descriptive statistics: most commonly used, includingcounts, proportions (%), measures of central tendencies(mean/mode/median) and variation (range/standarddeviation), cross tabulation (comparison betweendifferent audiences)– Correlations & regressions: used to showrelationships and predict outcomes and futurerelationshipsIn communication evaluation, descriptivestatistics are usually sufficient
  65. 65. 655. AnalysisBasics of analysing survey dataDescriptive statistics can be displayed in different waysComparisonRanking/prioritiesProportions
  66. 66. 665. AnalysisBasics of analysing survey dataDescriptive statistics can be displayed in different waysTrends / changesRelationships
  67. 67. 675. AnalysisBasics of analysing survey dataTallies/frequency counts: how many people fit into acategory – often displayed as percentage of a total:Shows onlypercentageBetter to showpercentage andactual number
  68. 68. 685. AnalysisBasics of analysing survey dataCross tabulation: Basic data split to show whichdemographic groups make up categories:Interestedto see whosaid “No”!Graph showsfrom whichcategories arethe “No”
  69. 69. 695. AnalysisBasics of analysing survey dataCross tabulation
  70. 70. 705. AnalysisBasics of analysing survey dataPutting responses together: Certain question responses(but not all) can be put together to provide a “total”score, e.g. “staff satisfaction”Responsesto thesequestioncan beadded upVerydissatisfiedDissatisfied Neithersatisfied nordissatisfiedSatisfied Verysatisfiedcurrent salary 4 1 2 2 1current benefits 3 4 1 1 1relations withcolleagues3 2 3 1 1relations withsupervisors1 1 2 3 3Overallsatisfactionscore(11) 27.5 % (8) 20 % (8) 20% (7) 17.5% (6) 15% Summatedtotal scalegenerated
  71. 71. 715. AnalysisBasics of analysing questionnaire dataHints on evaluating quantitative data– Simple counts, proportions and comparisons are sufficient “evidence”for most communication evaluation– Numbers and percentages must be reported in context, e.g:-90% satisfaction is impressive but less so when it’s 9 out of 10 people.-1,000 people say “yes” is impressive but less so when we know totalpopulation is 10,000.– Be cautious in making conclusions based on small samples– Qualitative data stronger when combined with quantitative data– More sophisticated statistical analysis can be made to checkcorrelation, relationships and statistical significance(chi square , t-test)
  72. 72. 725. AnalysisBasics of analysing survey dataEvaluating qualitative data– For survey, qualitative data is collected through open-ended questions– Often such data is consulted but not analysed– Data should be analysed using simplified content analysis
  73. 73. 735. AnalysisBasics of analysing survey dataSteps for evaluating qualitative data– 1. Read through the responses.– 2. Create response categories.– 3. Label each comment with one or several categories.– 4. Look at what you have.– 5. Think what are the responses about?– 6. Identifying the patterns and trends.
  74. 74. 745. AnalysisReporting and using results– Now you have made some analysis of the datayou are ready to write a report– How can you turn the data into findings andultimately recommendations?
  75. 75. 755. AnalysisReporting and using resultsHints on going from data to a report– Return to the original research question asked – how didyou split this up into variables (or topics) and then intoquestions?– What does the quantitative and qualitative data show youabout each topic? What are the conclusions you canreach?– You need to show a series of “evidence” that support yourconclusions – showing solid “proof” is rarely possible!
  76. 76. 765. AnalysisReporting and using resultsBest practices for report writing– Quantitative data is best represented in graphs – but fordata with small number of categories (e.g. yes/no) it can beentered into the text directly– Direct quotes from interviews and open questions can bepowerful testimonies in a report– Try and avoid just reporting survey results “as is” – youneed to interpret it for readers
  77. 77. 776. Conclusion– Some say that survey writing is more of an art than ascience– Always look towards existing, well-established surveys forexamples of question types and scales used– For further information and resources, consult my blog(search for “surveys”):http://intelligentmeasurement.wordpress.com
  78. 78. 78Contact detailsoneil@owlre.comglennoneilwww.owlre.com@glenn_oneilwww.intelligentmeasurement.wordpress.comContacts:

×