MSPA-APQuestionnaires and GuidelinesFeb 27, 20131Thursday, 30 May 13
The heart of Mystery ShoppingAsking the rightquestions.2Thursday, 30 May 13
WEBINAR• Approximately 50 registrations• New initiative from the MSPA Board• Complement face-to-face Workshops• Different ...
PRESENTER• Steven Di Pietro• Executive Director Service IntegrityMystery Shopping• Global and AP Board roles4Thursday, 30 ...
WEBINAR HOUSE RULES• Everyone is on Mute• You can hold up your hand• Or type written questions5Thursday, 30 May 13
AGENDA• The core features of a great questionnaire• The 12 17 deadly questionnaire mistakes• Building good guidelines6Thur...
EXPERTS• You’re all experts, maybe there’s onehint, message or question you hadn’tconsidered, or• Something you disagree w...
QUESTIONNAIRES• Core features8Thursday, 30 May 13
HAVE A GOAL• Too many questionnaires are aimless.  Just saying “Let’s find outwhat the customer thinks” is insufficient.  As...
WHO IS IT AIMED AT?• Senior Managers? Store Staff?Regional Managers?• Drives the detail and frequency.10Thursday, 30 May 13
ACKNOWLEDGETHE ART• Creating questionnaires is an art.See section 211Thursday, 30 May 13
RELEVANCE• Only ask what is relevant. Not because aclient thinks they can jam it in.• > Accuracy12Thursday, 30 May 13
DYNAMIC• Balance trends with collecting relevance.• If the store had clear signage, it won’t change by the nextround.13Thu...
CONTROLLABLE ANDACTIONABLE• Store reporting should be onlyfor actionable items.• Un-actionable items should notbe reported...
MEANINGFUL SCORES• Does the overall score make sense?• You can say our satisfaction has increased 2% and peopleknow what i...
MEASURETHE IMPORTANT• Measure things important to the customer.• Survey customers.• E.g. Don’t ask about security guard un...
UNREALISTIC BENCHMARKS• Moving from 80% to 90%, then 90% to92%.• Enough is enough.17Thursday, 30 May 13
FOCUS ONTHE SYSTEM, NOTTHE INDIVIDUAL• Focus on the store system, or networksystem.• Not the person.18Thursday, 30 May 13
THETREND ISYOUR FRIEND• For restricted budgets with lowfrequencies.• Aim for less questions, with higherfrequency.19Thursd...
SKIP LOGIC• If Q3 = N then go to Q 15• Reduces queries. Faster for shoppers.20Thursday, 30 May 13
12 17TRAPS21Thursday, 30 May 13
1.ASKTHE QUESTION ONCE• Too many questions have repetition• e.g. asking about satisfaction after every sectionof the custo...
2.ASK ONLY ONE QUESTION• Avoid joining statements like ‘and’.• Was the staff member wearing the uniform and was it clean?2...
3. SMALL SCALES• Not 1-10• Use 1-5, or 1-4• Better still 1 or 0...Yes/No• Yes/No questions are the sign of a good question...
4.AVOID NEGATIVE• e.g Was the staff member untidy?• Difficult to score• Confuse shoppers, and staff• They equate a tick wit...
5.AVOID LEADINGQUESTIONS• Don’t set up the answer in thequestion.• We always try to ensure staff are wellpresented, was th...
6. REMOVE AMBIGUITY• Be precise of positive attributes.• Describe “clean”.• Describe “Posters in goodcondition?”• Was the ...
7. NO ENTRAPMENT• Don’t set staff up for failure.• Realistic scenarios.• Don’t ask shoppers to try tricking staff.28Thursd...
8. BALANCE FREQUENCYWITH REPORTING• Infrequent shops e.g. 6 monthly shouldbe used only for System widereporting. Don’t eve...
9. LESS IS MORE• Don’t ask too many questions.• 30-35 questions for memorised shops.• 3 physical attributes (e.g. 1 name, ...
10. USE NORMAL SCENARIOS• Scenarios should match normal consumer behaviour because:• otherwise shopper can be exposed,• ot...
11. STAY INTHE EXPERTISE• Shoppers should focus on the mystery shop, not giving retailadvice.• Would you employ this serve...
12.AVOID MEASUREMENTS -HUH?• Measurements overload shoppers.• Usually the information can beobtained from business system....
13. NO SCOREKEEPING• Don’t let shoppers score or evensee the scoring.• Creates inconsistencies and bias.34Thursday, 30 May...
14. BAN N/A• Not Applicable (N/A) answers are a signof a poorly constructed question.• And are an easy out for the shopper...
15. USETHE ADDRESS LISTING• Use the address listing to answer questions• Use the site listing to answer questions.• Should...
16. REMOVE SIMILARANSWERS• Answers should be discrete and different.• Was the staff member a) Polite b)Friendly, c)Warm?37...
17. REDUCE FREETEXT• Temptation (especially for clients) is to have free text aftereach question.• Creates inconsistencies...
SHOPPER GUIDELINES39Thursday, 30 May 13
LENGTH OF GUIDELINE• Long guidelines = poor questionnaire.40Thursday, 30 May 13
GUIDELINE OVERLOAD• Guidelines against each question = poorquestionnaire41Thursday, 30 May 13
UNNECESSARY GUIDELINES• Self explanatory questionsdon’t need a guideline.• Important guidelines losefocus.• e.g. - this is...
CONSISTENT WITHQUESTIONNAIRE• Obvious but some questionnaires areinconsistent with the guidelines.• e.g. Guidelines say ta...
LENGTH OF GUIDELINES• Shoppers cannot be expected to read40 pages of guidelines when they arebeing paid $15 for the wholea...
The heart of Mystery ShoppingAsking the rightquestions.45Thursday, 30 May 13
NEXT• Survey• Contact me directly• sdipietro@serviceintegrity.com.au46Thursday, 30 May 13
Upcoming SlideShare
Loading in …5
×

MSPA AP Webinar 1 Questionnaires and Guidelines

809 views

Published on

The first in the Webinar Series for Mystery Shopping Providers Association Members.

This Webinar focuses on how to create efficient questionnaires and great guidelines. We also share some of the pitfalls to avoid.

Published in: Business, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
809
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

MSPA AP Webinar 1 Questionnaires and Guidelines

  1. 1. MSPA-APQuestionnaires and GuidelinesFeb 27, 20131Thursday, 30 May 13
  2. 2. The heart of Mystery ShoppingAsking the rightquestions.2Thursday, 30 May 13
  3. 3. WEBINAR• Approximately 50 registrations• New initiative from the MSPA Board• Complement face-to-face Workshops• Different but similar to the workshops• Will be recorded• All materials belong jointly to the presenter and MSPA andcannot be copied without authorisation.3Thursday, 30 May 13
  4. 4. PRESENTER• Steven Di Pietro• Executive Director Service IntegrityMystery Shopping• Global and AP Board roles4Thursday, 30 May 13
  5. 5. WEBINAR HOUSE RULES• Everyone is on Mute• You can hold up your hand• Or type written questions5Thursday, 30 May 13
  6. 6. AGENDA• The core features of a great questionnaire• The 12 17 deadly questionnaire mistakes• Building good guidelines6Thursday, 30 May 13
  7. 7. EXPERTS• You’re all experts, maybe there’s onehint, message or question you hadn’tconsidered, or• Something you disagree with.7Thursday, 30 May 13
  8. 8. QUESTIONNAIRES• Core features8Thursday, 30 May 13
  9. 9. HAVE A GOAL• Too many questionnaires are aimless.  Just saying “Let’s find outwhat the customer thinks” is insufficient.  Ask yourself thefollowing questions:• What will change once the results are received?• Why am I collecting the data. For example – to get a broad gripof service, to improve contact centre service, for marketing, forretention, for staff KPI’s, to drive home strategies? etc.9Thursday, 30 May 13
  10. 10. WHO IS IT AIMED AT?• Senior Managers? Store Staff?Regional Managers?• Drives the detail and frequency.10Thursday, 30 May 13
  11. 11. ACKNOWLEDGETHE ART• Creating questionnaires is an art.See section 211Thursday, 30 May 13
  12. 12. RELEVANCE• Only ask what is relevant. Not because aclient thinks they can jam it in.• > Accuracy12Thursday, 30 May 13
  13. 13. DYNAMIC• Balance trends with collecting relevance.• If the store had clear signage, it won’t change by the nextround.13Thursday, 30 May 13
  14. 14. CONTROLLABLE ANDACTIONABLE• Store reporting should be onlyfor actionable items.• Un-actionable items should notbe reported at store level.14Thursday, 30 May 13
  15. 15. MEANINGFUL SCORES• Does the overall score make sense?• You can say our satisfaction has increased 2% and peopleknow what it means.  You cannot say the combined Service,Complaints and Efficiency target has increased 2%, because itmeans nothing.  If they must be combined, consider using anindex instead of a percentage.15Thursday, 30 May 13
  16. 16. MEASURETHE IMPORTANT• Measure things important to the customer.• Survey customers.• E.g. Don’t ask about security guard uniforms if they are notimportant for the scenario.16Thursday, 30 May 13
  17. 17. UNREALISTIC BENCHMARKS• Moving from 80% to 90%, then 90% to92%.• Enough is enough.17Thursday, 30 May 13
  18. 18. FOCUS ONTHE SYSTEM, NOTTHE INDIVIDUAL• Focus on the store system, or networksystem.• Not the person.18Thursday, 30 May 13
  19. 19. THETREND ISYOUR FRIEND• For restricted budgets with lowfrequencies.• Aim for less questions, with higherfrequency.19Thursday, 30 May 13
  20. 20. SKIP LOGIC• If Q3 = N then go to Q 15• Reduces queries. Faster for shoppers.20Thursday, 30 May 13
  21. 21. 12 17TRAPS21Thursday, 30 May 13
  22. 22. 1.ASKTHE QUESTION ONCE• Too many questions have repetition• e.g. asking about satisfaction after every sectionof the customer interaction22Thursday, 30 May 13
  23. 23. 2.ASK ONLY ONE QUESTION• Avoid joining statements like ‘and’.• Was the staff member wearing the uniform and was it clean?23Thursday, 30 May 13
  24. 24. 3. SMALL SCALES• Not 1-10• Use 1-5, or 1-4• Better still 1 or 0...Yes/No• Yes/No questions are the sign of a good question24Thursday, 30 May 13
  25. 25. 4.AVOID NEGATIVE• e.g Was the staff member untidy?• Difficult to score• Confuse shoppers, and staff• They equate a tick with doing a good job.25Thursday, 30 May 13
  26. 26. 5.AVOID LEADINGQUESTIONS• Don’t set up the answer in thequestion.• We always try to ensure staff are wellpresented, was the staff memberwearing the full uniform?• Leads to shopper sympathy and bias.26Thursday, 30 May 13
  27. 27. 6. REMOVE AMBIGUITY• Be precise of positive attributes.• Describe “clean”.• Describe “Posters in goodcondition?”• Was the ticketing clear? Referring tofont? Size? Position? Consistent?Present?27Thursday, 30 May 13
  28. 28. 7. NO ENTRAPMENT• Don’t set staff up for failure.• Realistic scenarios.• Don’t ask shoppers to try tricking staff.28Thursday, 30 May 13
  29. 29. 8. BALANCE FREQUENCYWITH REPORTING• Infrequent shops e.g. 6 monthly shouldbe used only for System widereporting. Don’t even collect staffnames.• Frequent shops e.g. monthly areappropriate for store level reportingwith staff names.29Thursday, 30 May 13
  30. 30. 9. LESS IS MORE• Don’t ask too many questions.• 30-35 questions for memorised shops.• 3 physical attributes (e.g. 1 name, 1 timemeasurement, 1 physical attribute such asmarketing material)30Thursday, 30 May 13
  31. 31. 10. USE NORMAL SCENARIOS• Scenarios should match normal consumer behaviour because:• otherwise shopper can be exposed,• otherwise store cannot perform,• measures the majority of transactions.31Thursday, 30 May 13
  32. 32. 11. STAY INTHE EXPERTISE• Shoppers should focus on the mystery shop, not giving retailadvice.• Would you employ this server?32Thursday, 30 May 13
  33. 33. 12.AVOID MEASUREMENTS -HUH?• Measurements overload shoppers.• Usually the information can beobtained from business system.33Thursday, 30 May 13
  34. 34. 13. NO SCOREKEEPING• Don’t let shoppers score or evensee the scoring.• Creates inconsistencies and bias.34Thursday, 30 May 13
  35. 35. 14. BAN N/A• Not Applicable (N/A) answers are a signof a poorly constructed question.• And are an easy out for the shopper.35Thursday, 30 May 13
  36. 36. 15. USETHE ADDRESS LISTING• Use the address listing to answer questions• Use the site listing to answer questions.• Should not ask about the condition of the bathroom if thestore doesn’t have a bathroom.36Thursday, 30 May 13
  37. 37. 16. REMOVE SIMILARANSWERS• Answers should be discrete and different.• Was the staff member a) Polite b)Friendly, c)Warm?37Thursday, 30 May 13
  38. 38. 17. REDUCE FREETEXT• Temptation (especially for clients) is to have free text aftereach question.• Creates inconsistencies, queries, longer QC, longer time forshopper, shoppers usually fabricate the story just to fulfil therequirements.38Thursday, 30 May 13
  39. 39. SHOPPER GUIDELINES39Thursday, 30 May 13
  40. 40. LENGTH OF GUIDELINE• Long guidelines = poor questionnaire.40Thursday, 30 May 13
  41. 41. GUIDELINE OVERLOAD• Guidelines against each question = poorquestionnaire41Thursday, 30 May 13
  42. 42. UNNECESSARY GUIDELINES• Self explanatory questionsdon’t need a guideline.• Important guidelines losefocus.• e.g. - this is a cleanbathroom• e.g. - this is an emptygarbage bin42Thursday, 30 May 13
  43. 43. CONSISTENT WITHQUESTIONNAIRE• Obvious but some questionnaires areinconsistent with the guidelines.• e.g. Guidelines say take two photo’s and thesystem questionnaire asks for 1 photo.43Thursday, 30 May 13
  44. 44. LENGTH OF GUIDELINES• Shoppers cannot be expected to read40 pages of guidelines when they arebeing paid $15 for the wholeassignment.44Thursday, 30 May 13
  45. 45. The heart of Mystery ShoppingAsking the rightquestions.45Thursday, 30 May 13
  46. 46. NEXT• Survey• Contact me directly• sdipietro@serviceintegrity.com.au46Thursday, 30 May 13

×