How people talk about us behind our backs

How
people
talk about
us behind
our backs
Annie Pettit, Chief Research Officer
Tweet! #WAPOR @LoveStats

Agenda
•Are we being
good citizens?
•How are people
reacting to us?
•Can we fix it?

Method
• Social Media Data
– Hypothesis generation
– N>200 000
– Global English
• Survey Data
– Hypothesis confirmation and precision
– N=400
– Non-probability incentivized panel, Sampled and
Weighted to US Census Rep  Not because
the target group is census rep but because
we’re familiar with census rep

Survey Instructions
•We are really interested in your honest answers.
Don’t worry about hurting our feelings or
speaking a scary truth. You will not be penalized
for sharing any of your opinions. We need to
hear them!
• (Please be honest. We will not penalize your
account.)
• (Please don't be scared to tell the truth. We will
not penalize your account.)
• (Please, please, tell us the truth. :)

Trust

Who to trust?
• Do ya think these paid survey thing is a scam?
• @aaa took an online survey once. Seemed legit
• i just took a survey.... is it a scam or is it just me.
• hey all pls confirm me is this trust worthy
• @bbb surveys are scams usually
• @ccc gallop survey totally fraud
• is opinion outpost trust worthy?
• I dont trust ipsos synovate..at all.

It’s not spam if…
• Harder to convince: Poor responders, Men, Older people
• Younger people trust their friends, social media comments

Remind People
•You’re not selling anything
– Particularly on purchase intent questions
– “We’re not asking you to buy something. We’re just curious if you have any
plans.”
•You don’t want their credit card info
•You don’t need them to download
anything
•Your registration, certification
credentials

Survey Length

Survey Length
• this a long survey man smh
• That's one long ass survey
• wow that was a long survey
• And another long time bump!
• This survey is sooo long >.<
• Taking this long ass survey.
• This Boring Ass Long Survey..
• Gotta do this long ass survey
• This survey is to damn long ..
• Thisss SURVEY long Ass Hell Man !
• Damnn that survey was long as shit
• Omg that survey thing was long af'
• Damnn that survey was long as shit
• online apps with long ass surveys<<
• Taking so long to do this survey ugh
• I got to do these 2 long ass surveys
• Filling out this long ass survey. <<
• Taking so long to do this survey ugh
• Mothafn' survey was long as shit.
• This survey is really really long lol
• Filled out that long ass survey.... :/
• This survey is gonna take a long while
• Dang I hate doing them long ahh surveys

It’ll Only Take Five Minutes
• SSI survey about mobile devices. The survey says
1 mins, but it will take much longer than that. Also,
survey is from "qualtrics", so don't speeding
through the survey or else you won't get credit.
btw, another purple 2 cents ad is up
• I got an email survey that said it would take 1 min
so I did it. It actually took about 20 - they made me
watch a stupid video about Audi and Le Mans.
• hate surveys that say "this will only take 2
minutes" and end up taking 15 because i like to
write detailed feedback

12 Minute Surveys
•Shorter surveys preferred by
women, older people

Lying

You Should Lie
• Oh my god, I made a male account & started seeing
shitloads of surveys compared shoulda done that
sooner :/
• Just lie on the surveys. Pretend you are a middle aged
Hispanic woman. I made 18,000 in three months.
• Oh hello telephone survey people, why yes my name
is monty and im a 28yr old commercial airline pilot with
a phd. Lol
• I love to answer polls, but I tend to lie on my
responses.
• Do I lie on this survey or do I be truthful? I dont wanna
incriminate my self...

27% Confess to Lying*
*Not confessing does not equal innocence

I Lied Because…
Answer
wasn’t
there
Trying
to
qualify
In a
rush
Tired Having
fun
Random
ly
clicking
Upset
with
hosting
Don’t
know

I Confess Also…
• Partials: Do not read, Speed, Distracted, Answer on behalf
• Poor responders: Don’t admit distraction, Admit to random
clicking

Always give an out
•None of the above
•Never
•Less than X
•More than Y
•Other
•Please specify: _____________

Transparency

Why demographics?
•Everytime a survey is done they
always ask if ure hispanic like why
does it fn matter! #whooocares
•Why do surveys ask your race and
include the option "other"? What
the f could other be? "Me? I'm
actually a bellydancing cactus."

Why Screening?
•Survey just asked me where am I
living, and screened me out
immediately when I said East
Midlands
•I was doing a survey for the trailer and
it looks boring AF. I gave my honest
opinion and because it was all
negative they screened me out.

Screen Too Late
•Just got 10 mins through a 15 min survey
only to be disqualified.... Ggrrrrrrr
•Got a survey on films, DON'T COMPLETE
IT! I got to end and screened me out,
losing 20 mins of time Survey# 266616
•Fed up of doing 15 minutes of a survey
then getting screened out, why can't they
ask the relevant questions at the
beginning

I don’t understand you
•Partials do NOT understand
•Poor responders do NOT understand

Demographics
•Useless
– “To classify responses”
– “To categorize the data”
•Useful
– “For example, to understand how men
and women, or older and younger
people differ
– “To ensure we listen to many different
types of people”

Screening
•Screen people out ASAP
•Take the extra minute and tell people why
– Those are all the questions we have
for you today.
–We are sorry to screen you out but we
have already heard from a number of
people with your characteristics and
need to make sure we also listen to
people with other characteristics

Screening
•Let people know they may be
screened because they DO use or
product or because they do NOT use
a product
•Enforce recruitment and invitation
caps so that certain groups don’t get
far more surveys than other groups
–You know that women get fewer
invites so recruit less of them

Incentives Pay for Boredom

They EARN Incentives
• ugh @ this boring ass survey.
• i hate filling these surveys in..boring
• Focus group today good money but so boringggg
• swagbucks is mindnumbingly boring got me 9 bucks lol
• Done with that Youth Risk Behavior Survey -_- shit boring!!
• online surveys are intensely boring but I want the Paypal
points...
• Just participated in a focus group. Boring but LOVE the
incentives! $$
• Tonight was the most boring 'live' market research I have
ever done!!!!
• UGh this survey is so boring but I'm making ten dollars an
hour so its kind of nice

Topics
Love
Hate

Money Ranks Highest
9
7
5
3
Bottom
Rank
•Partials rank every incentive
higher
1
Top
Rank

Don’t Be Boring
•Loosen up your grammar
– “What is the best idea you can think of?”
•Use more casual language
– “Tell me why you like it.”
•Try a light joke or two
– “May the survey force be with you”

Responder Thoughts
• I absolutely loved this survey! Thank you for wanting my
opinion. :)
• I have enjoyed taking this survey, because it tells me that
some company actually cares about improving its surveys.
• I have to admit I was wondering if any company, was or
would ever pay attention to panelists concerns usually it
always about profit declared or not, but it's true.. But it's
comforting and showing some good faith to me, being
considered and that this company has seemed to notice
these concerns need to be addressed.. Thank you.
• I'm glad to see that you might have some concerns about
people not being completely truthful with their answers. I
always answer the questions truthfully. However,
sometimes the questions are asked in such a manner that
none of the above really apply.

Conclusions
•We think we are being
transparent but we are NOT
•We can be honest without
betraying the method
•Must write better
•Must not interpret data as
perfect

Thank you!
Annie Pettit
Chief Research Officer
Peanut Labs
annie@peanutlabs.com
@LoveStats on Twitter
www.peanutlabs.com
: )

How people talk about us behind our backs

Recommended

Recommended

More Related Content

What's hot

What's hot (10)

Similar to How people talk about us behind our backs

Similar to How people talk about us behind our backs (20)

More from Annie Pettit, Research Methodologist

More from Annie Pettit, Research Methodologist (20)

Recently uploaded

Recently uploaded (13)

How people talk about us behind our backs