SlideShare a Scribd company logo
1 of 4
Download to read offline
Wisdom of the Crowd versus Pollster
Pollsters are responsible for reporting findings based on a calculated process of gathering
results among a representative sample on a given topic. For the most part, where the
question is not contentious, we can be confident of the result within a particular margin of
error by ensuring the sample is representative and robust. So for example if we ask 1,000
people who have been selected to represent the total population accurately what their
favourite colour is, we can be confident the result will be accurate to within +/- 3.2%.
However what if the question involved reveals what a person may deem to be a socially
‘undesirable’ response? While some people are proud of their stance, and have no issues
about an attitude or belief that perhaps goes against the masses, this is not the case for all
– those who are less inclined to reveal their attitudes are referred to in polling terms as ‘shy
voters’.
It is not to say that ‘shy voters’ lie in their response to interviewers, however they do have
some difficulty in revealing to a stranger their disposition on a particular topic. As a result
they generally suggest they have not yet decided how they will vote, but some may claim
they will vote along the same lines as the masses.
The learning for researchers is that if you find yourselves with a high level of ‘Don’t Know or
Undecided’ respondents to your survey, it should be an immediate red flag that some level
of “shy voting” is at play.
So, here in lies the difficulty; if the very source of truth we rely on cannot be deemed to be
100% accurate how does a pollster overcome this to predict an accurate outcome?
In advance of the recent Marriage referendum all pollsters, including RED C, were reporting
high levels of support for the yes side, however there were concerns that with such a
contentious topic that those who supported the No side would be underrepresented in the
polls, either because they would be far more likely to tell us that they didn’t know how they
would vote or because they simply refused to take part on the poll at all. This was clearly
seen to be the case when we look at the final “standard” polls vs. the result, where the No
side were clearly underrepresented.
Including Undecided
Excluding
Undecided Sample Dates
Yes No Undecided Yes No
RED C 69% 25% 6% 69%* 31%* 1,009 11
th
– 13
th
May
Millward
Brown
53% 24% 23% 69% 31% 994 2
nd
– 15
th
May
Behaviour &
Attitudes
63% 26% 11% 71% 29% 927 1
st
– 11
th
May
Ipsos MRBI 58% 25% 17% 70% 30% 1,200 13
th
– 14
th
May
*(RED C did not re-allocate D/Ks, but suggested they would all vote NO)
Having witnessed the significant effect the ‘shy voter’ had recently on pollsters in the UK,
with actual outcome of UK elections versus poll outcomes quite different it was clear this
group couldn’t be ignored and needed some other technique to give a voice to the shy
voter.
RED C decided to use a technique known as ‘Wisdom of the Crowds’ to uncover the shy
voter. This method asks the respondent to consider all the conversations they may have
had, or heard friends and family having and on this basis to try and predict what they
believe the percentage outcome will be for and against the marriage referendum.
This removes any discomfort from the situation in which a shy voter may find themselves
and instead put the focus on the general population as opposed to one individual. The
result was a resounding success, as it predicted the outcome with 100% accuracy, 62% in
support and 38% against.
This “Wisdom of Crowds” method allowed the shy voter, whether in support of or against
the marriage referendum to lend some level of their own viewpoint, while disguised within
how the ‘masses’ will vote. As such it bridged the gap between standard polling and a
potential issue of the shy voter influencing the results.
So how does this transfer to general insights we gather for commercial clients or regular
political polling? The first thing we have to do is to always be aware of the topics we are
researching and the way that people may respond to them, and always ask are we doing
enough to get behind the topline stock answers we receive. For instance when we ask
people how likely they are to buy a product or service this can often be oversubscribed,
particularly among the genial Irish, who are likely to say they “might” buy something, even
if in reality they wont go on to do so. So we should also always be asking how likely they
think people in the population generally will be to buy that service or product as well, and
where large gaps exist take this into account in our analysis.
Within party support this also applies, and RED C use several techniques over and above
the standard questions to get the most accurate result possible – all parties are read out,
unlikely voters are removed, ‘undecided’ voters re-allocated and finally we look at past vote
behaviour and layer it with the current claimed intention to ensure an accurate sample.
This serves not to alter results, but to allow us report the most accurate results possible.
But maybe even this wont be enough? With the Irish elections looming we may also need
to look further at techniques such as the Wisdom of Crowds on party support to try to give
the ever present shy voter a voice.
It is not to suggest that standard polling is defunct, quite the contrary, it simply serves to
inform us as pollsters that although research can be quite systematic and indeed scientific,
we cannot forget the human in all of us and how best to overcome the nature of the
reluctant respondent.
Ciara Regan is an Associate Director with RED C Research.

More Related Content

Viewers also liked

Keeping Up: Digital Literacy Drop-In Sessions for Undergraduate Students
Keeping Up: Digital Literacy Drop-In Sessions for Undergraduate StudentsKeeping Up: Digital Literacy Drop-In Sessions for Undergraduate Students
Keeping Up: Digital Literacy Drop-In Sessions for Undergraduate StudentsErnesto Hernandez
 
Marketing Identity 2013: UX & UI Dizajn
Marketing Identity 2013: UX & UI DizajnMarketing Identity 2013: UX & UI Dizajn
Marketing Identity 2013: UX & UI DizajnTomáš Minarik
 
Søgning på nettet færdig
Søgning på nettet   færdigSøgning på nettet   færdig
Søgning på nettet færdigKirsten Møller
 
BestBoyfriend.me at Startup Weekend Bratislava
BestBoyfriend.me at Startup Weekend Bratislava BestBoyfriend.me at Startup Weekend Bratislava
BestBoyfriend.me at Startup Weekend Bratislava Tomáš Minarik
 
Agile Architecture Belfast Software Architecture User Group
Agile Architecture   Belfast Software Architecture User GroupAgile Architecture   Belfast Software Architecture User Group
Agile Architecture Belfast Software Architecture User GroupPaul Wallace
 
구글 글래스 + 아이워치가 가져올 UX혁명
구글 글래스 + 아이워치가 가져올 UX혁명구글 글래스 + 아이워치가 가져올 UX혁명
구글 글래스 + 아이워치가 가져올 UX혁명Dongchul Kim
 
적외선 인터페이스 프로젝트 특강
적외선 인터페이스 프로젝트 특강적외선 인터페이스 프로젝트 특강
적외선 인터페이스 프로젝트 특강Dongchul Kim
 
Canada country presentation_presentation
Canada country presentation_presentationCanada country presentation_presentation
Canada country presentation_presentationTeresa Maroto Ramos
 
증강현실 기술동향 및 전망
증강현실 기술동향 및 전망증강현실 기술동향 및 전망
증강현실 기술동향 및 전망Dongchul Kim
 
Project management office
Project management officeProject management office
Project management officedinafouad87
 

Viewers also liked (16)

Søgning på nettet
Søgning på nettet   Søgning på nettet
Søgning på nettet
 
Cual es el destino de tu vida 1,
Cual es el destino de tu vida 1,Cual es el destino de tu vida 1,
Cual es el destino de tu vida 1,
 
Keeping Up: Digital Literacy Drop-In Sessions for Undergraduate Students
Keeping Up: Digital Literacy Drop-In Sessions for Undergraduate StudentsKeeping Up: Digital Literacy Drop-In Sessions for Undergraduate Students
Keeping Up: Digital Literacy Drop-In Sessions for Undergraduate Students
 
Marketing Identity 2013: UX & UI Dizajn
Marketing Identity 2013: UX & UI DizajnMarketing Identity 2013: UX & UI Dizajn
Marketing Identity 2013: UX & UI Dizajn
 
Backupqr
BackupqrBackupqr
Backupqr
 
Els pneumàtics
Els pneumàticsEls pneumàtics
Els pneumàtics
 
Søgning på nettet færdig
Søgning på nettet   færdigSøgning på nettet   færdig
Søgning på nettet færdig
 
BestBoyfriend.me at Startup Weekend Bratislava
BestBoyfriend.me at Startup Weekend Bratislava BestBoyfriend.me at Startup Weekend Bratislava
BestBoyfriend.me at Startup Weekend Bratislava
 
Cual es el destino de tu vida 1
Cual es el destino de tu vida 1Cual es el destino de tu vida 1
Cual es el destino de tu vida 1
 
Agile Architecture Belfast Software Architecture User Group
Agile Architecture   Belfast Software Architecture User GroupAgile Architecture   Belfast Software Architecture User Group
Agile Architecture Belfast Software Architecture User Group
 
구글 글래스 + 아이워치가 가져올 UX혁명
구글 글래스 + 아이워치가 가져올 UX혁명구글 글래스 + 아이워치가 가져올 UX혁명
구글 글래스 + 아이워치가 가져올 UX혁명
 
HCI 기술특강
HCI 기술특강HCI 기술특강
HCI 기술특강
 
적외선 인터페이스 프로젝트 특강
적외선 인터페이스 프로젝트 특강적외선 인터페이스 프로젝트 특강
적외선 인터페이스 프로젝트 특강
 
Canada country presentation_presentation
Canada country presentation_presentationCanada country presentation_presentation
Canada country presentation_presentation
 
증강현실 기술동향 및 전망
증강현실 기술동향 및 전망증강현실 기술동향 및 전망
증강현실 기술동향 및 전망
 
Project management office
Project management officeProject management office
Project management office
 

Similar to Wisdom of the Crowd Vs. The Pollster

Audience research improved
Audience research improvedAudience research improved
Audience research improvedeleanor-dawson
 
The problem of so-called “push polls”When advocacy calls are made .pdf
The problem of so-called “push polls”When advocacy calls are made .pdfThe problem of so-called “push polls”When advocacy calls are made .pdf
The problem of so-called “push polls”When advocacy calls are made .pdfcallawaycorb73779
 
Survey/Audience analysis
Survey/Audience analysisSurvey/Audience analysis
Survey/Audience analysisJamie Kessel
 
Political Campaign Planning1
Political Campaign Planning1Political Campaign Planning1
Political Campaign Planning1garneth87
 
Being Primer on Sampling
Being Primer on Sampling Being Primer on Sampling
Being Primer on Sampling Peanut Labs
 
A Journalist’s Guide to Survey Research and Election Polls by Cliff Zuskin
A Journalist’s Guide to Survey Research and Election Polls by Cliff ZuskinA Journalist’s Guide to Survey Research and Election Polls by Cliff Zuskin
A Journalist’s Guide to Survey Research and Election Polls by Cliff ZuskinFincher Consulting
 
Assignment ExpectationThe Assignment is attachedThe pape.docx
Assignment ExpectationThe Assignment is attachedThe pape.docxAssignment ExpectationThe Assignment is attachedThe pape.docx
Assignment ExpectationThe Assignment is attachedThe pape.docxssuser562afc1
 
Television advertising effects on youth
Television advertising effects on youthTelevision advertising effects on youth
Television advertising effects on youthjay dave
 
Political Campaign Planning parte2
Political Campaign Planning parte2Political Campaign Planning parte2
Political Campaign Planning parte2garneth87
 
The Narrative Project - Overview Deck July 2014
The Narrative Project - Overview Deck July 2014The Narrative Project - Overview Deck July 2014
The Narrative Project - Overview Deck July 2014James North
 
15055_GSG_ForgottenMan_D5_02
15055_GSG_ForgottenMan_D5_0215055_GSG_ForgottenMan_D5_02
15055_GSG_ForgottenMan_D5_02Brittany Hill
 
Government factors that influence election outcomes and voter participation
Government factors that influence election outcomes and voter participationGovernment factors that influence election outcomes and voter participation
Government factors that influence election outcomes and voter participationSuayni Biggs
 

Similar to Wisdom of the Crowd Vs. The Pollster (20)

Audience research
Audience research Audience research
Audience research
 
Audience research improved
Audience research improvedAudience research improved
Audience research improved
 
The problem of so-called “push polls”When advocacy calls are made .pdf
The problem of so-called “push polls”When advocacy calls are made .pdfThe problem of so-called “push polls”When advocacy calls are made .pdf
The problem of so-called “push polls”When advocacy calls are made .pdf
 
Survey/Audience analysis
Survey/Audience analysisSurvey/Audience analysis
Survey/Audience analysis
 
Political Campaign Planning1
Political Campaign Planning1Political Campaign Planning1
Political Campaign Planning1
 
Three Polls
Three PollsThree Polls
Three Polls
 
Being Primer on Sampling
Being Primer on Sampling Being Primer on Sampling
Being Primer on Sampling
 
Alan smith
Alan smithAlan smith
Alan smith
 
A Journalist’s Guide to Survey Research and Election Polls by Cliff Zuskin
A Journalist’s Guide to Survey Research and Election Polls by Cliff ZuskinA Journalist’s Guide to Survey Research and Election Polls by Cliff Zuskin
A Journalist’s Guide to Survey Research and Election Polls by Cliff Zuskin
 
Assignment ExpectationThe Assignment is attachedThe pape.docx
Assignment ExpectationThe Assignment is attachedThe pape.docxAssignment ExpectationThe Assignment is attachedThe pape.docx
Assignment ExpectationThe Assignment is attachedThe pape.docx
 
Television advertising effects on youth
Television advertising effects on youthTelevision advertising effects on youth
Television advertising effects on youth
 
Political Campaign Planning parte2
Political Campaign Planning parte2Political Campaign Planning parte2
Political Campaign Planning parte2
 
Critical approaches (1)
Critical approaches (1)Critical approaches (1)
Critical approaches (1)
 
Alan smith
Alan smithAlan smith
Alan smith
 
The Narrative Project - Overview Deck July 2014
The Narrative Project - Overview Deck July 2014The Narrative Project - Overview Deck July 2014
The Narrative Project - Overview Deck July 2014
 
Surveys
SurveysSurveys
Surveys
 
Voter targeting
Voter targetingVoter targeting
Voter targeting
 
Marketing to Seniors
Marketing to SeniorsMarketing to Seniors
Marketing to Seniors
 
15055_GSG_ForgottenMan_D5_02
15055_GSG_ForgottenMan_D5_0215055_GSG_ForgottenMan_D5_02
15055_GSG_ForgottenMan_D5_02
 
Government factors that influence election outcomes and voter participation
Government factors that influence election outcomes and voter participationGovernment factors that influence election outcomes and voter participation
Government factors that influence election outcomes and voter participation
 

Wisdom of the Crowd Vs. The Pollster

  • 1. Wisdom of the Crowd versus Pollster Pollsters are responsible for reporting findings based on a calculated process of gathering results among a representative sample on a given topic. For the most part, where the question is not contentious, we can be confident of the result within a particular margin of error by ensuring the sample is representative and robust. So for example if we ask 1,000 people who have been selected to represent the total population accurately what their favourite colour is, we can be confident the result will be accurate to within +/- 3.2%. However what if the question involved reveals what a person may deem to be a socially ‘undesirable’ response? While some people are proud of their stance, and have no issues about an attitude or belief that perhaps goes against the masses, this is not the case for all – those who are less inclined to reveal their attitudes are referred to in polling terms as ‘shy voters’. It is not to say that ‘shy voters’ lie in their response to interviewers, however they do have some difficulty in revealing to a stranger their disposition on a particular topic. As a result they generally suggest they have not yet decided how they will vote, but some may claim they will vote along the same lines as the masses. The learning for researchers is that if you find yourselves with a high level of ‘Don’t Know or Undecided’ respondents to your survey, it should be an immediate red flag that some level of “shy voting” is at play. So, here in lies the difficulty; if the very source of truth we rely on cannot be deemed to be 100% accurate how does a pollster overcome this to predict an accurate outcome? In advance of the recent Marriage referendum all pollsters, including RED C, were reporting high levels of support for the yes side, however there were concerns that with such a contentious topic that those who supported the No side would be underrepresented in the polls, either because they would be far more likely to tell us that they didn’t know how they would vote or because they simply refused to take part on the poll at all. This was clearly seen to be the case when we look at the final “standard” polls vs. the result, where the No side were clearly underrepresented.
  • 2. Including Undecided Excluding Undecided Sample Dates Yes No Undecided Yes No RED C 69% 25% 6% 69%* 31%* 1,009 11 th – 13 th May Millward Brown 53% 24% 23% 69% 31% 994 2 nd – 15 th May Behaviour & Attitudes 63% 26% 11% 71% 29% 927 1 st – 11 th May Ipsos MRBI 58% 25% 17% 70% 30% 1,200 13 th – 14 th May *(RED C did not re-allocate D/Ks, but suggested they would all vote NO) Having witnessed the significant effect the ‘shy voter’ had recently on pollsters in the UK, with actual outcome of UK elections versus poll outcomes quite different it was clear this group couldn’t be ignored and needed some other technique to give a voice to the shy voter. RED C decided to use a technique known as ‘Wisdom of the Crowds’ to uncover the shy voter. This method asks the respondent to consider all the conversations they may have had, or heard friends and family having and on this basis to try and predict what they believe the percentage outcome will be for and against the marriage referendum. This removes any discomfort from the situation in which a shy voter may find themselves and instead put the focus on the general population as opposed to one individual. The result was a resounding success, as it predicted the outcome with 100% accuracy, 62% in support and 38% against.
  • 3. This “Wisdom of Crowds” method allowed the shy voter, whether in support of or against the marriage referendum to lend some level of their own viewpoint, while disguised within how the ‘masses’ will vote. As such it bridged the gap between standard polling and a potential issue of the shy voter influencing the results. So how does this transfer to general insights we gather for commercial clients or regular political polling? The first thing we have to do is to always be aware of the topics we are researching and the way that people may respond to them, and always ask are we doing enough to get behind the topline stock answers we receive. For instance when we ask people how likely they are to buy a product or service this can often be oversubscribed, particularly among the genial Irish, who are likely to say they “might” buy something, even if in reality they wont go on to do so. So we should also always be asking how likely they think people in the population generally will be to buy that service or product as well, and where large gaps exist take this into account in our analysis. Within party support this also applies, and RED C use several techniques over and above the standard questions to get the most accurate result possible – all parties are read out, unlikely voters are removed, ‘undecided’ voters re-allocated and finally we look at past vote behaviour and layer it with the current claimed intention to ensure an accurate sample. This serves not to alter results, but to allow us report the most accurate results possible.
  • 4. But maybe even this wont be enough? With the Irish elections looming we may also need to look further at techniques such as the Wisdom of Crowds on party support to try to give the ever present shy voter a voice. It is not to suggest that standard polling is defunct, quite the contrary, it simply serves to inform us as pollsters that although research can be quite systematic and indeed scientific, we cannot forget the human in all of us and how best to overcome the nature of the reluctant respondent. Ciara Regan is an Associate Director with RED C Research.