This document discusses the need for market research to adapt to changes in technology and consumer behavior. It argues that researchers should embrace new methods like social media research while still addressing concerns about data quality. The key is making tradeoffs - gaining new insights through more engaged, naturalistic approaches while balancing concerns about representation and bias. A new model is proposed where research is conducted in real time, engages participants, provides nuanced details at scale, and evolves continuously. Researchers are encouraged to move out of their comfort zone by embracing pragmatism over purity and prioritizing actionable insights over statistical significance. Specific examples from companies demonstrate how approaches like transparency, relationships, and purposeful sampling can enhance quality and produce relevant findings.

1. :

2. written by
Manila Austin, Ph.D., Director of Research
Julie Wittes Schlack, Senior Vice President, Innovation & Design

3. Leaving Our Comfort Zone: 21st
Century Market Research | 3
executive
summaryAdvances in social media, the empowerment of everyday consumers,
and the need for more actionable insights fuel a mandate for market
research to do more, faster. These developments create great
opportunity for researchers to exercise strategic leadership, to
inspire and innovate by bringing the voice of the customer to life,
to apply new insights to complex business problems, and to
produce creative, timely and actionable recommendations to
drive business results.
But the use of social media-driven research also fuels the quality
debate that’s been raging for years, creating worries about declining
response rates, questionable respondents, sample size, and
projectability. Market researchers need to consider and address
these legitimate concerns, while also recognizing the ways in which
online, social, community-based research can actually strengthen
validity and enhance quality.
To take that leap, it’s helpful to think in terms of tradeoffs, to
understand what researchers are risking—and gaining—by
shifting their focus and methods.
The choice facing the industry need not be to invest blind faith in
old, authoritarian research techniques or in uppity, untested new
ones. This paper sets out to collaboratively build a foundation
for a 21st
Century understanding of market research—what it can
accomplish and how. We seek to pose some provocative questions,
offer some initial thinking, and engage the industry as a whole
in an ongoing conversation about how to embrace the blurred
boundaries between marketing and market research, and activate
the ability to quickly garner and act on customer insight.
A New Model for a New Age
The emergence of social media challenges us to recognize and
figure out how to intelligently embrace a new way of doing research—
one that is sure to generate insight, to both inspire and inform,
and to provide strategic value. We see an integrative paradigm
emerging—a 21st
Century model—in which research is:
• Conducted in real time, so that it’s relevant and actionable
• Participatory and engaging, which means adopting
humanistic and consumer-centric methods
• Textured and nuanced, with the potential of getting
rich detail on a really large scale
• Continually evolving to meet new marketplace
demands from consumers, clients, and competition
• More dynamic, where we will rethink and re-invent to
drive innovation on an ongoing basis
Taken together, these criteria feel pretty different from the
somewhat dry language many of us grew up with. The online era
challenges many of our assumptions about data quality—validity,
projectability, bias—and it represents a significant change in how
we think and go about our work. As an industry—and to varying
degrees as individuals—we are being nudged, or shoved, out of
our comfort zone.
TRADE
Purity for Pragmatism
Anonymity for Transparency
General for Specific
Distance for Relationship
Control for Collaboration
Artificial for Natural
Randomness for Purpose
Looking Backward
for Looking Forward
BECAUSE
Pragmatic = Actionable
Transparency = Engagement
Specific = Relevant
Relationship = Candor
Collaboration = Creativity
Natural = Authentic
Purpose = Productivity
Looking Forward = The Future

4. Leaving Our Comfort Zone: 21st
Century Market Research | 4
The New Roads to
Quality Require
Challenging Old
Assumptions And
Making Informed
Trade-offs.
Quality is an urgent issue but there’s a more important
issue to do with what needs to be done going forward,
because the industry isn’t keeping pace with the change
going on around us. People are too focused on
quality, people are too focused on probability and
non-probability samples, people are too focused
on respondent engagement—this is all about making
minor changes to what we are doing right now.
Those are all necessary, but they’re not sufficient
conditions for the success of the function.
“(COCA-COLA’S) STAN STHANUNATHAN ON WHY QUALITY DOESN’T MATTER,”
RESEARCH., 22 OCTOBER 2009

6. Leaving Our Comfort Zone: 21st
Century Market Research | 6
Pragmatic =
Actionable
My call to action is that we would figure out the
way to return to the consumer’s backyard…We
have to rebuild that trust and we have to gain
much better insights than we are today…We need to
listen to them on their time, their turf, and in the
ways they want to communicate to us, not in ways
that we choose to communicate with them. And we
need to re-establish the trust and the confidence
that they deserve to have in us…
Kim Dedeker, ARF Leadership Forum 2008
It is time to decouple the notion of “quality” from purity. Today, it is
more important for research to be actionable than irrefutable. This
means shifting our focus—aiming not for the perfect, bias-free study,
but for an approach that pragmatically applies a range of methods to
generate and test hypotheses. Integrating elements of both humanistic
and experimental approaches allows us to produce timely, “good
enough” research targeted to specific business needs. Rather than
itemizing the statistical significance of individual data points, we
need to focus on synthesizing findings that are relevant, insightful,
and actionable.
We have learned that a longitudinal, iterative approach—one that
combines humanistic, person-centered approaches with more
traditional, experimentally derived ones—is most effective.
Humanistic methods (such as ethnographic-type activities that treat
participants as active co-investigators, as opposed to simply passive
survey respondents) are consumer-centric, reflecting how real people
want to engage with researchers. They are discovery-oriented and
exploratory, most suitable for uncovering connections, insights, and
nuances that lead to innovation and competitive advantage. And
iterative, as opposed to episodic research, supports researchers and
participants in an ongoing discovery process that allows everyone to
ask new questions as they uncover and reflect upon what they learn.
Top-down, researcher-centric approaches are best for generating
feedback, for confirming what is already known or suspected. By
design, they do not expand a problem space nor do they generate
knowledge outside of the researcher’s frame of reference. They also
run the risk of alienating the very people—everyday consumers—to
whom companies desperately need to listen. Are there circumstances
under which you want and need to hear from the population of
less-engaged, more neutral customers? Of course, but they are
not and should not be the only game in town.
We believe that there clearly is a place for both kinds of knowledge—
exploratory and confirmatory. But while researcher-centric
approaches are necessary for some purposes, numbers don’t tell
the whole story, or even close to it. By grounding knowledge
generation in the humanistic tradition, we ensure that we are
focusing on the consumers’ world view and framework, not just
the brands’. And combining elements of both traditions in an
iterative, agile, and pragmatic way allows researchers to move
between exploring and testing, generating and confirming
to produce the most actionable information in support of
specific business needs.
Delivers:
Insight and Meaning
Integrating consumer- researcher-centric approaches
Delivers:
Confirmation and Numbers
Consumer-
centric Design
researchER-
centric Design
Discovery
Exploring
Hypothesis generation
Feedback
Focusing
Hypothesis testing

8. Leaving Our Comfort Zone: 21st
Century Market Research | 8
Market researchers have been concerned with the quality of online
data collection since the dawn of the Internet, largely due to fears
that online consumers fail to accurately represent the general
population. These concerns, however, are becoming increasingly
irrelevant for two main reasons. First, more people are accessing the
Internet, making distinctions between online and offline groups less
meaningful. Second, if relevancy of insight is an important quality
factor (which we believe it is), then researchers have more to gain
by listening to the “right” group of people than they do by trying to
generalize findings to a generic population.
There is plenty of evidence showing that, except for a few discrete
segments, the Internet population in the U.S. is quickly becoming the
general population. Many European countries’ Internet penetration is
higher than what we have in the U.S., and projections for developing
markets (with the advent of smartphones) indicate that soon,
two billion people will be online.1
Online versus offline is quickly
becoming a non-issue.
More importantly, quality research must produce relevant findings;
and we have learned that listening to targeted, specific groups of
customers is the surest road to relevancy.
If you want to deepen customer loyalty, who better to engage than
members of your brand’s loyalty program? If your goal is to broaden
your brand’s appeal, then hearing from fans of your competitors’
brands may be the most useful approach.
Researchers can be more confident in taking action when they
trust they have the right people assembled to address their specific
objectives. We advise our clients that whether or not they generalize
from communities depends on the community composition and on
the particular question they’re trying to answer. Many questions—like
the ones featured in the United Airlines example shown here—can be
explored and findings effectively generalized when the community is
specific and the target market is defined enough.
1. www.internetworldstats.com/stats.htm
SPECIFIC =
RELEVANT
On REPRESENTATIVE SAMPLES
We recruited a community of our most valuable
customers—a relatively small population—
from our customer list. Because the community
accurately reflects my population of interest,
and because the purpose of the community is to
understand our most loyal customers, I believe
the results of my community’s research are
reflective of the larger target population. So even
though it’s not a traditional panel, I’m comfortable
using the community more quantitatively.
Dan Comenduley, Project Manager, Customer Metrics Insights,
United Airlines

10. Leaving Our Comfort Zone: 21st
Century Market Research | 10
NATURAL =
authenticTrading artificial approaches for more naturalistic (private, online,
at-home) research settings can strengthen validity and data quality,
especially when researchers are asking people to reveal intimate
aspects of their lives.
We have found that the freedom and relative safety of private
online communities allow for the iterative exploration of the
most intimate content. We have seen that in naturalistic settings,
regular people openly share detailed information about their
financial situations, experiences with serious illnesses, stresses
and hopes, relationship worries, and even embarrassing quirks
and habits.
Another benefit of naturalistic settings is that they not only provide
more authentic contexts for engagement and discovery, but they
strengthen researchers’ ability to generalize findings. Participating
in focus groups and anonymous surveys requires people to step out
of their daily lives; thus findings generated from these approaches
do not always translate to real-life situations. And with so much
public distrust in how companies use electronic information (e.g.,
identity theft, subversive marketing, etc.), people are less likely
to be truly open and forthcoming if they don’t know or cannot
trust the researcher. But a willingness on the part of researchers
to model the kind of self-disclosure that we hope to elicit from our
participants, coupled with the kind of ongoing connection enabled
by social media, make it possible to build a trusting relationship
between researcher and “subjects” over time.
Perhaps one of the most overlooked ways to enhance quality is
leveraging the power of a naturalistic research setting. Natural
settings promote authentic participation because the researcher
engages people on their own terms, refraining from barging into
people’s lives or extracting them from their homes to answer
questions they had no say in generating. One benefit of online
research—and private communities in particular—is that it allows
people to participate on their own time, on their own terms, and
from their own homes or smartphones. They are able to use social
technologies to bring their own lives to the researcher.
On intimacy
We formed a community of newly diagnosed cancer
patients and primary caregivers who participated
no matter where they lived, whatever hour of the
day and regardless of their condition. In contrast
to typical market research, the richest ‘aha’s’ came
when the patients and caregivers initiated their
own discussions, and we had the opportunity to
really just listen—observing how the members
supported each other and learning from the
stories they shared. We were a fly on the wall
in the treatment room, which for healthcare
marketers is very unusual.
Alana Brody, Former SVP Strategic Development,
National Comprehensive Cancer Network (NCCN)

12. Leaving Our Comfort Zone: 21st
Century Market Research | 12
On Anonymity
We found that in an unbranded community we were
continually challenged with talking directly to
our members without revealing our identity, and
thus being forced to find ‘creative’ ways to ask
questions and conduct ‘decoy’ research. Our
real objective was to have a direct dialogue
with our consumers and gain valuable insights
into their needs as shoppers. When we did move
forward and unveil our identity, becoming a
branded community, we found member engagement
increased and conversations become much more
relevant and valuable to both parties.
Michelle Laslo, Senior Manager,
Customer Strategy and Insights, PepsiCo
transparency =
ENGagement
One way to get quality information is to trade anonymity for a
transparent, open approach. Researchers are often concerned
that findings will not be valid if participants speak freely with one
another, know who the sponsoring company is, or are active
co-participants in the research process. Our experience and our
five-year program of research-on-research suggest this is actually
not true. We have found that when companies trade in anonymity,
they gain better engagement, more textured insights, and increased
value overall.
First, transparency in research is just simply more engaging for
people. When companies are upfront in disclosing their identity,
when they invite people into the fold, and when they demonstrate
that they are truly listening, people respond in kind. The research
we have done on community member participation clearly shows
that branded communities outperform unbranded ones.2
And our
research on corporate listening shows that community members
value feeling that their voice is heard and that their contributions
are making a difference.3
Engagement is critical for quality; when
people are engaged they try harder, they do and share more, and
go to great lengths for companies when they know who they are
talking to.
Second, companies undermine the ROI of research when they fail
to be transparent. Participation is lower, researchers must concoct
and field “dummy” research to disguise their identity (which
waters down the learning agenda and wastes time), and members’
energy is often diverted into guessing games that are not valuable
for the client.
And last, many of our clients have conducted parallel studies
comparing results from communities with those generated from
blinded approaches (e.g., panel surveys, focus groups, etc.) We
have collected over 25 examples across a range of clients, and the
comparison results are consistent: findings from data collected
through a transparent, community approach are directionally
similar to those obtained by other methods.
2. Katrina Lerman and Manila Austin, The Fifth ‘P’ of Marketing:
Participation (Communispace whitepaper, 2007)
3. Katrina Lerman and Manila Austin, What Companies Gain
from Listening (Communispace whitepaper, 2006)

14. Leaving Our Comfort Zone: 21st
Century Market Research | 14
relationship =
candorIt may seem counter-intuitive, but quality can be heightened
when researchers trade distance and objectivity for closeness and
relationship. A common fear market researchers share is the
possibility that the feedback from people participating in branded
research communities will be tainted, inflated or somehow not
trustworthy as a result of their ongoing involvement.
Our research shows that, if anything, members become more honest
and forthright as their tenure increases. Over time, members do come
to view company sponsors more positively, but this does not affect their
ability to provide valuable—and critical—feedback.
For example, we have developed a method for coding open-ended
responses for both “candor” and “richness” and have found that
members continue to provide critiques and textured detail in
their contributions, regardless of tenure.
We have also found that the relationship that develops between
company sponsors and community members can actually increase
clients’ confidence and trust in what they hear. Because communities are
transparent, composed of the “right people” for the business objective,
and because companies really know who they are hearing from, clients
feel more comfortable acting on the advice of the research community.
On candor
In a past job at a major food company, we had
members react to a new product concept early
in development. Internally, we really loved the
concept we were fielding, and had high hopes
for it. Members responded quickly to the concept
test—within 72 hours—and they hated it, totally
rejecting the idea. But it wasn’t just a gut
response. Because they felt they knew us, were
invested in us, and didn’t want to see us screw
up, they also provided us with ample, very clear
feedback on why. Our loyal users saw a fatal flaw
in the product that we had missed in our own
excitement. Based on their responses, we pulled
the product idea within three weeks, saving the
company costs on further development. In this
case our community helped us fail faster, and
allowed us to feel good about the decision
because we knew we could trust that
critical feedback.
Adrian C. Bing-Zaremba, Consumer Insights and Market Intelligence,
Boehringer Ingelheim Consumer Health Care Products

16. Leaving Our Comfort Zone: 21st
Century Market Research | 16
purpose =
productivity
We also need to re-evaluate our fear of the dreaded practice
effect—the idea that repeated participation in research necessarily
erodes quality. Recent research and our experience suggest just
the opposite.
There is mounting evidence that practiced research participants—
people who are motivated and engaged, and well-versed in how to
best contribute—actually produce better results. A study released
by the ARF’s Online Research Quality Council actually found
that increasing panel membership lowered “bad” survey taking
behavior (such as straight-lining or speeding).4
“Professional”
respondents, then, do not necessarily threaten—and may actually
improve—quality.
This sounds counter-intuitive, but we have also found that
experience participating in research—especially when the purpose
is transparent—produces better results. Members are motivated,
proficient, and simply more productive.
This is certainly the case with new product development
communities, but it is also true for insight communities.
Our participation research also shows no relationship between
greater monetary incentives and increased participation.5
Intrinsic motivation drives engagement. There is something
inherently energizing about a shared purpose and goal-directed
activity; when research participants know why they are being
asked to make electronic collages, take videos of a family dinner,
or brainstorm ideas, they are motivated to do a better job.
Concern about professional respondents has been acute in recent
years, but it is time to realize that quality requires trading in the
myth of the “fresh,” unpracticed consumer, and focus instead on creating
purposeful relationships with people to help them do a better job of what
they are already doing anyway. The notion that there is a “random”
population of people in the world that do not take surveys, answer
marketers’ questions, post and read reviews, or engage with brands on
a regular basis is wishful thinking in the 21st
Century. Consumers today
are extremely marketing savvy; and if there are untapped consumers out
there, then they certainly aren’t representative of the general population.
On practice
I like problem solving, and this community is about
problem solving, in terms of identifying what people
need, what they want to see done differently, and
how you can meet their expectations.
After a few months, I guess the [incentive] became
tiresome and the conversation kind of addictive.
I like many of the topics and hearing guys’ thoughts
on them, as well as having the opportunity to give
my own perspective.
Members of two Communispace Communities
4. Robert Walker, Raymond Pettit and Joel Rubinson, Foundations
of Quality Knowledge Brief (The Advertising Research
Foundation, 2009)
5. Katrina Lerman and Manila Austin, The Fifth ‘P’ of Marketing:
Participation (Communispace whitepaper, 2007)

18. Leaving Our Comfort Zone: 21st
Century Market Research | 18
collaboration =
creativityMarketers must also consider trading sterility and one-to-one tactics for
dynamic collaboration based on many-to-many interactions. To reap
the benefits of collaborative creation, however, researchers need to get
comfortable with—and figure out how to leverage—the group dynamics
that naturally arise when people get together for any purpose. The
likelihood that research participants will influence one another
throughout the research process is an understandable concern;
because interaction is encouraged and happens transparently in online
communities, we worry that members are subject to “group think.”
But the days of isolated, pristine research are over. In this era of tweeting
and lifestreaming and rating and review sites and human billboards,
everyone is subject to influence from their peers. Rather than attempting
to isolate people or control their interaction (or worse, constraining the
naturalistic community setting by virtually hiding responses or forcing
anonymity), we need to ask ourselves how to observe influence behavior
and learn from it.
The alli®
example demonstrates the creative potential in building on
group processes for breakthrough solutions.
On Influence
When GSK Consumer Healthcare introduced alli, we
did so knowing that this was a product that could
elicit some intense emotional responses. Our private
communities gave us a chance to pose questions of
users and non-users alike, but more importantly, to
see what questions alli non-users posed of users,
and of how passionate alli consumers answered
them, described their own experiences, and made
their own recommendations. I’d like to think that we
were visionary market researchers doing cutting-
edge work, but honestly, I think we were just being
realistic about the fact that consumers have
unprecedented opportunity to influence one
another online. So our goal wasn’t to pre-empt
‘group think’ so much as to understand it, to see
the peer-to-peer influence process in action and
learn from it.
Andrea Harkins, Manager, Integrated Insights,
GlaxoSmithKline consumer healthcare

20. Leaving Our Comfort Zone: 21st
Century Market Research | 20
looking forward =
the future
On Co-Creation
Scholastic Book Clubs recently won a Forrester
Research Groundswell award for our work with
a customer community comprised of parents and
teachers. Our goal was to redesign the Book Club
flyer in an effort to improve the way parents, kids,
and teachers find and buy the right book for the
right child. By zooming out and exploring not just
how teachers evaluate books, but how parents
evaluate their kids’ readiness and interest in
reading, this group helped us effectively redesign
our iconic flyer. This fruitful co-creation couldn’t
have happened if we had simply solicited feedback
on flyer designs from one group or the other in
isolation. The transparency and opportunity
for mutual influence, along with active, visible
facilitation is what made this process so productive.
Judy Newman, President of Scholastic Book Clubs
The industry’s historical focus on producing irrefutable, nationally
representative data points is meaningful only if we limit the role of
market research to testing and confirmation. But if market research
is to win a permanent seat at the executive table, if it is to be integral
to brand strategy, then it has got to be about creation, not just
prediction. It is time to trade a backward-looking and confirmatory
stance for a forward-looking and generative approach.
It is no longer clear in today’s long-tailed, filtered, personalized
world that it is actually, scientifically possible to accurately predict
behavior. But what we can do is co-create with our consumers,
rapidly, ideally one step ahead of them, but at least with them.
Markets are becoming more diverse and will continue to change
rapidly. So generating insights and engaging in co-creation
upstream in the development process and doing it in an agile
way (with short time frames and a focus on niche markets as
they emerge)—these are the ways market researchers will keep
pace with customers and “go where they go.” And those market
researchers who can “do” upstream creation as well as prediction
will play strategic roles in driving business.

21. Leaving Our Comfort Zone: 21st
Century Market Research | 21
trading off
can mean
trading up.
The industry has moved away from just being
seen as a quasi-scientific activity, providing
hard quantitative measurement that is detached
from the creative process and the complexities
of intuitive decision-making. Today, it is seen as
also embracing a much more pragmatic approach
that requires high levels of creativity and
imagination in order to tease out key insights.
DVL Smith, University of Hertfordshire, U.K.
It’s about understanding the human condition. We’re
too focused on understanding consumption behavior
and shopping behavior. We need to understand the
human condition, which you’ll only know by observing,
listening, synthesizing and deducing.
“(COCA-COLA’S) STAN STHANUNATHAN ON WHY QUALITY DOESN’T MATTER,”
RESEARCH., 22 OCTOBER 2009

22. Leaving Our Comfort Zone: 21st
Century Market Research | 22
summary
The kinds of results we have observed over the years are specific to communities as we do them
at Communispace, where insights are generated in the context of continuous, longitudinal,
intimate, purpose-driven groups, and where community members forge real relationships
with one another and with our clients over time.
But beyond our own experience, we generally believe that an online, iterative, consumer-centric
approach mitigates some of the risks and challenges of conventional market research, can actually
enhance quality, and uncovers relevant insights quickly in a way that is fun and authentic for real
people. By leveraging emerging technologies that foster connection, researchers can avoid
the pitfalls of barging into peoples’ lives and instead meet them where they are. As a result,
research efforts are likely to yield more spontaneous and revealing insights. And as an added
advantage, there are also efficiencies and cost-savings researchers can achieve by capturing a
high volume of rich, open-ended data at a relatively low cost.
By using humanistic, transparent approaches—in essence, by encouraging consumers to become
engaged in the form as well as the substance of the research—we get really engaged, motivated
participants. By involving customers as actors, not just as subjects; by bringing their voices into
every organizational function, market researchers will enable consumer-led growth. They’ll
ensure that their companies generate solutions that are relevant to customers—in design,
function, packaging, and messaging—and in so doing, drive growth and innovation.
21st
century method
Human, transparent approaches
Consumer-centric settings (leveraging online,
mobile, and other technologies)
Fast, targeted inquiries
Large scale “qualitative”
Building relationships that
endure over time
Potential Gain
Engaged, motivated participants who
generate higher quality data
“Naturalistic” settings that feel safe, maximize
comfort, and encourage intimacy
Research findings that are relevant, timely,
and actionable
Collecting an unprecedentedly large number
of open-ended data at a relatively low cost
Deep knowledge of participants as real people,
leading to greater insight and increased
confidence (you can trust that you really
know the people participating)