To lead a successful research practice in your organization, you need access—to customers, staff resources, and the availability to identify patterns surfaced over time. Unfortunately, many content strategists, marketers, and designers find themselves with either ad-hoc or sporadic opportunities to really practice strategic research. This session won’t focus on methods or approaches to research, or why research is important to build better products. But people new to research or those in mature research teams can both learn new perspectives to making research an organizational priority.

1. RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECHCHAT

2. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Why we should talk to our customers, users, clients, etc
• Best practices to conduct research
• Research deliverables
What we won’t be talking about
INTRO

3. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
What we won’t be talking about
INTRO

5. RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECHCHAT

6. During Investor Day, we also said
that at NASDAQ our fundamental mission first and
foremost is to serve our customers. We accomplished this
through intensive engagement and feedback with
these customers.
We also received much feedback from
the early adopters, most of it
overwhelmingly positive, but this feedback loop
is also helping us to develop and quickly
deploy innovative new features.
Bob Greifeld
CEO, Nasdaq Inc.
NASDAQ INC Q1 2016 EARNINGS CONFERENCE CALL
APRIL27, 2016 8:00AM ET

7. The first thing we have to do is listen
to our customers, and in fact, great design …
starts with a really deep listening to the customer.
What are the biggest problems they have in doing their job today, or
what do they wish they could do better, or where do they wish they
could take their relationships with their own clients?
We’ve found things that when we’ve shown it to the client, they said,
oh, that’s exactly what I need, but I didn’t even know I needed it.
So that’s great design.
Adena Friedman
COO & President, Nasdaq Inc.
OPENING REMARKS, PRODESIGN CONFERENCE
JANUARY30 2015

8. RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECHCHAT

10. SURVEYSAYS…

11. SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Location RoleEmployer Team size

12. SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Do you have access to artifacts, transcripts, and
other assets from any research activities?

13. SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
How frequently is research included in your
process or projects?

14. SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Select the barriers that prevent you from
conducting research:

15. SURVEY RESULTS rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
Do you have a governance strategy in place for
research outcomes, assets, artifacts, and transcripts?

16. LAGGARD MATUREPROGRESSING
ExecAttitude
EARLY
• Limited to user feedback
• Market research
• Ad-hoc
• Late-stage testing existing products w/
clients
• User/advisory groups
• Log analysis
• Limited to one business or product
• Discovery research w/ client base
• Consistent, iterative usability testing
• Monitored Beta & Pilot programs
• Comprehensive analytics & data collection
• Participatory design w/ clients, users, etc
• Discovery research w/ clients, non-clients
• Consistent evaluation of current products
• Explores new markets, audiences
• Multi-channel touchpoints
• Spans across organization
• Not limited to functionality: includes customer
experience, recruiting & onboarding, etc
• Find what customers don’t like • Usability test to predict the future (CYA) • Confirm current product decisions are sound
• Establish shared understanding of customers
and markets by product team
• Prioritize where to focus new innovations
• Inform customer/market segmentation
• Learn where to adjust product strategy
• Identify unmet needs
• No dedicated research team
• Product management or marketing
interacts with clients
• May hire perception/market research
firm
• Designers lead guerrilla usability tests
• May outsource research to agency
• Dedicated research person or team within
organization
• Product management & design support the
research team
• Management level position to lead alongside
product, marketing, commercial etc
• Resources available for non-research teams
to conduct research, report findings
• Individual product team • Product management
• Marketing
• Unconvinced • Cautious • Enabling • Expecting
• Product management
• Marketing
• Design
• Engineering
• Research processes are managed, consistent
• Follow best-practices
• Findings are available for wider teams
• Little pre-research planning
• Reports are emailed to stakeholders
• Little awareness of best practices
• none • Decisions are made consulting existing design
research or create need for more research
• Strategy, tactics, findings are consistently
applied across projects, orgs, businesses
• C-suite (CEO, CMO, etc)
• Commercial leaders
• Corporate Strategy
• External partners
What does research cover?
What are some of the
methods that are being
used?
How far does research
reach in the organization?
Why is the organization
investing in design
research?
Who makes up the research
team in the organization?
Who uses the findings that
come from design research?
What are the policies to how
research will be conducted
and consistently applied
across teams, projects, and
the organization?
+
+
+
+
+
Scope
Staffing
Purpose
Governance
Audience

17. WHATDOWEDOASDESIGNERS
THATPREVENTUSFROMDOING ?MORERESEARCH
MOREFREQUENTLY

18. NO CLEAR RESEARCH
STRATEGY
NO CLEAR RESEARCH
STRATEGY

19. ASSUME ONLYWE CAN
DO THE RESEARCH
ASSUME ONLYWE CAN
DO THE RESEARCH

20. ASSUME ONLYNEWRESEARCH
WILLTELLUSWHATWE NEED
TO KONW
ASSUME ONLYNEWRESEARCH
WILLTELLUSWHATWE NEED
TO KNOW

21. DOGMADOGMA

22. SELF-SERVING PRIORITIESSELF-SERVING PRIORITIES

23. NO USEFULCONCLUSIONNO USEFULCONCLUSION

24. IGNOREWHATCUSTOMERS
TELLYOUWHAT THEYWANT
IGNOREWHATCUSTOMERS
TELLYOUWHAT THEYWANT

25. • No clear strategywhyto research
• Assume onlywe can do the research
• Demand newresearch
• dogma
• self-serving priorities
• No usefulconclusions
• ignore customers

26. methodsarelemons
RESEARCH
Some

28. You’re Doing It
Well, actually,
WRONG

29. MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Outsource research to 3rd parties
• Little knowledge sharing beyond hiring
client team
• Difficult to adjust logistics, script,
scope, reach on the fly without
incurring new costs, contracts, etc
• Much insight leaves with the agency
when the project is over
• Check-the-research-box mentality:
one shot to do everything
• Make a case to reallocate funds in-
house or to *your* team
• Create a cross-project, cross-team
repository of reports/assets/etc
• Build elasticity into SOW
Yes,
and…

30. MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Other internal teams do all the research
• Research and findings exist in silos
• Teams who need research the most
may not be able to contribute
• Research priorities may be different
than yours
• Show connections between their
research and your work
• Invite those teams to your meetings
where design decisions, prioritization,
etc are made
• Find ways to help—reporting, recruiting,
managing research assets
Yes,
and…

31. MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Focus groups
• Groupthink
• Self-conscious or deliberately
secretive around strangers
• Little exploration or elaboration
• No additional research necessary
• Get in the room
• Influence the script
• Get leads for more interviews
• No inadmissible evidence:
Don’t fully discount what you hear
Yes,
and…

32. MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Surveys
• May give false sense of security
• Can be written to skew one way
• Harder to write than they seem
• No observable human behavior
• Provide opt-in to be contacted for more
comprehensive interviews later
• Mine for verifiable information
• Write questions that are appropriate for
surveys (duh)
Yes,
and…

33. MAKING THE BEST OF IT rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Old Research
• Org may think research has been done
and doesn’t need more
• May not address evolving marketplace,
expectations, reality
• May have been discovery research, but
you need usability testing
• Show gaps in understanding or
where things have changed
• Don’t discount research just
because it’s old
• If available, talk to people who
did or paid for the research to
ask what’s missing
Yes,
and…

34. • Be involved
• Influence the script
• Use any outreach as an opportunity to
establish a relationship for later
• Don’t dismiss the value just if the Internet
doesn’t think it’s the best way
Make THe best of it

36. D
Change
MANAGEMENT- -

37. klllll lllllm
1Find an
advocate
#WOCINTECHCHAT

38. IMPORTANCE Political support
Control over resources
Expertise
Same team
#WOCINTECHCHAT

39. Selecting the right
challenge is paying attention to
who else cares about it

41. klllll lllllm
ESTABLISHASENSE OF
URGENCY

42. klllll lllllmEstablish a sense of urgency
• Too much past success
• Lack of visible crises
• Low performance standards
• Insufficient feedback from
external, trusted sources
Traitsof
complacent orgs

43. klllll lllllm
Find external data to refute a comfortable status quo
Talk to unhappy or former customers
Show how profitable future opportunities are unobtainable
Establish a sense of urgency

44. klllll lllllmEstablish a sense of urgency
There were 70,000
searches for that topic last
quarter—people can’t find this
content!

45. klllll lllllmCelebrate short-term wins

46. klllll lllllm
• Visible to outsiders
• Unambiguous
• Tied directly to change effort, vision
Celebrate short-term wins

47. klllll lllllm
• Provide evidence you’re on the right track
• Help hone the vision and long-term strategies
• Build momentum
• Keep bosses/advocates on your side
Celebrate short-term wins

48. klllll lllllmCelebrate short-term wins
We interviewed over 50
customers this quarter!

49. klllll lllllmCelebrate short-term wins
We researched
patterns across 80% of our
product portfolio—almost
everyone is on board!

50. klllll lllllmCelebrate short-term wins
Our beta program
uncovered over 50 bugs we never
found in QC! Good thing we caught
them when we did!

51. klllll lllllmCelebrate short-term wins
Our prototypes
saved over $1MM of at-risk
business after those clients saw
what was coming!

52. D
Change
MANAGEMENT
Find Advocates
Create UrgencY
Celebrate The Wins
- -

53. tactics
Tactics

54. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Plan for iterative research
TACTICS

55. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Address research in the project plan
• Plan to iterate, redo, refine, based on the research
• Align research with what’s getting built and when
• Be as consistent as makes sense across projects
Plan for iterative research
TACTICS

56. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Frame why you’re
researching for yourself,
your team, and stakeholders
• Applies focus
• Sets expectations
Transparent research strategy
TACTICS

57. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Who’s who
• Themes to explore
• Duration of campaign
• Qualifying criteria for
participants
Transparent research strategy
TACTICS
• Confirmed participants
• Methods to be used
• Script, if available
• Estimated date to deliver
findings

58. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
What do we want to learn
and what will we do with it?
Transparent research strategy
TACTICS

59. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Most effective research doesn’t happen in a vacuum
• If dev informs the campaign upfront, they’re more prepared
for outcomes of campaign
• Marketing teams will be more receptive to your findings if
they know ahead of time
Encourage Dev/Product/Marketing collaboration
TACTICS

60. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Critical these teams know about your efforts so you don’t
contact same people
• Gold mine for referrals/leads for recruiting
• Expect to educate these teams on why you want to talk to
“their” clients
Service, sales, and support are allies
TACTICS

61. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Show you’re on the right track
• Share recordings & video clips
• Don’t sweat cherry-picking
Show execs the good stuff
TACTICS

62. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Before/After case study
• Show evolution of features/products based on user research
• Include quotes/metrics showing success
• Consider audience skeptical and unaware; don’t use jargon
or assume buy-in by default
Keep a case-study nearby
TACTICS

63. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Big data
• Machine Learning
• Network analysis
• Perception studies
Tie campaigns to larger business projects
TACTICS
}
• Listening campaigns
• Sentiment analysis
• New people to talk to

64. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
• Benefits of research
• How to:
• conduct an interview
• synthesize research
• prepare a report
Coach your organization
TACTICS:
WORKSHOPS RESOURCES
• Script templates
• Mad-Libs cheat sheet
• Report templates
• Persona templates
tt tt tt tt

65. rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs
qrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Market segmentation | Buyers | Promoters, saboteurs, etc
Expand the scope
TACTICS:

66. Plan for iterative research
Transparent research strategy
Encourage Product/Dev involvement
Sales, Service, & Support are allies
Show execs the good stuff
Keep a case study nearby
Tie campaigns to larger business projects
Train & coach to scale research
Expand the scope
Rethink your presentation
abab
abab

67. I
Transparency Collaboration
Planning Promotion
I

68. ONEMORETHING

69. NORESEARCHGOVERNANCESTRATEGY

70. MOSAIQFree Wordpress theme to organize & manage all
your research activities and assets
Perfect for teams and agencies
Tie transcripts, notes, and participants to findings,
personas, and more
Download today at NasdaqDesign.com/Mosaiq

71. RESEARCH
FREQUENTLY
I
oooooooooooooooooop
noooooooooooooooooomore
more
TUUUUUUUUUUUUUUUUUUUUUUVChris Avore @erova #confabMN
#WOCINTECHCHAT