Want Easier Decision Making Just Listen to Your Users

Healthcare Analytics Best Practices Summit
Want easier decision-making?
Just listen to your users.

© 2015 American Medical Association. All rights reserved.
Introductions
• Jayne Gretz, VP of Market Research & Analytics
• Nicholas Abruzzo, VP of Digital Strategy & Services
2

We know physicians.
Physicians don’t have time
• They are accessing the internet more often, but the amount of time per
session is decreasing
• They do not read, they scan
• They don’t like video as much as consumers because they have to
invest too much time to even evaluate its worth
– They would prefer to scan the transcript first to decide if it’s worth their time
3

We know physicians.
Physicians want to curate their own content
• They prefer to use search over navigation
• They ignore suggested results – they prefer to filter
• Even though they have preferred, trusted sites they aren’t the best for every
question
• They do not bookmark, they want to select the best sites vis-à-vis their
current question
• After they arrive at your site, they’ll give you two clicks then they’re back out
to the search results page – too many other options that can deliver what
they want
4

© 2015 American Medical Association. All rights reserved. 5
“I use Google mostly.
Or I go directly to the
site I’m interested in, but
I mostly just Google
search everything.”
“If there isn’t a great
search mechanism, it
gets frustrating. I will
just give up and go
somewhere else.”
“I prefer to use search – it’s
just easier. If I use the tabs,
the information under the tabs
is not always intuitive.”
“The suggestions
seems too
commercial to me,
like an
advertisement, as
opposed to a place
for resources.”
“Those featured topics
feel like an ‘editor’s pick.’
You need to get rid of
some of that fluff and find
a way to make it more
targeted toward the
audience looking for
particular resources.”
“I ignore the suggested
results mostly because
I ignore sponsored
links. Those
suggestions look like
ads and sponsored
sites.”

These learnings have implications for how we engage with physicians on our site and social
media platforms. The goal is an outside-in experience where the needs of our stakeholders
define the objectives of our digital ecosystem.
6

How we approach our digital strategy
• Both VoC and digital user behavior metrics are valuable in
understanding what our users want and expect from us
• How do you decide when to use which methodologies to get at your
answer?
– Web analytics deep dives and A/B/n tests are good for quick delivery, but are
limited to the scope of what the user was presented – nothing exploratory
– Usability testing and Market Research are good for thorough user input, but
will require extra time and budget
7

Blended approach
• Since we want the best of both worlds, we frequently partner to tackle
complex digital questions and projects
• Following case studies give a taste of how each team feeds the other to
improve our analysis and recommendations
8

Case 1 | Social Login
• Offering social login as an option is becoming a best practice
• But will this hold up with our audience?
10
Digital Analytics Approach:
Test it with our users
Market Research Approach:
Ask our users

Market Research Approach
• Deployed email survey to 250 physicians, residents, and students
• Gauge awareness and usage of social media platforms for both
personal and professional use
• Probed into likelihood to use social login on AMA website
11

About half of medical students (55%), residents (55%) and PCPs
(55%) say they are not likely to use social login on the AMA site.
12
26%
29%
14%
24%
8%
36%
19%
10%
26%
10%
28%
27%
11%
21%
13%
Not At All Likely
Not Very Likely
Neither Likely nor Unlikely
Somewhat Likely
Very Likely
Med Students (N=71)
Residents (N=75)
PCPs (N=103)
Likelihood of using social login
Q. If presented as an option, how likely are you to use your social media account(s) to log into or register with the AMA website? N=249
Data timeframe: August 2014

“I have a social
media account,
but have not used
it in several
months.”
“I do not like to use social
media, which I use for
pleasure, for any aspect of
my professional life.”
“I don't like giving people
access to my social
media accounts or any
more information about
me than necessary.”
“I never use
social media
accounts to
log into
anywhere.”
“I have no interest in the
AMA site, and am much
more likely to log onto
AAP.org, Sermo, Medscape
or Epocrates.”
“I would not want my
Facebook status to
constantly let
everyone know when
I have signed in or
used the AMA
website.”
Data timeframe: August 2014

• Results: neutral to negative
• Action: determined we should not move forward with investing in social
login
• Update: because of licensing changes, considering deploying as option
in a/b test format to gauge usage/interest among our total audience
population since a large number of our visitors are non-physicians
– This will now incorporate Digital Analytics Approach
14

Case 2:
Corporate Site
Usability
15

Case 2 | Corporate Site Usability
• Tasked with improving overall user experience
• How do we identify biggest pain points and where to start?
16
Digital Analytics Approach:
Web analytics deep dive
Market Research Approach:
Ask our users

Blended Approach
• Used web metrics to identify pain points and areas where there was a
gap between user expectations and content delivered
• Formal usability tests conducted – tasks selected were informed by
web metrics deep dive
• Deployed Site Intercept surveys on our Audience pages
17

Web Behavior Analysis
36,385 pageviews
Top 5 Previous Actions
Home page pageview 49%
Search results pageview 12%
Entered the site 10%
“Is Your Doctor an AMA
Member?” pageview
2%
Patient Resources
pageview
2%
Top 5 Next Actions
Exit the site 25%
Conduct a search 24%
Home page pageview 14%
Patient Resources
pageview
11%
CME pageview 3%
Physician Audience Page Navigation Summary
Among top 5 actions, this is the
only behavior taken from content
within the page Data timeframe: Q2’14

“I can not find how to get to
the code helper.”
“Can't find the rvu medicare
payment rate.”
“I am seeking a summary of
the Figures and Tables
contained within the AMA5
PI Guides for medico legal
reporting purposes. I do
not require the entire
publication but need a
reference list of what page
each of the Figures and
Tables is on for Permanent
Impairment reports.”
“This page is
confusing.”
“Impossible to find
a place to enter a
search.”
“I am still looking for the
way to search and learn
if the organization has
board certified Medical
Doctors.”
Site Intercept Survey
Data timeframe: Q3-Q4’14

Usability Tests
Data timeframe: Q4’14
INSIGHT // AUDIENCE-SPECIFIC CONTENT DOESN’T MATCH EXPECTATIONS
 Audience sections do not match that audience’s
career path and lifecycle, e.g. residents have a
very defined career course, and expected to find
content to match their career milestones.
 Opening a practice (found in the Medical Students
and Residents sections) was actually expected to
be found in the Physicians section, since users in
that stage in their career are free to consider
opening a practice.
Maybe add more tabs in the residents
section to show the different options you
have after the residency.
- P5 Resident
“
If I’m looking for information on
opening a practice, I would go to
‘physicians’, because they are the ones
who would be opening practice. I would
not categorize it as a resource for
medical students, because right now
they are focused on getting through
medical school and finding a residency.
- P1 Student
“

Results drove action:
• UX enhancement work streams & projects
– EX: Site navigation re-architecture & Jargon removal
• Further research into content preferences among audiences
– EX: This is drive the Audience page redesign
• Additional web analytics tracking enhancements
– EX: Usage of accordion menus and scroll tracking
• New site intercept surveys
– EX: Are you a physician or a member of the healthcare team?
21

Case 3 | Onsite Search
Key Business Question:
• What content on our website do people search for the most?
23

With web analytics
alone, it is hard to
determine what is
the most commonly
searched content
areas are.
24

Blended Approach
• Pulled 2014 onsite search data from Google Analytics
• Using SAS text miner, we’re working to identify answers to our research
objectives which can now be more complex
25

Research Objectives
1. What categories of content are searched for the most?
2. As changes are made to the site, how do the search trends for content
categories change? (ex: As some content is made easier to find, will they
be searched less often?)
3. How can we match categories back with authenticated users of the
website to provide an indicator of their “interest”?
4. For unauthenticated / “other” users, how can the categories be grouped
to provide more insight into who they might be (i.e. content focused on
specific practice settings, phase of career, etc)?
26

Hypotheses
1. Since the majority of the traffic to our homepage is from branded
organic search, users have been trained to use our homepage as a
search engine because they do not find it easy to discover the content
they are looking for through our navigation.
2. The top categories among authenticated physicians will vary from the
general population of visitors to the site.
3. Searches for consumer/patient focused terms (ex: medical marijuana,
obesity) will have the highest level of refinement and exits from the page
because of the lack of content on our site.
27

Results are currently being analyzed and will be used
to inform the AMA’s enterprise digital strategy.
28

Want Easier Decision Making Just Listen to Your Users

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to Want Easier Decision Making Just Listen to Your Users

Similar to Want Easier Decision Making Just Listen to Your Users (20)

More from Mikan Associates

More from Mikan Associates (6)

Recently uploaded

Recently uploaded (20)

Want Easier Decision Making Just Listen to Your Users