Qualitative Research in Results-Based Financing: The Promise and The Reality
InventoryReview_UserNeeds
1. User needs assessment and
user profiling
A review of the DIGICOM inventory
John Lewis
2. Who took part
• 29 countries took part in the DIGICOM survey
to produce the inventory
• 20 countries provided information that related
to user needs assessment and user profiling:
DK, LT, SK, UK, NL, EE, LV, HU, FI, EL, HR,
AT, PL, IT, ES, SE, CZ, NO, LU, BE & PT
3. Differing methods
• Practices and experience varied across the
respondents
• Varying descriptions of the techniques used
by different NSIs
• Identified 19 different methods used by NSIs
to assess and understand user needs
4. Techniques
Focus groups Meetings Advisory boards Feedback at
conferences
and events
Consultations User forums Surveys Questionnaires
Web forms Analysis of web
search terms
Analysis of web
metrics/traffic
Personas
User profiling Contact centres Phone calls E-mail/mail/fax
feedback
Monitoring
social media
Usability lab User research
5. Techniques
Direct feedback – not too resource intensive (depending on
how conducted), can provide both general and
specific/tailored insights
Surveys – relatively quick and cheap to set-up, allows you
to reach a large number of people, but inflexible, subject to
misinterpretation (of questions and answers) and not suited
for capturing details
Web feedback – often detailed and specific feedback, but
due to this it can be time consuming and resource intensive
to review the feedback
6. Techniques
Personas – time and resource to develop, but once
completed offer ongoing benefits for user-driven design and
understanding user needs
Monitoring Comms & Monitoring Social Media – similar
to web feedback, requires time and resource for analysis
User research/testing – can offer quality, detailed insights
and evidence with a targeted approach, although specific
resource is required
7. Grouping into themes
• The techniques used by NSIs can be grouped
into common themes
• The themes are based on the similarity in
approach, setting and potential findings
8. Themes
Group Description Technique
Direct feedback Usually face-to-face
feedback from
users at a pre-
arranged event
Focus groups
Meetings
Advisory boards
Feedback at
conferences
Consultations
User forums
9. Themes
Group Description Technique
Surveys Asking users
questions and
analysing
responses
Surveys
Questionnaires
Web Feedback Analysing how
users use the
website (also
includes web based
feedback via
forms/buttons)
Web forms
Analysis of
search terms
Analysis of web
metrics/traffic
10. Themes
Group Description Technique
Personas Creating customer
profiles based on what
different users groups
need and how they use
resources
Personas
User profiling
Monitoring
comms
Recording and
analysing requests and
contact from users
Contact centres
Phone calls
E-mail/mail/fax
feedback
11. Themes
Group Description Technique
Monitoring Social
Media
Recording and
analysing
comments and
contact from users
via social media
Monitoring social
media
User
research/testing
Testing, piloting,
trialling products
with users and
analysing
feedback/findings
Usability lab
User research
12. Most popular
The most popular techniques are:
1. Direct feedback
• Biggest theme – composed of 6 individual methods
• Face to face/real time
• Good quality tailored feedback
• Allows you to target specific user groups, topics, etc
2. Monitoring communications
• Feedback through existing channels
• Lack of context and inability to tailor feedback
• Generally negative
13. Other countries – Australia’s personas
Tourist – irregular, just wants the
popular/topical stuff
Harvester – more regular and understands the
data, knowledgeable of their topic area
Miner – regular data user, wants to look at it at
the deepest level, experts
Very similar to Denmark’s personas: Tourist, Farmer & Miner
How did they both develop such similar personas?
14. Findings
• No single method can provide all of the
answers
• Need to use a variety of methods to get a
clear picture of user needs
• Best practice techniques:
• Direct feedback
• Usability testing
• Monitoring communications and social media
• Personas
The key is to build a relationship with users, so
there can be two way conversation to
manage and understand expectations
15. Conclusion
The key is to effective user needs assessment
and user profiling:
• Understand who your users are
• Build a relationship with your users through
communication/conversation
• Understand and manage user expectations
16. Further questions
• Further investigation into the persona
development of Australia and Denmark (and
other countries – what are the similarities)
• Understand how NSIs each undertake the
best practice techniques – are they operating
in the same way, do they offer the same
results?