An improved hiring process would benefit most UX teams, but hiring managers have much to consider when deciding how to interview UXR candidates. How many stages of interviews? What are the best activities at each stage? Should we do portfolio reviews, take-home projects, neither, or both? How long should the process take? The interview process is critical to all involved; hiring managers need an approach that accurately assesses a candidate’s skills, and candidates need an opportunity to demonstrate their experience that respects their time. In this presentation, we will review data collected from UXR job-seekers and hiring managers, inspect trends and attitudes of both groups, present an illustrative overview of the “average” UXR interview process, and discuss the implications our findings have for your team.

1. How teams hire UX
Researchers:
UXPA
2023
THOMAS STOKES & LAWTON PYBUS
AUSTIN, TX
JUNE 21, 2023
A survey of
current trends &
best practices

2. Thomas Stokes
USER EXPERIENCE RESEARCH
CONSULTANT, USERTESTING*
Lawton Pybus
* Views expressed in this presentation are held by the
speakers and do not necessarily reflect those of any
employer or other affiliated organization.
USER EXPERIENCE RESEARCH
MANAGER, USERTESTING*

3. Why we’re
here today
WHY ARE WE HERE?

4. WHY ARE WE HERE?
What is the best way to evaluate UX
research job candidates? How do
we go about doing this today?
How can we elevate the experience
of both job-seekers AND hiring
teams?
Our findings revealed concrete ways we can
advance our hiring practices.

5. You might think we have a firm
grasp on best practices for job
interviews.
Prevalent dissatisfaction from
both sides made us wonder,
what could we do better?

6. JOB SEEKER
“It's horrible… [I]
complain to my wife
constantly about this…
the UX of job
applications is not
friendly…”
HIRING MANAGER
“It was horrible… we
got all of the wrong
people.”

7. “Perhaps designing hiring
process with empathy as a north
star might shift UX interview
narratives into more
empowering experiences, not
where dreams and confidence
are dashed with no justification
other than 'it is a tough market'.”
$36,600
Average cost of hiring the
wrong UXR candidate*
ACKNOWLEDGING THE
IMPLICATIONS
JOB SEEKER
* Numbers based on average costs,
salaries, & benchmarks

8. Our approach
How we explored the current
state of UX research hiring and
interview processes
01
Unexpected
alignment
Good news: hiring managers
and job-seekers generally
want the same things
02
Applications and screening
candidates can be full of friction
03
Early-funnel
friction
OVERVIEW
Lack of structure &
support
Structured evaluations are
underused
Interviewers don’t have access to
enablement
04
Key takeaways &
next steps
What you can put into practice
Resources
Our next steps
05
Time for your questions
06
Discussion

9. Our approach
HOW WE INVESTIGATED
THIS ISSUE

10. INTERVIEWS
Secondary Research
Primary Research
APPROACH:
SURVEY
COGNITIVE
INTERVIEWS
UNMODERATED INTERVIEWS

11. APPROACH:
Primary Research:
Mixed Methods
Design
Qualitative Interviews
• Attitudes & experiences
regarding UXR job
interview loops
○ n=7 hiring managers
(HMs)
○ n=10 job seekers (JSs)
Quantitative Survey
• Attitudes & experiences
regarding UXR job
interview loops
• N=160
○ n=56 hiring managers
○ n=104 job-seekers

12. APPROACH:
Secondary
Research
We reviewed existing
research & interview
guides including…
• Academic research
• Industry reports
• UX specific articles
• Articles from HR
• Books

13. Unexpected
alignment
WHAT DO WE WANT OUT OF
THE INTERVIEW PROCESS?

14. UNEXPECTED ALIGNMENT
Hiring managers and job seekers
are more alike than you might
think when it comes to what they
want from interviews.

15. INTERVIEW ACTIVITIES:
Earlier
INITIAL PHONE SCREEN
COMPUTER INTERVIEW/
VIRTUAL INTERVIEWS
TECHNICAL INTERVIEW
/ TECHNICAL Q&A’S
1-TO-1 INTERVIEW WITH
THE HIRING MANAGER
BEHAVIORAL
INTERVIEW
PORTFOLIO OR CASE
STUDY PRESENTATION
1-TO-1 INTERVIEW WITH
RESEARCH TEAM
PANEL INTERVIEWS
WHITEBOARDING
EXERCISES
TAKE HOME TEST
PROJECTS
1-TO-1 INTERVIEWS
WITH MEMBERS OF
OTHER TEAMS
More Common
Less Common
Later
INTERVIEW
ACTIVITIES
WHAT HIRING
MANAGERS USE &
WHEN
What hiring managers
use and when

16. HIRING ACTIVITIES:
A typical UXR
interview process
The “typical” UXR interview process can be
split into an early and “later” portion.
Six activities were popular with
hiring-managers.
Initial phone screen
1-to-1 interview with the hiring manager
Portfolio or case study presentation
Behavioral interview (e.g. to evaluate
collaboration/working styles/culture fit rather than
technical skills)
1-to-1 interviews with research team members
Technical interview/technical Q&A's
Phone Screen
1-to-1 interview with
hiring manager
Early
Technical interviews
& questions
Portfolio
presentation OR
Take-home AND/OR
Whiteboarding
Behavioral
interviews &
questions
Interviews with
other team
members
(researchers &
others)
Later

17. INTERVIEW ACTIVITIES:
Hiring managers
vs. job seekers
rankings
Whiteboarding exercises
1-to-1 interview with the
hiring manager
Initial phone screen
Initial phone screen
1-to-1 interview with the hiring manager
Portfolio or Case study
presentation
Technical interview/Technical Q&As
1-to-1 interview with the
research team
Portfolio or Case study presentation
Technical
interview/Technical Q&As
1-to-1 interview with the research team
Take home test projects
Take home test projects
Whiteboarding exercises
1-to-1 interviews with
members of other teams
Panel interviews
Panel interviews
1-to-1 interviews with members of other teams
Computer interview/Virtual
interviews
Computer interview/Virtual interviews
HIRING MANAGERS JOB SEEKERS

18. “LET’S CUT TO THE CHASE”
Job-seekers STRONGLY prefer
2-3 phases of interviews
Hiring managers select more assessment
methods but combine multiple methods
into a single interview phase.
5.5
Average number of
assessment
methods included
in processes.
“Some of these were
combined into one, so I would
not want to give the
impression that there were 8
interviews…more like
3 rounds”
HIRING MANAGER

19. Hiring
managers &
job seekers
share in the
outcome
01
Job-seekers want
an opportunity to
demonstrate
competency &
Hiring managers
want to evaluate
competencies
02
Hiring managers
were once job
seekers
03

20. Early funnel
friction
HOW DOES THE EXPERIENCE START?

21. “The UX of
applying to UX jobs
sucks”
WE HEARD ISSUES ON
BOTH SIDES
• Time-consuming
• Vague job descriptions
• Producing application materials
• Frustrating applications systems (e.g., Workday)
• Opaque interview loops
• Lack of feedback (e.g. “ghosting”)
Recruiters can
help, but the
experience varies
• Time-consuming
• Vague job descriptions
• Producing application materials
• Frustrating applications systems (e.g., Workday)
• Opaque interview loops
• Lack of feedback (e.g. “ghosting”)
• In sync recruiters streamline the hiring process;
effectively screen and identify qualified
candidates.
• A recruiter lacking understanding of candidate
qualities may present unsuitable candidates,
wasting time and effort.
• Without a recruiter, you bear the burden of the
entire candidate search process; a time
consuming distraction.

22. Let’s focus on what is
in our control

23. EARLY FUNNEL FRICTION
• Don’t require anything
you don’t need/review
• Hiring managers vary in
their use of cover letters,
portfolios
Ask for the right
application materials
• Where possible (e.g.,
notwithstanding onerous
approvals)
• Duty and role specificity
helps candidates self-sort,
improving the funnel
Tailor our job
descriptions
• Candidates value
feedback, but org policies
differ
• Timeliness
Nudge toward quicker feedback
What we
can do for
Job Seekers
• At least, “you are no
longer under
consideration” is
feedback

24. EARLY FUNNEL FRICTION
• Division of labor: you are
responsible for identifying
great researcher
candidates
• But a recruiter can do the
heavy lifting of finding
and screening them
• Be specific about what
you need and want (e.g.,
must haves and nice to
haves)
Make your recruiter a trusted partner
What we
can do for
Ourselves
• Provide examples
• Share sample interview
questions
• Make sure they know
your process
• Agree on “SLAs”

25. Lack of
structure &
support
HOW DO WE EFFECTIVELY INTERVIEW?
DO WE HAVE EVERYTHING WE NEED?

26. LACK OF STRUCTURE &
SUPPORT
Let’s look at evaluation
methods again
• We have preferred selection methods for UXR
• Phone screens, 1-to-1 with hiring managers,
and portfolio presentations are popular
• How do we know what is effective?

27. LACK OF STRUCTURE &
SUPPORT
On validity
1. How preditive a method is
2. Every selection method has a level of validity
3. Validity is additive
In short: interview processes can be designed
to be more valid by using more evaluations OR
changing the methods we use

28. Structured Interviews
LACK OF STRUCTURE &
SUPPORT
Questions and evaluation criteria are standardized
and predetermined
Usually include behavioral or situational questions,
rubrics, scoring techniques, programmatic process,
etc.
Significantly higher predictive validity compared to
unstructured interviews
BUT our survey revealed a underutilization of
structuring techniques.
Only 50% of HM respondents
have been provided rubrics
Which is similar to other findings from
NN/g…
36% used rubrics
52% had standard questions
22% had a scoring system

29. LACK OF STRUCTURE &
SUPPORT
Work sample tests
● Highest predictive
validity of all assessment
method types*
● Take many forms
REF: Schmidt & Hunter, 1999
Pros Cons
Portfolio Reviews Represents actual work
Showcase of
communication skills
Repurposable
Wide variance between
candidates
May not reveal candidates
actual contribution
Often unstructured
(increased bias and
decreased validity)
Whiteboard
Exercises
Relevant to some
role-specific competencies
Demonstrate thinking
process
Simulate collaborative
environment
Structured and
comparable*
Limited scope decreases
validity
Stress inducing
Take-home tests Relevant to many
role-specific competencies
Work at your own pace
Structured and
comparable*
Widely disliked,
Time investment isn’t
transferable

30. TAKEAWAYS
Follow these steps to make
your interviews structured
1. Perform a job-role analysis & generate task-list
statements
2. Rank tasks in survey
3. Write questions that get to these tasks
4. Create a scoring rubric
5. Pilot test most interviews
6. Create interview guides
7. Select interviewers. & brief them
8. Use 1-hour interviews with candidates
9. 30-min follow-up with interviewers & collect scores
10. Calculate scores
Structure your portfolio
reviews as well
1. Clear instructions on length, time, structure, what
you are evaluating on
2. Who is the audience?
3. Create questions & scoring rubrics

31. LACK OF STRUCTURE &
SUPPORT
Lack of enablement
limits interviewers’
effectiveness
The only enablement materials a majority of
hiring managers had access to were interview
guideline docs (60%) and job rubrics (50%)
Almost 66% get no training at all
13% didn’t have access to any resources
12% are not sure what resources they have

32. Key takeaways
& next steps
THE TLDR + WHAT’S NEXT FOR US

33. Why is there prevalent
dissatisfaction from both sides?
What can we do better?

34. SUMMARY
What we find when we
take a closer look
Some good news…
Hiring managers and job-seekers desires
are more aligned than not.
Early friction
Job-seekers encounter arduous application
process and hiring managers may/may not
have effective help screening candidates.
BUT…
Lack of structuring techniques
We don’t consistently use structure in our
evaluation methods.
Lack of enablement
Interviewers aren’t provided with
enablement and support.

35. TAKEAWAYS
What it means for you
Help job-seekers
Make the application process less arduous for
job-seekers
● Am I asking for only what I need to screen
applicants?
● Does my job description match what the role
entails?
● Do candidates get timely responses and/or
feedback?

36. TAKEAWAYS
What it means for you
Make recruiters your partner
Recruiters as an extension of your team
● Have I been clear about role expectations?
● Do I have clear must-have and ideal
qualifications?
● Have I provided examples?
● Do we have clear working dynamics and
opportunity for feedback?

37. TAKEAWAYS
What it means for you
Introduce structure
Maximize validity and minimize bias
● Am I making use of the best interview
techniques?
● Do we use structure in (almost) every interview
stage?
● How might we introduce structure where there
currently isn’t any?

38. TAKEAWAYS
What it means for you
Seek enablement & resources
Make use of existing resources and seek enablement
to improve interview skills
● Am I fully enabled to be an effective interviewer?
● What homework do I have to improve?
● Are there any resources I haven’t tapped into?

39. NEXT STEPS
What we’re doing next
Continued research
Improvements to UXR-specific best practices
will take continued effort
Some things we’re keen to
address include…
● How can we add structure to
increase the effectiveness of
portfolio reviews?
● How can we create job descriptions
that help job-seekers decide if it is
right fit for them?
● How can we provide job-seekers
with the feedback they desire?

40. Thomas Stokes
USER EXPERIENCE RESEARCH
CONSULTANT, USERTESTING*
Lawton Pybus
USER EXPERIENCE RESEARCH
MANAGER, USERTESTING*
Thank you

41. Resources &
Write-up
→ TheUXRsAnnotations.Substack.com