Designing for Hope: A Review of 4 Career Websites

DESIGNING FOR HOPE
A REVIEW OF FOUR CAREER WEBSITES

UPA BOSTON MINI-CONFERENCE
JUNE 9, 2010

Presented by:
Niyati Gupta & Michelle Kwasny

Agenda

!   Job Seeking, Emotions, & User Experience
!   Project Overview
!   Project Stages
>  Overall Job Search Affect
>  Positive Affect
>  Design
>  Design & Affect
!   Overall Findings & Next Steps

2

JOB SEEKING,
EMOTIONS &
USER EXPERIENCE

Short Exercise

Think of the last time you were job hunting,
By choice or by necessity…

What emotions did you experience?

4

Emotions
cheerful sad active angry at self disgusted
calm guilty enthusiastic attentive afraid
joyful downhearted bashful tired nervous
sheepish sluggish amazed lonely distressed
daring shaky sleepy blameworthy surprised
happy excited determined strong timid
hostile frightened scornful alone proud
astonished relaxed alert jittery interested
irritable upset lively loathing delighted
angry ashamed confident inspired bold
at ease energetic fearless blue scared
concentrating disgusted shy drowsy dissatisfied
with self with self

* Emotions taken from the Positive and Negative Affect Schedule – Expanded 5
Form (PANAS-X)

Job Search Feedback

6

Job Search Feedback

7

Job Search Feedback

“Another
day
searching
job

listings...

Another
day
where
I
am
qualified

for
nothing...

Except
maybe
selling

Lay-‐Z-‐Boy
recliners
on

commission.

I
hate
myself. ”

8

Job Search Feedback

I
have
been
looking

for
3
months
now
and

It
is
TOUGH
out

I
just
cannot
find

there.

anything.

I’m

really
frustrated!

The
job
market
is
Looking
for
a

not
what
it
used
to
specific
field
and

be
–
with
the
just
can’t
find

economy
the
way
it
those
jobs
online.

is
and
all!
What
do
I
do
now?

9

Emotions and User Experience

>  “It is impossible to act or think without engaging, at least
unconsciously, our emotions.” (Mehrabian & Russell, 1974)

>  “For designers it is important to design products that ‘fit’ the
emotions of the users, that is, products that elicit the emotions that
the user would like to experience.” (Desmet & Overbeeke, 2001)

>  “While research concludes that emotion is a fundamental
component of being human, the HCI community -- a community
which studies the interaction between humans and computers --
has mostly overlooked emotion as a component of user
experience.” (Agarwal & Meyer, 2009)

10

Research Questions
Job Search Affect
1

What emotions do users bring with them to career
websites?
Design
2

What are career websites doing to ease users’ minds?

Positive Affect
3

What positive affect are users looking for?

Design & Affect
4

Are any of the career websites’ designs/features
successful in building hope & easing frustration?

Research Questions
Job Search Affect
1
1

What emotions do users bring with them to career
websites?
Design
2


Positive Affect
3

What positive affect are users looking for?

Design & Affect
4

Are any of the career websites’ designs/features
successful in building hope & easing frustration?

Method
1
1

!   Collected overall emotions about job search using the
PANAS-X scale (Watson & Clark, 1994)
!   Collected positive job search situations & associated
2

emotions
!   Created a survey
!   Launched on Amazon’s Mechanical Turk
3
4

MTurk Demographics
2009 US Census
1
1

MTurk Abstract
median age 31 (US only) 36
% with college degree 63% (US only) 25%
female 69% (US only) 51%
2

US vs. India 57% vs. 32%

employed full time vs. unemployed 38% vs. 31%

mean annual income ~ $25,000
3

average time using mturk 1-6 mo

activity per week 1-5 hrs per week
earnings per week $1-$5 per week
4

“While the MTurk population may perhaps be representative of the U.S.
internet-using population, it cannot truly be seen to be a microcosm of the
country as a whole.” (Ross, Zaldivar, Irani, &Tomlinson, 2010)

* Results above from Ross, Zaldivar, Irani & Tomlinson, 2010;
Consistent with Ipeirotis, 2010.

Welcome Survey
Background Screening Qs
Introduction

Job Search Affect
(PANAS-X)

Positive Affect
(Positive Experiences)

N=62 Debrief
Within-Subjects Design

Survey Demographics
!  Represented 21 States
AL CT IN MI PA WA
1
1

AZ FL KY NC TN
CA HI LA NY TX
CO IL MA OH VA

!  Are you currently employed?
2

Employed
35%

Unemployed
65%
3

!  How many jobs have you applied to in the past 2 months?
Over 15
jobs
22%
4

1-5
11-15 jobs
jobs
10%
55%

5-10 jobs
13%

PANAS-X Scale
!  Likert-Like Rating scale
1
1

1 2 3 4 5
Very slightly or a little moderately quite a bit extremely
not at all

!  60 emotion words
2

cheerful sad active angry at self
disgusted calm guilty enthusiastic
attentive afraid joyful downhearted
bashful tired nervous sheepish
sluggish amazed lonely distressed
3

daring shaky sleepy blameworthy
surprised happy excited
determined
strong timid hostile
frightened
scornful alone proud
astonished
relaxed alert jittery
irritable upset lively interested
4

delighted angry ashamed loathing
inspired bold at ease confident
fearless blue scared energetic
disgusted with self shy drowsy concentrating

Job Search Affect Question
1
1
2
3
4

PANAS-X Scale
General Dimension Scales
1
1

Negative Affect (10) afraid, scared, nervous, jittery, irritable, hostile, guilty, ashamed, upset, distressed
Positive Affect (10) active, alert, attentive, determined, enthusiastic, excited, inspired, interested, proud,
strong

Basic Negative Emotion Scales
Fear (6) afraid, scared, frightened, nervous, jittery, shaky
2

Hostility (6) angry, hostile, irritable, scornful, disgusted, loathing
Guilt (6) guilty, ashamed, blameworthy, angry at self, disgusted with self, dissatisfied with self
Sadness (5) sad, blue, downhearted, alone, lonely
Basic Positive Emotion Scales
3

Joviality (8) happy, joyful, delighted, cheerful, excited, enthusiastic, lively, energetic
Self-Assurance (6) proud, strong, confident, bold, daring, fearless
Attentiveness (4) alert, attentive, concentrating, determined
Other Affective States
Shyness (4) shy, bashful, sheepish, timid
4

Fatigue (4) sleepy, tired, sluggish, drowsy
Serenity (3) calm, relaxed, at ease
Surprise (3) amazed, surprised, astonished

Job Search Affect Results
Positive Affect
1
1

Positive Affect = active, alert, attentive, determined, enthusiastic, excited,
inspired, interested, proud, strong

Overall Positive and Negative Affect
2

extremely 5

quite a bit 4
3

moderately 3

a little 2 2.97 2.79
2.74 2.61 2.58
2.21
4

Very slightly or
not at all
1
Basic Positive Affect Basic Negative Affect
Combined Unemployed Employed
N=62 N=40 N=22

Positive emotion = ATTENTIVENESS
1
1

Attentiveness= alert, attentive, concentrating, determined

Posi?ve
Emo?ons
during
Job
Search

extremely 5

2

quite a bit 4

3

moderately 3

3.24
3.41

a little 2
3.15

2.72

2.27
2.38
2.20

2.08
1.98

4

Very slightly or 1

not at all
Joviality
Self-‐assurance
A>en?venes

Combined
Unemployed
Employed

N=62 N=40 N=22

Negative emotion: SADNESS
1
1

Sadness: sad, blue, downhearted, alone, lonely

Nega?ve
Emo?ons
during
Job
Search

5

2

extremely

quite a bit 4

moderately 3

3

a little 2
2.95

2.62
2.76
2.37
2.39
2.61
2.63
2.67

2.29
2.15

2.00

1.69

Very slightly or 1

4

not at all
Fear
Hos?lity
Guilt
Sadness

Combined
Unemployed
Employed

N=62 N=40 N=22

Affective State = FATIGUE
1
1

Fatigue= sleepy, tired, sluggish, drowsy

Other
aﬀec?ve
states
during
job
search

extremely 5

2

quite a bit 4

moderately 3

3

a little 2

2.28
2.41
2.06
2.37
2.40
2.33
2.23
2.09
2.47

1.82
1.84
1.81

Very slightly or
1

not at all
4

Shyness
Fa?gue
Serenity
Surprise

Combined
Unemployed
Employed

N=62 N=40 N=22

Positive Affect
1
1

!   Respondents
shared three positive situations that they
experienced in their job search…
2
3
4

Positive Affect
1
1

!   And indicated what emotions they felt when the
positive experiences occurred.
2
3
4

* Emotions taken from the Positive and Negative Affect Schedule – Expanded
Form (PANAS-X)

Positive Affect Results
1
1

“Family
member
took
“Former
employee
had

time
out
and
helped
new
positions
“I
saw
all
of
my

me
rewrite
my
resume
available
that
might
accomplishments”

professionally”
suit
me”

2

Help from family Help from past
& friends colleagues or company Self-Achievements
3

“Application
was

“I
got
an
interview,

responded
to
very

and
it
went
well”

promptly”

4

Getting/acing Good experience with
the interview potential employer

1
1

Posi?ve
emo?ons
and
feelings
based
on
respondent's
posi?ve
experiences

(N=62)

Average No. of Respondents
2

30

23
23

3

14

4

Joviality
A9en:veness
Self-‐Assurance
Serenity

Posi?ve
emo?on
card
groups

1
1

3 Positive emotions = CHEERFUL, HAPPY, INTERESTED

Positive emotions based on positive experiences
2

(N=62)

43 41
No. of Respondents

39 38 38 38
36 34 34
33
28
3

25
21 20 19
16 16 15 14
13 12 12
10 8
4

Positive emotions

Conclusion
!   Job
seekers are slightly more positive than negative
1
1

about job searching

!   When thinking about overall job search, job seekers
are…
2

>  ATTENTIVE
>  SAD
>  FATIGUED
3

!   Whenthinking about positive situations during job
search, job seekers are…
4

>  JOVIAL (cheerful, happy, interested)

Design thoughts
!   Encourage positive emotions
1
1

>  Direct people to the right information at the right time
>  Direct attention to important and relevant information through
visual hierarchy
2

!   Reduce negative emotions
>  Include some self-assuring messages and content
>  Keep task flows short to eliminate fatigue
3
4

Method
Design
1


>  Reviewed four websites – Career Builder, LinkedIn, Simply
2

Hired and Monster

>  Collected features that we hypothesized would elicit positive
and negative emotions in job seekers.
3
4

CareerBuilder
1

It knows what jobs are
near me.
2

Homepage shows me
recent jobs in my
location.
3
4

CareerBuilder
1

I am confused!
2

Search results do not
match search
keywords.
3

Did I enter something
wrong?
4

LinkedIn
1

WOW! someone from
TCS looked at my
2

profile. I know TCS is
hiring!

I like to see who is
3

visiting my profile and
which company they
are working at.
4

LinkedIn
1

I met David at UPA
conference, but I
2

cannot add him!
BUMMER!

I cannot immediately
3

connect to people who
I do not know.
4

SimplyHired
1

Staples is hiring, let
me contact Katelyn as
2

she works there!

I like to see my
LinkedIn and
3

Facebook connections
who work at the
companies that are
hiring.
4

SimplyHired
1

Usability Associate
position at Vistaprint is
2

listed twice here- is it
the same positing or
do they have two
openings?
3

I do not like to see
repeated job postings.
4

Monster
1

After I finish applying
to a particular job,
2

Monster directs me to
related jobs that
interest candidates
like myself.
3

I like to see more
relevant job postings.
4

Monster
1

Which search/browse
jobs option I should
2

use?

Which one will give
me the most relevant
3

search results?
4

Method
1

!   Focused on positive features from each of the four
websites
!   Pleasure Arousal Dominance (PAD) scale to measure
2

emotion elicited from feature/design
!   Created a survey
!   Launched on Amazon’s Mechanical Turk
3
4

Welcome Survey
Background Questions
Introduction

SimplyHired Monster LinkedIn CareerBuilder
“shows connections” “also applied to” “who viewed profile” “jobs on homepage”

Emotional Emotional Emotional Emotional
Response Response Response Response

Positive Designs Only
Within-Subjects Design Debrief
Designs Randomized

“When I
search for a
job, I am
shown who I
know at this
company
based on my
LinkedIn and
Facebook
connections.”

“After I apply to a job, there is a link that says "Candidates for
this job also applied for..." which shows me other jobs I may
be interested in.”

“When I log in, I can see who has viewed my profile.”

“When I go to
the homepage
for the first
time, the
website knows
my location and
shows me job
openings
there.”

Pleasure Arousal Dominance Scale
1

Pleasure Arousal Dominance
unsatisfied - satisfied calm - excited cared for - in control
2

tense - relaxed sleepy - wide awake influenced - influential
melancholic - contented unaroused - aroused submissive - dominant
despairing - hopeful relaxed - stimulated controlled - controlling
annoyed - pleased guided - autonomous
3

unhappy - happy
unfriendly - friendly
4

Survey Demographics
!  24 States Represented
1

AL FL MD MO NU TN
AZ IL ME NC OH TX
CA KY MI NH OR UT
2

CT MA MN NV PA WA

!  Are you currently employed?
3

Unemployed;
31%
4

Employed; 69%

Survey Demographics
How many jobs have you applied to in the last 2 months?
1

! 
Over 15
jobs;
12%

11-15
jobs; 7%
2

1-5 jobs;
48%

6-10
jobs;
33%

Which of the following websites have you used in the past year to
3

! 
look for jobs? (check all that apply)
4

71% 19% Craigslist
55% 50% 14% Government Site
5% Indeed
17% 12%

Monster CareerBuilder LinkedIn SimplyHired Other

Results
Emotional Profiles of Features
1

(N=44)
9
high

8

7
2

6
Average Rating

5

4
3

5.37 5.50 5.39 5.47
3 4.93 5.11 5.17 5.01 5.12
4.50 4.60 4.62

2

low 1
4

"shows connections" "also applied to" "who viewed profile" "jobs on homepage"


Results
PAD by Feature
(N=44)
1

9
high

8

7
2

6
Average Rating

5

4
3

5.50 5.37 5.39 5.47
3 4.93 5.17 5.12 5.11 5.01
4.50 4.60 4.62

2

low 1
4



Results
Pleasure Differentials by Feature
1

(N=44)
9
high

8

7
2

6
Average Rating

5

4
3

3

2

low 1
4

unsatisfied - tense - melancholic - despairing - annoyed - unhappy - unfriendly -
satisfied relaxed contented hopeful pleased happy friendly


Results
Arousal Differentials by Feature
1

(N=44)
9
high

8

7
2

6
Average Rating

5

4
3

3

2
4

low 1
calm - excited sleepy - wide awake unaroused - aroused relaxed - stimulated


Results
Dominance Differentials by Feature
1

(N=44)
9
high

8

7
2

6
Average Rating

5

4
3

3

2

low 1
4

cared for - in influenced - submissive - controlled - guided -
control influential dominant controlling autonomous


Conclusion
1
1

!   Emotional profiles of all positive features were very
similar
2

!   Positive
features elicited slightly more AROUSAL than
PLEASURE, and slightly more PLEASURE than
DOMINANCE
3

!   Emotions are difficult to elicit without interaction
4

Discussion
1

!   No meaningful differences could mean…
>  Job seekers cannot relay their emotions accurately after only
viewing the feature
2

>  The designs/features are very similar
>  It is difficult to evoke an intense positive reaction

!   In the future we should…
3

>  Have job seekers interact with the website in context
>  Look at designs that have more differences
>  Look at negative features
4

OVERALL FINDINGS
& NEXT STEPS

Overall Findings
Job Search Affect & Positive Affect
!   Overall,
job seekers show slightly more positive affect
than negative affect

!   Job seekers are
>  Attentive
>  Sad
>  Fatigued

!   Positive experiences make job seekers
>  Jovial (Cheerful, Happy, Interested)

Overall Findings
Design & Affect
!   Emotional profiles of all positive features were very
similar

!   Positive
features elicited slightly more AROUSAL than
PLEASURE, and slightly more PLEASURE than
DOMINANCE

!   Emotions are difficult to elicit without interaction

Next Steps

!   Conduct additional research on Design & Emotion
>  Have job seekers interact with the website in context
>  Look at designs that have more differences
>  Look at negative features
!   Consider personality as a factor
!   Investigate the effect of emotion in behavioral
response
>  Approach-Avoidance
!   Look at non-verbal emotional scales
!   Investigate the social component of job seeking and
emotions

Citations

Agarwal, A., and Meyer, A. Beyond usability: evaluating emotional response as an
integral part of the user experience. In: Proceedings of ACM CHI 2009 (Boston
USA, May 2009), ACM Press, 2919-2930.
Desmet, P.M.A., Overbeeke, C.J., & Tax, S.J.E.T. (2001). Designing products with
added emotional value; development and application of an approach for research
through design. The Design Journal, 4(1), 32-47.
Mehrabian, A., & Russell, J.A. An approach to environmental psychology. M.I.T.
Press, Cambridge, MA, USA, 1974.
Ipeirotis, Panagiotis G. (2010). “Demographics of Mechanical Turk,” New York
University Working Paper No. CeDER-10-01, 2010.
http://hdl.handle.net/2451/29585, (accessed May 2010).
Ross, J., Irani, I., Silberman, M. Six, Zaldivar, A., and Tomlinson, B. (2010). "Who are
the Crowdworkers?: Shifting Demographics in Amazon Mechanical Turk". In: CHI
EA 2010. (2863-2872).
Watson, D. & Clark, L.A, (1994). “The PANAS-X Manual for the Positive and
Negative Affect Schedule – Expanded Form”. Retrieved from
http://www.psychology.uiowa.edu/, (accessed May 2010).

THANK YOU
Niyati.Gupta@Monster.com
Michelle.Kwasny@Monster.com

ACKNOWLEDGEMENTS
We would like to acknowledge Denise Nangle, Sandra Teare, the Monster UX Team,
Yanling Zhang, and Ryan Powell for their support with this project.

Positive emotion = ATTENTIVENESS: ATTENTIVE
1
1

Average
Ra?ngs
for
Emo?ons
related
to
A>en?veness

extremely 5

2

quite a bit 4

moderately 3

3

a little 2
3.68

3.50
3.64

3.23
3.40
3.35
3.20

3.13
3.08
2.97
3.09

2.90

Very slightly or 1

not at all
4

0

alert
a9en:ve
concentra:ng
determined

Combined
Unemployed
Employed

Negative emotion: SADNESS: ALONE
1
1

Average
Ra?ngs
for
Emo?ons
related
to
Sadness

extremely 5

2

quite a bit 4

moderately 3

3

a little 2

3.18
3.05

2.81
2.68
2.73
2.85
3.05
2.80

Very slightly or 2.39
2.5
2.55

not at all 1
2.14
2.14
2.09

1.86

4

0

sad
blue
downhearted
alone
lonely

Combined
Unemployed
Employed

Affective State = FATIGUE: TIRED
1
1

Average
Ra?ngs
for
Emo?ons
related
to
Fa?gue

extremely 5

2

quite a bit 4

moderately 3

3

a little 2

3.06
3.08
3.05

2.18
2.25
2.35
2.38
2.32

Very slightly or 1
2.05
1.90
1.90
1.91

not at all
4

0

sleepy
:red
sluggish
drowsy

Combined
Unemployed
Employed

Designing for Hope: A Review of 4 Career Websites

Recommended

Recommended

More Related Content

Similar to Designing for Hope: A Review of 4 Career Websites

Similar to Designing for Hope: A Review of 4 Career Websites (20)

Recently uploaded

Recently uploaded (20)

Designing for Hope: A Review of 4 Career Websites