Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO Forum, in Pittsburgh July 2013

©2009CarnegieMellonUniversity:1
Leveraging Human Factors
for Effective Security Training
ISSA CISO Forum 2013
Jason Hong
Associate Professor
Carnegie Mellon University
CTO and Co-Founder
Wombat Security Technologies

Interactions Can Be Successful

Interactions Can Also Fail

Human
Robot
Interaction
Social
Web
Cognitive
Tutors
New
Interaction
Techniques

Human Factors Issues
in Cybersecurity
• Studying human factors issues in
cybersecurity for 9+ years
– Why do people fall for phishing scams?
– How can we train people in a manner that
is fun, effective, and measurable?
– How can we build better user interfaces
and security warnings?

Influenced
MSIE
Warnings
Wombat
Security
Technologies
SciAm
&
CACM
APWG
Landing
Page

Today’s Talk
• Discuss some of our research findings
– Better user interfaces for avoiding attacks
– Teaching people effectively
• A model for thinking about
cybersecurity awareness and education
• Three cross-cutting strategies for
effective cybersecurity training

• Every browser now has basic
anti-phishing detection built in
• Are these user interfaces effective?
• Our 2008 study on warnings
• And what does it mean for training?

Screenshots
Internet Explorer 7 – Passive Warning

Screenshots
Internet Explorer 7 – Active Block

Screenshots
Mozilla Firefox – Active Block

Tested These Four Interfaces
• Shopping study
– IE Passive Warning
– IE Active Block
– FireFox Active Block
– Control (no warnings or blocks)
• Overall results
– Passive warning completely ineffective
– About half of people still fell for IE warning
– No one fell for FireFox warning
S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An Empirical
Study of the Effectiveness of Web Browser Phishing Warnings. CHI 2008.

Analyzing the Results
• C-HIP model for
real-world warnings
– See the warning?
– Understand it?
– Believe it?
– Motivated?
– Can and will act?

Screenshots
• MSIE 7 Active Block
• Half still fell for phish
despite the warning (?)
• Habituation (similar warnings)
• Two pathological cases
• Most saw the warning, but
many did not believe it
• “Since it gave me the option of
still proceeding to the website, I
figured it couldn’t be that bad”

Two Takeaways
• Better interfaces can dramatically
reduce security problems
• Model for warnings also relevant for
cybersecurity in general
– See the warning?
– Understand it?
– Believe it?
– Motivated?
– Can and will act?

Basis for the Cybersecurity
Training Model
Aware of the security issue?
Knowledge of what
actions to take?
Motivated to act?

Cybersecurity Training Model
Example: Passwords
Knowledge of what
actions to take?
Motivated to act?
Don’t reuse passwords
Common security risk
How to change
Secure and memorable
Stories of breaches
Require changes

Example: Smartphone Security
Knowledge of what
actions to take?
Motivated to act?
Have a PIN on device
(about 50% don’t)
How to do it on device
Avoiding bad PINs
At end of training
Start with upper mgt

Knowledge of what
actions to take?
Motivated to act?
• Most training
starts with
awareness
• Unfortunately,
most training
also stops with
awareness

Most Posters not Effective
http://mindfulsecurity.com/2009/09/19/free-threats-security-awareness-posters/

• Effective training
needs to address
all these steps
• Strategy #1
– Foster better
mental models
Knowledge of what
actions to take?
Motivated to act?

Mental Models
• People inevitably build
models of how things work
– Ex. me and my car
– Ex. children & computers
– Ex. maps of New York
and Boston

Mental Models Impact Security
• Ex. visibility in Facebook
– Suppose you have a private
Facebook album, but tag
someone. Can that person
see it or not?
• Ex. app stores
– All apps are vetted by
Google, so they are all
safe to download. Correct?

So, we just have to foster
the right mental model
and then we’re done?

There’s not Always a “Right”
Mental Model
• Experts can disagree on
• We asked 10 experts about malware

Incomplete Mental Models
Can Still Be Useful
• Rick Wash’s work on folk models
– Hackers are technical geeks that do it for fun
– Hackers seek personal info
– Hackers only target big fish
– Hackers only look for big databases of info
– People took different precautions
• Incomplete models may still be an
improvement over current state
– Degrees of better and worse

Cybersecurity Training
• Cybersecurity
education should
foster better
mental models
– Awareness
– Who and why?
– Fixing common
misconceptions
– Actionable items
Knowledge of what
actions to take?
Motivated to act?

Case Study: Phishing Attacks
• Interviewed 40 people as part of an
“email study” (Downs et al, SOUPS 2006)
• Only 55% of participants said they had
ever noticed an unexpected or strange-
looking URL
– Most did not consider them to be suspicious

Example: Phishing Attacks
• 55% of participants reported being
cautious when email asks for sensitive
financial info
– But very few reported being suspicious of
email asking for passwords
• Knowledge of financial phish reduced
likelihood of falling for these scams
– But did not transfer to other scams, such
as an amazon.com password phish

• Strategy #2: Tailor delivery of training
for your audience
– We’re all busy
– A lot of training is boring (wall of text)
– Little chance to test what you just learned
Teachable Moments Micro-Games

PhishGuru Simulated Phishing
• Create teachable moments thru
simulated phishing emails
• If recipient falls for it, show intervention
that teaches what cues to look for
– Useful for people who don’t know what
they don’t know (low awareness)

Subject: Revision to Your Amazon.com Information

Subject: Revision to Your Amazon.com Information
Please login and enter your information

• Why am I seeing this?
• How was I tricked?
• How to protect myself?
• Who and how?

Evaluation of PhishGuru
• Is simulated phishing effective?
– We’ve done 4 peer-reviewed studies
showing embedded training works well
– About 50% decrease in falling for phish
after one training
P. Kumaraguru et al. Protecting People from Phishing:
The Design and Evaluation of an Embedded Training
Email System. CHI 2007.
P. Kumaraguru et al. School of Phish: A Real-Word
Evaluation of Anti-Phishing Training. SOUPS 2009.

Results of One Study
• Tested 500+ people in one month
– 1 simulated phish at beginning of month,
testing done at end of month
• ~50% reduction in falling for phish
– 68 out of 85 surveyed recommend continuing
doing this sort of training in the future
“I really liked the idea of sending [org] fake
phishing emails and then saying to
them, essentially, HEY! You could've just
gotten scammed! You should be more careful –
here's how...”

• Strategy #2: Tailor delivery of training
for audience
– Create “teachable moments”
– Micro-games for training
– Just sending training via email (ineffective)
– Attending all day classes (boring, can’t test
skills)
– Watching videos (can’t test skills)

Strategy #3: Use Concepts
from Learning Science
• Area of research examining learning,
retention, and transfer of skills
• Example principles
– Learning by doing
– Immediate feedback
– Conceptual-procedural
– Reflection
– … many others

What About Motivation?
Knowledge of what
actions to take?
Motivated to act?
• Training also
needs to
address
motivation
• Open question
as to best
approaches for
cybersecurity

What Motivates People?
• Extrinsic factors (outside factors)
– Pay
– Privilege, Reputation
– Certificates, trophies
– Punishment
• Can’t just slap it on,
has to be appropriate
and thought through

• Intrinsic value of task
– Fun
– Curiosity
– Challenge, mastery
• Same as before, can’t just slap it on
• Cybersecurity and intrinsic motivation
may be hard to reconcile
• Intrinsic and extrinsic may conflict

• Social factors
– Reciprocity (you help me, I help you)
– Altruism
– Norms
– Social proof
– Identification with group
• Large untapped potential, but open
question as to how to best leverage

Energy Consumption

Summary
• Better user interfaces
• Cybersecurity training
model
– Better mental models
– Tailor delivery
– Learning science
• Lots of opportunities
for motivating people,
but still open question

Thanks, where can
I learn more?
Find more at
wombatsecurity.com
jasonh@cs.cmu.edu

Timing Matters Too
• Teachable moments
• Right after training
• Repeat enough times, becomes habit
(don’t have to appeal directly to
individual motivation anymore)

Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO Forum, in Pittsburgh July 2013

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO Forum, in Pittsburgh July 2013

Similar to Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO Forum, in Pittsburgh July 2013 (20)

Recently uploaded

Recently uploaded (20)

Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO Forum, in Pittsburgh July 2013

Editor's Notes