In this position paper, we argue that usable privacy and security is a grand challenge that needs more attention from the HCI community. We also discuss benefits to and new challenges for HCI, and use our research experiences to provide a critique of HCI.
3. Everyday Security Problems
Setting File Permissions
• In 2003, one Senate
Judiciary staffer found
that files for that
subcommittee were
readable to all users,
rather than just to
Democrats or
Republicans
See Reeder et al CHI 2008
5. Costs of Unusable Privacy & Security High
• People not updating
software with patches
-> Spyware, viruses, worms
• Too many passwords!!!
-> Easy to guess, and
wasted time resetting them
• Hard to configure systems
-> WiFi boxes returned
-> Misconfigured firewalls
• Ubicomp sensing systems
scare a lot of people
-> Less potential adoption
6. Usable Privacy and Security
“Give end-users security controls they can
understand and privacy they can control for
the dynamic, pervasive computing environments
of the future.”
- Grand Challenges in Information Security & Assurance
Computing Research Association (2003)
More research needed on how “cultural and social
influences can affect how people use computers
and electronic information in ways that increase
the risk of cybersecurity breaches.”
- Grand Challenges for Engineering
National Academy of Engineering (2008)
7. Talk Outline
Why Usable Privacy and Security
Highlights: My Experiences with Anti-Phishing
Open Challenges in Usable Privacy and Security
A Lens for Critiquing HCI
10. Phishing is a Plague on the Internet
• Estimated ~$3b direct losses a year
– Does not include damage to reputation, lost sales, etc
– Does not include response costs (call centers, recovery)
– Rapidly growing
• Spear-phishing and whaling attacks escalating
11.
12. Phishing Becoming Pervasive
• Stealing corporate secrets
• Damaging national security
• Targeting:
– universities
– Online social networking sites (Facebook, MySpace)
– Social media (Twitter, World of Warcraft)
13. Project: Supporting Trust Decisions
• Goal: help people make better online trust decisions
– Specifically in context of anti-phishing
• Large multi-disciplinary team project at CMU
– Economics, public policy, computer security,
social and decision sciences, human-computer interaction,
machine learning, e-commerce
14. Our Multi-Pronged Approach
• Human side
– Interviews and surveys to understand decision-making
– PhishGuru embedded training
– Anti-Phishing Phil game
– Understanding effectiveness of browser warnings
• Computer side
– PILFER email anti-phishing filter
– CANTINA web anti-phishing algorithm
– Machine learning of blacklists
– Social web + machine learning to combat scams
Automate where possible,
support where necessary
15. Impact of Our Work
• Game teaching people about phish played 100k
times, featured in over 20 media articles
• Study on browser warnings -> Internet Explorer 8
• Our filter is labeling several million emails per day
• Our evaluation of anti-phishing toolbars cited by
several companies, presented to Anti-Phishing
Working Group (APWG)
• PhishGuru embedded training undergone field trials at
three companies, variant in use by large email
provider, and used in APWG’s takedown page
16. Outline
• Human side
– Interviews and surveys to understand decision-making
– PhishGuru embedded training
– Anti-Phishing Phil game
– Understanding effectiveness of browser warnings
How to train people not to fall for phish?
17. PhishGuru Embedded Training
• A lot of training materials are boring and ignored
• Can we “train” people during their normal use of
email to avoid phishing attacks?
– Periodically, people get sent a training email by admins
– Training email looks same as a phishing attack
– If person falls for it, intervention warns and highlights
what cues to look for in succinct and engaging format
20. Everyday Privacy and Security Problem
Learning science principles
• Learning by Doing
• Immediate feedback
• Conceptual-Procedural Knowledge
21. Evaluation of PhishGuru
• Is embedded training effective? Yes!
– Study 1: Lab study, 30 participants
– Study 2: Lab study, 42 participants
– Study 3: Field evaluation at company, ~300 participants
– Study 4: Ongoing at CMU, ~500 participants
• In first study, examined what kind of intervention
– Comic strip telling a story most effective
• Will highlight study #2 in next slides
P. Kumaraguru et al. Protecting People from Phishing: The Design
and Evaluation of an Embedded Training Email System. CHI 2007.
P. Kumaraguru et al. Getting Users to Pay Attention to Anti-Phishing
Education: Evaluation of Retention and Transfer. eCrime 2007.
22. Study #2
• Questions:
– Have to fall for phishing email to be effective?
– How well do people retain knowledge?
• Experimental protocol
– Role play as Bobby Smith at Cognix Inc, go thru 16 emails
to study how people read email
• Embedded condition means have to fall for our email
• Non-embedded means we just send the comic strip
• Suspicion means got a warning about phish from friend
• Control means they got no warnings or training
– Also had people come back after 1 week
23.
24. Results of Evaluation #2
• Have to fall for phishing email to be effective?
• How well do people retain knowledge after a week?
25. Results of Evaluation #2
• Have to fall for phishing email to be effective?
• How well do people retain knowledge after a week?
26. Results of Evaluation #2
• Have to fall for phishing email to be effective?
• How well do people retain knowledge after a week?
27. Discussion of PhishGuru
• Act of falling for phish is teachable moment
– Just sending intervention not effective
• PhishGuru can teach people to identify phish better
– People retain the knowledge
– People aren’t resentful, many happy to have learned
• 68 out of 85 surveyed said they recommend CMU
continue doing this sort of training in future
• “I really liked the idea of sending CMU students fake
phishing emails and then saying to them, essentially,
HEY! You could've just gotten scammed! You should
be more careful -- here's how....”
28. APWG Landing Page
• CMU helped Anti-Phishing Working Group develop
landing page for phishing sites taken down
– Already in use by several takedown companies
– Seen by 31,000 people already in past 4 months
29. Anti-Phishing Phil
• A game to teach people not to fall for phish
– Embedded training about email, this game about web browser
– Also based on learning science principles
• Goals
– How to parse URLs
– Where to look for URLs
– Use search engines for help
• Try the game!
– Search for “phishing game”
S. Sheng et al. Anti-Phishing Phil: The Design and Evaluation of a
Game That Teaches People Not to Fall for Phish. In Proceedings of
the 2007 Symposium on Usable Privacy and Security, Pittsburgh, PA,
July 18-20, 2007.
36. Evaluation of Anti-Phishing Phil
• Is Phil effective?
• Study 1: 56 people in lab study
• Study 2: 4517 people in field trial
• Brief results of Study 1
– Phil about as effective in helping people detect phishing
web sites as paying people to read training material
– But Phil has significantly fewer false positives overall
• Suggests that existing training material making people
paranoid about phish rather than differentiating
37. Evaluation of Anti-Phishing Phil
• Study 2: 4517 participants in field trial
– Randomly selected from 80000 people
• Conditions
– Control: Label 12 sites then play game
– Game: Label 6 sites, play game, then label 6 more,
then after 7 days, label 6 more (18 total)
• Participants
– 2021 people in game condition, 674 did retention portion
38. Anti-Phishing Phil: Study 2
• Novices showed most improvement in false negatives
(calling phish legitimate)
40. Outline
• Human side
– Interviews to understand decision-making
– PhishGuru embedded training
– Anti-Phishing Phil game
– Understanding effectiveness of browser warnings
Do people see, understand,
and believe web browser warnings?
44. How Effective are these Warnings?
• Tested four conditions
– FireFox Active Block
– IE Active Block
– IE Passive Warning
– Control (no warnings or blocks)
• “Shopping Study”
– Setup some fake phishing pages and added to blacklists
– We phished users after purchases (2 phish/user)
– Real email accounts and personal information
S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An
Empirical Study of the Effectiveness of Web Browser Phishing
Warnings. CHI 2008.
45. How Effective are these Warnings?
Almost everyone clicked, even those
with technical backgrounds
47. Discussion of Phish Warnings
• Nearly everyone will fall for highly contextual phish
• Passive IE warning failed for many reasons
– Didn’t interrupt the main task
– Slow to appear (up to 5 seconds)
– Not clear what the right action was
– Looked too much like other ignorable warnings (habituation)
– Bug in implementation, any keystroke dismisses
49. Discussion of Phish Warnings
• Active IE warnings
– Most saw but did not believe it
• “Since it gave me the option of still proceeding to the
website, I figured it couldn’t be that bad”
– Some element of habituation (looks like other warnings)
– Saw two pathological cases
52. A Science of
Warnings
• See the warning?
• Understand?
• Believe it?
• Motivated?
• Can and will act?
• Refining this model for
computer warnings
53. Talk Outline
Why Usable Privacy and Security
Highlights: My Experiences with Anti-Phishing
Open Challenges in Usable Privacy and Security
A Lens for Critiquing HCI
54. Helping End-Users Cope
• Personal info fragmented across devices and
services
– Each with different UIs, notifications, policies
• More and more information being collected
– Surveillance in workplace and public places,
search engines, ubicomp sensors, etc
• Better division of labor for privacy and security?
– Think email spam: ISP, local sysadmin, email client, user
• Lots of ideas in literature, when to use what?
– Rules, ambiguity, translucency, deniability, invisible,
optimistic vs pessimistic privacy and security
– Is there really such a thing as informed consent?
55.
56. Understanding Attitudes and Behaviors
• Science of warnings
• Decision making / Behavioral economics
– I just got a dancing bear in email? I really want to see it now!
– vs unknown probability in future of unknown level of harm
• How (and why) attitudes and behaviors change
over time regarding privacy
– Cameras and phones, RFIDs and sensors in future
– Food for thought: Facebook Newsfeed
• Same info as before but easier -> huge protest
• Facebook put in “privacy placebos”, waited a while
• Barely a peep about Newsfeed privacy today,
probably increased utility and popularity of Facebook
57. Helping Organizations Cope
• How to train organizations regarding security?
– Social engineering and Insider threat, b/c no defenses today
• Better tools for helping organizations maintain
privacy of consumer data?
– Tools to help comply with privacy policies and laws
• How to get people to share more personal info,
but also feel safer about who it is shared with?
– Too much privacy can harm adoption of system
– Caller ID example, People Finder example
– Privacy corollary to Grudin’s law: when those who share
personal information do not benefit in proportion to the
perceived risks, the technology is likely to fail
58. Toolbox Perspective
D
esign
Prototype
Evaluate
• Design
– Better models of individuals and organizations
• Science of warnings (perception, attention, motivation)
– Better design patterns for usable privacy and security
• Evaluate
– Better methods for realistic evaluations
• Conventional HCI does not assume
intelligent and active adversary
• Big brother vs Little Sister adversaries
– Discount usability as well
• Heuristic eval, cognitive walkthru, etc
59. Talk Outline
Why Usable Privacy and Security
Highlights: My Experiences with Anti-Phishing
Open Challenges in Usable Privacy and Security
A Lens for Critiquing HCI
60. Usable Privacy & Security is Good for HCI
• Usable privacy and security can increase
perceived relevance of HCI
– Our usable privacy and security course has introduced many
people to HCI, who would not normally take such a course
– Also easy to argue that privacy and security are critical
to companies and national security
– Possible strategy: more bridges to other national priorities
• Security, electrical grid, emergency response,
health care, developing countries
• Things that we can pinpoint costing $billions that have
HCI failures
61. Thoughts from Working on Startup
• One of my motivations for startup was that I felt too
many CHI papers ended up only as CHI papers
– Not as much impact on products and practice as desired
– Even within the conventional wisdom of 15 years
– Compare #startups in HCI vs DB / Systems / Networking
– Compare $$ going to HCI, HCI is underperforming
63. Thoughts from Working on Startup
• Business professor: feature, product, business?
– Is it a big enough problem that people would pay money?
– Easier to get small inoffensive paper in than big paper
• Incentive is for researchers to aim for smaller papers
• More body of knowledge makes narrow papers easier
– Note: this doesn’t measure quality of the science
• Big ideas need love too!
– Put a cap on “interaction technique” papers
– Put a cap on “last 10%” papers
– Special sessions at conferences for big ideas
• We need to encourage more things like SketchPad,
Memex, Engelbart’s NLS, without sacrificing quality
– More alcohol + rump sessions on outrageous ideas
at UIST and CSCW
64. Summary
• Usable Privacy and Security critical to continue
getting benefits of Information Communication Tech
• Whirlwind tour of our work on anti-phishing
– Effective training mechanisms, warnings
• Fertile research areas for HCI
– Helping end-users, attitudes and behaviors,
helping organizations, toolbox
• Improving the HCI community
– Bridges, tech adoption
65. Acknowledgments
• Alessandro Acquisti
• Lorrie Cranor
• Sven Dietrich
• Julie Downs
• Mandy Holbrook
• Norman Sadeh
• Anthony Tomasic
• Umut Topkara
Supported by NSF, ARO, CyLab, Portugal Telecom
• Serge Egelman
• Ian Fette
• Ponnurangam
Kumaraguru
• Bryant Magnien
• Elizabeth Nunge
• Yong Rhee
• Steve Sheng
• Yue Zhang
66. HCI Folk and Security and Privacy Folk
Have Much in Common
• Both require holistic view of entire system
– Bad usability in one small part can ruin interaction
– Bad security in one small part can compromise entire system
• Both lament being done at end of design process
– “Can’t just sprinkle security dust on a system”
• Both lack widely accepted metrics
– Outside of encryption, security does not have
good ways of demonstrating something is secure
National Academy of Engineering (NAE) included “secure cyberspace” in their 2008 Grand Challenges for Engineering, arguing that more research is needed on the psychology of computer users, how people interact with their computers, and how “cultural and social influences can affect how people use computers and electronic information in ways that increase the risk of cybersecurity breaches.”
Biz week http://www.businessweek.com/magazine/content/08_16/b4080032218430.htm
The e-mail message addressed to a Booz Allen Hamilton executive was mundane—a shopping list sent over by the Pentagon of weaponry India wanted to buy. But the missive turned out to be a brilliant fake. Lurking beneath the description of aircraft, engines, and radar equipment was an insidious piece of computer code known as "Poison Ivy" designed to suck sensitive data out of the $4 billion consulting firm's computer network.
The Pentagon hadn't sent the e-mail at all. Its origin is unknown, but the message traveled through Korea on its way to Booz Allen. Its authors knew enough about the "sender" and "recipient" to craft a message unlikely to arouse suspicion. Had the Booz Allen executive clicked on the attachment, his every keystroke would have been reported back to a mysterious master at the Internet address cybersyndrome.3322.org, which is registered through an obscure company headquartered on the banks of China's Yangtze River.
PK
Steve
Serge
Thus far, our work has generated a great deal of interest and collaboration from a number of partners. Our automated email filter is undergoing a field trial at ****** main email servers, where it is labeling several million emails per day.
Our research evaluating anti-phishing toolbars has been cited by several companies, with ongoing evaluations being presented to the Anti-Phishing Working Group, a consortium of companies “committed to wiping out Internet scams and fraud.”
Design suggestions from our studies to understand browser warnings have been incorporated into the latest version of Microsoft’s Internet Explorer 8.
PhishGuru’s methodology of sending fake phishing emails to train individuals has undergone field trials at three different companies, and been cited by two different companies trying to commercialize the work. PhishGuru’s training materials have also been adopted by APWG on their landing page, a page that ISPs and web sites can show after taking down a phishing web site. Anti-Phishing Phil has been played by over 100,000 people, licensed by two companies, demoed at many security days meant to teach people about good security practices, and translated into Portuguese with several more translations underway. Finally, our group is commercializing all of this work through a startup we have founded, named Wombat Security Technologies.
S. Sheng, B. Magnien, P. Kumaraguru, A. Acquisti, L. Cranor, J. Hong, and E. Nunge. Anti-Phishing Phil: The Design and Evaluation of a Game That Teaches People Not to Fall for Phish. In Proceedings of the 2007 Symposium On Usable Privacy and Security, Pittsburgh, PA, July 18-20, 2007.
Phil needs to score 6 / 8 to move on to the next rounds, and the end of the round, phil got a chance to reflect what he missed.
In between rounds, we also have short tutorials to teach Phil better strategies to identify phishing. In this example, Phil’s father teaches Phil how to use a search engine.
S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. CHI 2008.