0
Jason Hong, PhD
Carnegie Mellon University
Wombat Security Technologies
Teaching Johnny Not
to Fall for Phish
Everyday Privacy and Security Problem
This entire process
known as phishing
How Bad Is Phishing?
Consumer Perspective
• Estimated ~0.5% of Internet users per year
fall for phishing attacks
• Conserv...
How Bad Is Phishing?
Perspective of Corporations
• Direct damage
– Loss of sensitive customer data
How Bad Is Phishing?
Perspective of Corporations
• Direct damage
– Loss of sensitive customer data
– Loss of intellectual ...
How Bad Is Phishing?
Perspective of Corporations
• Direct damage
– Loss of sensitive customer data
– Loss of intellectual ...
General Patton is retiring next week,
click here to say whether you can
attend his retirement party
Phishing Increasing in...
Phishing Increasing in Sophistication
Targeting Your Organization
• Around 40% of people in our experiments at CMU
would f...
Phishing Increasing in Sophistication
Targeting You Specifically
• Type #2 – Uses info specifically about you
– Social phi...
Phishing Increasing in Sophistication
Targeting You Specifically
Here’s a video I took of your
poster presentation.
Phishing Increasing in Sophistication
Targeting You Specifically
• Type #2 – Uses info specifically about you
– Whaling – ...
Phishing Increasing in Sophistication
Combination with Malware
• Malware and phishing are becoming combined
– Poisoned att...
Protecting People from Phishing
• Human side
– Interviews and surveys to understand decision-making
– PhishGuru embedded t...
Results of Our Research
• Startup
– Customers of micro-games featured include
governments, financials, universities
– Our ...
Outline of Rest of Talk
• Rest of talk will focus on educating end-users
• PhishGuru embedded training
• Anti-Phishing Phi...
User Education is Challenging
• Users are not motivated to learn about security
• Security is a secondary task
• Difficult...
But Actually, Users Are Trainable
• Our research demonstrates that users can learn
techniques to protect themselves from p...
How Do We Get People Trained?
• Solution
– Find “teachable moments”: PhishGuru
– Make training fun: Anti-Phishing Phil,
An...
PhishGuru Embedded Training
• Send emails that look like a phishing attack
• If recipient falls for it, show intervention ...
Subject: Revision to Your Amazon.com InformationSubject: Revision to Your Amazon.com Information
Subject: Revision to Your Amazon.com InformationSubject: Revision to Your Amazon.com Information
Please login and enter yo...
Evaluation of PhishGuru
• Is embedded training effective?
– Study 1: Lab study, 30 participants
– Study 2: Lab study, 42 p...
Study #4 at CMU
• Investigate effectiveness and retention of
training after 1 week, 2 weeks, and 4 weeks
• Compare effecti...
Study design
• Sent email to all CMU students, faculty
and staff to recruit participants (opt-in)
• 515 participants in th...
Effect of PhishGuru Training
Condition N % who clicked
on Day 0
% who
clicked on
Day 28
Control 172 52.3 44.2
Trained 343 ...
Discussion of PhishGuru
• PhishGuru can teach people to identify phish better
– People retain the knowledge
• People train...
APWG Landing Page
• CMU and Wombat helped Anti-Phishing Working
Group develop landing page for taken down sites
– Already ...
Anti-Phishing Phil
• A micro-game to teach people not to fall for phish
– PhishGuru about email, this game about web brows...
Anti-Phishing Phil
Evaluation of Anti-Phishing Phil
• Is Phil effective? Yes!
– Study 1: 56 people in lab study
– Study 2: 4517 people in fie...
Evaluation of Anti-Phishing Phil
• Study 2: 4517 participants in field trial
– Randomly selected from 80000 people
• Condi...
Anti-Phishing Phil: Study 2
• Novices showed most improvement in false negatives
(calling phish legitimate)
Anti-Phishing Phil: Study 2
• Improvement all around for false positives
Anti-Phishing Phyllis
• New micro-game just released by Wombat Security
• Focuses on teaching people about what cues
to lo...
Summary
• Phishing is already a plague on the Internet
– Seriously affects consumers, businesses, governments
– Criminals ...
Acknowledgments
• Ponnurangam Kumaraguru
• Steve Sheng
• Lorrie Cranor
• Norman Sadeh
Screenshots
Internet Explorer – Passive Warning
Screenshots
Internet Explorer – Active Block
Screenshots
Mozilla FireFox – Active Block
How Effective are these Warnings?
• Tested four conditions
– FireFox Active Block
– IE Active Block
– IE Passive Warning
–...
How Effective are these Warnings?
Almost everyone clicked, even those
with technical backgrounds
How Effective are these Warnings?
Discussion of Phish Warnings
• Nearly everyone will fall for highly contextual phish
• Passive IE warning failed for many ...
Screenshots
Internet Explorer – Passive Warning
Discussion of Phish Warnings
• Active IE warnings
– Most saw but did not believe it
• “Since it gave me the option of stil...
Screenshots
Internet Explorer – Active Block
Internet Explorer 8 Re-design
A Science of
Warnings
• See the warning?
• Understand?
• Believe it?
• Motivated?
• Can and will act?
• Refining this mode...
Outline
• Human side
– Interviews and surveys to understand decision-making
– PhishGuru embedded training
– Anti-Phishing ...
Detecting Phishing Web Sites
• Industry uses blacklists to label phishing sites
– But blacklists slow to new attacks
• Ide...
Robust Hyperlinks
• Developed by Phelps and Wilensky to solve
“404 not found” problem
• Key idea was to add a lexical sign...
Fake
eBay, user, sign, help, forgot
Real
eBay, user, sign, help, forgot
Evaluating CANTINA
PhishTank
Machine Learning of Blacklists
• Human-verified blacklists maintained by Microsoft,
Google, PhishTank
– Pros: Reliable, ex...
Machine Learning of Blacklists
• Approach #1: Use hashcodes of web page
– Simple, good against fast flux
– Easy to defeat ...
Machine Learning of Blacklists
• Use Shingling
• Protect against false positives
– Phishing sites look a lot like real sit...
Tells people why they are
seeing this message, uses
engaging character
Tells people why they are
seeing this message, uses...
Tells a story about what
happened and what the
risks are
Tells a story about what
happened and what the
risks are
Gives concrete examples of
how to protect oneself
Gives concrete examples of
how to protect oneself
Explains how criminals conduct
phishing attacks
Explains how criminals conduct
phishing attacks
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010
Upcoming SlideShare
Loading in...5
×

Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010

618

Published on

An overview of our group's research on anti-phishing.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
618
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 2-3.5 billion http://www.gartner.com/it/page.jsp?id=498245
  • 2-3.5 billion http://www.gartner.com/it/page.jsp?id=498245
  • 2-3.5 billion http://www.gartner.com/it/page.jsp?id=498245
  • 2-3.5 billion http://www.gartner.com/it/page.jsp?id=498245
  • http://www.nytimes.com/2008/04/16/technology/16whale.html
  • Biz week http://www.businessweek.com/magazine/content/08_16/b4080032218430.htm The e-mail message addressed to a Booz Allen Hamilton executive was mundane—a shopping list sent over by the Pentagon of weaponry India wanted to buy. But the missive turned out to be a brilliant fake. Lurking beneath the description of aircraft, engines, and radar equipment was an insidious piece of computer code known as "Poison Ivy" designed to suck sensitive data out of the $4 billion consulting firm's computer network. The Pentagon hadn't sent the e-mail at all. Its origin is unknown, but the message traveled through Korea on its way to Booz Allen. Its authors knew enough about the "sender" and "recipient" to craft a message unlikely to arouse suspicion. Had the Booz Allen executive clicked on the attachment, his every keystroke would have been reported back to a mysterious master at the Internet address cybersyndrome.3322.org, which is registered through an obscure company headquartered on the banks of China's Yangtze River.
  • Thus far, our work has generated a great deal of interest and collaboration from a number of partners. Our automated email filter is undergoing a field trial at ****** main email servers, where it is labeling several million emails per day. Our research evaluating anti-phishing toolbars has been cited by several companies, with ongoing evaluations being presented to the Anti-Phishing Working Group, a consortium of companies “committed to wiping out Internet scams and fraud.” Design suggestions from our studies to understand browser warnings have been incorporated into the latest version of Microsoft’s Internet Explorer 8. PhishGuru’s methodology of sending fake phishing emails to train individuals has undergone field trials at three different companies, and been cited by two different companies trying to commercialize the work. PhishGuru’s training materials have also been adopted by APWG on their landing page, a page that ISPs and web sites can show after taking down a phishing web site. Anti-Phishing Phil has been played by over 100,000 people, licensed by two companies, demoed at many security days meant to teach people about good security practices, and translated into Portuguese with several more translations underway. Finally, our group is commercializing all of this work through a startup we have founded, named Wombat Security Technologies.
  • ASSUME THAT THIS IS YOUR EMAIL INBOX AND AMONG OTHER EMAILS.. YOU THIS EMAIL FROM AMAZON THAT JUST LOOKS LIKE THE LEGITIMATE EMAIL FROM AMAZON. WHEN YOU OPEN THE EMAIL ….
  • YOU WILL SEE THIS.. WHICH LOOKS LEGITIMATE.. AND WITH THE DATA THAT WE HAVE .. WE KNOW THAT MOST OF THE USERS WILL CLICK ON THE LINK.. WHEN THEY CLICK ON THE LINK THEY WILL SEE ….
  • P. Kumaraguru et al. Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System. CHI 2007. P. Kumaraguru et al. Getting Users to Pay Attention to Anti-Phishing Education: Evaluation of Retention and Transfer . eCrime 2007.
  • TO ADDRESS SOME OF THE LIMITATIONS IN THIS STUDY, I AM CURRENTLY DOING THIS EXCITING STUDY AMONG CMU STUDENTS/FACULTY/STAFF WHERE I AM PHISHING THEM FOR THE LAST 4 WEEKS… I WAS INTERESTED IN STUDYING LONG TERM RETENTION .. MORE THAN 1 WEEK.. SO IN THIS STUDY WE ARE STUDYING 4 WEEK RETENTION.. IN PREVIOUS STUDY WE STUDIED 1 TRAINING MATERIAL… HERE WE ARE STUDYING 2 MESSAGES… THIS STUDY IS REALLY IN THE WILD AND WE ARE COLLECTING LOT OF DATA…. I M STILL IN THE DATA COLLECTION MODE IN A FEW WEEKS, I SHOULD HAVE SOME RESULTS FROM THIS STUDY…
  • Spear phishing emails are targetted phishing emails COLLECTING VARIETY OF INFORMATION (HR, COMPLAINTS THAT ARE BEING LOGGED TO HELP CENTERS AND ISO) COUNTERBALANCING THE EMAILS COLLECTING DATA FOR LEGITIMATE EMAILS TO SEE WHETHER TRAIING INCREASES CONCERN
  • The idea in this slide is to show that training conditions did better than control conditions and it was significantdifferenc… There is an improvement of 50% among people in PhihsGuru training
  • S. Sheng, B. Magnien, P. Kumaraguru, A. Acquisti, L. Cranor, J. Hong, and E. Nunge. Anti-Phishing Phil: The Design and Evaluation of a Game That Teaches People Not to Fall for Phish . In Proceedings of the 2007 Symposium On Usable Privacy and Security, Pittsburgh, PA, July 18-20, 2007.
  • Phil needs to score 6 / 8 to move on to the next rounds, and the end of the round, phil got a chance to reflect what he missed.
  • In between rounds, we also have short tutorials to teach Phil better strategies to identify phishing. In this example, Phil’s father teaches Phil how to use a search engine.
  • S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. CHI 2008.
  • THE USER WILL SEE THIS INTERVENTION… WHICH TELLS THEM HOW TO AVOID FALLING FROM PHISHING EMAILS… I WILL DESCRIBE IN DETAIL WHAT INFORMATION IS IN THIS INTERVENTION IN A COUPLE OF MINUTES. You have the printout of this intervention…
  • THE USER WILL SEE THIS INTERVENTION… WHICH TELLS THEM HOW TO AVOID FALLING FROM PHISHING EMAILS… I WILL DESCRIBE IN DETAIL WHAT INFORMATION IS IN THIS INTERVENTION IN A COUPLE OF MINUTES. You have the printout of this intervention…
  • THE USER WILL SEE THIS INTERVENTION… WHICH TELLS THEM HOW TO AVOID FALLING FROM PHISHING EMAILS… I WILL DESCRIBE IN DETAIL WHAT INFORMATION IS IN THIS INTERVENTION IN A COUPLE OF MINUTES. You have the printout of this intervention…
  • THE USER WILL SEE THIS INTERVENTION… WHICH TELLS THEM HOW TO AVOID FALLING FROM PHISHING EMAILS… I WILL DESCRIBE IN DETAIL WHAT INFORMATION IS IN THIS INTERVENTION IN A COUPLE OF MINUTES. You have the printout of this intervention…
  • THE USER WILL SEE THIS INTERVENTION… WHICH TELLS THEM HOW TO AVOID FALLING FROM PHISHING EMAILS… I WILL DESCRIBE IN DETAIL WHAT INFORMATION IS IN THIS INTERVENTION IN A COUPLE OF MINUTES. You have the printout of this intervention…
  • Transcript of "Teaching Johnny Not to Fall for Phish, for ISSA 2010 on May 2010"

    1. 1. Jason Hong, PhD Carnegie Mellon University Wombat Security Technologies Teaching Johnny Not to Fall for Phish
    2. 2. Everyday Privacy and Security Problem
    3. 3. This entire process known as phishing
    4. 4. How Bad Is Phishing? Consumer Perspective • Estimated ~0.5% of Internet users per year fall for phishing attacks • Conservative $1B+ direct losses a year to consumers – Bank accounts, credit card fraud – Doesn’t include time wasted on recovery of funds, restoring computers, emotional uncertainty • Growth rate of phishing – 30k+ reported unique emails / month – 45k+ reported unique sites / month • Social networking sites now major targets
    5. 5. How Bad Is Phishing? Perspective of Corporations • Direct damage – Loss of sensitive customer data
    6. 6. How Bad Is Phishing? Perspective of Corporations • Direct damage – Loss of sensitive customer data – Loss of intellectual property
    7. 7. How Bad Is Phishing? Perspective of Corporations • Direct damage – Loss of sensitive customer data – Loss of intellectual property – Fraud – Disruption of network services • Indirect damage – Damage to reputation, lost sales, etc – Response costs (call centers, recovery) • One bank estimated it cost them $1M per phishing attack
    8. 8. General Patton is retiring next week, click here to say whether you can attend his retirement party Phishing Increasing in Sophistication Targeting Your Organization • Spear-phishing targets specific groups or individuals • Type #1 – Uses info about your organization
    9. 9. Phishing Increasing in Sophistication Targeting Your Organization • Around 40% of people in our experiments at CMU would fall for emails like this (control condition)
    10. 10. Phishing Increasing in Sophistication Targeting You Specifically • Type #2 – Uses info specifically about you – Social phishing • Might use information from social networking sites, corporate directories, or publicly available data • Ex. Fake email from friends or co-workers • Ex. Fake videos of you and your friends
    11. 11. Phishing Increasing in Sophistication Targeting You Specifically Here’s a video I took of your poster presentation.
    12. 12. Phishing Increasing in Sophistication Targeting You Specifically • Type #2 – Uses info specifically about you – Whaling – focusing on big targets Thousands of high-ranking executives across the country have been receiving e-mail messages this week that appear to be official subpoenas from the United States District Court in San Diego. Each message includes the executive’s name, company and phone number, and commands the recipient to appear before a grand jury in a civil case. -- New York Times Apr16 2008
    13. 13. Phishing Increasing in Sophistication Combination with Malware • Malware and phishing are becoming combined – Poisoned attachments (Ex. custom PDF exploits) – Links to web sites with malware (web browser exploits) – Can install keyloggers or remote access software
    14. 14. Protecting People from Phishing • Human side – Interviews and surveys to understand decision-making – PhishGuru embedded training – Micro-games for security training – Understanding effectiveness of browser warnings • Computer side – PILFER email anti-phishing filter – CANTINA web anti-phishing algorithm – Machine learning of blacklists – Social web + machine learning to combat scams
    15. 15. Results of Our Research • Startup – Customers of micro-games featured include governments, financials, universities – Our filter is labeling several million emails per day • Study on browser warnings -> MSIE8 • Elements of our work adopted by Anti-Phishing Working Group (APWG) • Popular press article in Scientific American
    16. 16. Outline of Rest of Talk • Rest of talk will focus on educating end-users • PhishGuru embedded training • Anti-Phishing Phil micro-game • Anti-Phishing Phyllis micro-game
    17. 17. User Education is Challenging • Users are not motivated to learn about security • Security is a secondary task • Difficult to teach people to make right online trust decision without increasing false positives “User education is a complete waste of time. It is about as much use as nailing jelly to a wall…. They are not interested…they just want to do their job.” Martin Overton, IBM security specialist http://news.cnet.com/21007350_361252132.html
    18. 18. But Actually, Users Are Trainable • Our research demonstrates that users can learn techniques to protect themselves from phishing… if you can get them to pay attention to training P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish. CyLab Technical Report CMU CyLab07003, 2007.
    19. 19. How Do We Get People Trained? • Solution – Find “teachable moments”: PhishGuru – Make training fun: Anti-Phishing Phil, Anti-Phishing Phyllis – Use learning science principles
    20. 20. PhishGuru Embedded Training • Send emails that look like a phishing attack • If recipient falls for it, show intervention that teaches what cues to look for in succinct and engaging format • Multiple user studies have demonstrated that PhishGuru is effective • Delivering same training via direct email is not effective!
    21. 21. Subject: Revision to Your Amazon.com InformationSubject: Revision to Your Amazon.com Information
    22. 22. Subject: Revision to Your Amazon.com InformationSubject: Revision to Your Amazon.com Information Please login and enter your informationPlease login and enter your information
    23. 23. Evaluation of PhishGuru • Is embedded training effective? – Study 1: Lab study, 30 participants – Study 2: Lab study, 42 participants – Study 3: Field trial at company, ~300 participants – Study 4: Field trial at CMU, ~500 participants • Studies showed significant decrease in falling for phish and ability to retain what they learned P. Kumaraguru et al. Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System. CHI 2007. P. Kumaraguru et al. Getting Users to Pay Attention to Anti-Phishing Education: Evaluation of Retention and Transfer. eCrime 2007.
    24. 24. Study #4 at CMU • Investigate effectiveness and retention of training after 1 week, 2 weeks, and 4 weeks • Compare effectiveness of 2 training messages vs 1 training message • Examine demographics and phishing P. Kumaraguru, J. Cranshaw, A. Acquisti, L. Cranor, J. Hong, M. A. Blair, and T. Pham. School of Phish: A Real-World Evaluation of Anti-Phishing Training. 2009. SOUPS 2009.
    25. 25. Study design • Sent email to all CMU students, faculty and staff to recruit participants (opt-in) • 515 participants in three conditions – Control / One training message / Two messages • Emails sent over 28 day period – 7 simulated spear-phishing messages – 3 legitimate (cyber security scavenger hunt) • Campus help desks and IT departments notified before messages sent
    26. 26. Effect of PhishGuru Training Condition N % who clicked on Day 0 % who clicked on Day 28 Control 172 52.3 44.2 Trained 343 48.4 24.5
    27. 27. Discussion of PhishGuru • PhishGuru can teach people to identify phish better – People retain the knowledge • People trained on first day less likely to be phished • Two training messages work better – People weren’t less likely to click on legitimate emails – People aren’t resentful, many happy to have learned • 68 out of 85 surveyed said they recommend CMU continue doing this sort of training in future • “I really liked the idea of sending CMU students fake phishing emails and then saying to them, essentially, HEY! You could've just gotten scammed! You should be more careful -- here's how....” • Contrast to US DOJ and Guam
    28. 28. APWG Landing Page • CMU and Wombat helped Anti-Phishing Working Group develop landing page for taken down sites – Already in use by several takedown companies – Seen by ~200,000 people in past 20 months
    29. 29. Anti-Phishing Phil • A micro-game to teach people not to fall for phish – PhishGuru about email, this game about web browser – Also based on learning science principles • Goals – How to parse URLs – Where to look for URLs – Use search engines for help • Try the game! – Search for “phishing game” S. Sheng et al. Anti-Phishing Phil: The Design and Evaluation of a Game That Teaches People Not to Fall for Phish. In SOUPS 2007, Pittsburgh, PA, 2007.
    30. 30. Anti-Phishing Phil
    31. 31. Evaluation of Anti-Phishing Phil • Is Phil effective? Yes! – Study 1: 56 people in lab study – Study 2: 4517 people in field trial • Brief results of Study 1 – Phil about as effective in helping people detect phishing web sites as paying people to read training material – But Phil has significantly fewer false positives overall • Suggests that existing training material making people paranoid about phish rather than differentiating
    32. 32. Evaluation of Anti-Phishing Phil • Study 2: 4517 participants in field trial – Randomly selected from 80000 people • Conditions – Control: Label 12 sites then play game – Game: Label 6 sites, play game, then label 6 more, then after 7 days, label 6 more (18 total) • Participants – 2021 people in game condition, 674 did retention portion
    33. 33. Anti-Phishing Phil: Study 2 • Novices showed most improvement in false negatives (calling phish legitimate)
    34. 34. Anti-Phishing Phil: Study 2 • Improvement all around for false positives
    35. 35. Anti-Phishing Phyllis • New micro-game just released by Wombat Security • Focuses on teaching people about what cues to look for in emails – Some emails are legitimate, some fake – Have to identify cues as dangerous or harmless
    36. 36. Summary • Phishing is already a plague on the Internet – Seriously affects consumers, businesses, governments – Criminals getting more sophisticated • End-users can be trained, but only if done right – PhishGuru embedded training uses simulated phishing – Anti-Phishing Phil and Anti-Phishing Phyllis micro-games • Can try PhishGuru, Phil, and Phyllis at: www.wombatsecurity.com • Will show free demo of Phil and Phyllis to anyone who can explain to me what’s going on in Lost
    37. 37. Acknowledgments • Ponnurangam Kumaraguru • Steve Sheng • Lorrie Cranor • Norman Sadeh
    38. 38. Screenshots Internet Explorer – Passive Warning
    39. 39. Screenshots Internet Explorer – Active Block
    40. 40. Screenshots Mozilla FireFox – Active Block
    41. 41. How Effective are these Warnings? • Tested four conditions – FireFox Active Block – IE Active Block – IE Passive Warning – Control (no warnings or blocks) • “Shopping Study” – Setup some fake phishing pages and added to blacklists – We phished users after purchases (2 phish/user) – Real email accounts and personal information S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. CHI 2008.
    42. 42. How Effective are these Warnings? Almost everyone clicked, even those with technical backgrounds
    43. 43. How Effective are these Warnings?
    44. 44. Discussion of Phish Warnings • Nearly everyone will fall for highly contextual phish • Passive IE warning failed for many reasons – Didn’t interrupt the main task – Slow to appear (up to 5 seconds) – Not clear what the right action was – Looked too much like other ignorable warnings (habituation) – Bug in implementation, any keystroke dismisses
    45. 45. Screenshots Internet Explorer – Passive Warning
    46. 46. Discussion of Phish Warnings • Active IE warnings – Most saw but did not believe it • “Since it gave me the option of still proceeding to the website, I figured it couldn’t be that bad” – Some element of habituation (looks like other warnings) – Saw two pathological cases
    47. 47. Screenshots Internet Explorer – Active Block
    48. 48. Internet Explorer 8 Re-design
    49. 49. A Science of Warnings • See the warning? • Understand? • Believe it? • Motivated? • Can and will act? • Refining this model for computer warnings
    50. 50. Outline • Human side – Interviews and surveys to understand decision-making – PhishGuru embedded training – Anti-Phishing Phil game – Understanding effectiveness of browser warnings • Computer side – PILFER email anti-phishing filter – CANTINA web anti-phishing algorithm – Machine learning of blacklists Can we improve phish detection of web sites?
    51. 51. Detecting Phishing Web Sites • Industry uses blacklists to label phishing sites – But blacklists slow to new attacks • Idea: Use search engines – Scammers often directly copy web pages – But fake pages should have low PageRank on search engines – Generate text-based “fingerprint” of web page keywords and send to a search engine Y. Zhang, S. Egelman, L. Cranor, and J. Hong Phinding Phish: Evaluating Anti-Phishing Tools. In NDSS 2007. Y. Zhang, J. Hong, and L. Cranor. CANTINA: A content-based approach to detecting phishing web sites. In WWW 2007. G. Xiang and J. Hong. A Hybrid Phish Detection Approach by Identity Discovery and Keywords Retrieval. In WWW 2009.
    52. 52. Robust Hyperlinks • Developed by Phelps and Wilensky to solve “404 not found” problem • Key idea was to add a lexical signature to URLs that could be fed to a search engine if URL failed – Ex. http://abc.com/page.html?sig=“word1+word2+...+word5” • How to generate signature? – Found that TF-IDF was fairly effective • Informal evaluation found five words was sufficient for most web pages
    53. 53. Fake eBay, user, sign, help, forgot
    54. 54. Real eBay, user, sign, help, forgot
    55. 55. Evaluating CANTINA PhishTank
    56. 56. Machine Learning of Blacklists • Human-verified blacklists maintained by Microsoft, Google, PhishTank – Pros: Reliable, extremely low false positives – Cons: Slow to respond, can be flooded with URLs (fast flux) • Observation #1: many phishing sites similar – Constructed through toolkits • Observation #2: many phishing sites similar – Fast flux (URL actually points to same site) • Idea: Rather than just examining URL, compare content of a site to known phishing sites
    57. 57. Machine Learning of Blacklists • Approach #1: Use hashcodes of web page – Simple, good against fast flux – Easy to defeat (though can allow some flexibility) • Approach #2: Use shingling – Shingling is an approach used by search engines to find duplicate pages – “connect with the eBay community” -> {connect with the, with the eBay, the eBay community} – Count the number of common shingles out of total shingles, set threshold
    58. 58. Machine Learning of Blacklists • Use Shingling • Protect against false positives – Phishing sites look a lot like real sites – Have a small whitelist (ebay, paypal, etc) – Use CANTINA too
    59. 59. Tells people why they are seeing this message, uses engaging character Tells people why they are seeing this message, uses engaging character
    60. 60. Tells a story about what happened and what the risks are Tells a story about what happened and what the risks are
    61. 61. Gives concrete examples of how to protect oneself Gives concrete examples of how to protect oneself
    62. 62. Explains how criminals conduct phishing attacks Explains how criminals conduct phishing attacks
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×