Downloaded 14 times
Uploaded byAlejandro Correa Bahnsen, PhD
black hat deephish
This document discusses using machine learning and deep learning techniques to detect malicious URLs and TLS certificates. It describes building recurrent neural network models to classify URLs and certificates based on their content. The models were able to achieve over 98% accuracy on URL classification and can be used to detect phishing URLs and malicious TLS certificates that aim to mimic legitimate ones. A demo of these techniques is also mentioned. The goal is to develop more robust detection of malicious AI that is trying to simulate legitimate behavior.
More Related Content
Similar to black hat deephish
black hat deephish
- 1. DeepPhish Simulating MaliciousAI Alejandro Correa Bahnsen, PhD VP, AI & Research – CyxteraTechnologies
- 2. About Me • IndustrialEngineer • PhD in Machine Learning • Passionate about open-source • Scikit-Learn contributor • Organizer of Data Science Meetups
- 3. Who I’ve workedwith
- 4. Agenda • Phishing URLDetection using Machine Learning • Malicious Cert Detection using Deep Learning • DeepPhish: Simulating Malicious AI • Demo
- 5.
- 7. Why Phishing Detectionis Hard? Original Website Only Using Images Subtle Changes
- 8. Ideal Phishing DetectionSystem Is It Phishing? Machine Learning Algorithm
- 9. Ideal Phishing DetectionSystem Issues with full content analysis: • Time consuming • Impractical to process millions of websites per day • Hard to implement for small devices
- 10. There is alwaysthe need for an URL
- 11. Database of URLs http://moviesjingle.com/auto/163.com/index.php 1,000,000Legitimate URLs from Common Crawl http://paypal.com.update.account.toughbook.cl/8a30e847925afc5975161aeabe8930f 1/?cmd=_home&dispatch=d09b78f5812945a73610edf38 http://msystemtech.ru/components/com_users/Italy/zz/Login.php?run=_login- submit&session=68bbd43c854147324d77872062349924 https://www.sanfordhealth.org/ChildrensHealth/Article/73980 http://www.grahamleader.com/ci_25029538/these-are-5-worst-super-bowl-halftime- shows&defid=1634182 http://www.carolinaguesthouse.co.uk/onlinebooking/?industrytype=1&startdate=201 3-09-05&nights=2&location&productid=25d47a24-6b74 1,000,000 Phishing URLs from Phish Tank
- 12.
- 13. Recurrent Neural NetworksRNN URL h t t p : / / w w w . p a p a y a . c o m One hot Encoding … … … … … … … … … … … … … … … … … … … … … Embedding 3.2 1.2 … 1.7 6.4 2.3 … 2.6 6.4 3.0 … 1.7 3.4 2.6 … 3.4 2.6 3.8 … 2.6 3.5 3.2 … 6.4 1.7 4.2 … 6.4 8.6 2.4 … 6.4 4.3 2.9 … 6.4 2.2 3.4 … 3.4 3.2 2.6 … 2.6 4.2 2.2 … 3.5 2.4 3.2 … 1.7 2.9 1.7 … 8.6 3.0 6.4 … 2.6 2.6 6.4 … 3.8 3.8 3.4 … 3.2 3.3 2.6 … 2.2 3.1 2.2 … 2.9 1.8 3.2 … 3.0 2.5 6.4 … 2.6 LSTM LSTM LSTM LSTM Sigmoid …
- 14. URL Classification Results 3-FoldCV Accuracy Recall Precision Average 98.76% 98.93% 98.60% Deviation 0.04% 0.02% 0.02%
- 15. URL Classification Results 3-FoldCV Accuracy Recall Precision Average 98.76% 98.93% 98.60% Deviation 0.04% 0.02% 0.02%
- 16. Detecting Malicious URLs IsNot Enough!!
- 18.
- 19. What is aWeb Certificate?
- 20.
- 21. Forrester survey askedusers: “Some websites receive the following browser user interface security indicator in the browser. What do you think the security indicator is intended to tell users?” The website is safe: 82% The website is encrypted: 75% The website is trustworthy: 66% The website is private: 32% Secure | https://ultrabank.com
- 22. Forrester survey askedusers: “Some websites receive the following browser user interface security indicator in the browser. What do you think the security indicator is intended to tell users?” The website is safe: 82% The website is encrypted: 75% The website is trustworthy: 66% The website is private: 32% Secure | https://ultrabank.com
- 23. Hunting Malicious TLS Certificateswith Deep Neural Networks
- 24. Database of TLSCertificates 1,000,000 Legitimate Certificates from Common Crawl 5,000 Phishing Certificates CN = *.stackexchange.com, O = Stack Exchange, Inc., L = New York, S = NY, C = US CN = localhost, L = Springfield CN = slack.com, O = Slack Technologies, Inc., L = San Francisco, S = CA, C = US CN = *.trello.com, O = Trello Inc., L = New York, S = New York, C = US CN = localhost.localdomain CN = example.com, L = Springfield
- 25.
- 26. Deep Learning Algorithm Onehot encoding One hot encoding Embedding Embedding LSTM LSTM Dense/ReLu Dropout Dropout Dropout Concatenate Dense/ReLu Dropout Dense/Logit score Subject Principal Issuer Principal Extracted Features
- 27. Malicious Cert ClassificationResults 5-Fold CV Accuracy Recall Precision Average 86.41% 83.20% 88.86% Deviation 1.22% 3.29% 1.04%
- 30.
- 31. The Experiment: SimulatingMalicious AI Identify individual threat actors Run them through our own AI detection system Improve their attacks using AI
- 32. Uncovering Threat Actors •Objective: We want to understand effective patterns of each attacker to improve them through a AI model • As we can not know attackers directly, we must learn from them through their attacks • Database with 1.1M confirm phishing URLs collected from Phishtank
- 33. Threat Actor 1 naylorantiques.com 406URLs http://naylorantiques.com/components/com_contact/views/c ontact/tmpl/62 http://naylorantiques.com/docs/Auto/Atendimento/5BBROPI 6S3 http://naylorantiques.com/Atualizacao Segura/pictures/XG61YYMT_FXW0PWR8_5P2O7T2U_P9HND PQR/ http://naylorantiques.com/zifn3p72bsifn9hx9ldecd8jzl2f0xlwf 8f http://www.naylorantiques.com/JavaScript/charset=iso-8859- 1/http-equiv/margin-bottom Keywords atendimento, jsf, identificacao, ponents, views, TV, mail, SHOW, COMPLETO, VILLA, MIX, ufi, pnref, story, tryy2ilr, Autentico 106 domains naylorantiques.com, netshelldemos.com, debbiebright.co.z, waldronfamilygppractice.co.uk , avea-vacances.com , psncodes2013.com uni5.net , 67.228.96.204, classificadosmaster.com.br, ibjjf.org Visual Check Check in database Visual Check
- 34. Threat Actor 2 Vopus.org 13URLs http://www.vopus.org/es/images/cursos/thumbs/tdcanadatr ust http://www.vopus.org/ru/media/tdcanadatrust/index.html http://vopus.org/common/index.htm http://www.vopus.org/es/images/cursos/thumbs/tdcanadatr ust/index.html http://vopus.org/descargas/otros/tdcanadatrust/index.html Keywords tdcanadatrust/index.html 19 domains friooptimo.com, kramerelementary.org, kalblue.com, vopus.org, artwood.co.kr, stephenpizzuti.com, heatherthinks.com, corvusseo.com, natikor.by, optioglobal.com, backfire.se, fncl.ma, greenant.de, mersintenisakademisi.com, cavtel.net Check in database Visual Check Visual Check
- 35. The Experiment: SimulatingMalicious AI Identify individual threat actors Run them through our own AI detection system Improve their attacks using AI
- 36.
- 37. Threat Actors Effectiveness 0.24%0.69% 4.91% All Attacks (1,146,441) Threat Actor 1 (1,009) Threat Actor 2 (102)
- 38. The Experiment: SimulatingMalicious AI Identify individual threat actors Run them through our own AI detection system Improve their attacks using AI
- 39. DeepPhish Algorithm -Training Non Effective URLs Effective URLs Encoding … … … … … Model Az Rolling Window Concatenate andcreate Transform Train http://www.naylorantiques.com/content/centrais/fon e_facil http://kisanart.com/arendivento/menu-opcoes-fone- facil/ http://naylorantiques.com/atendimento/menu- opcoes-fone-facil/3 http://www.naylorantiques.com/con tent/centrais/fone_facilhttp://kisana rt.com/arendivento/menu-opcoes- fone- facil/http://naylorantiques.com/aten dimento/menu-opcoes-fone-facil/3
- 40. DeepPhish LSTM Network URL h t t p : / / w w w . p a p a y a . c o m Onehot Encoding … … … … … … … … … … … … … … … … … … … … … LSTM LSTM LSTM LSTM Softmax … tanH tanH tanH tanH …
- 41. DeepPhish Algorithm -Prediction Compromised Domains Allowed Paths + Model Filter paths Predict Next Character Iteratively Synthetic URLs /arendipemto/nenu- opcines-fone-facil vfone/faci/ Atondime http:// + www.naylorantiques.com + /arendipemto/nenu-opcines-fone- facilvone/facil/Atondime Create
- 42. Traditional Attacks vs.AI-Driven Attacks 0.69% Traditional DeepPhish 4.91% Traditional DeepPhish Threat Actor 1 Threat Actor 2 20.90% 36.28%
- 43. Traditional Attacks vs.AI-Driven Attacks 0.69% Traditional DeepPhish 4.91% Traditional DeepPhish Threat Actor 1 Threat Actor 2 20.90% 36.28%
- 44.
- 45. What’s Next?? AI poweredAttacks are real, as we probed with Deep Phish experiment. We need to enhance our own AI detection systems to account for the possibility of attackers using AI.
- 47. Thanks!! Alejandro Correa Bahnsen,PhD VP, AI & Research alejandro.correa@cyxtera.com @albahnsen