SlideShare a Scribd company logo
1 of 27
Download to read offline
httpscreenshot
A Tool for Both Teams
Outline
• Who we are
• The problem
• Our solution
• Demo
• How we’ve used it
• Q&A
Who We Are – Steve Breen
• Senior penetration tester
• Former “Enterprise” developer
– Current hacky script developer
• Vulnerability and exploit development hobbyist
by night
@breenmachine
breenmachine.blogspot.com
Who we are – Justin Kennedy
• Lifelong security hobbyist, actively for ~15 years
• Intern -> Computer Tech -> Help Desk -> SOC ->
Network Security -> Junior PT -> Senior PT ->
Team Lead
• Terrible at making slides look pretty… if you came
to see pretty slides, you may be in the wrong talk.
If you came to see an awesome tool that is
available (and is OSS) as of today, you’re in the
right place.
• @jstnkndy / juken (freenode)
Before we get started
_o/
Blue Team challenge
Let’s start off with a question…
Can you account for every device or application
on your network?
Why not?
Blue Team challenge reasons
1. You’ve inherited an infrastructure that you didn’t
build and (of course) not everything was documented by
your predecessor.
2. You work in an environment where business units
don't necessarily communicate as much as they
should and another business unit has spun up some
demo or test application without telling you.
3. You forgot about that old NT4 or tomcat box that no
one has touched in the past 10 years.
4. Or someone just plugged some shit into a network
jack.
Red Team challenge
1. We are constantly attempting to compromise
organizations that we don't know anything
about (besides our recon).
2. It's our job to identify what the target attack
surface looks like.
3. Anyone in here ever masscan a /8 for
common web ports?
4. Let’s face it, we don’t always have as much
time as we’d like on an assessment.
*Disclaimer: Some of these may look
familiar
Shmoocon 2015 - httpscreenshot
root
calvin
Shmoocon 2015 - httpscreenshot
Shmoocon 2015 - httpscreenshot
Shmoocon 2015 - httpscreenshot
Our Solution
HTTPScreenshot/Cluster
• HTTPScreenshot: A python script to
screenshot thousands of websites really
quickly (and reliably)
• Cluster: A script to do “fuzzy matching” on
HTML pages. Produce immediately usable
output with “similar” pages grouped together.
HTTPScreenshot.py
• Goals: Fast, Thorough, Automagic
• Challenges: Code was hacked together during
assessments – needs some TLC
• Fun features:
– Input is nmap/masscan output
– Javascript parsed and executed
– SSL autodetect
– SSL Certificate domain scraping for vhosts
– Headless (configurable fail-over to FireFox)
– Threaded
– Saves PNG and HTML (good for grep’ing)
– Attempts tls 1.0 and falls back to sslv3 when necessary
Cluster.py
• Identify “similar” websites and group them together
• Displays the resulting groups in a useful way (HTML output with JS
“hoverzoom”)
• Algorithm - Reduces to DBSCAN:
– Needed a clustering algorithm that didn’t require definition of “k”
– Uses HTML tag/attr values – computes a “similarity” score for two
sites
• href, name, src, id, class, title, h1
– Works fairly well – could DEFINITELY be improved upon
• Supports “diff” reports. Sites that have been
changed/added/removed since the last scan
Demo!
How we’ve used it
Story 1 - SQLi
Story 2 – WTF is SuperGoose?
Story 2 – WTF is SuperGoose?
Shmoocon 2015 - httpscreenshot
Story 3 – Bug Bounties
Cheat Sheet: Usage
• masscan --iL scope.txt --oG out.gnmap –rate 100000
• httpscreenshot.py -i out.gnmap --headless --
workers 50 --timeout 45 --autodetect --vhosts --
retries 1
• cluster.py –d <httpscreenshot directory>
masshttp
#!/bin/sh
~/tools/masscan –p1-65535 -iL scope.txt -oG out.gnmap --rate
100000
mkdir httpscreenshot
cd httpscreenshot
python ~/tools/httpscreenshot.py -i ../out.gnmap -p -t 50 -w 45 -
a -vH -r 1
python ~/tools/httpscreenshot.py -i ../out.gnmap -p -t 10 -w 5 -a
-vH -r 1
cd ..
python ~/tools/cluster.py -d httpscreenshot/
Questions / Suggestions?
Code: github.com/breenmachine/

More Related Content

What's hot

DevOps, CLI, APIs, Oh My! Security Gone Agile
DevOps, CLI, APIs, Oh My!  Security Gone AgileDevOps, CLI, APIs, Oh My!  Security Gone Agile
DevOps, CLI, APIs, Oh My! Security Gone AgileMatt Tesauro
 
DevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi Douglen
DevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi DouglenDevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi Douglen
DevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi DouglenDevSecCon
 
Top Security Challenges Facing Credit Unions Today
Top Security Challenges Facing Credit Unions TodayTop Security Challenges Facing Credit Unions Today
Top Security Challenges Facing Credit Unions TodayChris Gates
 
SecDevOps: The New Black of IT
SecDevOps: The New Black of ITSecDevOps: The New Black of IT
SecDevOps: The New Black of ITCloudPassage
 
Integrating DevOps and Security
Integrating DevOps and SecurityIntegrating DevOps and Security
Integrating DevOps and SecurityStijn Muylle
 
How to Leverage Log Data for Effective Threat Detection
How to Leverage Log Data for Effective Threat DetectionHow to Leverage Log Data for Effective Threat Detection
How to Leverage Log Data for Effective Threat DetectionAlienVault
 
Automated Security Testing
Automated Security TestingAutomated Security Testing
Automated Security Testingseleniumconf
 
Automating security tests for Continuous Integration
Automating security tests for Continuous IntegrationAutomating security tests for Continuous Integration
Automating security tests for Continuous IntegrationStephen de Vries
 
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...Abhay Bhargav
 
Continuous Security Testing - DevSecCon
Continuous Security Testing - DevSecConContinuous Security Testing - DevSecCon
Continuous Security Testing - DevSecConStephen de Vries
 
DevSecCon London 2017: Hands-on secure software development from design to de...
DevSecCon London 2017: Hands-on secure software development from design to de...DevSecCon London 2017: Hands-on secure software development from design to de...
DevSecCon London 2017: Hands-on secure software development from design to de...DevSecCon
 
DevSecCon Singapore 2018 - Remove developers’ shameful secrets or simply rem...
DevSecCon Singapore 2018 -  Remove developers’ shameful secrets or simply rem...DevSecCon Singapore 2018 -  Remove developers’ shameful secrets or simply rem...
DevSecCon Singapore 2018 - Remove developers’ shameful secrets or simply rem...DevSecCon
 
Ch 6: Attacking Authentication
Ch 6: Attacking AuthenticationCh 6: Attacking Authentication
Ch 6: Attacking AuthenticationSam Bowne
 
WTF is Penetration Testing
WTF is Penetration TestingWTF is Penetration Testing
WTF is Penetration TestingNetSPI
 
DevSecOps Fundamentals and the Scars to Prove it.
DevSecOps Fundamentals and the Scars to Prove it.DevSecOps Fundamentals and the Scars to Prove it.
DevSecOps Fundamentals and the Scars to Prove it.Matt Tesauro
 
Hacker Proof web app using Functional tests
Hacker Proof web  app using Functional testsHacker Proof web  app using Functional tests
Hacker Proof web app using Functional testsAnkita Gupta
 
DevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly Davidoff
DevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly DavidoffDevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly Davidoff
DevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly DavidoffDevSecCon
 
Principles Of Chaos Engineering - Chaos Engineering Hamburg
Principles Of Chaos Engineering - Chaos Engineering HamburgPrinciples Of Chaos Engineering - Chaos Engineering Hamburg
Principles Of Chaos Engineering - Chaos Engineering HamburgNils Meder
 
Continuous Security Testing in a Devops World #OWASPHelsinki
Continuous Security Testing in a Devops World #OWASPHelsinkiContinuous Security Testing in a Devops World #OWASPHelsinki
Continuous Security Testing in a Devops World #OWASPHelsinkiStephen de Vries
 

What's hot (20)

DevOps, CLI, APIs, Oh My! Security Gone Agile
DevOps, CLI, APIs, Oh My!  Security Gone AgileDevOps, CLI, APIs, Oh My!  Security Gone Agile
DevOps, CLI, APIs, Oh My! Security Gone Agile
 
DevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi Douglen
DevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi DouglenDevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi Douglen
DevSecCon Tel Aviv 2018 - Value driven threat modeling by Avi Douglen
 
Top Security Challenges Facing Credit Unions Today
Top Security Challenges Facing Credit Unions TodayTop Security Challenges Facing Credit Unions Today
Top Security Challenges Facing Credit Unions Today
 
SecDevOps: The New Black of IT
SecDevOps: The New Black of ITSecDevOps: The New Black of IT
SecDevOps: The New Black of IT
 
Security testautomation
Security testautomationSecurity testautomation
Security testautomation
 
Integrating DevOps and Security
Integrating DevOps and SecurityIntegrating DevOps and Security
Integrating DevOps and Security
 
How to Leverage Log Data for Effective Threat Detection
How to Leverage Log Data for Effective Threat DetectionHow to Leverage Log Data for Effective Threat Detection
How to Leverage Log Data for Effective Threat Detection
 
Automated Security Testing
Automated Security TestingAutomated Security Testing
Automated Security Testing
 
Automating security tests for Continuous Integration
Automating security tests for Continuous IntegrationAutomating security tests for Continuous Integration
Automating security tests for Continuous Integration
 
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
 
Continuous Security Testing - DevSecCon
Continuous Security Testing - DevSecConContinuous Security Testing - DevSecCon
Continuous Security Testing - DevSecCon
 
DevSecCon London 2017: Hands-on secure software development from design to de...
DevSecCon London 2017: Hands-on secure software development from design to de...DevSecCon London 2017: Hands-on secure software development from design to de...
DevSecCon London 2017: Hands-on secure software development from design to de...
 
DevSecCon Singapore 2018 - Remove developers’ shameful secrets or simply rem...
DevSecCon Singapore 2018 -  Remove developers’ shameful secrets or simply rem...DevSecCon Singapore 2018 -  Remove developers’ shameful secrets or simply rem...
DevSecCon Singapore 2018 - Remove developers’ shameful secrets or simply rem...
 
Ch 6: Attacking Authentication
Ch 6: Attacking AuthenticationCh 6: Attacking Authentication
Ch 6: Attacking Authentication
 
WTF is Penetration Testing
WTF is Penetration TestingWTF is Penetration Testing
WTF is Penetration Testing
 
DevSecOps Fundamentals and the Scars to Prove it.
DevSecOps Fundamentals and the Scars to Prove it.DevSecOps Fundamentals and the Scars to Prove it.
DevSecOps Fundamentals and the Scars to Prove it.
 
Hacker Proof web app using Functional tests
Hacker Proof web  app using Functional testsHacker Proof web  app using Functional tests
Hacker Proof web app using Functional tests
 
DevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly Davidoff
DevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly DavidoffDevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly Davidoff
DevSecCon Tel Aviv 2018 - End2End containers SSDLC by Vitaly Davidoff
 
Principles Of Chaos Engineering - Chaos Engineering Hamburg
Principles Of Chaos Engineering - Chaos Engineering HamburgPrinciples Of Chaos Engineering - Chaos Engineering Hamburg
Principles Of Chaos Engineering - Chaos Engineering Hamburg
 
Continuous Security Testing in a Devops World #OWASPHelsinki
Continuous Security Testing in a Devops World #OWASPHelsinkiContinuous Security Testing in a Devops World #OWASPHelsinki
Continuous Security Testing in a Devops World #OWASPHelsinki
 

Similar to Shmoocon 2015 - httpscreenshot

The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class Chris Gates
 
CloudStack and testing
CloudStack and testingCloudStack and testing
CloudStack and testingShapeBlue
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryPriyanka Aash
 
Creating Havoc using Human Interface Device
Creating Havoc using Human Interface DeviceCreating Havoc using Human Interface Device
Creating Havoc using Human Interface DevicePositive Hack Days
 
Protect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesProtect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesLeo Loobeek
 
My tryst with sourcecode review
My tryst with sourcecode reviewMy tryst with sourcecode review
My tryst with sourcecode reviewAnant Shrivastava
 
Introduction to red team operations
Introduction to red team operationsIntroduction to red team operations
Introduction to red team operationsSunny Neo
 
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
DevOops Redux Ken Johnson Chris Gates  - AppSec USA 2016DevOops Redux Ken Johnson Chris Gates  - AppSec USA 2016
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016Chris Gates
 
PAC 2019 virtual Christoph NEUMÜLLER
PAC 2019 virtual Christoph NEUMÜLLERPAC 2019 virtual Christoph NEUMÜLLER
PAC 2019 virtual Christoph NEUMÜLLERNeotys
 
Zane lackey. security at scale. web application security in a continuous depl...
Zane lackey. security at scale. web application security in a continuous depl...Zane lackey. security at scale. web application security in a continuous depl...
Zane lackey. security at scale. web application security in a continuous depl...Yury Chemerkin
 
Using AWS, Terraform, and Ansible to Automate Splunk at Scale
Using AWS, Terraform, and Ansible to Automate Splunk at ScaleUsing AWS, Terraform, and Ansible to Automate Splunk at Scale
Using AWS, Terraform, and Ansible to Automate Splunk at ScaleData Works MD
 
The Clash Between Devops and Quality Assurance
The Clash Between Devops and Quality AssuranceThe Clash Between Devops and Quality Assurance
The Clash Between Devops and Quality AssuranceWebcsonsultsEU
 
DevOps: Automate all the things
DevOps: Automate all the thingsDevOps: Automate all the things
DevOps: Automate all the thingsMat Mannion
 
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
AD113  Speed Up Your Applications w/ Nginx and PageSpeedAD113  Speed Up Your Applications w/ Nginx and PageSpeed
AD113 Speed Up Your Applications w/ Nginx and PageSpeededm00se
 
Hacklu2011 tricaud
Hacklu2011 tricaudHacklu2011 tricaud
Hacklu2011 tricaudstricaud
 
Netflix oss season 2 episode 1 - meetup Lightning talks
Netflix oss   season 2 episode 1 - meetup Lightning talksNetflix oss   season 2 episode 1 - meetup Lightning talks
Netflix oss season 2 episode 1 - meetup Lightning talksRuslan Meshenberg
 
Build automation best practices
Build automation best practicesBuild automation best practices
Build automation best practicesCode Mastery
 

Similar to Shmoocon 2015 - httpscreenshot (20)

The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
 
CloudStack and testing
CloudStack and testingCloudStack and testing
CloudStack and testing
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec glory
 
Creating Havoc using Human Interface Device
Creating Havoc using Human Interface DeviceCreating Havoc using Human Interface Device
Creating Havoc using Human Interface Device
 
Protect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying TechniquesProtect Your Payloads: Modern Keying Techniques
Protect Your Payloads: Modern Keying Techniques
 
My tryst with sourcecode review
My tryst with sourcecode reviewMy tryst with sourcecode review
My tryst with sourcecode review
 
Introduction to red team operations
Introduction to red team operationsIntroduction to red team operations
Introduction to red team operations
 
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
DevOops Redux Ken Johnson Chris Gates  - AppSec USA 2016DevOops Redux Ken Johnson Chris Gates  - AppSec USA 2016
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
 
PAC 2019 virtual Christoph NEUMÜLLER
PAC 2019 virtual Christoph NEUMÜLLERPAC 2019 virtual Christoph NEUMÜLLER
PAC 2019 virtual Christoph NEUMÜLLER
 
Zane lackey. security at scale. web application security in a continuous depl...
Zane lackey. security at scale. web application security in a continuous depl...Zane lackey. security at scale. web application security in a continuous depl...
Zane lackey. security at scale. web application security in a continuous depl...
 
Using AWS, Terraform, and Ansible to Automate Splunk at Scale
Using AWS, Terraform, and Ansible to Automate Splunk at ScaleUsing AWS, Terraform, and Ansible to Automate Splunk at Scale
Using AWS, Terraform, and Ansible to Automate Splunk at Scale
 
The Clash Between Devops and Quality Assurance
The Clash Between Devops and Quality AssuranceThe Clash Between Devops and Quality Assurance
The Clash Between Devops and Quality Assurance
 
DevOps: Automate all the things
DevOps: Automate all the thingsDevOps: Automate all the things
DevOps: Automate all the things
 
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
AD113  Speed Up Your Applications w/ Nginx and PageSpeedAD113  Speed Up Your Applications w/ Nginx and PageSpeed
AD113 Speed Up Your Applications w/ Nginx and PageSpeed
 
Hacklu2011 tricaud
Hacklu2011 tricaudHacklu2011 tricaud
Hacklu2011 tricaud
 
Don't Mind the Gap by Galen Emery
Don't Mind the Gap by Galen EmeryDon't Mind the Gap by Galen Emery
Don't Mind the Gap by Galen Emery
 
Don't Mind the Gap by Galen Emery
Don't Mind the Gap by Galen EmeryDon't Mind the Gap by Galen Emery
Don't Mind the Gap by Galen Emery
 
Introduction to threat_modeling
Introduction to threat_modelingIntroduction to threat_modeling
Introduction to threat_modeling
 
Netflix oss season 2 episode 1 - meetup Lightning talks
Netflix oss   season 2 episode 1 - meetup Lightning talksNetflix oss   season 2 episode 1 - meetup Lightning talks
Netflix oss season 2 episode 1 - meetup Lightning talks
 
Build automation best practices
Build automation best practicesBuild automation best practices
Build automation best practices
 

Recently uploaded

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 

Recently uploaded (20)

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 

Shmoocon 2015 - httpscreenshot

  • 2. Outline • Who we are • The problem • Our solution • Demo • How we’ve used it • Q&A
  • 3. Who We Are – Steve Breen • Senior penetration tester • Former “Enterprise” developer – Current hacky script developer • Vulnerability and exploit development hobbyist by night @breenmachine breenmachine.blogspot.com
  • 4. Who we are – Justin Kennedy • Lifelong security hobbyist, actively for ~15 years • Intern -> Computer Tech -> Help Desk -> SOC -> Network Security -> Junior PT -> Senior PT -> Team Lead • Terrible at making slides look pretty… if you came to see pretty slides, you may be in the wrong talk. If you came to see an awesome tool that is available (and is OSS) as of today, you’re in the right place. • @jstnkndy / juken (freenode)
  • 5. Before we get started _o/
  • 6. Blue Team challenge Let’s start off with a question… Can you account for every device or application on your network? Why not?
  • 7. Blue Team challenge reasons 1. You’ve inherited an infrastructure that you didn’t build and (of course) not everything was documented by your predecessor. 2. You work in an environment where business units don't necessarily communicate as much as they should and another business unit has spun up some demo or test application without telling you. 3. You forgot about that old NT4 or tomcat box that no one has touched in the past 10 years. 4. Or someone just plugged some shit into a network jack.
  • 8. Red Team challenge 1. We are constantly attempting to compromise organizations that we don't know anything about (besides our recon). 2. It's our job to identify what the target attack surface looks like. 3. Anyone in here ever masscan a /8 for common web ports? 4. Let’s face it, we don’t always have as much time as we’d like on an assessment.
  • 9. *Disclaimer: Some of these may look familiar
  • 15. Our Solution HTTPScreenshot/Cluster • HTTPScreenshot: A python script to screenshot thousands of websites really quickly (and reliably) • Cluster: A script to do “fuzzy matching” on HTML pages. Produce immediately usable output with “similar” pages grouped together.
  • 16. HTTPScreenshot.py • Goals: Fast, Thorough, Automagic • Challenges: Code was hacked together during assessments – needs some TLC • Fun features: – Input is nmap/masscan output – Javascript parsed and executed – SSL autodetect – SSL Certificate domain scraping for vhosts – Headless (configurable fail-over to FireFox) – Threaded – Saves PNG and HTML (good for grep’ing) – Attempts tls 1.0 and falls back to sslv3 when necessary
  • 17. Cluster.py • Identify “similar” websites and group them together • Displays the resulting groups in a useful way (HTML output with JS “hoverzoom”) • Algorithm - Reduces to DBSCAN: – Needed a clustering algorithm that didn’t require definition of “k” – Uses HTML tag/attr values – computes a “similarity” score for two sites • href, name, src, id, class, title, h1 – Works fairly well – could DEFINITELY be improved upon • Supports “diff” reports. Sites that have been changed/added/removed since the last scan
  • 18. Demo!
  • 20. Story 1 - SQLi
  • 21. Story 2 – WTF is SuperGoose?
  • 22. Story 2 – WTF is SuperGoose?
  • 24. Story 3 – Bug Bounties
  • 25. Cheat Sheet: Usage • masscan --iL scope.txt --oG out.gnmap –rate 100000 • httpscreenshot.py -i out.gnmap --headless -- workers 50 --timeout 45 --autodetect --vhosts -- retries 1 • cluster.py –d <httpscreenshot directory>
  • 26. masshttp #!/bin/sh ~/tools/masscan –p1-65535 -iL scope.txt -oG out.gnmap --rate 100000 mkdir httpscreenshot cd httpscreenshot python ~/tools/httpscreenshot.py -i ../out.gnmap -p -t 50 -w 45 - a -vH -r 1 python ~/tools/httpscreenshot.py -i ../out.gnmap -p -t 10 -w 5 -a -vH -r 1 cd .. python ~/tools/cluster.py -d httpscreenshot/
  • 27. Questions / Suggestions? Code: github.com/breenmachine/