Internet-wide Scanning
Jamie O’Hare @TheHairyJ
Orbital
Reconnaissance
Exhaustively discovers publicly
accessible risk prone assets
2 phobos.io
Deloitte left their Active Directory exposed.
With RDP enabled.
3
4
A French Hydroelectric plant’s control panel was exposed.
Remained online despite flooding in the area.
So you want to Scan the Internet?
5
No, you don’t
The Scanners
The Usual Suspects
Censys
Founded in 2015, from
a research project from
University of Michigan
Shodan
Started in 2009 by
John Matherly as a
market research tool
ZoomEye
Launched in 2013, a
product from
Knownsec
7
“
8
Anyone have any documentation or insight on ZoomEye? What is available on their website
isn’t as in depth as I am looking for
What can I do for you?
I am specifically looking for the location of the crawlers, scanning procedure, ports scanned…
There are no work documents for these issues
The With
Censys
Created by the same
research group
Faster and more
random than Masscan
Shodan
“Something similar but
not ZMap”
ZoomEye
XMap & WMap
For both infrastructure
and web-application
scanning
9
Stateless Scanning
Get faster speeds by
splitting the scanning
process in two
Management of responses
can be achieved using SYN
Cookies
10
SYN
seq: x
SYN-ACK
ack: x + 1
The What
Censys
27
Limited additional
support
11
Shodan
512
Support for DBs, RDP
and much more
ZoomEye
1000
NMap Top 1000
however, uses XMap...
The Data
12
▪ Port 80 - HTTP
▫ Apache 2.4.10
▫ <!DOCTYPE html>
▫ WordPress
▪ Port 443 - HTTPS
▫ Heartbleed Check
▫ Certificate Information
▪ Port 554 - RTSP
▪ Port 11211 - Memcache
The How
Horizontal
Single port across
multiple systems
Vertical
Numerous ports across
the same system
13
The When
Shodan and ZoomEye are 24/7
Censys uses regiemented scans
▪ Daily, biweekly, weekly
▪ Take place over 24 hours
14
The Where
15
Grey Noise Intelligence
Collects data on benign
scanners such as Shodan,
as well as malicious actors!
Has a Splunk app now for
SIEM integration
16
Inherent Latency
There is an inherent latency
with using Internet-wide
scanning data
Responses need indexed
and uploaded, this varies
across platforms
17
Summary
Scanning Location Services
Censys
Regimental Horizontal
ZMap USA 27
Shodan
Continuous
Vertical/Horizontal
ZMap-like
Worldwide 512
ZoomEye
Continuous
Vertical/Horizontal
XMap and WMap (?)
China(?) 1000(?)
18
The Use Cases
Interesting Discoveries
Exposed Databases
292 Databases found
within JANET
Infected Services
Watch in ‘real-time’
the spread of malware
across the Internet
Scary Stuff
Crematorium, rail
signal controllers and
nuclear power plants
20
NCSC’s Minimum Cyber Security Standard
25th June 2018
“Ensure that any
infrastructure is not
vulnerable to common
cyber-attacks”
1st October 2018
Using Censys, one can
identify a number of
services vulnerable to
Heartbleed on JANET
21
autonomous_system.asn: 786 and 443.https.heartbleed.heartbleed_vulnerable: true
NCSC’s Minimum Cyber Security Standard 2
25th June 2018
“Support TLS v1.2 for
sending and receiving
email securely”
1st August 2018
Using Censys, one can
identify plenty of
services on JANET not
adhering to this
22
autonomous_system.asn: 786 and 110.pop3.starttls.tls.version: TLSv1.0
Identifying Services that could be used in DRDoS attacks
17th January 2014
US-Cert issues an alert
listing the services
which could be used in
DRDoS
23
20th September 2018
I wrote a blog post,
investigating said
services within JANET
I found 6204 services,
which collectively
could amount to a
2242824 amplification
factor
thehairyj.github.io
Bug Bounties
Twitter
$280
4 SMTP services
vulnerable to POODLE
via Shodan
@omespino
Grab
$5000
Analytics database
exposed due to
misconfigured firewall
via Censys
@vinodsparrow
Twitter
$10080
Private Docker registry
tied to Vine, hosted on
AWS via Censys
@avicoder
24
The Research
The Researchers
University of Arizona
Published multiple
exceptional works all
across the topic
26
ICS and SCADA
Majority of work is
focused here
Vulnerability Scanning
Using the information
provided to find
vulnerabilities
The Vulnerability Scanning
It’s different
▪ Gaining information earlier
▪ Compliment with additional info
▪ Remember, pinch of salt
27
Reconnaissance
Passive Reconnaissance
Active Reconnaissance
Scout: a Contactless Active Reconnaissance Tool
Using Censys data,
Scout associates
Internet-wide
scanning results with
National Vulnerability
Database entries
28
When compared to OpenVAS, Scout was able to return
results with an effectiveness of 74%!
The Know-how
There is more than Shodan
Expand your tool box
Use Internet-wide Scanning for good
Keep an eye on your digital footprint
Don’t advertise your services
Make it require more effort
30
31
THANKS!
Any Questions?
Jamie O’Hare @TheHairyJ

Internet-wide Scanning

  • 1.
  • 2.
  • 3.
    Deloitte left theirActive Directory exposed. With RDP enabled. 3
  • 4.
    4 A French Hydroelectricplant’s control panel was exposed. Remained online despite flooding in the area.
  • 5.
    So you wantto Scan the Internet? 5 No, you don’t
  • 6.
  • 7.
    The Usual Suspects Censys Foundedin 2015, from a research project from University of Michigan Shodan Started in 2009 by John Matherly as a market research tool ZoomEye Launched in 2013, a product from Knownsec 7
  • 8.
    “ 8 Anyone have anydocumentation or insight on ZoomEye? What is available on their website isn’t as in depth as I am looking for What can I do for you? I am specifically looking for the location of the crawlers, scanning procedure, ports scanned… There are no work documents for these issues
  • 9.
    The With Censys Created bythe same research group Faster and more random than Masscan Shodan “Something similar but not ZMap” ZoomEye XMap & WMap For both infrastructure and web-application scanning 9
  • 10.
    Stateless Scanning Get fasterspeeds by splitting the scanning process in two Management of responses can be achieved using SYN Cookies 10 SYN seq: x SYN-ACK ack: x + 1
  • 11.
    The What Censys 27 Limited additional support 11 Shodan 512 Supportfor DBs, RDP and much more ZoomEye 1000 NMap Top 1000 however, uses XMap...
  • 12.
    The Data 12 ▪ Port80 - HTTP ▫ Apache 2.4.10 ▫ <!DOCTYPE html> ▫ WordPress ▪ Port 443 - HTTPS ▫ Heartbleed Check ▫ Certificate Information ▪ Port 554 - RTSP ▪ Port 11211 - Memcache
  • 13.
    The How Horizontal Single portacross multiple systems Vertical Numerous ports across the same system 13
  • 14.
    The When Shodan andZoomEye are 24/7 Censys uses regiemented scans ▪ Daily, biweekly, weekly ▪ Take place over 24 hours 14
  • 15.
  • 16.
    Grey Noise Intelligence Collectsdata on benign scanners such as Shodan, as well as malicious actors! Has a Splunk app now for SIEM integration 16
  • 17.
    Inherent Latency There isan inherent latency with using Internet-wide scanning data Responses need indexed and uploaded, this varies across platforms 17
  • 18.
    Summary Scanning Location Services Censys RegimentalHorizontal ZMap USA 27 Shodan Continuous Vertical/Horizontal ZMap-like Worldwide 512 ZoomEye Continuous Vertical/Horizontal XMap and WMap (?) China(?) 1000(?) 18
  • 19.
  • 20.
    Interesting Discoveries Exposed Databases 292Databases found within JANET Infected Services Watch in ‘real-time’ the spread of malware across the Internet Scary Stuff Crematorium, rail signal controllers and nuclear power plants 20
  • 21.
    NCSC’s Minimum CyberSecurity Standard 25th June 2018 “Ensure that any infrastructure is not vulnerable to common cyber-attacks” 1st October 2018 Using Censys, one can identify a number of services vulnerable to Heartbleed on JANET 21 autonomous_system.asn: 786 and 443.https.heartbleed.heartbleed_vulnerable: true
  • 22.
    NCSC’s Minimum CyberSecurity Standard 2 25th June 2018 “Support TLS v1.2 for sending and receiving email securely” 1st August 2018 Using Censys, one can identify plenty of services on JANET not adhering to this 22 autonomous_system.asn: 786 and 110.pop3.starttls.tls.version: TLSv1.0
  • 23.
    Identifying Services thatcould be used in DRDoS attacks 17th January 2014 US-Cert issues an alert listing the services which could be used in DRDoS 23 20th September 2018 I wrote a blog post, investigating said services within JANET I found 6204 services, which collectively could amount to a 2242824 amplification factor thehairyj.github.io
  • 24.
    Bug Bounties Twitter $280 4 SMTPservices vulnerable to POODLE via Shodan @omespino Grab $5000 Analytics database exposed due to misconfigured firewall via Censys @vinodsparrow Twitter $10080 Private Docker registry tied to Vine, hosted on AWS via Censys @avicoder 24
  • 25.
  • 26.
    The Researchers University ofArizona Published multiple exceptional works all across the topic 26 ICS and SCADA Majority of work is focused here Vulnerability Scanning Using the information provided to find vulnerabilities
  • 27.
    The Vulnerability Scanning It’sdifferent ▪ Gaining information earlier ▪ Compliment with additional info ▪ Remember, pinch of salt 27 Reconnaissance Passive Reconnaissance Active Reconnaissance
  • 28.
    Scout: a ContactlessActive Reconnaissance Tool Using Censys data, Scout associates Internet-wide scanning results with National Vulnerability Database entries 28 When compared to OpenVAS, Scout was able to return results with an effectiveness of 74%!
  • 29.
  • 30.
    There is morethan Shodan Expand your tool box Use Internet-wide Scanning for good Keep an eye on your digital footprint Don’t advertise your services Make it require more effort 30
  • 31.

Editor's Notes

  • #2  Name, Twitter, Napier, Undergraduate now MSc, President of ENUSEC, Internet-wide scanning, Talk at 2600Edinburgh in December, Tweet, Blog I did my dissertation on this topic last year, and now that piece of work is being published at a conference MORE PROVOCATIVE NAME
  • #3 Orbital Reconnaissance much cooler than Internet-wide Scanning . Astronaut, NASA, Sir Patrick Moore Mortician, CyberSec So what can you find via Orbital Recon
  • #5 For all actors this is an interesting opportunity . A good bounty for researchers . Cred and Noterierity for skids . Assets for Hacktivists or Nation State actors . So you might start thinking I should probably start scanning the internet….
  • #6 That is me warning you Operational costs, angry ISP and potential legal costs . So why not just use someone’s data who is already doing it
  • #7 So let’s meet the scanners!
  • #8 So there is 3 . . Starting with Shodan, . Then Censys . ZoomEye - Need to preface something
  • #9  conflicts Language barrier . So I asked twitter, and the ZoomEye lead replied . may not be 100% correct . Nevertheless how do they scan?
  • #10 Starting with Censys Masscan is another IWS tool, talk at DEFCON 22 , fastest tool/but shite . But how do these tools work
  • #11 Well as you can imagine, can’t wait for responses So SYN Cookies are used . So what is being scanned?
  • #12 Censys, DBs, HTTPs, ICS Shodan does much more, VNC, RDP, RTSP . ZoomEye has 0 of these features, but tries to show you potential vulnerabilities based on query . Chinese, great bunch of lads
  • #13 Some examples . getting the data APIs, REST, Python . Now how they scan
  • #14 Starting with Censys Masscan is another IWS tool, talk at DEFCON 22 , fastest tool/but shite . But how do these tools work
  • #15 Shodan and ZoomEye Masscan is another IWS tool, talk at DEFCON 22 , fastest tool/but shite . who, with, what, how and when. The only thing left is Where
  • #16 Colour coded for each tool You can see Shodan is the only worldwide scanner, Censys in USA, ZoomEye in China . so where am I sourcing this data.
  • #17 GreyNoise Intellegence Shows the people scanning the Internet, including Shodan,Cenys, Zoomeye . It also shows when scans are taking place, which is interesting because
  • #18 Internet-Wide Scanning is slow Inherent latency, esp Mobile networks . Finally, to summarize this section
  • #19 In summary ZoomEye* . Now moving on to what this data can be used for fully
  • #20 Why would you use Internet-wide scanners in more depth
  • #21 Internet-wide scanning has lead to a lot of interesting discovers Personally, I have found a lot of misconfigured databases within academic networks . Some researchers has found some scary stuff, famous examples include Crematoriums, French Hydro Electric plants, Rail Signal controls and even nuclear power plants People watch for spread of malware through banner changes ala Dyn DDoS . Along the same vein it has been used to check for implementation of security standards...
  • #22 For example here is the NCSC Minimum security standard It requires infrastructure to not be vulnerable to common cyber attacks/ known vulnerabilities . In case of Janet (which technically) however, does not even do this well . At the bottom here you can see the query used, and just like blue peter, here is that query i prepared earlier
  • #23 Another example Says only use TLS v1.2 cause the other versions are more flawed . But lets have a snoop around JANET to see if anyone is doing this properly . Using the query at the bottom, here are results I prepared earlier
  • #24 A different example, identifying amplification factors Using the US-Certs list and Shodan . Find 6204 services, which potentially could be used to generate 2242824
  • #25 I’ve seen internet-wide scanners been used in Bug Bounties Here is 3 examples../. . Moving past Use cases and now into the academic domain
  • #26 The Research
  • #27 So if you are a student, or love an academic paper UoA... The main focus of research is... . However, there is a small fringe group looking at... . Which in fact is where my research is based
  • #28 So just to self indulge for a second Using the vulnerability scanning features of IWS, or generating vuln data from the IWS data is novel as it gets information earlier . So getting what ports are open, what services are running, what software is used and what vulnerabilities they are susceptible to before even communicating with that system . But with the inherent latency and the lack of communication, you cannot be certain that the data is correct
  • #29 Furthering this self indulgence My research resulted in a tool known as Scout . Scout is the first... . Now with the masturbatory section is over we move onto the conclusions
  • #30 The know-how The takeaways
  • #31 The takeaways from the talk (hopefully) And thats me
  • #32 Thanks for listening I’m open to any questions you have