Algorithms for extraction and visualization of metadata from Domain Name Server records -- 2010 Third International Conference on Advances in Mesh Networks

1. 2010 Third International Conference on Advances in Mesh Networks
Algorithms for extraction and visualization of
metadata from Domain Name Server records
Arley Barros Leal da Silveira 1 Nuno M. Garcia 1,2,3
1 2
Universidade Lusófona de Humanidades e Instituto de Telecomunicações,
3
Tecnologias, Lisbon, Portugal Universidade da Beira Interior,
arleybls@gmail. Covilhã, Portugal
ngarcia@professores.ulusofona.pt
Abstract — Nowadays, security awareness is on every industry In this paper we present a tool named TXDNS that
agenda. Mitigating threats and assessing risks is key to addresses these issues, by generating names that are fed to a
successful security posture and business continuity. Penetration DNS name resolver.
tests are state-of-the-art and one of the main processes used by
Previous tools perform similar tasks, such as dnsmap [3],
security professionals to discover, assess and evaluate a given
infrastructure for potential security risks. On this paper we DioNiSio [4], dnsenum [5], Fierce Domain Scanner [6] and
discuss Domain Name Server (DNS) information Maltego [7]. Table 1 shows a comparison list between
reconnaissance, a critical phase of any penetration test TXDNS and other similar tools.
assessment, and present a tool that implements four different
techniques to extract DNS metadata from a given domain Table 1. Comparison of features between TXDNS and other similar tools.
namespace: TLD Rotation, Dictionary Attack, Brute Force
Fierce
Permutation and Typos. We present each algorithm and discuss TXDNS dnsmap DioNiSio dnsenum Domain Maltego
the details of implementation of the tool. Finally, the integration Scanner
of the tool with a powerful visualization engine to provide Multiple
seamless graphical representations of the extracted data is DNS Yes No No No Yes No
discussed. The source code of the tool is available to further Servers
research. Brute Permuta-
No No No No No
Force tion
Dictionary Yes Yes Yes No Yes Yes
I. INTRODUCTION
Including
Penetration tests may be classified in three categories TLD
nd No No No No Yes
Rotation 2 Level
according to the degree of knowledge one has of the Domains
infrastructure to be analyzed: Black Box, Gray Box or Crystal
Typos Yes(*) No No No No No
Box. Black Box and Gray Box tests assume the lack of
previous knowledge of the network information, either totally Graphical
Represen- Maltego No No No No Yes
or partially. Crystal Box tests, in opposition, assume that the tation
tester has full knowledge of the network information. This
Reverse
work falls in the first category of tests, as we assume Queries
Yes No Yes No Yes No
incomplete or no knowledge of the network infrastructure.
One of the advantages of this approach is that it allows the Specified Specified
Threading No No NS, MX Yes
assessment of the degree of exposure for a particular by User by User
infrastructure. The knowledge of the DNS data is a critical Resource
A, CNAME,
HINFO, A, MX, NS, A, MX, NS,
phase of any penetration test assessment, and the tool Records MX, NS, A A, MX, NS A
SOA SOA
presented here implements four different techniques to Supported SOA e TXT
extract DNS metadata from a given domain namespace (*)Transposition, Missing or Wrong Chars
[1][2].
There are several methods to collect information regarding This introduction concludes the first section. The
DNS service machines, such as queries in public search remainder of this paper is organized as follow: Section II
engines, website crawling or social engineering, just to name presents the selected inference and discovery algorithms;
a few. Nevertheless, the amount and type of collected Section III presents the results; Section IV presents the
information is limited by the nature of the methods used, and integration of the tool with the visualization application, and
usually is limited to the information provided by regular DNS finally, Section V concludes the paper with conclusions.
records.
978-0-7695-4092-4/10 $26.00 © 2010 IEEE 81
DOI 10.1109/MESH.2010.20

2. II. INFERENCE ALGORITHMS FOR DNS DATA uusofona.pt or ulusofon.pt instead of ulusofona.pt.
Defined by RFC 1034 [8] and RFC 1035 [9], the Domain With the abovementioned algorithms and some
Name System (DNS) protocol is currently the name persistence, the tool is able to infer a good part of the domain
resolution protocol for the Internet. infrastructure, as the algorithms may be activated one or
It embodies a distributed and hierarchical architecture that several at run-time.
allows a fast response to DNS queries. The DNS service The tool was developed in ANSI/ISO C. The
supports several types of queries, being the Address (A) the implementation of the resolver of the DNS tool was achieved
most common. The tool we describe here focus on the using the DNS API from Microsoft Windows, included in the
following DNS fields, Mail Exchanger (MX), Canonical dnsapi.lib library. Socket management was implemented
Name (CNAME), Text (TXT), Host Information (HINFO), using the wsock32.lib from Microsoft Windows. The
Name Server (NS) e Start of Authority (SOA). programming tool used was Microsoft Visual Studio 2008.
The use of standard tools (e.g. nslookup) to discover the The integration of the tool with the visualization
addresses of machines running services such as web services application was achieved using XML. Also to support a large
or email services is not adequate to discover usually less number of queries, the tool deploys threads that run the
visible sub-domains. To infer a map of the available DNS algorithms in simultaneous; otherwise, queuing the generated
information, we implemented four algorithms: Top Level names to a single DNS resolver would result in a long
Domain (TLD) rotation, Brute Force, Dictionary Attack and execution time.
Typing Errors. These algorithms are described as follows: In summary, the main characteristics of the tool include:
TLD Rotation: in this algorithm, the TLD domain suffix 1) Speed: The tool uses multithreading to faster
of the queried address is changed, e.g., while querying processing of the name query queue. IBM Rational
the domain ulusofona.pt, it will also check all other Quantify & PurifyPlus [12] was used to measure
possible combinations, such as, ulusofona.com, execution time and identify memory leaks;
ulusofona.net, and so on. The TLD list is retrieved from 2) Efficient data structures: The tool implements a linked
IANA and IETF [10][11]. list to manage efficiently the semaphores of the
Brute Force: this algorithm checks for all possible threads;
combinations of defined length for the sub-domains of 3) Ease of operation: The tool may be accessed from the
the queried domain. The character set used for the command line and does not require special skills for
combinations is {a..z}, {0..9} or {{a..z}, {0..9}}, e.g., its operation. The output is easily understandable and
for the ulusofona.pt it will check the a.ulusofona.pt, interpretable;
bb.ulusofona.pt, zzz.ulusofona.pt subdomains and so on. 4) Reliability: The effort on efficient memory and error
Dictionary Attack: this algorithm uses a file containing management make the tool very reliable;
words that will be used as a sub-domain query for a 5) Portability: Despite the use of Microsoft Windows
given domain. The quality of the output of this libraries, the core of the tool is ANSI C, so it is
algorithm depends directly from the quality of the used portable to other platforms.
dictionary. For the ulusofona.pt domain, the dictionary
may include words that result in queries such as At this point, we have not done performance comparison
school.ulusofona.pt, university.ulusofona.pt, and so on. with other tools, mostly for two reasons: firstly, the
Typing Errors: with the increase of phishing attacks, it is algorithms that generate the names later to be fed to the DNS
increasingly common for enterprises to register domain resolver are not complex, thus the architecture of the tool
names that resemble the original domain name, as a itself is not determinant on the performance of the tool (some
form of protection against loss of traffic and attacks that other tools also implement threading); secondly, the DNS
may result from users mistyping the correct address in server response time is several orders of magnitude bigger
the address bar. This algorithm uses a combination of than the capability of the name generation algorithms, and so,
three techniques to simulate typing errors: transposition, the performance of the tool is limited by the speed of the
double typing, and missing of a character, and are response of DNS servers. As the DNS server infrastructure a
described as follows: common resource to all the tools, the performance assessment
1) Transposition: this technique foresees the typing of a particular tool has no interest, as any assessment will be
mistake that occurs when the user types the key that is limited by the same performance restriction, the DNS
on the side of the originally intended key. For infrastructure.
example, instead of ulusofona.pt the user types
ylusofona.pt or uluaofona.pt, and so on; III. RESULTS
2) Double typing: pressing the same key twice is a This section presents the results obtained with this tool.
common mistake. This technique generates names that The main goal of the tool is to allow the discovery of less
include a double typing mistake, e.g. uulusofona.pt or visible topologies and relations [13] using the data stored in
ulusoofona.pt and so on; the DNS, and finally to use this information to prevent
3) Missing a key: similarly as before, missing a key is system break-ins [14].
also common. This feature generates names such as Figure 1 shows some results when the tool is used with the
82

3. TLD Rotation algorithm for the domain ulusofona.pt. As the
type of DNS information requested is not specified, the tool D:>txdns -rt ulusofona.pt -rr MX
assumes that it must retrieve the Address (A) field from the -----------------------------------------------
TXDNS (http://netlab.ulusofona.pt/id) 0.1.2
DNS record. It is visible in Figure 1 as the use of this -----------------------------------------------
algorithm brings 37 new hosts to our name catalogue. > ulusofona.mp - forwarding.chi.mp
Figure 2 shows a similar query, but specifying for the > ulusofona.tk - MX-HOST.DOT.tk
> ulusofona.pt - mtorga.ulusofona.pt
retrieval of Mail Exchange records (MX). > ulusofona.nhs.uk - mail.nhs.uk
Figure 3 shows a brute force query on the microsoft.com ----------------------------------------------
domain. The length of the sub-domain is defined with a Resolved names: 4
minimum of 1 and a maximum of 3. As the character set for Failed queries: 1445
Total queries: 1449
the combinations is not specified, the tool assumes the ------------------------------------------------
{{a..z}, {0..9}} character set. This run-time includes the Fig. 2. Using the Top Level Domain rotation algorithm for search of (MX)
definition of 20 threads. records on the DNS database for the ulusofona.pt domain.
Viewing the results shown in Figure 3, one may find odd
D:>txdns -bb --min 1 --max 3 microsoft.com -x 20
that the well-known www.microsoft.com host is not detected. -----------------------------------------------
To look for this, we will run the tool again, but this time we TXDNS (http://netlab.ulusofona.pt/id) 0.1.2
will search for the Canonical Name (CNAME) record of the -----------------------------------------------
> m.microsoft.com - 65.55.186.23
DNS entry. Figure 4 shows the output. > ea.microsoft.com - 131.107.88.60
> eu.microsoft.com - 207.46.197.32
D:>txdns -rt ulusofona.pt > ea.microsoft.com - 131.107.88.60
------------------------------------------------ > fs.microsoft.com - 131.107.0.125
TXDNS (http://netlab.ulusofona.pt/id) 0.1.2 > ga.microsoft.com - 207.46.197.32
------------------------------------------------ > ga.microsoft.com - 207.46.232.182
> ulusofona.com.ph - 203.119.4.28 …(partially omitted results) …
> ulusofona.cg - 64.18.138.88 > sip.microsoft.com - 131.107.106.16
> ulusofona.kr - 222.231.8.226 > smm.microsoft.com - 65.55.100.45
> ulusofona.mp - 75.101.130.205 > tag.microsoft.com - 207.46.140.29
> ulusofona.net.ph - 203.119.4.28 > vua.microsoft.com - 65.54.96.220
> ulusofona.ph - 203.119.4.28 > vua.microsoft.com - 65.54.96.220
> ulusofona.ngo.ph - 203.119.4.28 > wer.microsoft.com - 65.55.22.188
> ulusofona.org.ph - 203.119.4.28 > wsp.microsoft.com - 207.46.248.105
> ulusofona.i.ph - 203.119.4.38 -----------------------------------------------
> ulusofona.mil.ph - 203.119.4.28 Resolved names: 54
> ulusofona.nu - 62.4.64.119 Failed queries: 19706
> ulusofona.rw - 64.18.138.88 Total queries: 19760
> ulusofona.gouv.rw - 64.18.138.88 ------------------------------------------------
> ulusofona.st - 195.178.186.40 Fig. 3. Using the Brute Force algorithm to search of (A) records on the DNS
> ulusofona.tk - 193.33.61.2 database for the microsoft.com domain.
> ulusofona.co.st - 195.178.186.40
> ulusofona.ws - 64.70.19.33
> ulusofona.com - 208.73.210.27 D:> txdns -bb --min 1 --max 3 microsoft.com -x
> ulusofona.com.ba - 195.222.33.180 20 -rr CNAME
> ulusofona.com.cn - 218.241.97.60 -----------------------------------------------
> ulusofona.kr - 222.231.8.226 TXDNS (http://netlab.ulusofona.pt/id) 0.1.2
> ulusofona.a.nf - 88.191.93.163 -----------------------------------------------
> ulusofona.ph - 203.119.4.28 > s.microsoft.com - reroute.microsoft.com
> ulusofona.gob.ve - 150.188.4.235 > c.microsoft.com - c.microsoft.akadns.net
> ulusofona.vn - 72.52.194.126 > g.microsoft.com - g.msn.com
> ulusofona.biz.vn - 72.52.194.126 …(partially omitted results) …
> ulusofona.edu.vn - 72.52.194.126 > www.microsoft.com - toggle.www.ms.akadns.net
> ulusofona.gov.vn - 203.119.8.111 -----------------------------------------------
> ulusofona.net.vn - 72.52.194.126 Resolved names: 38
> ulusofona.org.vn - 72.52.194.126 Failed queries: 19722
> ulusofona.int.vn - 72.52.194.126 Total queries: 19760
> ulusofona.health.vn - 72.52.194.126 -----------------------------------------------
> ulusofona.ac.vn - 72.52.194.126 Fig. 4. Using the Brute Force algorithm to search of (CNAME) records on
> ulusofona.com.vn - 72.52.194.126 the DNS database for the microsoft.com domain.
> ulusofona.info.vn - 72.52.194.126
> ulusofona.name.vn - 72.52.194.126
The expected www.microsoft.com is now visible in Figure
> ulusofona.pro.vn - 72.52.194.126
------------------------------------------------ 4, and this may be used as an example of the importance of
Resolved names: 37 the different records on the DNS entry.
Failed queries: 1412 Figure 5 shows the results of a query using a dictionary on
Total queries: 1449
-----------------------------------------------
the microsoft.com domain. The efficiency of this algorithm
depends exclusively of the quality of the used dictionary. For
Fig. 1. Using the Top Level Domain rotation algorithm for search of (A)
records on the DNS database for the ulusofona.pt domain. this example, a dictionary of 580 words was used.
83

4. Dictionaries such as the one used are available on the 1. Dictionary query with TLD rotation query for (A)
Internet, in several languages. and for (TXT) records:
Figure 6 shows the use of the algorithm simulating typing txdns –rt -f namelist.txt dominio.com
errors on the google.com domain. It is visible how similar txdns –rt -f namelist.txt dominio.com –rr TXT
names generate DNS responses.
2. TLD rotation with typing errors for (MX) and
(HINFO) records:
D:> txdns -f namelist.txt microsoft.com
-----------------------------------------------
txdns –rt –t dominio.com –rr MX
TXDNS (http://netlab.ulusofona.pt/id) 0.1.2 txdns –rt –t dominio.com –rr HINFO
-----------------------------------------------
> accounting.microsoft.com - 207.46.131.251 3. Typing errors and dictionary queries for (SOA)
> agent.microsoft.com - 207.46.197.32
> billing.microsoft.com - 65.54.159.250
and (NS) records:
> channels.microsoft.com - 207.46.232.182 txdns –t –f namelist.txt dominio.com –rr SOA
> directory.microsoft.com - 131.107.115.87 txdns –t –f namelist.txt dominio.com –rr NS
> design.microsoft.com - 207.46.232.182
> example.microsoft.com - 207.46.197.32
…(partially omitted results) … Worthy of note is the fact that a query for a (A) record may
> rss.microsoft.com - 207.46.232.182 result in more than on IP address, e.g. the query for
> sharepoint.microsoft.com - 207.46.105.139 www.microsoft.com may return one or more IP addresses.
> services.microsoft.com - 207.46.132.190
The tool includes the possible use of the –v parameter that
> shop.microsoft.com - 207.46.232.182
> smtp.microsoft.com - 205.248.106.32 allows the display of detailed information on a given query.
> transfer.microsoft.com - 207.46.236.112 Figure 7 shows the result of a query for TLD rotation test on
> ts.microsoft.com - 131.107.106.15 the ulusofona.pt domain.
> windows.microsoft.com - 65.55.81.30
-----------------------------------------------
We can see as this query now returns not only one result
Resolved names: 42 for the ulusofona.pt domain, but two results, the first one for
Failed queries: 538 mtorga.ulusofona.pt and the second one for
Total queries: 580 smtp.empresas.novis.pt.
-----------------------------------------------
Fig. 5. Using the Dictionary algorithm to search of (A) records on the DNS
database for the microsoft.com domain. D:>txdns -rt ulusofona.pt -rr MX -v
----------------------------------------------
TXDNS (http://netlab.ulusofona.pt/id) 0.1.2
D:> txdns -t google.com ----------------------------------------------
----------------------------------------------- > ulusofona.mp - forwarding.chi.mp
TXDNS (http://netlab.ulusofona.pt/id) 0.1.2 > ulusofona.tk - mx-host.dot.tk
----------------------------------------------- > ulusofona.pt - mtorga.ulusofona.pt
> hoogle.com - 64.202.189.170 | smtp.empresas.novis.pt
> voogle.com - 85.17.35.48 > ulusofona.nhs.uk - mail.nhs.uk
> yoogle.com - 82.98.86.169 ----------------------------------------------
> boogle.com - 70.38.37.248 Resolved names: 4
> foogle.com - 64.13.232.120 Failed queries: 1445
> gkogle.com - 82.98.86.165 Total queries: 1449
> giogle.com - 208.87.33.151 -----------------------------------------------
…(partially omitted results) … Fig. 7. Using the TLD rotation algorithm to search of (MX) records on the
> gooogle.com - 74.125.91.104 DNS database for the ulusofona.pt domain, with the –v parameter allowing a
> gooogle.com - 74.125.91.104 complete view of all the data retrieved from the DNS resolver.
> ggoogle.com - 64.233.161.104
> googgle.com - 208.73.210.50
> googlr.com - 64.233.161.104
> googlle.com - 69.46.228.38 IV. INTEGRATION WITH THE VISUALIZATION TOOL
> googlee.com - 64.233.161.104 Matelgo [7] is an Open Source solution, developed in Java
-----------------------------------------------
Resolved names: 38 by the company Paterva. It is oriented to forensic analysis
Failed queries: 19722 and includes a sophisticated graphical engine that allows the
Total queries: 19760 graphical representation of the information in a user friendly
----------------------------------------------- and intuitive manner.
Fig. 6. Using the Typing errors algorithm to search of (A) records on the
DNS database for the google.com domain. The integration of the tool with Maltego was achieved by
implementing an option that modifies the standard output of
The use of a single algorithm generates a large volume of the program (stdout) to an XML stream formatted according
metadata, as one may deduce from the previously shown to Paterva rules.
figures. Yet the tool allows us to create scenarios that include To integrate the output, it is necessary to create inside
more than one algorithm. Some examples may be seen next, Maltego the customized transformation resources to our tool.
and as the extension of the output is long, the results shown Once created the resource, it is necessary to add a Domain or
in Figure 7 are clipped: DNSName object to Maltego workspace, applying the created
84

5. transformation resource. Figure 8 shows the visualization of Given the results the tool is able to return, we believe that
the result previously observed in Figure 1. it shows great potential for use in network security tests, such
Besides of the graphical representation of the results of the as, e.g. penetration tests. Further research will include
queries, Maltego allows us to create new queries on top of handling DNS records for IPv6. The tool and its source code
previous results. This may result in a complex tree of are available at http://netlab.ulusofona.pt/id.
information that exposes in a intuitive manner the DNS
structure of a given domain. ACKNOWLEGEMENTS
Figure 9 shows the graphical representation of the TLD
Authors acknowledge the help of Paterva, in particular
rotation query performed on the google.com domain,
Roelof Temmingh and Andrew MacPherson.
followed by a dictionary query on the same domain.
REFERENCES
[1] Stahl, M: RFC 1032: DOMAIN ADMINISTRATORS GUIDE (last
accessed the 7th April 2010), http://tools.ietf.org/html/rfc1032
[2] Lottor, M: RFC 1033: DOMAIN ADMINISTRATORS OPERATIONS
GUIDE (last accessed the 7th April 2010),
http://tools.ietf.org/html/rfc1033
[3] dnsmap, Passive DNS network mapper a.k.a. subdomains bruteforcer,
(last accessed 7th April 2010), http://code.google.com/p/dnsmap/
[4] dnsenum, enumerate DNS info about domains, (last accessed 7th April
2010) http://code.google.com/p/dnsenum/
[5] DioNiSio, DNS scanner, (last accessed 7th April 2010),
http://dionisio.sourceforge.net/
[6] Fierce Domain Scanner, (last accessed 7th April 2010),
http://ha.ckers.org/fierce/
[7] Paterva Maltego, (last accessed the 7th April 2010),
http://www.paterva.com/maltego/
[8] Mockapetris, P: RFC 1034: Domain Names – Concepts and Facilities
(last accessed 7th April 2010), http://tools.ietf.org/html/rfc1034
[9] Mockapetris, P: RFC 1035: Domain Names – Implementation and
Specifications (last accessed the 7th April 2010),
Fig.8. Graphical representation of the results obtained with TLD rotation http://tools.ietf.org/html/rfc1035
query for the (A) records on the ulusofona.pt domain. [10] IANA Root Zone Database, (last accessed the 7th April 2010),
http://www.iana.org/domains/root/db/#
[11] Eastlake, D., Panitz, A.: RFC 2606: Reserved Top Level DNS Names
(last accessed the 7th April 2010), http://tools.ietf.org/html/rfc2606
[12] IBM Rational PurifyPlus, (last accessed the 7th April 2010),
http://www-01.ibm.com/software/awdtools/purify/
[13] Samwalla, R., Sharma, R., Keshav, S.: Discovering Internet Topology,
Cornell University, (last accessed the 7th April 2010),
http://www.cs.cornell.edu/skeshav/papers/discovery.pdf
[14] Bellovin, S.: Using the domain name system for system break-ins,
1995, (last accessed the 7th April 2010), http://www.usenix.org/
publications/library/proceedings/security95/full_papers/bellovin.pdf
Fig.9. Graphical representation of the TLD rotation query on the (A)
records for the google.com followed by a dictionary query on the same
domain.
V. CONCLUSIONS
This research focused on harvesting DNS metadata, by
using inference algorithms for the generation of names that
were subsequently fed to a DNS name resolver. For that
purpose a specific tool was built, using a standard
programming language. Furthermore, as to allow a more
intuitive visualization of the results, the tool was integrated
with a commercial graphical package.
85