The document provides an overview of how the internet works by discussing various networking concepts and components involved in connecting devices and routing traffic from local networks to external networks and servers. It explains protocols like TCP, UDP, and IP and networking devices like routers, switches, firewalls, proxies, load balancers and VPNs and how they facilitate communication and security. The document is intended to help understand error logs and troubleshoot network issues by providing context on the underlying infrastructure.

1. “There and back again”
How The Internet Works
Photo: http://www.flickr.com/photos/86530412@N02/8210762750/ by StockMonkeys.com
A talk by Jon “The Nice Guy” Spriggs
First given at PHPNW December 2012

2. It's simple, right?

3. It's all about perspective
● The previous slide was right “from a certain
point of view”
● But it left out a lot of complicated bits
● Things like:
– What does your router do?
– How does your ISP reach your server?
– What about the network where the server is?
● So, let's drill down a bit

4. Much closer, but still not quite right

5. Getting across it all...
● Many of you will already refer to all your connections as being
TCP/IP connections
– They're not all TCP/IP, some are UDP/IP, or ICMP/IP, or just, IP, or
ARP, or GRE/IP, or IPSEC/IP or … well, lots of things.
– And that doesn't get us anywhere near the actual application protocol
● It's actually explained through a few different models
– Some refer to it as the OSI Model (ISO/IEC 7498-1)
– Some as the Internet Protocol Suite (RFC1112)
● But, the way you get from host to host, or host to network, is by
piling a few different things on top of each other
● So, let's look at how that is supposed to work

7. Let's start with getting on your
network. It's harder than it looks!
● We'll assume it's a CAT5e wired network with DHCP!
● Plug in both ends of the ethernet cable and provide power to the NIC
● Ethernet link (power & comms) detected, speed and duplex (optional), plus media type
(optional++) is negotiated
● When using a switch, it learns the MAC address of the devices behind each port, and
only sends packets for that MAC address to that port. Hubs used to be much more
common, because they wouldn't learn MAC addresses, and would broadcast the traffic
across all it's ports. An attacker could just attach to a hub and see all traffic, but with
switches they must convince the network they have the MAC of the router.
● Computer requests DHCP address using it's MAC address and the DHCP server
replies with an address, netmask, (optional) default route and a lease time.
● When the computer tries to connect to an IP address, it uses it's routing table. If the
address is “directly connected” on the same subnet, it requests the MAC address of the
IP address, otherwise it requests the MAC address of the gateway.
● The computer communicates at “Layer 2” with the MAC address it learned, and the rest
of the link is assumed to have worked*
● If the computer is resolving a DNS name, it's got to communicate with the DNS server
to ask for the IP address of the server, so it can start the IP dance again.

8. WOW, wasn't that hard!
● And this stuff happens EVERY time you
connect! To anything.
● It's a wonder anything on the internet EVER
works!
● And in that explanation you've not even got
past your LOCAL network.
● 99% of the time, you don't need to know about
this stuff, but sometimes it helps when you've
got some weird error log, or network issue to
understand how this all works.

9. TCP and UDP
● TCP is considered reliable, as it ensures a conversation can occur by
using a handshake to prove two-way connections.
– Used in HTTP, HTTPS, SMTP, LDAP, XMPP, FTP, SSH
● UDP is a “fire and forget” protocol – the connection is not guaranteed,
which makes it a faster and efficient protocol, while not always reliable.
– Used in DNS, TFTP, Syslog, NTP, VPNs*
● Some systems will use a combination of both TCP and UDP to perform
different roles, for example voice or video conferencing products will set
up the link using TCP, then share media over UDP.
● DNS uses UDP for client queries, and TCP to exchange updates
between authoritative nodes and replication targets.
● There are other L3 protocols, such as GRE or ESP which use neither
TCP or UDP to communicate.

10. So how does this routing
thing work then?
● Routes are defined in one of three ways
– Statically assigned
● Have I been told which way to go?
● A default gateway classes as this, as does anything added using route add
– Dynamically (e.g. BGP, EIGRP, RIP, etc.)
● Has something else, which I trust, told me what networks live beyond it?
– Local (IP address and NetMask defined)
● Does this address live in my subnet?
● A route has a “Metric” which defines the “cost” of using it, but there are
rules, for example:
– The metric is only relevant with equally specific networks, so, a route to the
network 10.8.0.0/24 is more specific than a route to the 10.8.0.0/16 network
– If we have two routes for 10.8.0.0/24, one with a Metric of 1, and the other with
a Metric of 2, the Metric 1 “wins”, even if the Metric 1 gateway is down
– If we have two routes for 10.8.0.0/24, with the same Metric, but where the
gateway for one is up, and the other isn't, the gateway which is up “wins”
– A local route can be overridden with a static route, but this gets messy FAST

11. WAN Accelerators
● A pair (or cluster) of WAN accelerators will tend to be
deployed between your last router before the WAN router
and the WAN router at either end of a high-volume or low
capacity WAN link
● Each end builds a data dictionary which is exchanged
with the other members and then sends just the
dictionary entries
● Consider, much of IP traffic relates to the various packet
headers, using a WAN accelerator can reduce the
amount of traffic being sent over a known link, especially
with high traffic targets (such as MS-AD servers or DNS)

12. Proxy
● Most of you will have seen/heard of these, particularly if you work
for a company of any size above 2 or 3 employees.
● Usually deployed to broker a connection between you and a web
server, a proxy will intercept the connection request to a remote
server, do “stuff” with the content (cache it, filter it, strip stuff out,
etc.) and then give you the content.
● It used to be very common before NAT was prevalent in network
connections, letting several machines connect to the internet,
appearing as one IP or service.

13. Reverse Proxy
●
A reverse proxy exposes several services as one device sharing
common ports
● Sometimes used to encrypt public traffic (HTTPS → HTTP),
while permitting the private traffic to be intercepted and actions
performed upon it (e.g. AV scans)
●
Mostly seen with web servers, but sometimes mail, FTP or even
several services on one port using a principal called multiplexing
● See also services such as pagekite which permits remote
devices to share their web services with a public URL

14. Host Based Firewall
● Usually best examples of these are IPTables for
Linux, Microsoft Firewall for Windows, PFSense
for BSD.
● They prevent inbound connections where
unexpected, and outbound connections where
specified.
● Good for when you're on public wifi, 3g or raw
internet. Useful if you've got a virus infection
elsewhere in your local network.

15. Firewall (L2)
● A layer 2 firewall looks at the IP headers only
(source IP and port, destination IP and port)
● It's called a layer 2 firewall, or sometimes a
“bump in the wire”, because it's invisible to the
devices either side of the firewall
● It will typically only have a management
address, and will be connected between a
switch and a router, or between a switch and a
server.

16. Firewall (L3)
● Most common firewall deployment.
● Inspect inbound and outbound connections
from a network, matching a white/black list.
● A common place to perform NAT
● This role, on a home network, is performed by
your cable modem or ADSL router, usually...

17. Firewall (L7)
● Otherwise known as an Application Firewall
● This inspects traffic in known protocols (e.g.
HTTP, HTTPS, FTP, SMTP, etc.) and applies
Accept/Deny/Drop rules to those protocols.
● It is usually considered to be slower than L3
firewalls (sometimes even 1/10th as fast)
● Frequently used to hand off AV scanning etc.
● Usually deployed after an L3 Firewall

18. Intrusion Protection System
● Much like the L2 Firewalls, IPS devices tend to
be “bump-in-the-wire” devices.
● They look for network anomalies
● Typically, IPS systems are updated more
frequently than firewalls
● Some IPS devices can update L2, L3 and L7
firewalls to protect immediately against
network threats.

19. Load Balancer
● A load balancer MAY
– Work with a device to see how much load it is under
– Look at how much traffic has been sent to a device
– Round-Robin traffic to each “up” member
– Poll each member to see whether it's still accepting requests
● And then will pass traffic from each request to one of
the members of a cluster.
● Frequently seen in front of DNS servers, Web Servers,
SMTP servers, etc.

20. VPN Terminators
● Usually seen in IPSec or SSL varieties
● Will terminate lots of incoming connections from
external workers
● Mostly implemented as a hardware appliance,
although many L3 firewalls will also perform the same
role on existing hardware
● IPSec VPNs largely being replaced with SSL VPNs for
“Road Warriors” and home workers, although Site-To-
Site VPNs are still the domain of IPSec and Hardware
gateways

21. Closer to your corporate networks

22. OK, that's enough devices, how
about troubleshooting some of this?
● Tools include:
– ifconfig/ipconfig
● Check your interfaces are up, and passing traffic OK
– netstat
● Check your routing table and active/listening connections
– ping/traceroute
● Check your connectivity from host to host
– nslookup/dig/whois
● Check your DNS responses
– nmap
● Check a device's listening services, your Network Topology, OS and App fingerprints
– Packet Captures (e.g. wireshark, tcpdump, snoop)
● Check the traffic looks right

23. If we've got time, we can talk about
some protocols, if you're interested?
● Hands up if you want to talk protocols
– Such as HTTP and HTTPS
– Or why SFTP is not the same as FTPS or FTP
● And what is the difference between Active and Passive FTP
– Why SSH is better than Telnet, and SCP is better than FTP
– How DNS works (I might need some help on this :D)
– How DHCP works
– Different VPN technologies (IPsec, OpenVPN, PPTP, SSH,
and more)
– Or something else...... you choose!

24. “There and back again”
How The Internet Works
Photo: http://www.flickr.com/photos/86530412@N02/8210762750/ by StockMonkeys.com
Questions?