IPsec is a suite of protocols designed to secure network connections, applicable to both IPv4 and IPv6, providing authentication, verification, and confidentiality of data. It includes protocols like Authentication Header (AH) and Encapsulating Security Payload (ESP), which can function in transport or tunnel modes, addressing different security needs. AH focuses on authenticating data without encryption, while ESP offers both authentication and encryption, suitable for secure VPN communications.

1. IPSec Basics – Level-1
Avadhesh Agrawal (avagrawal@gmail.com)
V1.0/Dec-2013
1

2. What is IPSec?
• Suite of Protocols for securing network connections
• Seen as extension to IP Protocol family
• Works pretty same way for IPv4 & IPv6
• Provides following basic services
Service
Purpose
Authentication & Verification
Authentication means that ensuring that data has come from
authentic user only
Verification means that ensuring that data is not altered during the
journey
Confidentiality
V1.0/Dec-2013
Ensures that data is not visible to third party – during the journey
2

3. IPSec Terminology (1/2)
• IPSec Protocols used
IPSec Protocol
Purpose
Authentication Header (AH)
Authenticates the data flowing over connection
Encapsulating Security
Payload (ESP)
Encrypts+Authenticate the data flowing over connection
• Note : These protocol are typically used independently – however can be used together (pretty uncommon) as well.
• Transport vs Tunnel Mode
Transport Mode
Tunnel Mode
Provides a secure connection between two
end-points
Provides a secure connection between two gateways
or either of the end is a gateway i.e. host-to-gateway
communication & vice versa
Encapsulates IP Payload only
Encapsulates complete IP packet (IP Header +
Payload)
Simply a secured IP connection
Primarily used for VPN
Both IPSec protocols (AH as well as ESP)
can function in Transport Mode
Both IPSec protocols (AH as well as ESP) can
function in Transport Mode
V1.0/Dec-2013
3

4. IPSec Terminology (2/2)
• Transport vs Tunnel Mode (Cont …)
Note : More on Transport/Tunnel mode later
V1.0/Dec-2013
4

5. IPv4 Datagram (refresher)
Field
Purpose
Ver
Protocol Verion (4 = IPv4)
Hlen
Header Length (as count of 32bit words). Excludes
payload size & other headers
TOS
Type Of Service Indicator
Pkt Len
Total packet length (in bytes). Including header length
in bytes.
ID
Used for associating fragmented packets
Flgs
Mainly used during fragmentation
Frag Offset
Identifies the position of this fragment in complete unfragmented packet.
TTL
Time To Live – in terms of Hops
Proto
Type of protocol (UDP/TCP/etc) as encapsulated in
payload. Addition for IPSec:
50 : IPSec : ESP
51 : IPSec : AH
Header Cksum
Cheksum of entire IP Header (Payload Excluded). Not
a cryptographic checksum. Purpose confined to detect
errors during transmission
Src/dst IP address
Source / Destination IP Address
IP Options
Optional (application specific information)
V1.0/Dec-2013
5

6. AH – Overview(1/N)
• As already discussed - through AH – authentication is done – but no - encryption
– Thus, sniffer can still read the contents of message
• Still serves three purposes
– Helps in ensuring that sender is a authentic one. In other words, we (as an receiver) are able to validate
that we have received data from a valid sender only.
– Helps in detecting any alterations to data during transit
– (Optionally) prevents replaying of data i.e. malicious user reading the data & then re-injecting same
data at later point in time
• Authentication is performed by computing cryptographic hash-based message
authentication code
– Nearly all fields of IP packet (Header as well payload) considered
• Fields liable to change during transit are excluded – like TTL, Header Checksum
– Intermediate hops cannot (as they will not have IPSec related information for this particular connection)
and need not re-calculate authentication code
V1.0/Dec-2013
6

7. AH – Header(2/N)
Field
Length
(Bytes)
Purpose
Next Header
1
Protocol type of next payload. Note that, in Tunnel Mode, next payload will be IP Packet (i.e. IP
Header + IP Payload (i.e. Protocol (say TCP/UDP etc) Header + Protocol Payload)s
Payload Len
1
Specifies AH Header Length. Note Don’t get confused with the field name “Payload Len”.
Defined in, 32-bit words – “minus 2”.
Reserved
2
For future use – must be set to “0”
SPI
4
Security Parameter Index. In simple words, identifies security parameters associated with a
given connection.
Sequence
Number
4
Continuously increasing number – with every packet. Primarily to avoid replay attacks. On
reaching maximum value, rather than wrapping-around – connection is re-negotiated
Variable
Integrity Check Value. Cryptographic hash of entire packet – however some fields are left.
ICV
V1.0/Dec-2013
7

8. AH – Transport Mode(3/N)
V1.0/Dec-2013
8

9. AH – Transport Mode(4/N)
• Key noticeable points:
– Majority of fields are authenticated. Fields those can get modified during transit are skipped.
– Original IP packet is modified – as new header viz AH Header, gets added between IP Header & IP
Payload.
– Note the shuffling/usage of protocol code in modified IP Packet.
• In original IP packet, proto field of IP Header was set to “TCP” where as in modified packet, proto field is set to “AH”
• Further, in modified packet, next field of AH Header is set to TCP. Helps receiver in identifying the actual protocol.
• Commonly referred as mechanism to link different headers.
– Lets see, how receiver node reconstructs the original IP packet as sent by sender node.
• Packet is Authenticated
• AH Header is removed
• Value from next (i.e. TCP) is restored in proto field of IP Header.
• Thus original packet is restored
V1.0/Dec-2013
9

10. AH – Tunnel
Mode(5/N)
V1.0/Dec-2013
10

11. AH – Tunnel Mode(6/N)
• Key noticeable points:
– Entire IP packet (IP Header + IP Payload) is encapsulated within another IP packet.
– Thus, modified packet looks like as follows:
• New IP Header (say H1) followed by
• AH header followed by
• Original entire IP packet. Note that, original IP packet remains un-modified in Tunnel Model.
– Note the shuffling/usage of protocol code
• In new IP Header, proto field of IP Header is set to “AH” – indicating that – this packet is of IPSec:AH type
• Further, next field of AH Header is set to IP. This is required because entire original IP packet is encapsulated.
– Lets see, how receiver node reconstructs the original IP packet as sent by sender node.
• Packet is Authenticated
• New IP header & AH Header is removed
• That’s it.
– Suited for VPN kind of environment where tunnel needs to be simply created
V1.0/Dec-2013
11

12. AH – Transport vs Tunnel Mode(7/N)
• How to distinguish between Transport Mode and Tunnel mode?
– Mode information not added explicitly
– Answer lies in the value stored in next field of AH header
• If next = IP then Tunnel Mode
• If next = <proto i.e. UDP/TCP/etc> then Transport Mode
• Authentication Algorithms
– SHA-1
– MD5
V1.0/Dec-2013
12

13. ESP– Overview(1/N)
• More complex that AH
• Allows Authentication as well as Encryption
– Authentication – Optional
• Header format is different than AH – discussed later
– Even has a trailer as well
• Provides Tunnel as Transport Mode – as in AH
• ESP Encryption algorithms commonly used
– DES, DES3, AES, Blowfish
– Algorithm & key used during encryption, is already negotiated during connection establishment phase
(a new world in itself – discussed separately)
• ESP Authentication algorithms commonly used
– Same as used in AH i.e. SHA-1, MD5
V1.0/Dec-2013
13

14. ESP Header/Trailer
(w/o Auth)(2/N)
Field
Length
(Bytes)
Purpose
ESP Header
ESP Trailer
4
Security Parameter Index. In simple words, identifies security parameters associated with a
given connection.
Sequence
Number
4
Continuously increasing number – with every packet. Primarily to avoid replay attacks. On
reaching maximum value, rather than wrapping-around – connection is re-negotiated
Encrypted
Payload
V1.0/Dec-2013
SPI
Variable
Depending upon the mode – Transport or Tunnel, Either IP payload (Transport) or entire IP
Packet (Tunnel) gets encrypted here. Note: ESP Header (i.e. SPI & Sequence No.) is NOT
encrypted.
Padding
0-255
Place-holder for aligning block-oriented algorithms.
Pad Len
1
Length of padding bytes
Next Header
1
Protocol type of next payload. Note that, in Tunnel Mode, next payload will be complete IP
Packet.
14

15. ESP Header/Trailer(with Auth)(3/N)
• Here, Authentication data has been added additionally.
• Only (i) ESP Header & (ii) Encrypted Payload is
authenticated
• Authentication Data field in trailer – not encrypted
• Presence/Absence of Authentication is known to sender.
Receiver gets to know – by virtue of SPI.
Field
Length
(Bytes)
Purpose
Authentication
Data
Variable
Same as ICV in case of AH.
V1.0/Dec-2013
15

16. ESP – Transport Mode (4/N)
V1.0/Dec-2013
16

17. ESP– Transport Mode(5/N)
• Key noticeable points:
– Original IP packet is modified – as new header viz ESP Header, gets added between IP Header & IP
Payload.
– Additionally ESP Trailer (consisting of padding, pad_len, next) gets added
– Optionally, Authentication data may also be added at the end of packet
– IP payload (i.e. TCP Header + TCP payload from previous figure) along with ESP trailer is encrypted
• ESP Header and Authentication Data (if present) – are excluded from encryption
– In case if Authentication is needed then
• ESP Header + encrypted payload + ESP Trailer - authenticated. ICV stored as Authentication Data at the end of
packet
• IP Header - excluded
– Note the shuffling/usage of protocol code in modified IP Packet.
• In original IP packet, proto field of IP Header was set to “TCP” where as in modified packet, proto field is set to “ESP”
• Further, in modified packet, next field of ESP Trailer is set to TCP. Helps receiver in identifying the actual protocol.
• Note that, actual protocol type is encrypted – hence hidden from packet sniffers
V1.0/Dec-2013
17

18. ESP – Tunnel Mode (6/N)
V1.0/Dec-2013
18

19. ESP– Tunnel Mode(7/N)
• Key noticeable points:
– Entire IP packet (IP Header + IP Payload) is encapsulated within another IP packet.
– Thus, modified packet looks like as follows:
• New IP Header (say H1) followed by
• ESP header followed by
• Original entire IP packet. Note that, original IP packet remains un-modified in Tunnel Model.
• Then ESP Trailer followed by
• Optional Authentication Data
– Key difference from ESP – Transport Mode is
• Original IP Header (along with IP payload) gets encrypted as well.
– Note the shuffling/usage of protocol code
• In new IP Header, proto field of IP Header is set to “ESP” – indicating that – this packet is of IPSec:ESP type
• Further, next field of ESP Trailer is set to IP. This is required because entire original IP packet is encapsulated.
V1.0/Dec-2013
19

20. More To Come …
• Security Association
• SPI
• Key Management - IKE
V1.0/Dec-2013
20

21. Resources
• Pretty good tutorial for beginners
– An illustrated guide to IPSec @ unixwiz.net
V1.0/Dec-2013
21