inside the ESP

Posted: August 24, 2013 in technical
Tags: , , , ,

Let’s go somewhere where the bad guy should never go 😛

As I am listening to Blind Guardian, I have such a taste to write some more about my dear IPSec. As many of my other IPSec-related posts, this one either couldn’t have been possible without the help of my colleague and IPSec guru, vmp.

The entire purpose of the fancy IKE (IKEv1, IKEv2) negotiation is to establish a secure tunnel between the 2 IPsec peers, in order to protect the traffic between the sensitive entities, which are:

a. computers from 2 subnets, each behind an IPSec gateway – tunnel mode case, site-to-site

b. a roadwarrior computer and the subnet behind a security gateway – tunnel mode case, remote-access

c. 2 independent computers – transport mode case

No matter whether we negotiate these associations or I establish them statically (manual keying), after the IKE negotiation, the IPSec entities have 2 security databases filled-in: SAD (Security Association Database) and SPD (Security Policy Database), which contain:

SAD: security parameters (encryption/authentication mechanism and keys) to secure (encrypt +/- authenticate) traffic between the entities protected

SPD: the traffic selectors – the identities protected: which are described in the above a.b.c. enumation

The “encryption” of the traffic is done via the ESP (Encapsulating Security Payload – RFC 2406) protocol or using AH (Authentication Header – RFC 2402) protocol. The first one does encryption +/ authentication (authentication only for the payload data), the second one does only authentication (of the entire packet, exception the mutable fields of the outer IP header, like: TTL or TOS).

This post hopes to show how to decrypt the ESP packets using Wireshark ( I have an 1.5.0 SVN, you should get at least a newer one, cuz this one still has issues with IKEv1 – ESP keys). Make sure your Wireshark is able to do this by following the check-up at this wiki link:

If I were to grab the ESP packets and load up the capture in Wireshark, we would see a messed up payload. How about the good guys do troubleshooting and want to look inside these ESP packets? (the bad guy should NEVER do this 😛 )?

The _good_ guys would need, of course, the ESP keys from the SAD. I am using Strongswan to log the keys.

There are 2 ways to get these keys:

1.a. IKEv1 case: debug logging and then look at pluto.log

1.b. IKEv2 case: debug logging and then look at the charon.log

2. if everything fails (I wouldn’t even bother with the 2 above), IKEv1/IKEv2: list the linux kernel’s database:

– use the FreeBSD setkey tool

– or, for newbies like myself, use ip xfrm state command

The ip xfrm state command will dump the SAD and you can take the keys from there.

Btw: to dump the SPD, use the ip xfrm policy command.

What else?: mwell, we need to know the IPSec peers.

Ok, we look into the logs, or better use the keys output by the above commands, then save them nearby. Remember, in a common case, we have ESP with AH, where we have 2 keys, 1 for ESP encryption and 1 for ESP authentication. We can also do ESP-NULL encryption, when we will only have 1 AH key.

Open wireshark. Go to Edit > Preferences … , expand the +Protocols tree in the left panel, go to ESP. You will have there 16 available information forms to fill-in, meaning that you can decapsulate 16 ESP flows at once. Let’s have a look at my wireshark.


I have here the my 2 IPSec peers, Alice ( and Bob (, or whatever you want to call them and whatever IP addresses you want to give them…so on…

We have to specify the type of outer IP of the packets (type of IP of the peers), here is IPv4, the IPSec peers and the SPI (Security Parameter Index, the index in the SAD for this SA tuple), which I simply defined as “any”. Then defined the proper algorithms I know to have used and the keys grabbed above from the kernel.

Now click OK and look at the ESP packets, you should be able to have content there, under the ESP header, not just encrypted “Payload Data” .

BUT: for a professional how to on this, use the actual wiki links describing this procedure.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s