Determining the Optimal MTU for GRE or IPsec Tunnels

A suboptimal MTU for your organization's GRE or IPsec tunnel results in severe performance degradation. Below you can learn how to determine the optimal MTU for your organization's tunnels.

Overview

When a user from your organization requests a web site, the user's traffic first travels from your organization's edge network appliance (for example, a router or firewall) to a Zscaler Enforcement Node (ZEN) via a primary or secondary GRE or IPsec tunnel. From there, the ZEN sends the traffic out to the requested destination web server if it complies with your organization's security and compliance policies. See the image below.

Overview

When you configure a GRE or IPsec tunnel to the ZEN, you must set a Maximum Transmission Unit (MTU) for the tunnel. The MTU determines the maximum packet size that can be sent over that tunnel, and setting an optimal MTU here is crucial. A suboptimal MTU for the tunnel results in significantly poor performance for your users.

An optimal tunnel MTU is equal to or lower than the following key values:

  • The Network Appliance MTU: The maximum total data per packet allowed by the edge network appliance from which the tunnel is built
  • The Path MTU: The maximum total data per packet allowed by appliances that stand in the path between your network appliance and the ZEN

If your tunnel MTU is larger than either value, the network or path appliance divides each packet into fragments. The appliance then places each fragment into its own packet, with its own header. (The appliance thus must ensure that the maximum size of each fragment is its own MTU minus the header size.) The appliance also records in the header the following information so that the receiving appliance can properly identify the fragments and reassemble them into the original packet that was sent.

  • Total Length: The size of the fragment
  • Identification: The value that identifies the original packet the fragment belongs to.
  • More Fragments (MF): A flag set to a 1 for all fragments except the last one, which is set to 0. A flag set to a 1 indicates to the receiving appliance that more fragments of this packet are coming, while a flag set to a 0 indicates that the appliance has received the last fragment of the packet.
  • Fragment Offset: A value that helps the receiving appliance reassemble the packet fragments into the right sequence

When this fragmentation process occurs for each packet sent through your tunnel, your users will experience significant performance issues.

To help avoid this scenario and ensure efficient packet transport, Zscaler recommends you complete the tasks below to determine and set the optimal MTU for your tunnels.

Configuration Instructions

Determining the Network Appliance MTU

Refer to your network appliance documentation to learn how to determine the appliance MTU. For example, if you have a Cisco appliance, you can find instructions here.

You must determine the network appliance MTU before proceeding to the next step.

Determining the MSS

 

Prerequisites

Before you begin, make sure you have the following information ready:

Instructions

Proceed with the steps below. You can also refer to the example provided.at the end of this section.

  1. Execute the following ping command to the ZEN or VPN host name using the appliance from which you're building the GRE or IPsec tunnels.
	ping -g [network appliance MTU value minus 50] -G 1600 -h 10 -D [destination]
  • This command allows you to discover a range for the maximum segment size (MSS) -- that is, a range for the maximum payload data per packet allowed by appliances that stand in the path between your network appliance and the ZEN. It directs your appliance to send to the destination sweeping pings -- a sequence of packets that incrementally increase in size (by 10 bytes in this case) -- until the packets reach a specified size, or until the packets reach a point at which adding another 10 bytes would make the packets exceed the MSS.
  • Below is a more detailed explanation of the command components and the values to use.
    • -g  = Packet size to start with when sending the sweeping ping.
      The value to plug in for g must equal the network appliance MTU minus 50. For example, if your network appliance MTU is 1450, the value is 1400.
    • -G = Packet segment size to end with when sending the sweeping ping.
      For this command, use the value 1600.
    • -h = Increment (in number of bytes) by which to increase the size of packets when sending the sweeping ping.
      For this command, use the value 10.
    • -D = Prevents the tunnel from fragmenting packets. This is critical to ultimately discovering the MSS. Even if the appliance doesn't reach the G value (the size with which to end the sweeping ping), because of this component, the appliance stops sending packets once it finds it has to fragment packets to keep them from exceeding the MSS. Without this limitation, the appliance would simply continue to send packets by fragmenting them -- for example, if the MSS is 1470, and your packet size was 1478, it would fragment that packet into two packets so that the first would be 1400 bytes, and the second packet 8 bytes.
    • [destination] = This is the packet destination. These are the IP addresses of the primary and secondary ZENs to which your organization forwards traffic.
  • For example, if your organization's network appliance MTU is 1450, and your destination IP address is 10.10.10.13, your ping command is:
ping -g [1400] -G 1600 -h 10 -D 10.10.10.13
  1. When the appliance ends the sweeping pings, identify the packet size at which your pings stopped. You now know that the MSS is somewhere between this value and this value plus 10.
  2. Execute the same ping command, but change the values entered for -g and -h.
    • For -g, enter the the packet size at which your appliance stopped sending packets, as identified in step 2.
    • For -h, use 1 so that the appliance increases the packet size by increments of 1.
ping -g [packet size at which appliance stopped sending packets, identified in step 2] -G 1600 -h 1 -D [destination]
  • For example, if the value you identified in step 2 was 1450, your ping command would be:
ping -g [1450] -G 1600 -h 1 -D 10.10.10.13
  1. Again, identify the packet size at which your pings stopped. That value is your MSS.

See an example.

Finding VPN host name

Example1

In this example:

  • The network appliance MTU is 1330
  • The destination ZEN IP address is 192.152.0.19

The ping command to execute in this case would be:

g 1330 -G 1600 -h 10 -D 192.152.0.19

Upon execution, you may see results similar to the content below.

ping -g 1330 -G 1600 -h 10 -D 192.152.0.19
PING 192.152.0.19 (10.152.0.19): (1330 ... 1600) data bytes
1338 bytes from 192.152.0.19: icmp_seq=0 ttl=121 time=418.883 ms
1348 bytes from 192.152.0.19: icmp_seq=1 ttl=121 time=441.258 ms
1358 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1368 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1378 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1388 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1398 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1408 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1418 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1428 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1438 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1448 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1458 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1468 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
ping: sendto: Message too long
ping: sendto: Message too long
ping: sendto: Message too long
  • The appliance sent to the IP address 192.152.0.19 pings that start at a packet segment size of 1330 bytes, increasing the size by increments of 10.
  • The appliance stopped sending packets once they reached 1468 bytes (even before they reached the G value of 1600 bytes). Since the command specified that packets could not be fragmented, the appliance stopped sending packets when adding another 10 bytes to 1468 would have made the packet size exceed the MSS -- in other words, the point at which the appliance would have had to begin fragmenting packets in order to transport them.
  • From this, it can be deduced that the MSS is somewhere between 1468 and 1478.
  1. With the information from the first ping command, you would execute the following second ping command:
ping -g 1468 -G 1478 -h 1 -D 192.152.0.19

Upon execution, you may see results similar to the content below.

ping -g 1468 -G 1478 -h 1 -D 192.152.0.19
PING 192.152.0.19 (10.152.0.19): (1330 ... 1600) data bytes
1468 bytes from 192.152.0.19: icmp_seq=0 ttl=121 time=418.883 ms
1469 bytes from 192.152.0.19: icmp_seq=1 ttl=121 time=441.258 ms
1470 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1471 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
1472 bytes from 192.152.0.19: icmp_seq=2 ttl=121 time=289.218 ms
ping: sendto: Message too long
ping: sendto: Message too long
ping: sendto: Message too long

From this result, you can conclude that your MSS is 1472 bytes.

Calculate the Path MTU

With your MSS, you can now calculate the path MTU -- the maximum packet size allowed by appliances that stand in the path between your network appliance and the ZEN. The path MTU is the MSS value plus the values for the IP header (20 bytes) and the ICMP header (8 bytes). Use the following calculation.

Path MTU = MSS + 20 Bytes (IP Header) + 8 bytes (ICMP Header) 

For example, if your MSS is 1472, your path MTU would be 1500 (that is, 1472 + 20 + 8).

Use the lower value as the MTU for your tunnel

Compare your network appliance MTU and your path MTU. Make your tunnel MTU equal to the lower of these two MTU values.

For example, if your network appliance MTU is 1500, and your path MTU is 1300, the value you set as the tunnel MTU would be 1300. This ensure that packets traveling through your GRE or IPsec tunnel do not exceed the packet size limitations of your network appliance or other appliances in the path between your network appliance and the ZEN.

NOTE: If you experience issues performing the tasks above, Zscaler recommends that you use a tunnel MTU of 1400.