After decades years of struggles, the IP fragmentation remains one of the persistent challenges in IP networks, particularly if you have to implement extra layers in the protocol stack (like MPLS or PPP over Ethernet) without changing the layer-2 MTU size, or if you use tunneling or IP encryption techniques. The generic solution to the IP fragmentation issues should be the Path MTU Discovery that was issued as an RFC in November 1990 and remained a draft standard ever since. However, misconfigured firewalls still prevent us from using this solution reliably.
Fortunately, TCP implementations don't have to rely exclusively on Path MTU Discovery - RFC 4821 describes an alternate probe-based mechanism.
If you cannot get PMTUD to work reliably in your network, you can at least fix the TCP sessions by setting the TCP Maximum Segment Size on the intermediate routers with the ip tcp adjust-mss interface configuration command. Broken UDP-over-IPv4 applications that pretend to use the PMTUD but ignore its results can be fixed with policy-based routing that clears the DF bit in UDP packets. There's no equivalent hack for UDP-over-IPv6.
The worst impact of IP fragmentation is in the router-to-router communication (GRE tunnels or IPSec encryption). If a router-to-router IP packet is fragmented somewhere in the path, the receiving router has to reassemble the original packet, resulting in significantly reduced switching performance. In these cases, it’s best to enable the router’s support for PMTUD with the tunnel path-mtu-discovery interface configuration command (assuming the end hosts support PMTUD as well). Worst case, you can still lower the tunnel MTU size as well as TCP MSS value, resulting in slightly higher switching overhead but ensuring that the GRE or IPSec packets will not be fragmented.
More to explore:
Related blog posts:
- The relationship between interface MTU, IP MTU and MPLS MTU
- All MTUs Are not the Same
- TCP MSS Clamping Explained
- MTU issues (and TCP MSS clamping) in residential IPv6 deployments
- The MPLS MTU challenges
- Asymmetric MPLS MTU Problem
- The impact of MTU mismatch on OSPF
- EIGRP MTU metric
- TCP Problems with Path MTU Discovery, RFC 2923
- Path MTU Discovery, RFC 1191
- IPv6 Path MTU Discovery, RFC 8201