Hi guys,
Just posting this incase someone else has the same problem.
I’m running MT 3.19 on an x86 server with 512mb running with 1 256mb Xen instance. The Xen is running the CentOS 5.1 distribution that MT prepared and is linked from the wiki page. The Xen has a static virtual ethernet running priviate IP space which is natted to an external IP by the MT host. All is working ok, except the networking to the CentOS seems really slow.
When I do a TCP dump from inside the CentOS I’m seeing lots of Jump frames being transmitted and the MT host replying with ICMP frag needed messages. The MTU on the CentOS interface is set at the default 1500, but it’s still transmitting frames such as 4K in size.
Here’s an example. 192.168.255.1 is the MT host virtual interface and 192.168.255.4 is the CentOS server. This tcpdump was done on the CentOS. The 92.96.x.x address is my DSL dynamic IP and I’m accessing over the internet.
07:05:20.161157 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: P 5896:7045(1149) ack 2914 win 11479
07:05:20.424021 IP 92.96.x.x.12537 > 192.168.255.4.pcsync-https: . ack 7045 win 17520
07:05:20.424072 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: P 7045:7071(26) ack 2914 win 11479
07:05:20.501044 IP 192.168.255.4.pcsync-https > 92.96.x.x.12544: . 8906:11810(2904) ack 1635 win 7992
07:05:20.501068 IP 192.168.255.4.pcsync-https > 92.96.x.x.12544: . 11810:13262(1452) ack 1635 win 7992
07:05:20.501221 IP 192.168.255.1 > 192.168.255.4: ICMP 92.96.x.x unreachable - need to frag (mtu 1500), length 556
07:05:20.791594 IP 92.96.x.x.12544 > 192.168.255.4.pcsync-https: . ack 8906 win 17449 <nop,nop,sack 1 {11810:13262}>
07:05:20.791615 IP 192.168.255.4.pcsync-https > 92.96.x.x.12544: . 13262:14714(1452) ack 1635 win 7992
07:05:20.944012 IP 92.96.x.x.12537 > 192.168.255.4.pcsync-https: . ack 7071 win 17494
07:05:21.126058 IP 92.96.x.x.12544 > 192.168.255.4.pcsync-https: . ack 8906 win 17449 <nop,nop,sack 1 {11810:14714}>
07:05:21.126086 IP 192.168.255.4.pcsync-https > 92.96.x.x.12544: P 14714:15775(1061) ack 1635 win 7992
07:05:21.261050 IP 192.168.255.4.pcsync-https > 92.96.x.x.12544: . 8906:10358(1452) ack 1635 win 7992
07:05:21.398072 IP 92.96.x.x.12544 > 192.168.255.4.pcsync-https: . ack 8906 win 17449 <nop,nop,sack 1 {11810:15775}>
07:05:21.960867 IP 92.96.x.x.12544 > 192.168.255.4.pcsync-https: . ack 10358 win 17520 <nop,nop,sack 1 {11810:15775}>
07:05:21.960889 IP 192.168.255.4.pcsync-https > 92.96.x.x.12544: . 10358:11810(1452) ack 1635 win 7992
07:05:22.294154 IP 92.96.x.x.12544 > 192.168.255.4.pcsync-https: . ack 15775 win 17520
07:05:22.307008 IP 92.96.x.x.12537 > 192.168.255.4.pcsync-https: P 2914:3554(640) ack 7071 win 17494
07:05:22.307025 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: . ack 3554 win 13245
07:05:22.314840 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: . 7071:11427(4356) ack 3554 win 13245
07:05:22.314986 IP 192.168.255.1 > 192.168.255.4: ICMP 92.96.x.x unreachable - need to frag (mtu 1500), length 556
07:05:23.041152 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: . 7071:8523(1452) ack 3554 win 13245
07:05:23.703738 IP 92.96.x.x.12537 > 192.168.255.4.pcsync-https: . ack 8523 win 17520
07:05:23.703769 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: . 8523:9975(1452) ack 3554 win 13245
07:05:23.703781 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: . 9975:11427(1452) ack 3554 win 13245
07:05:23.968150 IP 92.96.x.x.12537 > 192.168.255.4.pcsync-https: . ack 11427 win 17520
07:05:23.968171 IP 192.168.255.4.pcsync-https > 92.96.x.x.12537: . 11427:15783(4356) ack 3554 win 13245
07:05:23.968223 IP 192.168.255.1 > 192.168.255.4: ICMP 92.96.x.x unreachable - need to frag (mtu 1500), length 556
Now what I found out from some googling is that somewhere TCP Segmentation Offload (TSO) is broken (this is a new one on me). I’m not sure if it’s CentOS, or the Xen network driver in CentOS (don’t think it’s MT).
But either way, to fix this I issue:
ethtool -K eth0 tso off
Then all runs perfect and browsing to the CentOS webserver is much faster and no Jumbo frames are being transmitted.
Hope this helps someone.
Rich