I wanted to get a feel for how MT router in my environment would look and if it would solve a problem I had. So before buying the 4011 the people on this forum helped me decided on I decided to spin up CHR in my esx cluster. I created 1 interface per VLAN having this in the path seems to add around 300ms of latency
Before we start diving into my esx cluster this same cluster has F5 VE's running acting as a gateway for some devices. So the flow looks like this. PC - Switch - Switch - FW - SWITCH - SWITCH - ESX (F5) . I see almost no added latency with this setup compared to this flow PC - Switch - Switch - FW - Switch - Switch - PC. So while I won't 100% say the ESX network is perfect It seems to be able to handle packets at low latency.
I have no firewall rules, 1 static route 0/0 going out to the firewall so I could get my license. I have no bridging and maybe this is my problem? I have no fast path as looking at the docs that's not supported in CHR. This CHR VM has 8 cores at 2.4ghz and 16GB of ram, and runs off an NVME drive locally on the host. I migrated all vms that might cause disk io off before this test and this is never done for the F5 VE's that are on another host. So all the advantages I could give to this vm I have.
I have run the following two tests and the second one is very odd to me as the CHR should NOT be involved.
Wireless host on vlan 103 to wired host on vlan 102 both host manually set to use CHR as their gateway no other settings changed. Ping avgs 600ms
Remove CHS put the gateway back to the hardware firewall ping avgs 250-380ms.
wireless host on vlan103 wired host on vlan103 manually set gateway again to CHR nothing else changed. 600ms avg
change gateway on wired host back to hardware firewall 3 pings lost during network reload 600ms avg
change gateway on wireless host back to hardware firewall 5 pings lost during network reload 300ms avg
For test #2 I never stopped the ping never changed anything other than the gateway. Any thoughts welcome.