Having an RB450G, I did run my providers connection on one interface (eth1) and my internal network on another interface (eth2).

While optimizing this setup, I created a trunk to a single interface (eth1) with both my provider and internal network (tagging happens on the managed switch). While everything still worked, I however got strange results with speedtest.net ... my typical test site suddenly reported non stable results around 30 a 40 Mbits while it normally runs at 115 Mbps.

My initial idea was that the RB450G had a performance issue... however the load did not pass 5% and when I tested another site via speedtest.net I saw that it did runs at 115 Mbps ??? So the RB450G can handle the load, has no issues with the VLAN trunks, ... but still it does give a strange result ? Maybe the test site was bad... so I also tested on the managed switch directly... but this reported 115 Mbps for all test sites.

In the mean while I'm using again two interfaces (eth1 and eth5) both connected with a trunk to the switch... and now all is fine again ???

Anyone an idea how this can be explained ???

Yes... you're doing a hairpin on ether1. This is NOT a good idea for high bandwidth usage. You'll run into the same limitation if you try to use any combination of ports other than Ether1. Ports 2-5 are on a switch chip and share a single interface to the CPU.

In short, you pretty much have to use Ether1 for the WAN and Ether2-5 for the LAN.

Hope this helps!

But the bandwith is not an issue... using another test site (by which I mean a server location in speedtest.net) runs 115 Mbits fine with this setup and almost no CPU usage ?

In addition, the different VLAN's should seperate the traffic... it should not matter what goes in or out on which interface ?

I can't get any consistent results that proves that it is impossible

Sorry about that. I must have misunderstood. It sounded like you went from 115Mbit/s to 30Mbit/s with the hairpin configuration and went back to normal when you removed that configuration.

After re-reading, I think you're saying that with the hairpin configuration (still a damned odd way to set it up), you can get 115Mbit/s from some sites, but only 30Mbit/s from others.

If that's the case, I do not have an answer for you. Perhaps you can post a diagram/description showing exactly how everything is physically attached for each bandwidth test.

-Troy

While the trunks are not required in setup2 I left as much as possible identical between the two setups.

In setup1 two VLAN's are defined on eth1 on the RB45G

In setup2 the same VLAN's are defined on eth5 as well on the RB45G (in theory I would only need VLAN 100 on eth1 and VLAN200 on eth5 or not even VLAN's at all and let the switch do the work but this is just a simplified overview).

In both case following identical NAT rule is present...

action=masquerade chain=srcnat out-interface=eth1-VLAN100 to-addresses=0.0.0.0

Should I also specify an in-interface ?

In setup1, SOME (and this I don't understand) traffic to speedtest.net runs at instable speeds others at 115 Mbps depending on wich test server I choose.

In setup2, all traffic to speedtest.net runs at 115 Mbps no matter which test server I choose.

I don't understand either. While the 450 shouldn't have any problem pushing that much data in a hairpin configuration, the performance should be the same either way. The fact that it does fail some sites in the hairpin configuration is odd.

Either way though, its a good idea to only ever allow traffic to pass over a physical connection once. Doing it twice when you don't have to is asking for problems (even if it does work).

BTW, what switch are you using? Could this be an issue with the hairpin config on it rather than the 450?

Having traffic enter on one VLAN of a physical interface and leave on another VLAN of the same physical interface is common practice when routing among VLANs.

Check that ether 1 is set to auto negotiate with the switch and check that both ends believe that negotiation was successful and show the same speed/duplex settings,

The switch is a TP-Link SG3216... a managed 1 Gbit switch.

I also don't see why running two VLAN's over ether1 should be problematic, "strange" or asking for problems ?

I also don't see why the interface would negotiate differently having two VLAN's over it or not (but I will test that as soon as possible).

I also don't see why running two VLAN's over ether1 should be problematic, "strange" or asking for problems ?

It isn't.

I also don't see why the interface would negotiate differently having two VLAN's over it or not (but I will test that as soon as possible).

No suggestion that VLANs would cause it to negotiate differently, but worth checking that both sides are set correctly since a duplex issue would potentially show up with traffic going both directions on the same physical interface.

Having multiple VLANs and even routing between them on a single interface isn't a configuration problem. It is common and it does work. What I'm saying is that it can lead to performance problems. Some hardware can handle this better than other hardware.

Routing or bridging traffic back to the same physical interface increases the demand on that interface, and with a MIPS CPU, at least, this will NEVER perform as well as routing/bridging between separate physical interfaces.

The odd thing about this particular instance, is that the loss of performance is not consistent, so we cannot blame it on the hairpin configuration. The problem may still be on the MT or it may be on the TPLink. We've been through the MT and it does not appear to be at fault, so what's left?

---
Off Topic:

If you want to see the performance comparison between single and multiple interfaces, try this config on any 450/493/75x/95x:

ros code

/interface ethernet set ether3 master-port=ether2
/interface bridge add
/interface bridge port
  add port=ether1 bridge=bridge1
  add port=ether2 bridge=bridge1
  add port=ether4 bridge=bridge1
/ip firewall connection tracking set enabled=no
/ip address
  add interface=bridge1 address=192.168.1.1/24
  add interface=ether5 address=192.168.5.1/24

With connection tracking is turned off, we should have optimal performance. From here, run the following tests using two computers and a traffic generator (or maybe just FTP). If you do, please post your results.

1) Ether2-Ether3 (Switched, does not use CPU)

You should get very close to wire speed when using the switch. The CPU will not see any traffic here.

2) Ether1-Ether2 (Bridged, two CPU interfaces)

MT/Linux bridging is pretty solid. You should still get some pretty awesome performance bridging 2 physical interfaces. Additional load on the CPU may have some consequences.

3) Ether1-Ether5 (Routed, two CPU interfaces)

Routing definitely introduces some overhead, but the 600Mhz CPU in the RB450G should be able to handle this without much loss in throughput over the bridging test.

4) Ether2-Ether4 (Bridged, one CPU interface)

Here, we're introducing a significant bottle neck. All traffic will need to pass the same physical interface (between the CPU and Ethernet controller). It wouldn't surprise me to see >60% drop in performance over test #2.

5) Ether4-Ether5 (Routed, one CPU interface)

Like test #4, all traffic is passing over the same physical CPU interface. A performance drop of >60% from #3 would not be a surprise.

Again, I would be interested in seeing your actual results. I'd run the test myself, but do not have an appropriate device sitting on my bench right now.

PS: This is not bagging on MT at all. Across the board, they are hands-down the best bang for the buck. It just never surprises me to see people try these hairpin configurations and wonder why they don't get the performance they were expecting. Knowing the limitations and capabilities of the product will result in a much better user experience.

I'll try to do some tests one of these days... but the problem is dat I don't have a spare Routerboard to test with.