Community discussions

MikroTik App
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Cascading Routers

Wed Sep 01, 2004 11:45 am

Hi,
Does anyone know why I am seeing the throughput of cascaded routers drop so much compared to when they are back-to-back? i.e. I have four machines each running OSv2.8. I can route traffic through each pair of routers at up to 28Mbps, but when I put them in a cascade of four routers the throughput halves. I am not running any features to shape traffic or anything, just simple static routes.
Many thanks if anyone has any insight to offer!
Peter.
 
User avatar
mag
Member
Member
Posts: 376
Joined: Thu Jul 01, 2004 12:32 pm
Location: Cologne, NRW, Germany
Contact:

Re: Cascading Routers

Wed Sep 01, 2004 12:08 pm

do you see any delay in ping or traceroute? what type are the interconnections? what cpu load do you have while load-testing?
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Wed Sep 01, 2004 12:32 pm

Hi,
All the connections are cross-over ethernet CAT5E, except for the WLAN, which is on coax with inserted attenuation so that the Rx Level is around -65 dBm. Reported conection rate for the WLAN is 54 Mbps.

All ping delays are 2-3 mS.

The CPU loading is;
Box A & F 99% (WinXP running the Mikrotik BW Tester)
Box B (900MHz Pentium Encryptor) varying between 1-98%
Box C (RB230) varying 29-100%
Box D (RB230) varying 7-100%
Box E (700 MHz Pentium) varying 1-46%
CPU loadings seem to vary wildly during a UDP one-way test.
Please note, that even though the two Hosts running the routers are showing 99%, we have proven that they can measure 28Mbps over just the encryptors or just the RB230 Wlans.
Regards,
Peter.
 
User avatar
mag
Member
Member
Posts: 376
Joined: Thu Jul 01, 2004 12:32 pm
Location: Cologne, NRW, Germany
Contact:

Wed Sep 01, 2004 1:42 pm

is it assured, that every ethernet link works without errors. sometimes the half-duplex/full-duplex problem still occurs. (100baseT NICs should be set to auto-negotiation).

what NICs are you using?

Btw: if i understood you right, you do have a wireless (g or a) link in between. then around 28mbps netto data rate is all you will get... (to our experience)

[quote="pritchie"]
All the connections are cross-over ethernet CAT5E, except for the WLAN, which is on coax with inserted attenuation so that the Rx Level is around -65 dBm. Reported conection rate for the WLAN is 54 Mbps.
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Wed Sep 01, 2004 2:06 pm

Hi,
Actually, I need to clarify something. We can get 28 Mbps across the whole cascade when the encryptors have AES128 turned off, but it drops to 14 Mbps when we turn encryption on, even though we can get 28 Mbps through the encryptors when they are back-back with AES on.

i.e.
Encryptors back-back with AES128 on = 28Mbps
Wireless Link Routers back-back = 28 Mbps (as expected as you say)
Encryptors (AES OFF) cascaded with wireless routers = 28Mbps
Encryptors (AES ON) cascaded with wireless routers = 14 Mbps

So turning AES ON on the Encryptors seems to halve throughput even though we know the Encryptors themselves can do much more than this with AES ON. It seems almost that the Mikrotik RB230s have reduced throughput for AES-encrypted traffic even though they themselves are not doing the encryption.
 
User avatar
mag
Member
Member
Posts: 376
Joined: Thu Jul 01, 2004 12:32 pm
Location: Cologne, NRW, Germany
Contact:

Wed Sep 01, 2004 2:17 pm

ok, i see. this looks strange, as the routers between the encryptors should not have to do anything with the encrypted packets besides routing. i guess you are using ipsec with aes? so it is probably an ipsec-handling problem.
are there by chance any mangle-rules on the routers between?

i am missing the
Encryptors back-back with AES128 OFF = ? Mbps
did you try this too?
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Wed Sep 01, 2004 3:01 pm

We have not configured any mangle rules on the RB230 units, and yes, we are using ipsec with aes128 on Mikrotik OSv2.8.

The encryptors back-back (x-over ethernet) with AES OFF can do 49Mbps UDP one-way. Incidentally, we have proven that our 'HOST' machines can do something like 97 Mbps UDP one-way back-back over a x-over.
 
User avatar
mag
Member
Member
Posts: 376
Joined: Thu Jul 01, 2004 12:32 pm
Location: Cologne, NRW, Germany
Contact:

Wed Sep 01, 2004 10:32 pm

just an idea: what througput do you get if you build up an EoIP tunnel over the rb230: eth-(rb230-rb230)-eth

i would test this with and without ipsec. eoip is carried through GRE so it might work somehow... (eoip mtu 1500)
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Thu Sep 02, 2004 10:43 am

Thanks for the suggestion - unfortunately we have had to send the machines we were using as Encryptors out to an exhibition, so I can't do any further testing right now. I appreciate your input on this thread though - would you be interested to know further results once we get the encryptor machines back again?
 
tully
MikroTik Support
MikroTik Support
Posts: 502
Joined: Fri May 28, 2004 11:07 am

Thu Sep 02, 2004 11:31 am

If you are doing a UDP test, you shouldn't see any problems (unless the cpu is overloaded). TCP will slow down with latency, and the more routers and time added for routing, encyptions, filtering, and such, then the slower TCP.

If you think about it, when you have routers in the middle of this cascade, they have to unencrypt and encrypt (double the work of your first example), so it might be CPU overloaded or latency if you are doing TCP testing.

John
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Thu Sep 02, 2004 11:40 am

Hi Tully,
I wasn't anticipating that the routers in the middle of the cascade (RB230s) would be unencrypting and re-crypting. We certainly haven't configured them to and would not expect them to decrypt traffic that they haven't been configured to. The Routers in the middle (RB230s acting merely as a transparent wireless link) should just be routing the traffic straight through without examining the content. The 'stand-alone tests' seem to indicate that the pairs of machines are capable of the 28Mbps and the results quoted below are all for UDP one-way. Are you saying that an RB230 with wireless card will automatically decrypt the traffic before pushing it over the wireless connection?
Many thanks for any input.
 
User avatar
lastguru
Member
Member
Posts: 432
Joined: Fri May 28, 2004 9:04 pm
Location: Certified Trainer/Consultant in Riga, Latvia
Contact:

Thu Sep 02, 2004 1:47 pm

IPsec will fragment packets that have maximal size as encapsulation and encryption increase packet size. Please try decrease the packet size you use for testing and see if having RB's inbetween still halves the data rate. Then we can think of a solution.
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Thu Sep 02, 2004 1:59 pm

I understand what you are saying and that makes sense. I will try that as soon as I can get my system set up again - hopefully later today.
Many thanks.
 
User avatar
mag
Member
Member
Posts: 376
Joined: Thu Jul 01, 2004 12:32 pm
Location: Cologne, NRW, Germany
Contact:

Thu Sep 02, 2004 5:33 pm

yes, of course, thanks
by now we are using mostly special boxes for ipsec. but using mikrotik routers is in the queue.
thread though - would you be interested to know further results once we get the encryptor machines back again?
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Mon Sep 06, 2004 7:19 pm

We did the test again with MTU 1400 and throughput was back to maximum. Thanks very much everyone for your input. By the way, does anyone know of an actual 5.8GHz wireless card which has on-board hardware WEP/IpSec AES and is supported by Mikrotik OS?
Also, I understand that latest version of OS to support Atheros chipset with hardware AES is still in Beta-version and still has some issues to iron out on the wireless front? Please correct me if I'm wrong.
 
User avatar
lastguru
Member
Member
Posts: 432
Joined: Fri May 28, 2004 9:04 pm
Location: Certified Trainer/Consultant in Riga, Latvia
Contact:

Mon Sep 06, 2004 10:24 pm

We did the test again with MTU 1400 and throughput was back to maximum.
Do you want to restore MTU to its normal state? I would suggest playing with fast-frames option in wireless configuration and/or ip packet packer... Ideally, the MTU of all the setup would be unchanged 1500 bytes.
 
pritchie
just joined
Topic Author
Posts: 23
Joined: Tue Aug 10, 2004 12:47 pm

Tue Sep 07, 2004 11:53 am

OK, thanks for that extra info. I will experiment with that in due course.
Thanks again for your help.

Who is online

Users browsing this forum: evellin, Majestic-12 [Bot] and 91 guests