Community discussions

 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

RB1100 low bridging/shapping performance

Thu Feb 03, 2011 4:53 pm

Hello,

I've just bought an RB1100 router that I intended to use as a replacement for my Intel server traffic shaper running RouterOS 5.0rc4. I've saved the settings on the Intel, uploaded them on the RB1100 and off I went to move the cables from the Intel to the new shaper. I was lazy enough not to use a dedicated out-of-band management interface (useless anyway after you read further) so it took me quite a while to load the prefs in the Winbox. I took a look at the interfaces, it took about a minute for it to load the 13 interfaces and when it did, it only indicated about 120Mbps going out while on the Intel there are about 220-240Mbps passing through at peak times according to the 95th percentile graph from the Cisco port it's plugged into.
Next step was obviously to check the CPU load. 100% @ ~120Mbps out of the normal 240Mbps. I started to get emails from my Smokeping server in Germany, old respose time 35-40ms -> 250-350ms.
It's obvious not to expect much from an appliance using an 800Mhz single core CPU instead of the Intel's X3440 Xeon Quad Core but I wasn't expecting a 13 gigabit ports appliance not to be able to handle more than 120Mbps doing only that.
Some tech details about my old setup: About 10 drop or accept rules in the firewall, one L7 rule, about 500 simple pfifo queues all of these using about 15-20% of the Xeon. The intel has two NICs as ports of a bridge and it's only purpose is shaping the traffic in a transparent bridge setup.

My questions are: are there any special requirements for using the RB1100 in this current setup that I was not aware of? Because maybe there are things that need to be done in a certain way for the 800 MHz CPU to be able to handle the traffic.
Why are there 1Gbps ports for a device that's not able to handle at least 1Gbps. I would expect this to be oversubscribed like some Cisco switches are but I would also expect to be able to carry 1Gbps at least in half-duplex.
I was quite happy with the Intel except for the fact that it's NICs were on PCI, thus limiting the bridge throughput to the PCI bus speed (from my experience up to 350-400Mbps traffic).
I've read about the queues optimizations. I haven't found anything relevant anywhere, though I think I definitely read something about this.

Any pertinent opinion is highly appreciated.
 
hedele
Member
Member
Posts: 335
Joined: Tue Feb 24, 2009 11:23 pm

Re: RB1100 low bridging/shapping performance

Thu Feb 03, 2011 8:07 pm

Hi,

Simple Queues and L7 inspection slaughter the CPU. Please do not use them, or at least don't use hundreds or thousands of them.
No Routerboard will be able to cope with that kind of (useless, because redundant) work.

Try to convert your simple queues to a queue tree, or to PCQ simple queues. Drop the L7 rule, if it isn't 100% necessary.
You may also try overclocking your CPU to 1066 MHz, but note that this may make the board unstable.

Also, disable connection tracking, if you are not using any NAT configuration or advanced firewall features which require them (like connection-bytes or L7).
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 04, 2011 12:11 am

Hi,

Simple Queues and L7 inspection slaughter the CPU. Please do not use them, or at least don't use hundreds or thousands of them.
No Routerboard will be able to cope with that kind of (useless, because redundant) work.

Try to convert your simple queues to a queue tree, or to PCQ simple queues. Drop the L7 rule, if it isn't 100% necessary.
You may also try overclocking your CPU to 1066 MHz, but note that this may make the board unstable.

Also, disable connection tracking, if you are not using any NAT configuration or advanced firewall features which require them (like connection-bytes or L7).
Just for the sake of it, I already tried that even though it was useless to me. I've disabled all queues, I've disabled all firewall rules. I've also tried to overclock it, I don't recall the command exactly, the point is that the command was valid but a "Your system does not allow CPU clock settings" or something alike was returned. Anyway, overclocking it from 800 -> 1066 will mean what, bridging 200Mbps without loss but not more?

It just doesn't add up. I don't understand this:
http://routerboard.com/pdf/routerboard_ ... _tests.pdf
According to this benchmark Mikrotik says that the router can perform at it's worst 121000 pps. OK. The normal traffic through the Intel shaper as observed after the rollback was 20000-25000 pps with a total throuhput of 200 Mbps. Doing some simple math it turns out that under normal usage the average packet size can be approximated at a size of 1000 bytes.
So Mikrotik measured a maximum 500.32Mbps throughput for a constant 512 bytes per packet and 1335.84 for a 1500 bytes packet, it would seem reasonable to estimate the throughput for 1000 bytes at (500.32+1335.84)/2=918Mbps. I only need 200-300Mbps but the CPU goes nuts without any fancy work.

And may I also remind you that the above mentioned values are worst case scenarios according to the benchmark.

@Mikrotik, any clarifications please?
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6274
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 04, 2011 9:40 am

rb1100 does not use PCI bus.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1695
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: RB1100 low bridging/shapping performance

Fri Feb 04, 2011 12:12 pm

And may I also remind you that the above mentioned values are worst case scenarios according to the benchmark.
Ehm, it is best case scenario - with almost no configuration. So as soon as you start configure, something numbers will go down.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 04, 2011 1:41 pm

rb1100 does not use PCI bus.
What type of bus is using is irrelevant. What's relevat here is that it can't bridge more than 120Mbps through 2 gigabit ports from the same port group, nor between ports from two different groups.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 04, 2011 1:48 pm

Ehm, it is best case scenario - with almost no configuration. So as soon as you start configure, something numbers will go down.
I was refering to the fact that the numbers I took from the official Mikrotik benchmarks are the lowest possible values, under the worst case scenario. The performance is a lot better without any sort of routing/conntrack. The values I used as listed by Mikrotik are under the Firewall ON (no mention on how many rules) + Conntrack ON + RSTP Bridge Mode.

Thank you.
 
RK
Long time Member
Long time Member
Posts: 560
Joined: Tue Nov 21, 2006 11:22 am
Location: Winnipeg, Canada and Central America

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 11:18 am

I already tried that even though it was useless to me. I've disabled all queues, I've disabled all firewall rules.
Let me see if I understand this correctly.
You are saying that you disabled all queues and all firewall rules but you still were not able to bridge 200 Mbps of traffic?

Did you make sure that connection tracking was off?
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 3:31 pm

I already tried that even though it was useless to me. I've disabled all queues, I've disabled all firewall rules.
Besides the fact that the conntrack is something that I need, I disabled it anyway, it was roughly a 50% throughput gain, but then again, no shaping, no filters, no conntrack? Doesn't this nice and shiny box look more and more like a rubbish TP-Link unmanaged switch? Only about 4 times more expensive.

And please don't make me do this, but I'm seriously thinking of buying one of those, just to convince myself that the TP-Link will be able to handle 1Gbps.

@Mikrotik: If the sole purpose of 1Gbps port is to be able to handle more than 100Mbps per port (but not much above this value) I will completely understand and close this topic, I will just assume that your tests showed that the RB1100 CPU can handle more than 100Mbps per port and decided to go for gigabit ports, yet knowing that I will never be able to go as high as 1Gbps. Yet, the performance benchmarks are highly erroneous in respect to the advertised router performance, mine performs 4-5 times worse. And don't take this the wrong way, I love RouterOS, in fact I just bought a PRO/1000 ET card as a replacement for my curent shaper (low speed PCI bus and no RX/TX queues on the built-in NICs) and I will always have only good words about RouterOS. My problem is with your top-notch Routerboard. I'm still waiting for an official response from you in respect to the discrepancies between the real router and the advertised performance.

Thanks.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1368
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 8:56 pm

Hey mihaialdea, could you please provide a capture of the traffic you need to bridge?

Maybe it's some big size packet crazy thing.

I had ping time issues when bridging 802.1q, 802.1ad, EoIP and IPIP. I haven't tested other types. The issue could be NOT IN THE MikroTik bridge, I haven't had time to check more. I just untagged the traffic.

P.S. I once reloaded a router by reflashing from bootloader and then re-created all configs from withing WinBox without restoring a .backup from another platform. Then one problem went away.

Have you tried with v5 RC7 / RC8 ? There are improvements in RC7....

Hey if you can't find the RB1100 usable - send it to me :) :P
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
poli5681
newbie
Posts: 44
Joined: Fri Jul 03, 2009 11:47 am

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 9:15 pm

Hello mihaialdea,

I had similar problems with bridging and queues on my 1100s; I bridged Port 11-12 ,
enabled piping through fw, and added mangle rules and a complex queue tree for interface 12:
The result was a more or less completely unresponsible RB - the interfaces showed only a few Mbps of data and was hardly reachable on all other IFs.

I just changed the bridged ports to 6-7 and the queue on port 7; Everything is fine since then and i can easily get 200Mbps+ without exceeding 30% load.

This happens on all my 1100s; I thought it was an hardware issue and they fixed it and redesigned parts of the RBs (cause the RB1100 were unavailable for a couple of month).
But this still happens on my new ones i got last week.

Maybe this is related somehow to your problem.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1368
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 9:22 pm

poli5681

What are the affected RotuerOS versions and bootloaders?
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
poli5681
newbie
Posts: 44
Joined: Fri Jul 03, 2009 11:47 am

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 9:31 pm

Hello NetworkPro,

I´ve ROS 4.16 and BL2.29 on my last RB1100 and ROS 4.11 and BL2.27 on my first;
So i guess all versions in between are affected.
 
biker
just joined
Posts: 3
Joined: Mon Sep 10, 2007 4:45 am

Re: RB1100 low bridging/shapping performance

Sat Feb 05, 2011 9:35 pm

I solve this using the rb 1100 has a gigaswitch and the queue, conntrack and firewall rules with a X86 server with 2 giganics, the 1100 cpu can´t handle all the traffic.
 
RK
Long time Member
Long time Member
Posts: 560
Joined: Tue Nov 21, 2006 11:22 am
Location: Winnipeg, Canada and Central America

Re: RB1100 low bridging/shapping performance

Sun Feb 06, 2011 11:18 am

Hello mihaialdea,

I had similar problems with bridging and queues on my 1100s; I bridged Port 11-12 ,
enabled piping through fw, and added mangle rules and a complex queue tree for interface 12:
The result was a more or less completely unresponsible RB - the interfaces showed only a few Mbps of data and was hardly reachable on all other IFs.

I just changed the bridged ports to 6-7 and the queue on port 7; Everything is fine since then and i can easily get 200Mbps+ without exceeding 30% load.
Maybe bridging ports 5 and 6, on different switch chips, would be even better?
 
poli5681
newbie
Posts: 44
Joined: Fri Jul 03, 2009 11:47 am

Re: RB1100 low bridging/shapping performance

Sun Feb 06, 2011 2:42 pm

Maybe bridging ports 5 and 6, on different switch chips, would be even better?
Thank you RK; That might be even better.
I still have 1 RB1100 left to install; I´ll try and let you know my results.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:15 am

Hey mihaialdea, could you please provide a capture of the traffic you need to bridge?

Maybe it's some big size packet crazy thing.

I had ping time issues when bridging 802.1q, 802.1ad, EoIP and IPIP. I haven't tested other types. The issue could be NOT IN THE MikroTik bridge, I haven't had time to check more. I just untagged the traffic.

P.S. I once reloaded a router by reflashing from bootloader and then re-created all configs from withing WinBox without restoring a .backup from another platform. Then one problem went away.

Have you tried with v5 RC7 / RC8 ? There are improvements in RC7....

Hey if you can't find the RB1100 usable - send it to me :) :P
No, unfortunately I can't provide you with that because that would mean for me to break the NDA I've signed with my employer. For the time being it's only bridging the traffic for a single server which is a online game server. No more thant 30-40Mbps at any given time. The CPU ranges from from 40-80% usge. The client doesn't complain about anything, but if I start adding servers things will go poof.
I'm not using any sort of encapsulation, everything is in access mode.

Thanks.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:17 am

Hello mihaialdea,

I had similar problems with bridging and queues on my 1100s; I bridged Port 11-12 ,
enabled piping through fw, and added mangle rules and a complex queue tree for interface 12:
The result was a more or less completely unresponsible RB - the interfaces showed only a few Mbps of data and was hardly reachable on all other IFs.

I just changed the bridged ports to 6-7 and the queue on port 7; Everything is fine since then and i can easily get 200Mbps+ without exceeding 30% load.

This happens on all my 1100s; I thought it was an hardware issue and they fixed it and redesigned parts of the RBs (cause the RB1100 were unavailable for a couple of month).
But this still happens on my new ones i got last week.

Maybe this is related somehow to your problem.
My setup is similar, I have port 13 for out-of-band management and ports 1-2 for bridging. Yet I can't get more than 50-60 Mbps without packet loss.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:22 am

poli5681

What are the affected RotuerOS versions and bootloaders?
Hi,


[admin@MikroTik] > system routerboard print
routerboard: yes
model: "1100"
serial-number: "XXXXXXXXXXX"
current-firmware: "2.29"
upgrade-firmware: "2.27"


It's running RouterOS 4.13

Thank you.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:24 am

I solve this using the rb 1100 has a gigaswitch and the queue, conntrack and firewall rules with a X86 server with 2 giganics, the 1100 cpu can´t handle all the traffic.
:D Pretty expensive gigabit switch, huh?
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:26 am

Maybe bridging ports 5 and 6, on different switch chips, would be even better?
Thank you RK; That might be even better.
I still have 1 RB1100 left to install; I´ll try and let you know my results.

I've alreay tried that, I've bridged ports 1 and 6. Same thing...
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1368
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:27 am

- What does support@mikrotik.com say about these issues?
If you are going to ask them, include a supout.rif and description and link to this thread.

- Anyone cares to try v5RC8?
The scheduled downtime could be from 1 minute for the upgrade to normally 5 minutes to 1 hour if you need to recover the router. Prepare a .backup beforehand just in case or simply - remember the config or write down the details to be able to recreate.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:36 am

I've done some further tests that lead to new conclusions.
I put a dedicated machine behind it and started downloading an ISO from a local mirror server. The speed went up to ~ 500Mbps with everything turned off, no queues, no conntrack, no firewall, nothing.
It was still capable of handling about 200Mbps with conntrack on, one packet marking rule (just to test) and one queue. The only difference was that now I had a dedicated out-of-band dedicated port. I know that this should not be relevant, but anyway.
So tried to put it in a real life situation and I put an online gaming server behind it. 60-80% CPU usage at about 30Mbps traffic and about 10k active connections. So I believe that once you hit the 50 Mbps cap with normal traffic you will get packet loos and delays.

Excuse me for not thinking before about that but here's a print screen of the traffic/cpu usage. Now it's 9:35 AM and the gamers are sleeping, this is why the traffic is so low. Yet the CPU usage is abnormally high.
You do not have the required permissions to view the files attached to this post.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:39 am

- What does support@mikrotik.com say about these issues?
If you are going to ask them, include a supout.rif and description and link to this thread.

- Anyone cares to try v5RC8?
The scheduled downtime could be from 1 minute for the upgrade to normally 5 minutes to 1 hour if you need to recover the router. Prepare a .backup beforehand just in case or simply - remember the config or write down the details to be able to recreate.
I'll do that. I first asked the forums because you guys have real life experience with these toys while MT apparently only benchmarks them and puts the test results in a document.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Mon Feb 07, 2011 9:50 am

- Anyone cares to try v5RC8?
The scheduled downtime could be from 1 minute for the upgrade to normally 5 minutes to 1 hour if you need to recover the router. Prepare a .backup beforehand just in case or simply - remember the config or write down the details to be able to recreate.
I haven't tested it yet on RB1100 but I tested it on an Intel server and it had some very strange problems. Read more here: http://forum.mikrotik.com/viewtopic.php?f=2&t=48957

Thanks.
 
taduikis
Member
Member
Posts: 417
Joined: Sat Jul 07, 2007 12:09 pm

Re: RB1100 low bridging/shapping performance

Wed Feb 09, 2011 6:42 pm

Seems that something is seriously wrong in the config itself if you get that much CPU load under traffic so low.

BTW, did you notice these sexy routers under "Made for Mikrotik"?

Image

Network-optimized or not I doubt any PPC matches Quad Core setup. And if those Ethernet interfaces are at least on PCIe x1, they surely would fit your demands. Too bad they, IMO cost at least twice as much as 1100 :)
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Thu Feb 10, 2011 12:14 am

Seems that something is seriously wrong in the config itself if you get that much CPU load under traffic so low.

BTW, did you notice these sexy routers under "Made for Mikrotik"?

Network-optimized or not I doubt any PPC matches Quad Core setup. And if those Ethernet interfaces are at least on PCIe x1, they surely would fit your demands. Too bad they, IMO cost at least twice as much as 1100 :)
Hi,

I got the point, go for x86. And I did, you wouldn't imagine the experience... I was relatively OK with a SR1630GP with it's onboard NICs as long as I didn't threw more than 200Mbps at it. Due to the PCI interface limitations, it was limited to 200Mbps.
I do however need to shape about 1Gbps so I figured that this can't be done by one machine, I need two that could handle ~500Mbps. So I've bought a PRO/1000 ET with the 82576 chipset and plugged it in. The only MT version that sees the board is 5.0rc7 (for some reason rc8 doesn't load them properly). I had to do some tweaks such as lifting the interface MTU to 1504 because of multiple tagged VLANs I had there (while 4.16 didn't needed that, but didn't had the multiple-cpu support like 5.0).
The 5.0 is rather shabby in respect to shaping tagged VLANs, everything comes almost to a stall at some points, but after disabling "Use IP firewall on VLAN" in the bridge settings, all works OK.

Bottom line here is that RB1100 is too damn slow, going for top notch hardware isn't an option either due to the lack of support for the (I quote) "expensive uncommon chipset", like the 82576 is.
I've even opened a topic to find out the top config that someone is using in production. I already feel sorry for the money spent on the RB1100 and now I start to feel sorry about the money spent on the PRO/1000 ET, it's too damn new it seems for Mikrotik to fully suport it.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1368
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: RB1100 low bridging/shapping performance

Thu Feb 10, 2011 2:22 pm

..Due to the PCI interface limitations, it was limited to 200Mbps....

Could you elaborate a little bit more on this.

I read here a slightly different story http://ixbtlabs.com/articles2/gigeth32bit/index.html
...the throughput of the Gigabit Ethernet reaches 1000 Mbit/s which is approximately equal to 120MB/s, i.e. it's nearing the speed of the 33MHz 32bit PCI bus.
and right now I am unable to actually test transfer that passes through two PCI Gigabit cards that are on the same PC.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 7:13 am

It depends on the system, You can have multiple PCI channels in a single mobo or all of them can be lumped into one channel.

PCI 32bit/33mhz caps out at 1064mbit/sec
PCI 32bit or 64bit /66mhz caps out at 2128mbit/sec
PCI 64bit/66mhz caps out at 4264mbit/sec

So if you have a mobo with 2 Gbit NIC's in PCI slots and those slots share the same channel you would not be able to reach 1gbit full duplex of transfer speed, for that you would need PCI 66mhz slot in 32bit or 64bit flavors. However if each NIC was in a PCI with it's own PCI channel then with 32bit/33mhz you could do 1gbit full duplex.
 
RK
Long time Member
Long time Member
Posts: 560
Joined: Tue Nov 21, 2006 11:22 am
Location: Winnipeg, Canada and Central America

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 7:50 am

So tried to put it in a real life situation and I put an online gaming server behind it. 60-80% CPU usage at about 30Mbps traffic and about 10k active connections. So I believe that once you hit the 50 Mbps cap with normal traffic you will get packet loos and delays.
For comparison, I do over 50 Mbps with 30K active connections in the connection tracker and some firewall rules, but no queues, on a RB493AH with under 40% CPU.
 
taduikis
Member
Member
Posts: 417
Joined: Sat Jul 07, 2007 12:09 pm

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 10:36 am

I too was looking for a Mikrotik-made ready-to-go solution and did quite some research about it. It seems like the older 1333MHz RB1000 performs better than RB1100, but is more expensive and difficult to get. And when you're more into it, seems like it's not that expensive to build your custom x86 machine or buy that already made one in these good lookin' 19" rack cases.. What's more, in custom built machine you can put HDD which doesn't suffer from rewrite cycles and in general I've noticed that all my x86 based machines perform way better with RouterOS than any Mikrotik PPC or mipsbe..
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:00 pm

For comparison, I do over 50 Mbps with 30K active connections in the connection tracker and some firewall rules, but no queues, on a RB493AH with under 40% CPU.
It may have something to do with the tagged VLANs. I transport a few tagged VLANs to the switchport where the MT is plugged in. If I disable the "User IP firewall for VLAN" under the bridge settings I have <1ms ping reply to the hosts on the tagged VLANs while with the option enabled I get 20-40ms when there's about 200Mbps passing through. This happens on a x86 machine but I assume that the same thing happens on a Routerboard.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:25 pm

I too was looking for a Mikrotik-made ready-to-go solution and did quite some research about it. It seems like the older 1333MHz RB1000 performs better than RB1100, but is more expensive and difficult to get. And when you're more into it, seems like it's not that expensive to build your custom x86 machine or buy that already made one in these good lookin' 19" rack cases.. What's more, in custom built machine you can put HDD which doesn't suffer from rewrite cycles and in general I've noticed that all my x86 based machines perform way better with RouterOS than any Mikrotik PPC or mipsbe..
I'm seriously thinking of buying something from here:
http://www.itxdepot.com/xcart/home.php?cat=12000
for my future servers running RouterOS. While RouterOS is an absolutely astonishing product and it's worth every penny, I can't say the same thing about RB1100. If you have over 200Mbps traffic to pass through it, you'll have to start disabling connection tracking, don't do packet marking, don't use too many firewall rules and don't use simple queues. Basically if you have a couple of hundreds Mbps passing through, you need to turn the RB1100 into an unamanged switch. I'd go for a Linksys switch or a second-hand Cisco for that matter, at half the price :wink:
So yes, I would definitely recommend RouterOS + x86 and strongly advise against RB1100 for more than 50Mbps throughput. My personal advice after trying different stuff is not to aim for high-end devices (Quad Core Xeons with 82576 or 85675 network chips). For a bridge/router go for PRO/1000 GT NICs and dual core CPUs with speeds as high as possible (ex. Core 2 Duo @ 3.06GHz). This type of hardware is under production for many years and the above mentioned NICs are very stable according to the Mikrotik HCL. I haven't tested it myself but I rely only on this:

http://wiki.mikrotik.com/wiki/Supported_Hardware
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:27 pm

depending on the packet size, we can get even 1.3Gbit throughput with RB1100 and firewall and conntrack on. Maybe there is some other problem?
No answer to your question? How to write posts
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:37 pm

Thats great but how many firewall rules etc etc. The RB1100 is lacking in CPU power, It's aimed as a core or like-core router.

The reality is x86 is where you have to go when you have a core router handling MPLS with 50-100 BGP based VPLS circuits running, 20-40 PPPoE sessions coming in over a VLan and ~100 firewall mangle rules based on IP. Add in OSPF and a BGP system with 1000-1500 routes and want to push 100+mbit of traffic

We're currently at the limits of the RB800 we have in a outer core router running 1/2 that pushing 30mbit, The CPU's sitting at 70%+ all the time and we will be skipping the RB1100 upgrade and going straight to x86.

Here's hoping that the RB1200 is a multicore or multicpu system designed around ROS 5's SMP setup and can handle 1gbit of routed traffic in a setup close to what you see in a medium to large ISP's core
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:40 pm

I didn't say it will be more powerful than a 16 core Xeon router, currently we have no device to compete with that. But it is not so bad as the above poster implied, and we get much better results with them.
No answer to your question? How to write posts
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:44 pm

I should hope so with 16 cores :P

There are alot of problems with going the x86 path, ROS is rather picky and has had some interesting bugs with certain hardware in the past. It would be nice if MT produced a router aimed at medium to large ISP's that need the grunt but also need the backing from MT to support it. Something in the $1200-1600USD mark with options for 512mb/1gb/2gb/4gb of ram in it and enough CPU power to do 1gbit+ whilst doing bgp, routing, mpls etc

Either that or come up with a golden x86 1-2ru server that you can say without a doubt ROS wont have any issues on :)

Just an idea
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 12:55 pm

ok, on it

:)
No answer to your question? How to write posts
 
taduikis
Member
Member
Posts: 417
Joined: Sat Jul 07, 2007 12:09 pm

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 1:54 pm

I think no one here says that RB1100 is a bad product. It's just time to admit that it's intended for small to medium sized ISPs or companies that are running lots of computers and do not need complex solutions 1100 cannot handle. Many people are confused seeing expensive, high-end MikroTik router with 13 gigabit ports and so they expect it to perform as top-class router. Seems like gigabit is fitted there as the only means to have >100mbps traffic through a single port. And whats more, evasive action from Mikrotik always giving notes about packet size reminds me of Apple, when they said "Your holding it wrong".. :)

Besides, ain't x86 systems more flexible? I mean you can upgrade and replace various different parts if they stop functioning or doesn't seem to pack enough juice anymore.. When in the case of RB, you have to throw it all away :)
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 1:55 pm

for the most important links, connect each cable to a different switchgroup, then throughput will be better. ie. speed from port 3 to 4 is not as good as from 3 to 7
No answer to your question? How to write posts
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 2:48 pm

Whats the bus speed between the switch chip and the CPU interface? I remember us ditching the RB800 ethernet daughterboard because the board only had a single 100mbit interface into the RB800's cpu or other switch chips and thus those extra 16(?) ports were all limited to this 100mbit uplink when doing routing or pppoe
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:20 pm

for the most important links, connect each cable to a different switchgroup, then throughput will be better. ie. speed from port 3 to 4 is not as good as from 3 to 7
Hello,

You guys keep saying that and you seem to start more and more like politicians :-)

PLEASE once and for all, stop putting this on the bus, it's all about the CPU. The bottleneck is not the 1Gbps link from the two switchport groups to the CPU, it's the CPU not being able to handle properly a little over 100 Mbps while performing some simple tasks related to the traffic flow. I've said this a couple of times before, all these tweaks you are suggesting only bring the router to the capabilities of an unmanaged switch. It performs great, I can transfer a couple of hundreds Mbps through it, yet I can't do shaping, I can't mark packets, I can't use firewall rules, basically all your suggestions lead to taking the overhead from the CPU and put it on the switch chips. GREAT! but what about all the features available in the RouterOS Level 6 license?
I tend to think that offering a Level 6 license was for marketing purposes only. The difference between Level 4 and Level 6 is that you put Level 6 on a workhorse machine that's supposed to be able to handle lots of connections, lots of traffic, lots of encryption. Can the RB1100 CPU handle all these? From my perspective NOT. What's the point in having all these capabilities at our fingertips if we can't use them? Everything is related to that damn CPU.
If you want green computing, go for a 1U rackmount chasis and put a D525 Atom CPU inside and RouterOS 5.0 by default to enjoy the latest multi-cpu features rcently added in 5.0rcX. Alternatively build a top-notch appliance for the ones of us that need serious throughput. Finish the support for 82756, make it stable... (5.0rc7 detects it while 5.0rc8 doesn't).

Thanks.
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:21 pm

Whats the bus speed between the switch chip and the CPU interface? I remember us ditching the RB800 ethernet daughterboard because the board only had a single 100mbit interface into the RB800's cpu or other switch chips and thus those extra 16(?) ports were all limited to this 100mbit uplink when doing routing or pppoe
Supposedly each of the two switchport groups is connected to the CPU @ 1Gbps speed (don't know the exact bus figures) and the remaining three ports have a dedicated 1Gbps link to the CPU.
Last edited by mihaialdea on Fri Feb 11, 2011 3:38 pm, edited 1 time in total.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:30 pm

on RB1100, the whole switch group shares single Ethernet interface bandwidth (1Gbps full duplex) this Ethernet interface is built in into CPU, so don't have any PCI bus limitations
No answer to your question? How to write posts
 
mihaialdea
newbie
Topic Author
Posts: 28
Joined: Thu Jan 06, 2011 12:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:37 pm

on RB1100, the whole switch group shares single Ethernet interface bandwidth (1Gbps full duplex) this Ethernet interface is built in into CPU, so don't have any PCI bus limitations
Hello Normis,

I'm starting to get the feeling that I'm talking to the walls here. What part of "the bottleneck is not the bus, but the CPU" is not clear?

Thanks.
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:41 pm

Thats a good starting point, So the max you can get in theory being ether11 running to the 1st switch group and ether12 running to ether13 is 2gbit/sec full duplex. Does that sound right to you?

If so, what is the MT expected throughput max on an RB1100 runnning basic services with say 1-2 dozen firewall mangle rules (connection/packet marking based on IP) I see in your benchmarking that you have the RB800/1100 listed as doing 1.4gbit in Routing/Contrack, Since only MT knows the exact setup you used could you rerun the test's but this time have 2 mangle rules in place, 1 to mark the connection based on dst IP and the other to mark the packet based on connection mark and post the results?

I think a few people would be interested to see how much of a hit it takes in a simple config
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:41 pm

I corrected your statement, I have not said anything about the CPU yet. Currently we don't have any better CPUs, so like I said, you can buy a Xeon with 16 cores if you want better performance.
No answer to your question? How to write posts
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:49 pm

I corrected your statement, I have not said anything about the CPU yet. Currently we don't have any better CPUs, so like I said, you can buy a Xeon with 16 cores if you want better performance.
Be fair, you don't need 16 cores. A entery level Xeon 3040 and beat the RB1100, I've got x86 routers that are 7th gen Dell's that can push 600-800mbit routed with about 40 mangle rules and a dozen simple queue along with a pair of full bgp feeds
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 23797
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:52 pm

We have not advertised the RB1100 as better than all X86 systems, yes, if you need extra power, currently your option is to get a X86 system. Many of our MFM partners offer preassembled systems like that.

We are working on more powerful devices, but there is nothing to announce at the moment.
No answer to your question? How to write posts
 
Beccara
Long time Member
Long time Member
Posts: 606
Joined: Fri Apr 08, 2005 3:13 am

Re: RB1100 low bridging/shapping performance

Fri Feb 11, 2011 3:59 pm

Fair point but you haven't said what it's aimed for either, without that all people see is a 13 port gbit router. If it can't handle it then why not simply have ports 11-13 as gbit and ports 1-10 in 2 100mbit switch groups. That way bus issues are not there and it wouldn't give them impression of a powerful router

Who is online

Users browsing this forum: No registered users and 6 guests