Infos needed before jumping RouterOS wagon (BGP, bandwidth)

RouterOS General
1

Hi!

I for for a small ISP. We offer internet services with our own datacenter as well as access and security solutions.

What we have right now is this:

  1. our internet servers in a colocation which also runs our /19 in their multi-homed AS
  2. we provide internet access with layer 3 switches at our office location
    2.1 the switches are BGP capable, but have only 256 MB RAM (non-upgradable)
    2.2 right now we have a single 100 mbps upstream
    2.3 each customer is handled by its own VLAN on the switch (ip interface, bandwidth limitation)
    2.4 we pass all VLANs as single trunk to our carrier who unpacks and delivers it to our clients (untagged)
    2.5 pmacct is used on a server, attached to a mirror port to our vlan trunk to monitor bandwidth usage of each VLAN
    2.6 ethernet port based traffic is monitored via SNMP
  3. customers usually get a managed media converter (fiber to copper) or unmanaged VDSL-to-ethernet bridge (modem)

What I / we want is this:

  1. run our own AS (we have one already with RIPE, but inactive) multi-homed (>= 2 upstreams and peers; >= 1 of them with 1 Gbps)
  2. migrate our colocated servers to our office and running them through our AS
  3. migrate customers’ access VLANs including bandwidth management and eBGP etc. to new router
  4. deliver RB2011 device as CPE instead of media converters / modems
  5. maybe run BGP on some customers’ ends to provide access fault tolerance

I’ve currently made up my mind that RouterOS (I know it from using a 751 at home for access and branch office VPN) may be the most flexible and inexpensive way of achieving our goals while delivering (presumably / hopefully) similar stability as Cisco, Juniper, etc. who have very interesting products but are over our budget.

Next step will be evaluation and here is my first question:

When I read through the forums it seems ROS for CCR is not quite where it is on the RB1100 speaking of stability and making the most out of the hardware. So maybe I’d rather evaluate the RB1100 and wait for the CCR until it’s where it’s supposed to be. But would a RB1100 AHx2 be able to handle maybe 50 VLANs with customer bandwidth limitation and BGP with two full tables, two 1 Gbps upstreams, 1 Gbps downstream, mirror port(s) and SNMP (polling port stats every minute)?

The second question would render the first one unnecessary :wink: :
Is the CCR production ready for the given scenario or at leat pretty damn close to it?

Thanks guys!

2

According to your post, you will need a device which can handle more than 1gbps and your only choice is CCR in this case (or go for an x86 custom build machine)

3

You mean because the two 5 port switch units are “only” 1 gbps to the cpu each?

Let’s make this clear: Choosing between RB1100AHx2 and CCR now is not about money, I’m just concerned about stability. And the upgrade to multi-homed gbps upstreams may not come too soon. Personally I expect we will upgrade gradually from our current 100 mbps to 2x100mbps to 1 gbps + 100 mbps and at the end to 2x 1 gbps. I’d be perfectly okay with upgrading to CCR in a year or so when ROS has gotten rock stable and we have switched to gbps.

4

The only problem with CCR right now is that the RouterOS code is not optimized for multi core.
You need rephrase your question such as;

“When routing and firewall will be optimized for multi core support ?”

If Mikrotik does this, CCR is a good choice for your setup for BGP + 1gbps traffic.

Regarding the stability of CCR, I find it quite stable. We have 3xCCR in operation and they are ok.

If this is for a datacenter, you will prob have attacks to your network, and CCR might not be a good solution to be used as a firewall. It crashes under high pps when firewall is used. I hope it will be fixed when more multi core support is added in the future.

5

Thanks doush! May I ask with how much load (traffic, bgp table) you’re running your CCRs?

We haven’t thought about using the routers as firewalls as our servers are behind firewall appliances anyway.

6

doush, in 6.2 CCR should be more resilient against DoS/DDoS attacks

7

We do a large amount of Service Provider/Telco/Carrier design and implementation. Baltic Networks makes RouterOS based routers that are well suited to your needs - at least one of them has roughly 10 Gbps of forwarding capacity.

http://www.balticnetworks.com/manufacturers/maxxwave/routermaxx-routers-powered-by-mikrotik.html

We have put many of these in play for core, distribution and access and they perform very well and are stable.

8

I want to clarify, that “CCR crashes with high pps” was fixed in recent software updates.

9

Thank you guys!

After having a little chat with my boss and ordering a RB2011UAS-2HnD-IN as upgrade / replacement of my 751 at home we’ll start with some RB2011s and probably a CCR next month for further evaluation and testing. :slight_smile:

I’ll keep you updated (or will bug you with more questions :wink: ) and plan on doing some blogging with my / our experiences.