I am brand new to RouterOS and the MikroTik Cloud Core Routers. I just bought a CCR1036-12G-4S-EM because after trying very hard for more than one week the ERPro-8 still was not able to accomplish what I bought him for. Routing packets from 2VLANs thru 1. SFP WAN, Routing packets from 3 other VLANs thru 2. SFP WAN, some IGMP stuff for IPTV and of course all of this in Wire Speed (this might be the big issue since I got 2 Fiber SFP’s which are both fully 1Gbps capable). So now my question is what is the best practice to achive my goals. With the Edge Router I had multiple options:
Use the Load Balance Feature with two Load Balance Groups one where 1. SFP is the main connection one where the 2. SFP is the main connection. Than make a firewall modify rule to switch packets to each group based on there source VLAN This is a good feature the problem is that the Load Balancer does not support HW offloading correctly and maxes out at 450Mbps which is a quater of what I pay for…
Use the Static Routing feature and create two tables one for the GW of the 1. SFP one for the GW of the 2. SFP. After that also create a FW Modify rule to switch packets to each group based on there source VLAN. Also here HW offloading does not work. You can manually force it on, gives you offloading for incoming traffic… outgoing traffic still create 100% CPU usage also VLAN Intercommunication between mgmt VLAN and other VLANs creates 100% CPU usage on only 50% of Wire Speed.
Also one of my ISPs needs a special DHCP option client-option “send vendor-class-identifier “100008,0001”;” and has to be on VLAN10 but I guess this is no biggy with MikroTik.
I also attached my config from my EdgeRouter for the people who are interessted about the setup. But its pretty straight forward.
EdgeRouter.txt
Thanks a lot for your help
What you are looking to do is fairly basic and straight forward inside of a Mikrotik. You basically have two ways of going about this.
1.) In /ip route rules you can specify source addresses, and force them to use one routing table or another that you define when you create a routing entry with a routing mark.
2.) Use /ip firewall mangle to mark packets in the prerouting chain with the “mark routing” action based off of what VLAN that they come off of. Once again you will have a routing entry with a routing mark that matches the above rule.
This kind of gives you an overview of how option 2 works, but instead of it just being policy based routing, it uses some custom firewall matchers to do load balancing. http://wiki.mikrotik.com/wiki/Manual:PCC
First I have to say something I hope the Engineers of the Routerboards and the Developers of the RouterOS reads that. You guys are AWMSOME. I literally tried for at least a week even spent one day debugging the Edge Router etc. etc. found many bugs and flaws in the FW and was not able even with help of some of the developers to reach the “pretty easy” goals which are here described also. On RouterOS and the CCR1036 I was able to setup all my goals within a few hours (I had to get to know how RouterOS works). I guess know I could set it up within 30mins. And it works absolutely FLAWLESS! Even with my high speed connections the CCR1036 keeps calm. CPU load around 1 - 3%.
After I successfully done my initial setup is working stable. I wanted to ask some little follow up questions.
I used /ip firewall mangle to set the VLANs to specific /ip routing marks I created. The question here is I created the IP routing marks using the gateway address. But one of my Fiber WANs does have a dynamic IP so not only IP could change. Default GW could change also. Is there a better way than what I did?
I used IGMP Proxy for my IPTV, I actually would prefer PIM but I did not succeed since I did not completely understand the syntax. Can someone guide me in the right direction according to my config (see attachment)? Also in a next setup I would also need PIM anyway since my Sonos devices which I believe also need multicast for discovery are in a different VLAN than my WLAN devices who should be able to control them. Any idea’s on that?
I played around with port forwarding and found something a little bit odd. For testing purpose I created a NAT rule to pass thru port 8080 to a service. And it worked without creating a FW rule? Is this normal? Or did I misconfigured something on my FW end?
Thanks again guys for your help. Much appreciated.
1.) There are two ways that present themselves as to how to handle a dynamic IP change. You can use a script with a schedule that will run and check for an IP change, and if it happens, modify the routing table. There are a couple of scripts that are available on the Wiki that deal with dynamic IP addresses, you can take one of those and modify them to fit the need and update the configuration.
The second method would be to rely on the DHCP client, set the weight of the route to the least amount in the client, and anything that is not marked differently in the mangle rules will use that route by default, and what is specifically marked will use the other connection.
2.) I’m not familiar with IGMP, so I cannot be much help here.
3.) The primary function of a MikroTik is to route packets, so by default they accept everything. Looking at your configuration, you have only protected the input chain on the firewall (default settings). This will only protect the router itself as it only filters packets that are specifically destined for the router itself. The forward chain in /ip firewall filter is where you lock down what kind of traffic can flow over the router itself. Here is a basic stateful packet inspection firewall configuration.
That is a good place to start for a firewall. You will need to explicitly allow what you want before the drop everything else rules. Replace the in-interfaces to match your current configuration. Keep in mind that when doing a dst-nat action for NAT and port forwarding, the NAT action happens before the forward chain, so you need to have the appropriate firewall matchers in place to allow the traffic through.
Firewall seams the be working flawless now. Also the rules for the specific services are now taking action. How they should.
I have two reaming problems…
My IGMP stuff with the Sonos and the AppleTV / FireTV in the Multimedia VLAN .11 which is accessable thru the WiFi devices but multicast which I think is needed for communication is not working
VLAN issue on my CCR1036… I got 12Gigabit Ports on the CCR1036… So there is no need for my 8-Port GBit Switch which was next to my Previous Edge Router Pro… So can I do VLAN untagged Ports with the CCR1036? What I tried is I create all my VLANs on a new Bridge Interface. And than Add ETH1 ETH2 ETH3 ETH4 which are going to my diffrent switches… But ETH5 is an Untagged VLAN port for VLAN10… Which I was not able to create… any tips on that?
For number two. You cannot think of VLANs like you do for a switch with a MikroTik, the router sees them as another routed port, so it really doesn’t have the concept of a tagged or untagged port. If you create an interface called VLAN201, it can have every service that any other Ethernet like interface can have, just packets leaving that interface are tagged with a VLAN. So just think of the router receiving traffic for it to route. You can duplicate tagged and untagged switch functionality by making a bridge for the physical interface you want untagged and the VLAN you want tagged. The main thing is that the VLAN interface cannot be on the same bridge as the physical port it is a part of.
Think of how you want traffic to enter the router to decide what to do based off of that. If traffic is entering the router untagged, you do not need to assign a VLAN to that interface since the router has no way of knowing what VLAN was assigned to it, assuming that you’re not just dumping a lot of traffic towards the router. Since a switch can only have one PVID for a port, as the router goes to send reply packets and sends it to the switch, the switch will not be able to get back to where it needs to. However if you have one up-link port from a switch to the router, then that port needs to know about each VLAN, and the switch should have the port configured as a trunk port to maintain the VLAN tags. Then you can either bridge all of the VLANs together, and have one IP assigned to the bridge along with any other services that you desire, or have a separate IP/services for each VLAN interface. If you want the each VLAN to leave another interface of the CCR without it going through routing processes, then create a bridge for each VLAN and physical exit port pair.