WAN-capable ports on routers

RouterOS General
1

I own three MikroTik routers (one CCR2216-1G-12XS-2XQ and two CCR2004-1G-12S+2XS). I have noticed that in Quick Set dialog window, Internet section, Port dropdown I can only select two ports (typically named “Eth1” and “SPF1”) and not any other port available on the router. Hence I would like to clarify the following:

  1. Does it mean that only these two ports are WAN-capable and all other ports are not? In other words, it is a true limitation of the router or is it a limitation of the Quick Set dialog window that I can bypass by manually configuring the ports to be WAN-facing in GUI/CLI?
  2. If this is a limitation of the router itself (i.e. only two ports are currently WAN-capable), is this a hardware limitation or is there a hope that all other ports will become WAN-capable at some point in future with a RouterOS update?
  3. Why don’t the names of these two ports match with the actual port names as listed in /interface/ethernet? For instance, on CCR2216-1G-12XS-2XQ Quick Set dialog window calls these ports “Eth1” and “SPF1”, but in /interface/ethernet they are called “ether1” and “qsfp28-1-1” correspondingly. The reason this naming difference causes confusion is because there are two SPF ports on this router with “1” as a suffix (“qsfp28-1-1” and “sfp28-1”) and it takes trial and error to figure out which one of them is referred to as “SPF1” in Quick Set dialog window. Seems like it is something that could be fixed as a software update.
Device tests results - minimum value of "Mbps" for a single true 1Gbps connection
2
  1. and 2. All ports can be whatever you want them to be, It Is a limitation ( by design) of Quickset.
    The Quickset covers one or more “common” configurations, It Is intended as a Quick set :wink: , and It should not be used for anything but those basic configurations, in any case only once after a reset, better if not used at all, because It has a lot of limitations and possible side effects.
3

Please note that some router models, such as the L009UiGS-RM https://cdn.mikrotik.com/web-assets/product_files/L009UiGS-RM_230555.png, has a good switch chip with hardware offload connecting most of the ports, and one (usually ether1) or a few ports connecting directly to the CPU. In such cases it would be better to use the CPU linked port(s) as WAN port(s), instead of putting them in a LAN bridge together with the rest of the ports.

4

Thank you, this is very useful. Indeed, I am not using Quick Set (in fact, I don’t even use GUI or interactive CLI, I use scripting to configure all my devices), I was just puzzled when I looked at QuickSet. I will now experiment with other ports.

5

Thank you for advice, but may I ask why would it better to load CPU with all WAN-bound traffic? Provided that I can (and routinely do) make all ports assigned to LAN as a separate bridge anyway and never include WAN port into that bridge? Wouldn’t it be better to use one port from the hardware offload chip as a WAN port as long as I have it logically separated from the rest of the bridged LAN ports?

6

Yes … if switch chip supports L3HW offload … which switch chip in L009 doesn’t.

7

Thank you for advice, but may I ask why would it better to load CPU with all WAN-bound traffic? Provided that I can (and routinely do) make all ports assigned to LAN as a separate bridge anyway and never include WAN port into that bridge? Wouldn’t it be better to use one port from the hardware offload chip as a WAN port as long as I have it logically separated from the rest of the bridged LAN ports?

The switch chips in the MikroTik hardware only directly see the ports that it managed, and can only hardware offload one single bridge, and on that bridge only the ports directly connected to it. There is no hardware offload for:


  • Ports from outside of the switch being added to the bridge (associated with that switch chip)
  • Ports of the switch being moved out of the single bridge, either to be standalone, or to be member of another bridge

Which also means you should only create at most one bridge per available switch chip.

In the example of the L009 above, if you take ether2 out of the main bridge to make it a WAN port, then that port is no longer hardware offloaded and will use the CPU just like ether1. Furthermore, if you don’t use ether1 as a separate port, and decides to put that port into the main bridge as member, then that port of course still uses the CPU and will affect the other ports of the bridge too, broadcasts on that bridge will need to go over the CPU to be relayed to ether1.

And let’s say you completely ignore ether1 (keep it outside of the bridges) and take out ether2 from the bridge as WAN port. Not only does ether2 loses any HW offload capabilities and need to 100% use the CPU (exactly like ether1), it will take out the available throughput from the link between the CPU and the switch chip. From the diagram above you can see that ether1 has a separate 1Gbps full duplex link to the CPU, but ether2 shares a 2.5Gbps link to the CPU with all the other ports (through the switch chip). If you use ether2 for WAN then traffic between WAN and LAN must go up and down the same link, reducing throughput that might be needed for, for instance, inter-VLAN traffic between some other ports. L009 is a router with a weak CPU so probably will not be able to saturate that 2.5Gbps link with routing, but according to the numbers there might be ideal case (probably fasttrack/fast path with large packet size) where it can saturate the 3.5Gbps of both link. Something that it can’t if you ignore ether1 and use ether2 as WAN port.

Of course, ignoring the reduce total throughput, you can still keep ether2 in the main hardware offload bridge, and configure VLAN (bridge VLAN filtering is hardware offloaded on that L009 switch chip) to make it an access port of a bogus VLAN, then you can use that port as WAN port with hardware offload enabled. In fact, I am using such setup on my RB5009. But please note that outside of the high-end CCR 2xxx and CRS 3xx/5xx/6xx models, hardware offload with the switch chip is L2 only! Which means for most of the lower models, routing which is L3 will need to travel to the CPU anyway and does not benefit from hardware offload and will have no benefits over using ether2 as a separate WAN port.

8

Anyway the L009 specs have:
https://mikrotik.com/product/l009uigs_rm#fndtn-testresults
323.6 Mbps as routing speed for 512 bytes with 25 filter rules (this is the value that happens to be the most similar to real world usage).
I believe that at the most, maybe, in a lucky day, with no or very little hit firewall rules, it can get 500 Mbps.

BUT the models OP has:

CCR2216-1G-12XS-2XQ and two CCR2004-1G-12S+2XS

are slightly beefier, with 11655.6 and 4854.3 Mbps respectively int the same 512 bytes 25 ip filter rules, and the first with Fasttrack L3 HW offload is reported as reaching 189593.6 :open_mouth: .
so the limited CPU power of the L009 should not affect his setup.

9

The 25 rules 512 bytes value is representative for cases where fasttrack cannot be actively used. Like when using mangle mark-routing, or on previous RouterOS versions when using IPv6. For normal configs that can take advantage of fasttrack, throughputs nearer to the fast path value is achievable, and thus 1Gbps WAN routing is possible with the L009, even with PPPoE, and now even with IPv6, on the out-of-the-box defconf settings.

10

I was trying to say something slightly different and generic :blush: .

On low power devices such as the L009 settings need to be optimized to reach higher speeds in routing than what is represented in the 512 byte test result.

On beasts like the CCR2216 the throughput, even in the worst possible conditions, is very high so that one has a lot more slack space with “inefficient” settings.

11

That is interesting, is it based on your testing? To be frank I have never seen a company provide marketing information that downplayed the throughput available.
Seeing that fasttrack is default setting, makes it all the more strange.
Also the small print…
Test results show device maximum performance, and are reached using mentioned hardware and software configuration, different configurations most likely will result in lower results

@jaclaz I think your representation was accurate and more useful.
Now perhaps the L1009 is an anomaly but I have a friend that uses them in his business and with very light firewall rules (minimal) reaches 500-700 range and that is with fasttrack on.

12

Their marketing values are the one in the 1518 columns :slight_smile:. The 25-rule-1518-byte value can probably never be achieved in real life when there are meaningful firewall filters with fasttrack unavailable. That is where the “marketing” is and where they are overselling it. Usually, most speed tests whether with speedtest.net or iperf3 that we normally run use packets approaching the 1500 bytes IP MTU limit, which means 1518-byte ethernet frames. However, the results we normally obtain are the ones from the 512-byte column, which is the value for 3x the number of packets to be processed compared to 1518-byte. MikroTik has not downplayed the numbers at all :smiley:. And real-life results are indeed lower than MikroTik’s marketing numbers, as if the router had 3 times more overhead.

And yes, with many low- or mid-range MikroTik devices that 25-rule-512-byte value is pretty near the mesurable throughput when you use a firewall config similar to defconf but without the fasttrack rule (because maybe you use a few mangle rules), or when you used IPv6 with the defconf IPv6 firewall before 7.18 (where fasttrack was not available), I can confirm this with the hEX / hEX S, hAP ac², hAP ac³, 3011, L009. But you have to reduce the number from the table by 25% if that number was from ROS6 and you now run ROS7. For the hEX and hEX MikroTik has adjusted the value from ROS6 to ROS7 for the 25-rule row when they release the hEX refresh, as you can see from here:

https://web.archive.org/web/20240526122958/https://mikrotik.com/product/RB750Gr3#fndtn-testresults
https://web.archive.org/web/20241124075441/https://mikrotik.com/product/RB750Gr3#fndtn-testresults

But on ROS7 the hEX really ran tests on speedtest.net at about 250-300Mbps without fasttrack (IPv6 results were lower than IPv4 due to more overhead). Same with the hAP ac² at about 700-760Mbps (the 986Mbps number from MIkroTik is for ROS6).

However, with fasttrack active (by just using the defconf configuration from MikroTik), with RouterOS 7, all the devices that I listed above can saturate a 1Gbps WAN even with PPPoE, and since 7.18 with IPv6 too. Well, the hEX maybe at 98% instead of 100%, this post has my numbers with the RB750Gr3 with PPPoE http://forum.mikrotik.com/t/is-pppoe-still-slow/182072/1

Because with fasttrack active, the majority of the packets travel the “fast path” way, which means the throughput can in theory be closer to the values in the Routing none (fast path) row. And all the devices above have published numbers > 1Gbps for that row even if we look at the 512-byte column instead of 1518. Of course, even with fasttrack active some packets will still use the slow path (according to MikroTik), and the router has other overheads like processing DNS, WinBox, PPPoE and even fasttrack itself (which needs to track a few things), so my hEX cannot really reach 1843Mbps WAN routing throughput. That number is probably only possible for the case when the firewall tables are all completely empty and connection tracking is disabled as a result.

13

All well and good, I will continue to quote the numbers Jaclaz used so I dont phuck people in the ear when making purchases.

14

Thank you for your detailed response, it is very useful. I just wanted to clarify something: it seems like L009 became a distraction, as somebody else in the thread mentioned it as an example, but this is not the router I have. I have CCR2216-1G-12XS-2XQ and CCR2004-1G-12S+2XS, does everything you have said still apply to these models?

Here are their block diagrams:

15

No they are beasts!!

16

… and should they be too d@mn fast for internet :open_mouth: , you can still use the 1Gb (intended for management) ethernet port as WAN :laughing: .

Of course CGGXANNX is very correct in the analysis of the real world performance, but with those two beasts of routers it is unlikely that you will have speed issues even with many more firewall rules, mangles and what not.

Going to the usual automotive comparison, If you have a Lotus 7 you’d likely want to find all possible ways to squeeze each and every HP from it (+1), if you have a Lamborghini Huracan you would probably not care as much :wink: .

17

WRONG.… only ether1 which is NOT connected to the switch chip has no way to do HW offloading.

“RTL8367, 88E6393X, 88E6191X,88E6190, MT7621, MT7531, EN7562CT switch chips can use HW offloaded vlan-filtering since RouterOS v7.”

18

@anav, pay attention to details. I was talking about L3 HW offload, VLANs are L2. And this thread is about WAN ports, which usually mean routing - that’s L3 function.

19

Ahh sorry my bad, I was talking about HW offload and you are talking L3HW offload.

Can you tell ignorant me why L3HW offload is important?
Does this let a switch act as a router and not lose switch like throughput???
Which would only be useful if the WAN connection was greater than 1gig anyway.

20



  1. I would like to avoid using MGMT port for any production needs in order to keep it consistent between all MT devices on the racks. I have two racks, with one MT router and 5 MT 48-port switches in each rack, implementing two separate LANs within an office each with its own independent WAN access, so I am trying to keep everything as clean and consistent as possible in order not to create technical debt. This is exactly why I started asking about the best ways to designate WAN port on the router.
  2. Surely my routers have plenty of computing power, but I really prefer to understand how the equipment I am configuring actually works so that I can configure it in the most sensible and intelligent way. I am certainly learning a lot from this discussion thread.