CCR2004-1G-12S+2XS slow NAT performance [Fixed]

[Please read carefully. I don’t have enough hardware to really test bandwidth, so at some point I stopped investigating. I am very satisfied with Mikrotik hardware].

Dear all,

First I would like to thank Mikrotik and the community for these nice products.
I purchased a couple of Mikrotik products for my home lab (I am not a professional) :
1 x CCR2004-1G-12S+2XS
1 x CRS326-24G-2S
1 x CRS305-1G-4S+IN

Here are my questions :

  • How can I disconnect from webfig?
  • Is Mikrotik bootloader secure and signed?
  • Are Mikrotik packages signed when downloading and upgrading?
  • Is Mikrotik software open-source and reviewed by the community (Github like)?

Kind regards,
French Fries

  1. dont use webconfig, I use winbox
  2. bootloader no idea, what that is, I get stuff straight from their website, so assuming its Latvian super protected.
  3. packages signed… no ideas I get stuff straight from their website I certainly dont have to sign anything to get them.
  4. Open source, no friggen way. RouterOS is proprietary, only Normis and Putin have that access. :wink:
    If such a fraidy cat, suggest you go pfsense

Okay being a tad sarcastic, but only because I have zero interest in answering that type of question AFTER a purchase.
If this had been, I am thinking to get MT but would like some information first, that would have deserved a straight answer.
Also how do you know they are excellent products if not used yet??
I prefer tators anyway.

Disconnect from WebFig: there’s an icon (kind of a blue left arrow on brownish background) in the upper right corner of page which causes you to log out.

Is bootloader secure and signed: Bootloader is included in side RouterOS install images. If one deems ROS install package to be safe, then one doesn’t have to care about security of bootloader.

Are mikrotik packages signed: I can only guess. File format (.npk) is more or less proprietary. When ROS reads them, they do perform some checking (if package is corrupt, installation mostly fails with appropriate error message). How easy is it to construct a custom package which includes malware? That’s everybody’s guess. Only support@mikrotik.com could answer this question with confidence.

Is mikrotik software opensource: No, ROS is closed source.

@anav: I answered the questions despite having gut feeling that the post is a smartly disguised troll.

Haha okay,

In that case I will redirect your questions and ask the op to contact NORMIS at the following address
WELETANYTWAT@ONMTFORUMS.STEWPID

Thank you for your answers.
All I read is guess, if you don’t know I will contact support and ask.

Don’t tell Putin has access to a Latvian router, Latvia is part of the EU. This is one reason because I am supporting Mikotik : this is a European company and I don’t want to invest into foreing products (I am French and France is part of the EU). Same Country.

I am also witching from OPNsense to Mikrotik because of hardware acceleration and I need to go 10gb and I don’t want to use a computer as router.
From my point of view, ROS is based on Linux as most routers, switches and firewalls are (except Cisco which has its own OS).

My home lab has several VLANs and I am wondering how to filter inter VLAN traffic on 10Gb lines, these are my needs.
I will get back to you when I reach this point.

Thanks for Webfig.

Good to know, here is one excellent reference for setting up vlans on Router OS.
Although for 10G networks and switches there is another way to configure vlans.

REF for vlan filtering
http://forum.mikrotik.com/t/using-routeros-to-vlan-your-network/126489/1

Ref for switch chip method.
https://www.youtube.com/watch?v=Rj9aPoyZOPo

Gluck!

Thanks. Will test.

One question : I did a simple test in router mode with NAT between a 10Gb LAN and 10Gb WAN and output is only 500Mb/s. What is wrong with NAT?

FF

There is nothing wrong with NAT :wink:
What device is performing the NAT ? CCR ? CRS ?

Thanks. I am testing the CCR2004 first:
Here is my configuration

/interface bridge
add name=bridge1
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=10.90.21.100-10.90.21.200
/ip dhcp-server
add address-pool=dhcp interface=bridge1 name=main
/interface bridge port
add bridge=bridge1 interface=sfp-sfpplus2
/interface detect-internet
set detect-interface-list=all internet-interface-list=all lan-interface-list=all wan-interface-list=all
/interface l2tp-server server
set l2tpv3-circuit-id="" l2tpv3-cookie-length=0 l2tpv3-digest-hash=md5
/interface list member
add interface=sfp-sfpplus1 list=WAN
add interface=bridge1 list=LAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether1 network=192.168.88.0
add address=10.90.21.254/24 interface=bridge1 network=10.90.21.0
/ip dhcp-client
add interface=sfp-sfpplus1
/ip dhcp-server network
add address=10.90.21.0/24 dns-server=192.168.1.254 gateway=10.90.21.254 netmask=24 ntp-server=192.168.1.254
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip ssh
set host-key-size=4096
/system clock
set time-zone-name=Europe/Paris

Without context, not helpful
Need network diagram to show the relationship ahead, behind, and in line with the device.

How did you perform this “test” ? What is the traffic-pattern ? (1 client/IP or some traffic generator sourcing from 100’s of different source IP’s ?, packet sizes etc)
So by 500Mb/s you mean “500 megabits per second” right ? ( which is indeed not much for a box like CCR2004)

During NAT, can you provide info on the CPU-utilisation ?

https://wiki.mikrotik.com/wiki/Manual:Tools/[b]Profiler[/b]

Thank you.

Network pattern :
RouterOS 7.x latest, with eth1 network admin
spf+ 1 : WAN connected to 2.5 Gb ethernet connector of fiber line (speed is 2.5Gb)
spf+ 2 : bridge 10.90.21.254 with one port providing DHCP 10.90.21.x (tested with 1Gb and 5Gb same results).
Fiber box providing DNS
NAT[/list]
Gb = Gigabit

I can browse the Internet connecting to spf+ 2.
I did not perform a precise iperf3 test, only a speedtest.
Speed with NAT is around 500Gb/s.
Direct connection to fiber box gives maximum speed.

/tool profile 
Columns: NAME, USAGE
NAME          USAGE
www           0%   
ethernet      2.8% 
console       0%   
firewall      2.5% 
networking    5.1% 
management    0%   
routing       1%   
profiling     0%   
bridging     0.7% 
unclassified  2.3%

iperf3 -p 9225 -c iperf.par2.as49434.net

Connecting to host iperf.par2.as49434.net, port 9225
[  5] local 10.90.21.200 port 36218 connected to 193.177.162.41 port 9225
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  82.9 MBytes   695 Mbits/sec    0   3.00 MBytes       
[  5]   1.00-2.00   sec  80.0 MBytes   671 Mbits/sec    0   3.00 MBytes       
[  5]   2.00-3.00   sec  80.0 MBytes   671 Mbits/sec    3   1.56 MBytes       
[  5]   3.00-4.00   sec  80.0 MBytes   671 Mbits/sec    0   1.65 MBytes       
[  5]   4.00-5.00   sec  80.0 MBytes   671 Mbits/sec    0   1.72 MBytes       
[  5]   5.00-6.00   sec  80.0 MBytes   671 Mbits/sec    0   1.77 MBytes       
[  5]   6.00-7.00   sec  80.0 MBytes   671 Mbits/sec    1   1.58 MBytes       
[  5]   7.00-8.00   sec  81.2 MBytes   682 Mbits/sec    0   1.35 MBytes       
[  5]   8.00-9.00   sec  80.0 MBytes   671 Mbits/sec    0   1.42 MBytes       
[  5]   9.00-10.00  sec  80.0 MBytes   671 Mbits/sec    0   1.46 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   804 MBytes   675 Mbits/sec    4             sender
[  5]   0.00-10.02  sec   802 MBytes   672 Mbits/sec                  receiver

iperf Done.

There must be something wrong in my config as Hardware spec is much higher.

Same results when using two thread:

iperf3 -P2 -p 9225 -c iperf.par2.as49434.net
Connecting to host iperf.par2.as49434.net, port 9225
[  5] local 10.90.21.200 port 36278 connected to 193.177.162.41 port 9225
[  7] local 10.90.21.200 port 36280 connected to 193.177.162.41 port 9225
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  47.0 MBytes   394 Mbits/sec   48   1.27 MBytes       
[  7]   0.00-1.00   sec  39.0 MBytes   327 Mbits/sec   84   1.02 MBytes       
[SUM]   0.00-1.00   sec  86.0 MBytes   721 Mbits/sec  132             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  43.8 MBytes   367 Mbits/sec    1   1004 KBytes       
[  7]   1.00-2.00   sec  37.5 MBytes   315 Mbits/sec    0   1.14 MBytes       
[SUM]   1.00-2.00   sec  81.2 MBytes   682 Mbits/sec    1             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  31.2 MBytes   262 Mbits/sec    1    761 KBytes       
[  7]   2.00-3.00   sec  48.8 MBytes   409 Mbits/sec    0   1.23 MBytes       
[SUM]   2.00-3.00   sec  80.0 MBytes   671 Mbits/sec    1             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  30.0 MBytes   252 Mbits/sec    0    806 KBytes       
[  7]   3.00-4.00   sec  50.0 MBytes   419 Mbits/sec    0   1.30 MBytes       
[SUM]   3.00-4.00   sec  80.0 MBytes   671 Mbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  30.0 MBytes   252 Mbits/sec    1    592 KBytes       
[  7]   4.00-5.00   sec  51.2 MBytes   430 Mbits/sec    2    979 KBytes       
[SUM]   4.00-5.00   sec  81.2 MBytes   682 Mbits/sec    3             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec  30.0 MBytes   252 Mbits/sec    0    641 KBytes       
[  7]   5.00-6.00   sec  50.0 MBytes   419 Mbits/sec    0   1.02 MBytes       
[SUM]   5.00-6.00   sec  80.0 MBytes   671 Mbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.00   sec  33.8 MBytes   283 Mbits/sec    0    672 KBytes       
[  7]   6.00-7.00   sec  46.2 MBytes   388 Mbits/sec    2    785 KBytes       
[SUM]   6.00-7.00   sec  80.0 MBytes   671 Mbits/sec    2             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.00-8.00   sec  31.2 MBytes   262 Mbits/sec    1    512 KBytes       
[  7]   7.00-8.00   sec  48.8 MBytes   409 Mbits/sec    0    839 KBytes       
[SUM]   7.00-8.00   sec  80.0 MBytes   671 Mbits/sec    1             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.00-9.00   sec  30.0 MBytes   252 Mbits/sec    0    554 KBytes       
[  7]   8.00-9.00   sec  50.0 MBytes   419 Mbits/sec    0    874 KBytes       
[SUM]   8.00-9.00   sec  80.0 MBytes   671 Mbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec  32.5 MBytes   273 Mbits/sec    0    597 KBytes       
[  7]   9.00-10.00  sec  47.5 MBytes   398 Mbits/sec    0    897 KBytes       
[SUM]   9.00-10.00  sec  80.0 MBytes   671 Mbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   339 MBytes   285 Mbits/sec   52             sender
[  5]   0.00-10.02  sec   336 MBytes   282 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   469 MBytes   393 Mbits/sec   88             sender
[  7]   0.00-10.02  sec   467 MBytes   391 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   809 MBytes   678 Mbits/sec  140             sender
[SUM]   0.00-10.02  sec   804 MBytes   673 Mbits/sec                  receiver

NAT is done from the CPU, all packet must be modified and recalculated.
Both CRS can do at max near 600Mbit/s of NAT traffic, because mainly are Switches, not Routers.
Instead the CCR can achieve a NAT speed of near 4,5Gbit/s

If you need 10Gbit/s NAT speed, you must buy at least one CCR1036.

He is testing with a CCR2004, I believe that has plenty of juice for NAT..at least 5gigs worth and 1500 for ipsec…

Maybe that I should use VLANs to have LAN and WAN on the bridge so I can use switch hardware offloading?
However,

/interface ethernet switch print
Columns: NAME, TYPE, L3-HW-OFFLOADING
# NAME     TYPE              L3-HW-OFFLOADING
0 switch1  Marvell-98PX1012  no

I don’ t see any solution to reach 10Mbit/s routing as per spec.
For sure, I am quite surprised by the lack of hardware offloading of firewall rules and switching.

The router has spf+ interfaces, there must be something that I don’t understand.

You could enable fasttrack, it works for NAT as well. CCR should handle gigabit with ease without it, but may be worth trying out.


/ip firewall filter
add chain=forward action=fasttrack-connection connection-state=established,related
add chain=forward action=accept connection-state=established,related

Also, disable internet-detect. Is is rather useless and unpredictable (adds dynamic dhcp client etc..).


/interface detect-internet
set detect-interface-list=none internet-interface-list=none lan-interface-list=none wan-interface-list=none

If your device is shipped with v7, you may want to upgrade to v7.1rc2 as it contains the latest fixes in this branch.

Also, you’re talking about 500Gb/s which is not really possible with SFP+ and 10 Mbit/s, which is easily achievable :slight_smile:

Thank a lot!

Please note that WAN is not part of the bridge.

Same output, here is my detailed configuration, still far from 10Gb/s.

/export
# sep/04/2021 19:58:36 by RouterOS 7.1rc2
# software id = L1XN-2BCQ
#
# model = CCR2004-1G-12S+2XS
# serial number = D4F00E00064E
/interface bridge
add name=bridge1
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=10.90.21.100-10.90.21.200
add name=dhcp_pool1 ranges=10.90.21.100-10.90.21.200
add name=dhcp_pool2 ranges=10.90.21.100-10.90.21.200
/ip dhcp-server
add address-pool=dhcp_pool2 interface=bridge1 name=dhcp1
/interface bridge port
add bridge=bridge1 interface=sfp-sfpplus2
add bridge=bridge1 interface=sfp-sfpplus3
add bridge=bridge1 interface=sfp-sfpplus4
add bridge=bridge1 interface=sfp-sfpplus5
add bridge=bridge1 interface=sfp-sfpplus6
add bridge=bridge1 interface=sfp-sfpplus7
add bridge=bridge1 interface=sfp-sfpplus8
add bridge=bridge1 interface=sfp-sfpplus9
add bridge=bridge1 interface=sfp-sfpplus10
add bridge=bridge1 interface=sfp-sfpplus11
add bridge=bridge1 interface=sfp-sfpplus12
/interface l2tp-server server
set l2tpv3-circuit-id="" l2tpv3-cookie-length=0 l2tpv3-digest-hash=md5
/interface list member
add interface=sfp-sfpplus1 list=WAN
add interface=bridge1 list=LAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether1 network=192.168.88.0
add address=10.90.21.254/24 interface=bridge1 network=10.90.21.0
/ip dhcp-client
add interface=sfp-sfpplus1
/ip dhcp-server network
add address=10.90.21.0/24 dns-server=192.168.1.254 gateway=10.90.21.254 netmask=24 ntp-server=192.168.1.254
/ip firewall filter
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes
add action=accept chain=forward connection-state=established,related
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip ssh
set host-key-size=4096
/system clock
set time-zone-name=Europe/Paris
/system ntp client
set enabled=yes
/system ntp client servers
add address=192.168.1.254



iperf3 -p 9225 -c iperf.par2.as49434.net
Connecting to host iperf.par2.as49434.net, port 9225
[  5] local 10.90.21.200 port 44914 connected to 193.177.162.41 port 9225
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  82.7 MBytes   694 Mbits/sec    0   3.00 MBytes       
[  5]   1.00-2.00   sec  80.0 MBytes   671 Mbits/sec    0   3.00 MBytes       
[  5]   2.00-3.00   sec  81.2 MBytes   682 Mbits/sec    3   1.54 MBytes       
[  5]   3.00-4.00   sec  80.0 MBytes   671 Mbits/sec    0   1.64 MBytes       
[  5]   4.00-5.00   sec  80.0 MBytes   671 Mbits/sec    1   1.21 MBytes       
[  5]   5.00-6.00   sec  80.0 MBytes   671 Mbits/sec    0   1.29 MBytes       
[  5]   6.00-7.00   sec  80.0 MBytes   671 Mbits/sec    0   1.35 MBytes       
[  5]   7.00-8.00   sec  80.0 MBytes   671 Mbits/sec    0   1.39 MBytes       
[  5]   8.00-9.00   sec  81.2 MBytes   682 Mbits/sec    0   1.42 MBytes       
[  5]   9.00-10.00  sec  80.0 MBytes   671 Mbits/sec    0   1.43 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   805 MBytes   675 Mbits/sec    4             sender
[  5]   0.00-10.02  sec   803 MBytes   672 Mbits/sec                  receiver

iperf Done.

Solved, I feel ashamed : the router speed is limited by my ISP. I am supposed to have 5G/s now and 10G/s later and I only have 600Mb/s. Sorry for the confusion.

I measured NAT speed using Mikrotik speedtest : around 160Gbit/s
Quite and impressing speed indeed compared to my last firewall based on OPNsense.

Seems like I bought the right hardware buying a CCR2004…