Community discussions

MikroTik App
 
User avatar
sirbryan
Member
Member
Topic Author
Posts: 303
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

CCR2004-1G-12S+2XS Performance Update Dec 2022 - 7.4.1 = 19Gbps

Tue Dec 06, 2022 6:58 am

When I first got my 2004's, routing performance seemed limited to 2-3Gbps, a far cry from the potential 20-40Gbps mentioned on the website. So I set out to replicate MikroTik's lab results as best I could. I created a "sandwich" with two CCR2116's on the top and bottom and two CCR2004 (12S+) in the middle, connected by the SFP28 port. The 2116's each have two SFP+ DACs connecting ports 1 and 2 to the 2004's 1 and 2.

My purpose in doing this was to see what to expect performance-wise if I used these as aggregation "switches" (er, routers) on either end of a pair of fibers between two buildings (vs. installing a mux or replacing the 2-count fiber with a 12- or 24-count) since I already had them in stock.

Initially I configured the 2004's like a switch, with all the ports in a bridge, and each port tagged "natively" (PVID) to a VLAN (1 = 1001, 2 = 1002, 3 = 1003, etc.) with the SFP28 ports acting as the trunks. SFP+ port 1 on each CCR2116 was then connected to the other, as were ports 2, by way of the 2004 "switches." The 2116's 16-core CPUs were easily able to saturate each port with 9.6Gbps of UDP (and occasionally TCP) traffic, and when things got working well, we ended with 19+Gbps of traffic across the SFP28 link.
2116 No. 1-SFP+1==DAC==SFP+1 2004 No. 1        2004 No. 2==DAC==SFP+1 2116 No. 2
                                     SFP28==SFP28 
2116 No. 1-SFP+2==DAC==SFP+2 2004 No. 1        2004 No. 2==DAC==SFP+2 2116 No. 2
The next test involved routing through the 2004's instead of VLAN bridging. After assigning routes and IP's to all interfaces, with the same cabling configuration, test results were similar: up to 19.x Gbps before we started to run out of CPU.

At around 19Gbps both 2004's hit over the 90% mark, regardless of UDP or TCP traffic. Using the built-in speed tests on the 2116's, the TCP tests ranged from 4Gbps one-way to 8.8Gbps, with a very rare 9.2Gbps on occasion. There were zero firewall rules, queues, or anything else in all four routers, simply IP addresses and a couple of static routes.

I'd say, using RouterOS 7.4.1, the theoretical maximum for routing or bridging on these things is 20Gbps in the real world, with most people seeing less than that depending on what else they have the router doing.

On 7.6 and 7.7beta9, my throughput tests were much lower, like 6-7Gbps instead of 8-9Gbps per port. So while 7.7 claims to have fixed support for the various SFP+ rates, there's a performance hit somewhere else.

Also interesting to note, traffic going from the SFP28 out the SFP+ ports uses less CPU than traffic coming in via the SFP+ ports and going out the SFP28 ports.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2990
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: CCR2004-1G-12S+2XS Performance Update Dec 2022 - 7.4.1 = 19Gbps

Tue Dec 06, 2022 6:20 pm

in routing test ccr2004 was working on fast-path mode ?

have you tryed bridge without vlan to allow fast-path in bridge mode?
 
User avatar
sirbryan
Member
Member
Topic Author
Posts: 303
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: CCR2004-1G-12S+2XS Performance Update Dec 2022 - 7.4.1 = 19Gbps

Wed Dec 07, 2022 1:10 am

Both 2004's reported Fast Path was active, for both bridging and routing. (As of 7.2, fast path supports VLANs.)

I even tried fast forward on the bridge (2 ports only) with no real change in CPU loading; as I recall, Fast Path actually seemed better.

As I think about it, this 19Gbps is bidirectional. Since everybody likes to aggregate things, this would be 19 in and 19 out, totaling to 38Gbps, which is right close to the 40 that MikroTik's RFC2544 tests show.
 
paolobyte
newbie
Posts: 28
Joined: Wed Apr 05, 2017 6:00 am

Re: CCR2004-1G-12S+2XS Performance Update Dec 2022 - 7.4.1 = 19Gbps

Tue Mar 14, 2023 1:55 pm

How did you configure it? I guess you did not do bridge+vlan routing since it doesn't have any switch chip. What I mean is, VLAN on top of the physical interface?
 
User avatar
sirbryan
Member
Member
Topic Author
Posts: 303
Joined: Fri May 29, 2020 6:40 pm
Location: Utah
Contact:

Re: CCR2004-1G-12S+2XS Performance Update Dec 2022 - 7.4.1 = 19Gbps

Tue Mar 14, 2023 4:37 pm

I did it with both IP's on the interface as well as all ports bridged with VLAN's. In both cases, it's all done in CPU, so neither method has any hardware offload mechanism to leverage.

I also realized that the 19Gbps limitation was more about the fact that I was using two 10Gbps cables from upstream routers acting as source generators. It may be possible to get a bit more through if you're using SFP28 uplink and three SFP10's on the downlink side, since traffic going in that direction uses less CPU.

At 18+Gbps, CPU loading was in the 90's, and I didn't have a source that could generate more traffic, so I stopped pushing it past 19Gbps.
 
erlend
just joined
Posts: 2
Joined: Sun Apr 30, 2023 9:30 pm

Re: CCR2004-1G-12S+2XS Performance Update Dec 2022 - 7.4.1 = 19Gbps

Sun Apr 30, 2023 10:22 pm

Dear all

I am using the following configuration:
https://www.init7.net/de/support/router ... 2.auto.rsc

I havs made several tests, with RouterOS 7.8 using two computers on the lan, and against my internet provider.
Locally I get max 7 Gbps, and against the init7 iperf3 server (25G connection!), about 5.5 Gbps.

Its all very disappointing.
Is there anything that I can change in the configuration in order to improve the performance?

I had the impression that 15G was possible?

Is there no other option than to splash out on a CCR2216?
can I use the configuration from the ccr2004 in the CCR2216?
Does anybody know if the CCR2216 will be able to have a throughput of 25G from a single interface with this configuration?
How warm does the CCR2216 get?
I was unable to use the S+RJ10 modules in the CCR2004, as it got too warm (70 degrees when connected at 10G, and 60 when connected at 1G and 2.5G)
With a heatsink on the sfp28 modules, and one sfp+ 10G module as well as 3x 1G modules, the fans of the ccr2004 mostly stay silent. Would that be the same for the CCR2216?

Best regards
Erlend

Who is online

Users browsing this forum: 4l4R1, boocko, loloski, lurker888, sch and 34 guests