I have a seriuos problem with all my CCR2004 when traffic pass from giga interface to 10 giga interface and viceversa I have a packet loss aorund 1% or little less.

According with the block diagram https://i.mt.lv/cdn/product_files/CCR20 ... 200459.png I am thinking the problem is generated in the passive intelligent port extender

CCR2004 is not usable to mix different interface speeds :-(

how is it possible?

Do you experience packet loss in both directions equllky or is one direction distinctively worse?

I'd expect problems in packet direction 10G->1G where quite some buffering occurs and enabled flow control should help a lot (another matter is whether flow control actually works).

We have mainly packet loss pinging from the remote end to ccr2004 gigabit interface, the problem probably occurs when the upstream traffic from the 10G->1G fullfill the ethernet..

Enabling flow control not solve the problem.

buffer memory issue? I.e not enough buffer space when going from 10Gbps to 1Gbps and vice versa. Not sure how to get the stats from a Mikrotik on this.

There was a similar issue on the CCR1072 where the ethernet port was claimed only to be suitable for management traffic.
viewtopic.php?t=125361

We have mainly packet loss pinging from the remote end to ccr2004 gigabit interface, the problem probably occurs when the upstream traffic from the 10G->1G fullfill the ethernet..

So does the packet loss only occur when the 1G interface is full?

We have mainly packet loss pinging from the remote end to ccr2004 gigabit interface, the problem probably occurs when the upstream traffic from the 10G->1G fullfill the ethernet..

So does the packet loss only occur when the 1G interface is full?

No when the giga interface receive data from 10 giga interface: the slower port sometimes has to keep in pause the faster one before to receive new packets because the buffer is full or not have the buffer.

What's the lowest throughput (on the 1G) you've seen the packet loss?

The problem is how fast the data arrive on the giga port from the 10 giga port, you can loose packet also whit 100kb or less. If the number of packets arriving exeeds tha capibility of gigaport to send it you will lose packet.
Generally is around 1% or littble bit less. This is a big hardware poblem for ccr2004 and it is not usable to mix different speed ports.

The 1G port is for management (winbox, ssh to the router), as it's also labeled on the front panel. This port is somewhat limited, and is meant for what it says. The diagram also gives a clue to this.

Normis we are speaking about SFP ports

Sincerely I am very disappointed for CCR2004's performance.
We replaced a CCR1036 + 10G switch (with trunks) to a CCR2004 with directly put in DACs cables 10G.

We do about 4.5gb traffic through it.
20 firewall rules, most in RAW for "simple" packet filter at the border.
fasttrack enabled.


4x99% cpu, one 100%.
I tried disabling everything, just fastpath, 80% cpu.
even with 2g traffic the voip was chopping.
Replaced back the 1036, and we are with it now. I could consider the 1072 but I fear the watchdog issues that are on the forum. Till now the 1036 is 2+ years working solid.
we use the long-term ROS.

A router where you can't mix different sfp ports speeds it's not usable in production, if I deploy fiber I need to have 0% packets loss and no 1% like now.

I opened a ticket [SUP-30387] but for now no solution to the issue :-(

Have you tried changing the interface queues for both ports from hardware-only to ethernet-default ?

You need buffering when mixing interface speeds on any router and i'm not sure what the default buffer capabilities are for the CCR2004.

I would at least try a few different queue types for the interfaces to see if performance improves.

Where can I find precise documentation about queue types?
Well I read the wiki... but I have not found any REAL example of WHERE to use wich queue type or interface type

Have you tried changing the interface queues for both ports from hardware-only to ethernet-default ?

You need buffering when mixing interface speeds on any router and i'm not sure what the default buffer capabilities are for the CCR2004.

I would at least try a few different queue types for the interfaces to see if performance improves.

The first think that I have tried before to write here and to open a support ticket.

Sincerely I am very disappointed for CCR2004's performance.
We replaced a CCR1036 + 10G switch (with trunks) to a CCR2004 with directly put in DACs cables 10G.

But You did a downgrade! Take a look at the performance figures:

https://mikrotik.com/product/CCR1036-12 ... estresults
https://mikrotik.com/product/ccr2004_1g ... estresults

Yes, the 2004 is faster with pure fastpath and big packets, but only in this situation. Everything else, and the 1036 wipes the floor with it. It isn't meant as a direct substitute to the 1036 - it is more something above 1009 and just below 1016.

Yes, each one of its 4 cores is much faster than the 10XX ones - but not enough to offset 4 cores versus 36 cores. It will outperform the CCR10XX series in single threaded activities, but not in firewalling.

Hello Paternot, I have read carefully the performance figures, but sincerely I expected a lot more.
I did not expect a 100% cpu load !!!
However I am planning to replace the 1036 witha 1072 because I need the pure 10G ports for each link.
I have some fear to get a 1072 that has issues as in the threads about "watchdog and 1072".

I use conntrack, I use Fasttrack, it is one of my border router.

Hello Paternot, I have read carefully the performance figures, but sincerely I expected a lot more.
I did not expect a 100% cpu load !!!
However I am planning to replace the 1036 witha 1072 because I need the pure 10G ports for each link.
I have some fear to get a 1072 that has issues as in the threads about "watchdog and 1072".

I use conntrack, I use Fasttrack, it is one of my border router.

Well, I can't speak for the usage model on You network - but if it was taxing one CCR1036, I find remarkable that one CCR2004 can barely cope with the load. I mean, take a look at the results for 512 bytes and 25 ip rules: it's about 1/3 of the 1036 speed!

You may be right, I may have chosen the wrong product.
The 1036 was at about 15% average cpu...

Any update on this issue?

Any update on this issue?

Any update on this issue?

I would just like to say that I experienced the same issues.

I have x4 CCR2004's and when they were in my lab before being shipped to production I had them connected to each other using Mikrotik 1G S-RJ01 Ethernet modules. NSW-RTR-01 (172.16.4.1 - CCR2004) was connected to NSW-RTR-02 (172.16.4.2 - CCR2004) via a Mikrotik 1G S-RJ01 Ethernet module. I ran "ping 172.16.4.2 interface=sfp-sfpplus2 do-not-fragment count=100 size=1500" and received high packet loss. Since I did this a while ago now I would have to dig up this packet loss data if you want proof.

Alternatively Mikrotik you could grab x2 CCR2004's and connect them to each other via Mikrotik 1G S-RJ01 Ethernet modules and run the same ping test and see the packet loss yourself. I don't see any packet loss if the Router to Router connections use 10G interfaces instead. I haven't tested this but I assume 1G Fibre SFPs will be affected too. Hopefully shaping or setting the Tx/Rx Flow Control to on will resolve this issue.

Worth investigating.

I still have x2 CCR2004's that aren't in production yet so I did some testing on them. The CCR2004's are directly connected to each other via sfp-sfpplus2.

I tested with:
- 1G Ethernet module - Mikrotik S-RJ01 - 17% packet loss (First attempt)
- DIFFERNT 1G Ethernet module & Ethernet cable - Mikrotik S-RJ01 - 3% packet loss (Second attempt)
- 1G SM Fibre SFP - Ubqiuiti UF-SM-1G-S - 0% packet loss
- 1G MM Fibre SFP - Ubiquiti UF-MM-1G - 0% packet loss

Obviously something is wrong with the CCR2004 and 1G ethernet modules. In production I have the Router to Router connections as 10Gbps DAC cables and they get 0% packet loss.

Mikrotik S-RJ01:
[x@NSW-RTR-01] > ping 172.16.4.2 count=420 do-not-fragment size=1500 interface=sfp-sfpplus2
419 172.16.4.2 1500 64 0ms
sent=420 received=347 packet-loss=17% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

Mikrotik S-RJ01:
[x@NSW-RTR-01] > ping 172.16.4.2 count=420 do-not-fragment size=1500 interface=sfp-sfpplus2
419 172.16.4.2 1500 64 0ms
sent=420 received=406 packet-loss=3% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

Ubqiuiti UF-SM-1G-S:
[x@NSW-RTR-01] > ping 172.16.4.2 count=420 do-not-fragment size=1500 interface=sfp-sfpplus2
419 172.16.4.2 1500 64 0ms
sent=420 received=420 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

Ubiquiti UF-MM-1G:
[x@NSW-RTR-01] > ping 172.16.4.2 count=420 do-not-fragment size=1500 interface=sfp-sfpplus2
419 172.16.4.2 1500 64 0ms
sent=420 received=420 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

If I was you I would give shaping just below 1Gbps a shot and see if you still get the packet loss.

I don't use 1G Ethernet modules in productions so I don't have too much skin in the game, goodluck.

If I was you I would give shaping just below 1Gbps a shot and see if you still get the packet loss.

I don't use 1G Ethernet modules in productions so I don't have too much skin in the game, goodluck.

it not work

Can I have an official answer about this problem by mickrotik support? I am pinging every day for ticket [SUP-30387] wihtout any answer :-(

Any update on this issue?

Can I have an official answer about this problem by mickrotik support? I am pinging every day for ticket [SUP-30387] wihtout any answer :-(

If you're not getting response through official channels, then you surely won't get any response here. This is user forum (where MT employees might check from time to time and answer some questions) not official support forum.

I have a support tikcet opened without any answer too

Exactly the same problem here !
Tired of playing "Betatester" for Mikrotik. Paid for something that really not Works.. and no one help of them.... THX Mikrotik.
Lost time, lost patience, lost money... and you loose a costumer.

I am using back old CCR016 to connect gigabit site. CCR2004 is not usable in real because if we have problem to use different port speeds we can't use 25 Gb/s uplink ports with 10 gbis/s ports.

Any update on this issue?

Any update on this issue?

Not much will happen in 3 days ... specially so not over weekend.

There are a lot of users with same problem around and after 2 months no answer from mikrotik :-(

At this point I have to suppose that is è hw problem without solutions. CCR2004 is an expensive paperweight

Hello. I want to tell you my personal experience.
My experience with 2004 was bad because I replaced a 1036, but maybe the 2004 was too "light" for the purpose.

Now I have installed two 2004 with:

1x10g uplink
1x rj01 to AF24 ubiquiti
2x SFP+ 10G to other sites
everything in routing via BGP
0 issues and cpu low.
I tried also to disable conntrack for the routing (previously I used fasttrack everywhere) and 0 issues, just a higher cpu.

Please check the tx drop under the tab Tx Stats

Hello
I got 1649 tx drop on the interface with an RJ01 with AIRFIBER24 from UBNT.
it 4 days and 4 hours.

it is a 500mbit link.
no issues from the customers.

maybe the issue you are describing?
The interfaces are routed between each other, so no mixed speeds in the same bridge.

Hello
I got 1649 tx drop on the interface with an RJ01 with AIRFIBER24 from UBNT.
it 4 days and 4 hours.

it is a 500mbit link.
no issues from the customers.

maybe the issue you are describing?
The interfaces are routed between each other, so no mixed speeds in the same bridge.

You are loosing packets if you check on SFP28 you 'll find zero tx drops.

Now I have all 10 gigabit pots but I loose packets onyl on sfplus ports.

Hello.
I use:
Sfp1 1g sfp
Sfp2 10g link
Sfp3 1g on rj01 with airfiber
Sfp11 and sfp12 on a bridge. Two 1g links

I have drops only on the sfp3 1g.
I don't use 25g ports

It depens how fast the trafffic coming on the 10 gibi/s port to be routed or swithed to the other ports.

We are speaking about 0,1% of packet loss that is not to much for a wireless link but is too much for a fiber link.

Everything is routed!

I have a similar problem, CCR2004 post sfp-sfplus2 - SFP 1G insert, no auto, flow controll On ... and I have a lot of drop packages.
They start to appear when I exceed the speed of 100Mbit / s, at 200-500Mbit / s I have several thousand of them per minute, multicast transmission is interrupted (frame stop and IPTV artifacts).
Changing queue improves things a bit, but fails to get stability.

Hello
I never found final documentation in how changing queue helps.
I leave them at only-hardware-queue

Today I have tested all combinations with different queue sizes.
There is a difference, if you set ethernet-default and experiment with the size of the queue it may be enough for you.
I checked between 50-1000 packets and there was always a TX DROP problem, with queue values of 150-250 the least problem was. However, I am using multicast for IPTV, so dropping packets too often is unacceptable.
This problem occurs only with optical inserts with 1Gbps link speed, no matter if the insert itself is 1Gbps or 10Gbps.

I went back to CCR1036, wait for the new original MikroTik inserts and check it again.

We use only hardware queue
on the wiki I didnt found any interesting data about changing queue types on interfaces.

I have replaced both 2004 in heavy duty enveroniment (core)

I installed 2x 2004 with FASTPATH and bgp only.
heavy packet loss on 10G interfaces
replaced with a UBNT edgeswitch 16XP and 1036+vlans, work flawlessy
with just 1.5g traffic, cpu at 30% (fastpath) and irq at 15, heavy packet loss.
changed cables, ports. no change.
latest longterm, tried also stable, no differencies.

Hi,

We had issues with interfaces dropping using rj01 on 2004s. After replacing with fiber sfps the issues are gone. Could perhaps be power consumption.

/Mikael

I have the same problem and opened a ticket with no response over 2 weeks ago. Very annoying. The odd thing is that im only getting the tx drops on the interface going to my elan. At first I thought it was an mti mismatch but have since ruled that out. I even put a CRS309 in between my ELAN and my CCR2004 and the tx drops moved to the interface going to the ELAN on the CRS309 and stoped on my cloud core. I've tried finisar, mikrotik, and fiberstore modules with the same results. Even tried a 1G and 10G DAC from multiple vendors. My ISP for my ELAN checked their equipment and show no drops on their side or FCS errors. It's odd that the problem moved down to the interface on my CRS309 instead of staying on my CCR2004. Even odder is when I unplug the 10G DIA and switch over to my backup 1G DIA the TX drops go away and as soon as I plug my 10G DIA back in they come back. It is weird that the TX drops only show up once you open the interface and not in the column view when I select them. It always shows 0 in the column view on both the crs2004 and crs309 depending on whether or not im using the crs309 inline but shows up after double clicking the interface and going to stats. The other weird thing is that the tx drops don't show up on any other 1G interface on the CCR2004. I would expect it to be consistent at the very least. I set a lab up with all the smae routers and have yet to be able to reproduce the result in the lab and I believe this to be due to the fact that I am not able to tax the CPU load high enough to perturbate the issue. Please Mikrotik. HELP!!!!! My CCR2004 like many others is a paperweight as it is causing out of order packets and voice quality issues even thought the packet loss through the ccr2004 is low at 0.08 to 0.12 percent.

Give it up I have opened a ticket 3 months a go without any answer. No chance to solve this issue.

I even put a CRS309 in between my ELAN and my CCR2004 and the tx drops moved to the interface going to the ELAN on the CRS309 and stoped on my cloud core. … It's odd that the problem moved down to the interface on my CRS309 instead of staying on my CCR2004.

So you think the CCR2004 is causing tx drops on a completely separate device (the 309)? I don't think that's possible.

There is a problem with Tx-Drop on mikrotik 10Gbps interfaces in combination with any 1Gbps interface.
Even at speeds of 100Mbps, TX-drops arrive quickly.

I opened a support ticket on 12/21/2020 and still no response! This is obviously an issue and we deserve answers. Anyone else had luck with a support ticket?

I also have an open ticket, but I'm waiting longer ... no reaction.

I even put a CRS309 in between my ELAN and my CCR2004 and the tx drops moved to the interface going to the ELAN on the CRS309 and stoped on my cloud core. … It's odd that the problem moved down to the interface on my CRS309 instead of staying on my CCR2004.

So you think the CCR2004 is causing tx drops on a completely separate device (the 309)? I don't think that's possible.

I've also problem with tx drops on switch (CRS317). Never had this problem before with RB4011. Tx drop are only on interfaces connected to CCR2004. Tested "overloading" ports connected to server with CHR -> not drop, no issue.

I've also problem with tx drops on switch (CRS317). Never had this problem before with RB4011. Tx drop are only on interfaces connected to CCR2004. Tested "overloading" ports connected to server with CHR -> not drop, no issue.

In the case of connecting the 10G interface to the 1G interface and additionally overloading the 1G link with the test, the appearance of Tx-Drop is a rather natural situation.

It is not natural for them to appear even further up to 1G, e.g. at 200Mbps.

At this point both units are bricks to me and I am going to return the units I purchased as they are paper weights and no response from the vendor. I get that we have COVID going on right now but a brand new product with multiple people having the same issue deserves an answer I would think. My network can't take much more of this before I have to break down and spend big bucks for Juniper.

My carrier has been helpful on scheduling a 10G port upgrade on my 1G ELAN going to the CCR2004 to see if this helps on the 22nd. If this does not fix the issues I am shipping these back. Resume pulling hair out.

Oh also of note trying to generate support.rif crashes the router horribly and requires physically removing power from the unit. Lovely.

We have in production several CCR2004 and all have problems about packet loss on 10 Giga sfp+ ports no problem of packet loss on sfp28 ports used at 10 Giga.

It's an hardware problem without solutions.... :-(

Hello,

I appreciate this post but do not appreciate the lack of response from MT on this. I need these CCR2004 routers but am NOT ordering them from MT due to this issue being unresolved.

Again, thanks for the posts!

Hello,

I appreciate this post but do not appreciate the lack of response from MT on this. I need these CCR2004 routers but am NOT ordering them from MT due to this issue being unresolved.

Again, thanks for the posts!

Give it up it's normal from mikrotik the lack of response when they can't resolve the problem.