MikroTik

markonen

No experience of Mellanox optics specifically but my Mikrotik devices have had zero issues with any third party optics I've thrown at them, be it SFP, SFP+, SFP28, QSFP+ or QSFP28. Everything has just worked. The Mellanox side has sometimes been a bit trickier so I'd look at optics that have the Mel...

markonen

Can you help me figure out a napkin scenario where it actually does cause a loop? I tried to think up one but couldn’t, at least not with EBGP and sane tie-breaking rules (like those I described). The way you enable this on Junos (just add "multipath;" on the bgp level) and IOS (set "...

markonen

Okay, confusion confirmed, although I can't rule out the confusion being in my head :D I don't understand what this has to do with ADD PATH. The OP (like me) wants to have the defaults received from two BGP peers automatically result in ECMP routing. This is what's happening in all the other vendor ...

markonen

I'm sorry but I still don't understand why. My read was that the OP had two BGP sessions with a default coming from each, and wanted the ECMP route over those. So no BGP session is carrying more than one route per prefix. What did I miss, where's the need for ADD PATH?

markonen

ECMP hashes changing should not normally affect peering sessions. Sure, if the peering session is over the ECMP link (implying multihop) then different paths would get chosen, but that should not affect the TCP session itself. The case where resilient hashing is more useful is ECMP load balancing of...

markonen

I've been wondering if there's some sort of confusion at MikroTik about what exactly we're asking for. Maybe someone at MT thinks they need to implement BGP ADD_PATH before working on this? Some of the comments I've seen over the years from them seem to point that way. I bet most of us would be very...

markonen

7.14 completely disabled my WireGuard-based OOB management network.

The root cause seems to be that all wg interfaces are stuck in the main VRF even if you've specified another VRF for them.

Downgrading to 7.13.5 brought them back to the custom VRF and brought back the network.

markonen

It is because of incorrect configuration. For BGP best path selection to work, all routes must be received in the same BGP instance. Is there a downside to doing this on purpose? I have a case where I want to have ECMP for routes received over two BGP sessions. If the best path selection is per BGP...

markonen

Thank you, packet sniffer indeed showed connection attempts from my end. So I double checked the config and saw that I was missing a MD5 key... :D Recapping why I ended up writing to the forum about such a simple config error: - I remembered seeing even non-established connections in the Session men...

markonen

I have a CCR2004-1G-12S+2XS running RouterOS 7.13 with 20 BGP peers running. Today I added two new ebgp connections (to bring the total to 22), and they just don't appear in "/routing bgp session" at all. I mean, the issue is not that the sessions aren't up, but that nothing appears in the...

markonen

Here's my setup: - Ubiquiti PoE Adapter U-POE-AT providing 802.3at power to port 1. This is supplying 48VDC @ 0.65A - Ubiquiti U6-Mesh access point drawing power from port 5, max power consumption is 11.4W according to the data sheet - SFP module for network access (should be unrelated to the PoE st...

markonen

qsfp - use sub-interface configuration for establishing link (for 40Gbps and 100Gbps links, all sub-interfaces must be enabled); This looks like something that will affect me. Currently I've only enabled the -1 interface for each QSFP28 port and explicitly disabled -2, -3 and -4. Sounds like my con...

markonen

On the one hand choosing a GPL licensed platform for your products (Linux) and then re-implementing lots of standard features from scratch for which better open source implementations exist is admittedly a weird hill to die on but it's a business decision MikroTik has made because it must believe t...

markonen

Can you please elaborate? What is fixed in v7.7 and v7.8 betas.

No he can't, because that would be excessive quoting.

markonen

Even the brand new 100G switch CRS504 has just 16MB of storage so that limitation is not going away any time soon. This is a decision MikroTik has made and the price is them having to build everything from scratch because the standard approaches will not fit.

markonen

Sounds simple but ideally you'd want to implement it in the switching ASIC for L3HW. You can then really dial down those delays and still be reliable on devices with starved CPUs. (No idea whether Mikrotik is actually working on that).

markonen

I'm thinking of using CCR2004-1g-2xs-pcie to build 1U Supermicro server + router systems for remote deployment. My worry is that if a server is accidentally turned off from the BMC (connected to the console port), power to the CCR will be lost and turning the system back on will require a site visit...

markonen

2) Disks are formatted from RouterOS as ext4 for a while now.

That's good news! As of 7.3.1, Disks > Format Disk in WebFig only offers ext3 though, and the formatted disk also appears as ext3? Is this a WebFig text label issue only?

Screenshot 2022-06-16 at 18.59.43.png

markonen

Two quick questions about containers:

1) are veth interfaces mandatory, or can host networking be used?
2) the docs state that ext4 is supported, is this new? Any plans to add ext4 support to Disks > Format Drive?

markonen

But I am against promoting a new major release to "stable" status (and prominently recommending it for installation on the webpage) before it is even feature-complete relative to the previous one (and without mentioning that in the description). … There apparently is a (human) resource bo...

markonen

I went to apply some overdue upgrades on one of our workhorse CCR2004s and remembered this thread. This is with full BGP tables and 2-10G of 24/7 web traffic.

markonen

There is a new related item in the latest beta:

*) switch - improved resource allocation on 98PX1012 switch chip for CCR2004-1G-12S+2XS device;

markonen

Longest peer uptime is 32 days but IIRC they had scheduled maintenance then. I haven't really paid attention to these enough to really say whether it's the remote or some change we did that reset the other sessions (or if it was indeed a lockup, I guess).

markonen

Full feeds from one or two transits (over both IPv4 and IPv6, so 2-4 sessions) + some peering. So one CPU core is more or less pegged doing BGP.

markonen

I have two that are running 6.47.8 with 75 days and 66 days of uptime, as well as a bunch that I just upgraded from 6.47.4 to 6.48.1 that were between 80 and 120 days of uptime before the upgrade.

Just BGP, static routes and raw firewall rules.

markonen

CCR2004 needs RouterOS 7 to use the full 4GB of RAM. With ROS6 it can only use 1792MB. So if that has been enough for you, then the 1016 should work fine (in terms of RAM).

markonen

I run five 2004s in production with full BGP tables (and a couple more without BGP) and they don’t suffer from the reboots. I’ve tried to pattern match the things people are reporting in this thread and OSPF stands out—I don’t run OSPF but many people reporting reboots here seem to.

markonen

Can you elaborate on the packet loss issue?

Here's the thread about it:
viewtopic.php?f=3&p=842145#p842145

markonen

switch - improved packet transmit between CPU and 98PX1012 for CCR2004-1G-12S+2XS device

This sounds like a fix to the CCR2004 packet loss issue. Would someone from Mikrotik like to give a bit more detail about what was done here? Thank you!

markonen

I don't think we can expect a somewhat stable fully functional v7 within 1.5 years Also bear in mind: it is common knowledge that things are only stable in the 'long term' branch so it could be many years from now before it will be really stable. It's worth noting that MikroTik already sells hardwa...

markonen

I even put a CRS309 in between my ELAN and my CCR2004 and the tx drops moved to the interface going to the ELAN on the CRS309 and stoped on my cloud core. … It's odd that the problem moved down to the interface on my CRS309 instead of staying on my CCR2004. So you think the CCR2004 is causing tx dr...

markonen

2 x 10gbps 10Gtek SFP+ MM Modules in a LAG

Sorry to hijack the thread, but what's the performance like on that LAG, and the CPU load?

I've been reluctant to set up bonding on my 2004s as its done in software vs my current setup of hardware bonding on a CRS317.

markonen

Have y'all had these issues with the v7 betas?

I know it's a hard question since v7's own bugs can mask this. But as the 2004 was designed for ROS7, I wonder if some of these problems are specific to ROS6.

Unfortunately I don't have any spares to try v7 on...

markonen

Went and looked at the uptimes of our six longest-deployed 2004s: 11w3d18h44m51s, S/N D4F00... 10w6d18h10m42s, S/N D4F10... 8w1d16h32s, S/N D4F10... 5w1d16h1m47s, S/N D4F10... 1w6d17h29m41s, S/N D4F00... 1d22h28m59s, S/N C8A60... The last two have clearly rebooted without human intervention, not 100...

markonen

I'm sorry if I'm off topic. I can't find any real test results in CCR2004 anywhere. Please tell me how much traffic you have going through the most loaded CCR2004 and what is the load on the CPU ? Our CCR2004s currently do BGP (full table) and have a simple Raw firewall (conntrack disabled). So wit...

markonen

I took counter samples from three of the seven CCR2004s I have in production and the TX drop rates for the busiest interfaces were: * 1/23536 packets dropped (0.0042%) * 1/76566 packets dropped (0.0013%) * 1/104768 packets dropped (0.00095%) These are all on 10G ports with 10G fiber links and ROS 6....

markonen

Conntrack is beneficial when the device can comfortably handle the number of connections. On your core router, that seems to be the case for normal traffic. Each connection has a memory cost though, and that makes conntrack a potential weak point in DoS attacks. So with conntrack/fasttrack the route...

markonen

Hi, I'm (ab)using a bonding interface on a CRS309 to load balance incoming traffic onto three Linux servers and it's working great! However, ARP link monitoring is giving me trouble. My configuration (with RouterOS 6.47.1) is: /interface bonding add arp-ip-targets=10.0.0.50 link-monitoring=arp mode=...

Search found 38 matches

Re: CCR2004-1G-12S+2XS SFP28 25G from mellanox [SOLVED]

Re: BGP ECMP (multipathing)

Re: BGP ECMP (multipathing)

Re: BGP ECMP (multipathing)

Re: BGP ECMP resilient hashing

Re: BGP ECMP (multipathing)

Re: v7.14 [stable] is released!

Re: How to disable ECMP on BGP?

Re: BGP connections not appearing in sessions

BGP connections not appearing in sessions

Fried a hEX S with 802.3at PoE

Re: v7.12beta [testing] is released!

Re: v7 and BFD, any ETA?

Re: v7.8beta [testing] is released!

Re: v7.7rc is released!

Re: v7 and BFD, any ETA?

Re: Using CCR2004-1G-2XS-PCIe from a RouterOS host (CHR/x86)

Re: v7.4beta [testing] is released!

Re: v7.4beta [testing] is released!

Re: v7.3beta [testing] is released!

Re: The big CCR2004 reboot thread (was 2004 hardware issues?)

Re: CCR2004 packet loss

Re: The big CCR2004 reboot thread (was 2004 hardware issues?)

Re: The big CCR2004 reboot thread (was 2004 hardware issues?)

Re: The big CCR2004 reboot thread (was 2004 hardware issues?)

Re: CCR2004 replacement advice

Re: The big CCR2004 reboot thread (was 2004 hardware issues?)

Re: v6.49beta [testing] is released!

Re: v6.49beta [testing] is released!

Re: v7.1beta3 [development] is released!

Re: CCR2004 packet loss

Re: 2004 hardware issues?

Re: 2004 hardware issues?

Re: 2004 hardware issues?

Re: CCR2004 packet loss over 10G Interfaces

Re: CCR2004 packet loss over 10G Interfaces

Re: BGP and conntrack

Server load balancing with bonding