Community discussions

MikroTik App
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Packet loss to loopback - OSPF / MPLS

Mon Apr 26, 2021 4:50 am

Okay, I have 2 CHRs(running vrrp), and a ccr2004. Mpls, vpls, ospf between the 3(full mesh) across 2 different connections, 1 is a wireless connection, the other a fiber connection. Wireless interface cost is 20, fiber is 10 on all 3 devices.

They all have a bridge (no ports on it) acting as the loopback for ospf.

I have smokeping running on a third server in the same location as the 2 CHRs, with a ping probe doing 5 pings 20sec. it shows random packet loss with no obvious pattern to the loopback of the 2004, but not to either of the physical interfaces on the device(one connected to fiber, one connected to wireless). All addresses on the Chr show no loss. Unfortunately this presents a problem as there is vpls between the loopback of all of the devices, and there is corresponding loss over the vpls tunnel.

Does anyone have any suggestions on steps to troubleshoot this or have an idea of what it could be? My best guess is it's related to ospf or mpls.
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Packet loss to loopback - OSPF / MPLS

Mon Apr 26, 2021 9:23 am

What type of hypervisor are you using for the CHR? There are some known issues with MPLS and the CHR.

The 2004 has certainly had some issues - what version of ROS are you running on it?
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Re: Packet loss to loopback - OSPF / MPLS

Mon Apr 26, 2021 3:06 pm

What type of hypervisor are you using for the CHR? There are some known issues with MPLS and the CHR.

The 2004 has certainly had some issues - what version of ROS are you running on it?
The hypervisor is proxmox, multi queue enabled, using host cpu.

Running on 6.48.1 on all.
 
JJT211
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Sun Apr 28, 2019 9:01 pm

Re: Packet loss to loopback - OSPF / MPLS

Mon Apr 26, 2021 3:59 pm

I would start with running the LTS firmware 6.47.9
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Re: Packet loss to loopback - OSPF / MPLS

Mon Apr 26, 2021 5:49 pm

What type of hypervisor are you using for the CHR? There are some known issues with MPLS and the CHR.

The 2004 has certainly had some issues - what version of ROS are you running on it?
I was thinking about what you said with MPLS having issues, and it occurred to me that the 2 physical interfaces would still be running through mpls, and they do not show any loss.
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Packet loss to loopback - OSPF / MPLS

Tue Apr 27, 2021 2:04 pm

Running on 6.48.1 on all.

You need to be running at least 6.48.2 because it has specific fixes for packet loss and reboots for the 2004 series. Normally I run Long Term on everything, but for the 2004, it needs to be this specific version for now.

What's new in 6.48.2 (2021-Apr-09 10:17):

###clipped###

*) switch - improved resource allocation on 98PX1012 switch chip for CCR2004-1G-12S+2XS device;
*) switch - improved system stability with 98PX1012 switch chip for CCR2004-1G-12S+2XS device;
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Re: Packet loss to loopback - OSPF / MPLS

Tue Apr 27, 2021 7:24 pm

Running on 6.48.1 on all.

You need to be running at least 6.48.2 because it has specific fixes for packet loss and reboots for the 2004 series. Normally I run Long Term on everything, but for the 2004, it needs to be this specific version for now.

What's new in 6.48.2 (2021-Apr-09 10:17):

###clipped###

*) switch - improved resource allocation on 98PX1012 switch chip for CCR2004-1G-12S+2XS device;
*) switch - improved system stability with 98PX1012 switch chip for CCR2004-1G-12S+2XS device;
I Just installed that, I'll report back the results.
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Re: Packet loss to loopback - OSPF / MPLS

Wed Apr 28, 2021 12:57 pm

Unfortunately, that was not the solution. I'm going to try running smokeping through a vyos gateway instead of the chr and see if that has the same pproblem. I put an rb4011 out there, and it was significantly better but does appear to still be dropping packets, so it's very possible its the Chr.
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Re: Packet loss to loopback - OSPF / MPLS

Thu Apr 29, 2021 4:11 am

Further news: after looking into the problem further, I see it is clearly happening at the vmbr, the drop count is consistent with what smokeping is showing.

Anyone familiar with proxmox/linux bridges?
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Packet loss to loopback - OSPF / MPLS

Thu Apr 29, 2021 9:17 am

You're likely hitting the limitation of running MPLS in a CHR that's not running on Hyper-V. Proxmox deals with MTU differently.

https://mum.mikrotik.com/presentations/ ... 448059.pdf

There was supposedly a fix for this issue a few months back in ROS but I haven't tried it. Also, I've seen a number of people switch to OVS in ProxMox and claim that it's a workaround for the MTU issue
 
antleo
newbie
Topic Author
Posts: 47
Joined: Thu Dec 19, 2019 3:27 am

Re: Packet loss to loopback - OSPF / MPLS

Mon May 17, 2021 4:27 am

So after further investigation, there was some unknown issue with the ospf routing - we had 1 wireless link, and 1 fiber link. According to the rib on both sides it was routing over fiber with an appropriate next hop. However we shut down the wireless link, and suddenly there was no route to the loopbacks being announced at all. I added the interface as passive, and added a network statement, and it began working again, which is odd as in this particular instance we are redistributing connected routes, which should include the loopback.

TLDR - if you run into this issue and are using redistribute, try using passive/network instead.
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Packet loss to loopback - OSPF / MPLS

Mon May 17, 2021 8:28 pm

Glad you figured it out.

As a general rule, you don't want to use redistribute with OSPF except for actual external routes like Static, BGP, etc. It breaks the area boundaries and obscures the SPF topology info.

Who is online

Users browsing this forum: No registered users and 11 guests