If WAN fails over, MTR shows packet loss on 1st hop (router)

Hi all, Not sure how best to explain the issue, but here goes.

Below is a copy of my routing table. #2 is unreachable. This leaves the default route gateway as 46.31.224.4, and the gateway for route-mark WAN2 as 46.31.224.4 also.

 
#      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0   S  ;;; Routing Mark - WAN2 : Failover
        0.0.0.0/0                          WAN1 - BT VDSL            2
 1   S  ;;; Routing Mark - WAN2
        0.0.0.0/0          46.31.226.148   46.31.224.4               1
 2   S  0.0.0.0/0                          WAN1 - BT VDSL            1
 3 A S  0.0.0.0/0          46.31.226.148   46.31.224.4               2
 4 ADC  10.10.10.0/24      10.10.10.251    LAN-Bridge                0
 5 ADC  46.31.224.4/32     46.31.226.148   WAN2 - VF ADSL            0
 6  DC  172.17.0.0/24      172.17.0.1      07 - Management         255

If I then MTR out of my network, the first hop shows lots of packet loss, shown below

  Host 									Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 10.10.10.251						97.8%    46    0.4   0.4   0.4   0.4   0.0
 2. 46.31.224.4							0.0%    46   17.8  18.5  17.3  31.4   2.2
 3. 46.31.224.1							0.0%    46   17.7  20.9  17.5  88.3  13.1
 4. ge-1-0-11.rt0.sov.uk.goscomb.net	0.0%    46   18.1  18.5  17.5  29.2   1.7
 5. xe-0-0-1-0.rt0.thn.uk.goscomb.net	0.0%    46   18.3  19.3  17.8  46.7   4.7
 6. 195.66.224.125 						0.0%    46   37.8  19.5  17.8  37.8   3.8
 7. 209.85.240.61						0.0%    46   18.6  18.9  17.9  23.4   1.1
 8. 209.85.253.94						0.0%    46   18.7  19.3  18.5  28.1   1.4
 9. 72.14.242.166						0.0%    46   28.4  28.4  27.7  29.4   0.3
10. 72.14.238.43						0.0%    45   28.0  28.8  27.8  53.5   3.8
11. ???
12. google-public-dns-a.google.com 		0.0%    45   27.8  28.1  27.6  28.9   0.3

If I disable the route for traffic route-marked for WAN2, the MTR then looks as it should.

 
#      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0   S  ;;; Routing Mark - WAN2 : Failover
        0.0.0.0/0                          WAN1 - BT VDSL            2
 1 X S  ;;; Routing Mark - WAN2
        0.0.0.0/0          46.31.226.148   46.31.224.4               1
 2   S  0.0.0.0/0                          WAN1 - BT VDSL            1
 3 A S  0.0.0.0/0          46.31.226.148   46.31.224.4               2
 4 ADC  10.10.10.0/24      10.10.10.251    LAN-Bridge                0
 5 ADC  46.31.224.4/32     46.31.226.148   WAN2 - VF ADSL            0
 6  DC  172.17.0.0/24      172.17.0.1      07 - Management         255



  Host 									Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 10.10.10.251						0.0%    46    0.4   0.4   0.4   0.4   0.0
 2. 46.31.224.4							0.0%    46   17.8  18.5  17.3  31.4   2.2
 3. 46.31.224.1							0.0%    46   17.7  20.9  17.5  88.3  13.1
 4. ge-1-0-11.rt0.sov.uk.goscomb.net	        0.0%    46   18.1  18.5  17.5  29.2   1.7
 5. xe-0-0-1-0.rt0.thn.uk.goscomb.net    	0.0%    46   18.3  19.3  17.8  46.7   4.7
 6. 195.66.224.125 						0.0%    46   37.8  19.5  17.8  37.8   3.8
 7. 209.85.240.61						0.0%    46   18.6  18.9  17.9  23.4   1.1
 8. 209.85.253.94						0.0%    46   18.7  19.3  18.5  28.1   1.4
 9. 72.14.242.166						0.0%    46   28.4  28.4  27.7  29.4   0.3
10. 72.14.238.43						0.0%    45   28.0  28.8  27.8  53.5   3.8
11. ???
12. google-public-dns-a.google.com 		0.0%    45   27.8  28.1  27.6  28.9   0.3

The mangle rules are as follows-

/ip firewall mangle
add action=mark-connection chain=prerouting in-interface="WAN2 - VF ADSL" new-connection-mark=WAN2_conn
add action=mark-routing chain=prerouting dst-address-list=Voipfone dst-port=5060-5070,10000-60000 new-routing-mark=WAN2 passthrough=no protocol=udp
add action=mark-routing chain=output comment=VFVDSL connection-mark=WAN2_conn new-routing-mark=WAN2 passthrough=no

Also, if #2 becomes reachable, the MTR also shows 0% loss for the first hop.

What is causing this? It’s almost as if the two routes clash when the primary route is down? I can script things to avoid this, but I would like to know in more detail why it doesnt work before I make changes.

Any help is much appreciated, thanks.

I’ve noticed similar result. But if you exit mtr and run it again right after disabling the first default route, you’ll see the new route being used. So I belive this has impact only on route tracing, not on the connection itself.

Thanks for the input but that sounds like a state not being cleared. With my issue, it would show 100% packet loss, even after restarting MTR.

I think this is a bug with in ROS, or an issue with my understanding.

My theory is… if there is a default route, and a ‘route-marked’ route using the same gateway, it causes the problem.


Cheers