Secondary IPv6 route weirdness with Hurricane Electric tunnel

Strykar · Wed May 15, 2024 6:44 am

My ISP provides a /64 prefix via DHCPv6 on the pppoe-out1 interface which I enable and configure to set a default route.
I offer this via SLAAC on the LAN bridge0 interface and LAN clients receive a GUA from the /64 and IPv6 works.

I wished to use HE's routed /64 as a secondary IPv6 route with a much higher routing metric / distance.
This is the suggested configuration by HE for their routed /64:

/interface 6to4 add comment="Hurricane Electric IPv6 Tunnel Broker" disabled=no local-address=1xx.xxx.xxx.xx0 mtu=1280 name=sit1 remote-address=216.218.221.42
/ipv6 route add comment="" disabled=no distance=1 dst-address=2000::/3 gateway=2001:470:35:30c::1 scope=30 target-scope=10
/ipv6 address add address=2001:470:35:30c::2/64 advertise=no disabled=no eui-64=no interface=sit1

The only change I made was changing distance to 15 and target-scope to 30, I assumed this would make ROS send traffic over it only if my ISP's native /64 route was down.
However, I have confirmed that as soon as I add the suggested route, IPv6 stop working for LAN clients.

What am I missing here, how do I configure HE's tunnel as a secondary / failover IPv6 route for the LAN?

Strykar · Wed May 15, 2024 7:59 pm

I tested this some more and the issue appears to only affect failing over to the HE.net tunnel route.

Disabled HE's suggested (2000::/3) route with a default gw (::/0) instead, assigned it a distance of 2.

Instead of trying to route the standard /64 prefix, I took a /64 from the /48 HE assigns and announced it via bridge0.

Here's what I found:
If I disable the pppoe-out1 interface and reboot the router, the HE.net tunnel works
If pppoe-out1 is now brought online, it moves LAN clients off the HE.net tunnel, as expected.

However, if I now disable pppoe-out1, the HE.net tunnel takes 10-15 mins to switch to / come online for LAN clients, all IPv6 traffic fails without any ICMP messages for ping etc.
Happy to hear any tips to troubleshoot this :/

mkx · Thu May 16, 2024 9:01 am

The problem with IPv6 (as compared to IPv4 with NAT) is that IPv6 normally doesn't do NAT. So when you have two public IPv6 prefixes, every client needs to have IPv6 addresses from different prefixes and then it's up to client to select which IPv6 address it wants to use for a particular connection. And connections using "wrong" IPv6 address can't work over the fail-over route.

What you see (with the long time-out) is likely that RAs for the (now) non-working route time-out and clients can't renew it from router (router ceases to advertise it). So after that timer expires, clients are left with only one prefix which hopefully works.

There are mechanisms which may help to get your fail-over work without such a long gap. But I can't test them (I only have one IPv6 prefix and route), perhaps some other forum member will be able to help.

Strykar · Thu May 16, 2024 9:38 am

What you see (with the long time-out) is likely that RAs for the (now) non-working route time-out and clients can't renew it from router (router ceases to advertise it). So after that timer expires, clients are left with only one prefix which hopefully works.

There are mechanisms which may help to get your fail-over work without such a long gap. But I can't test them (I only have one IPv6 prefix and route), perhaps some other forum member will be able to help.

Thank you, that is probably it, but if it were true, how come fallback to my ISP's native IPv6 feels instant?
It is only failing over to the tunnel that is slow.

mkx · Thu May 16, 2024 10:45 am

... if it were true, how come fallback to my ISP's native IPv6 feels instant?

As I wrote I don't have any 1st hand experience. Since this is an issue which involves both router and clients, it would be necessary to do analysis on both ...

CGGXANNX · Thu May 16, 2024 11:31 am

Thank you, that is probably it, but if it were true, how come fallback to my ISP's native IPv6 feels instant?
It is only failing over to the tunnel that is slow.

It's not really "fallback" when talking about your native ISP IPv6, because that route has lower distance (lower than the 2 that you manually set for HE). Which means it's the default route.

* When pppoe-out1 was disabled (and after the reboot like you wrote). Your clients on bridge0 only have the IPv6 prefix from HE, and traffics are routed through the HE route as expected.

* Now when you bring pppoe-out1 up, DHCPv6 client gets the prefix from the ISP for the pool, which is then announced on bridge0 immediately, clients have both prefixes and probably prefer the more recent one (there is also the prefix preference order from RFC 6724 https://datatracker.ietf.org/doc/html/rfc6724 that is applied first, on Windows you can see it by running

netsh interface ipv6 show prefixpolicies

which by default outputs

Precedence  Label  Prefix
----------  -----  --------------------------------
        50      0  ::1/128
        40      1  ::/0
        35      4  ::ffff:0:0/96
        30      2  2002::/16
         5      5  2001::/32
         3     13  fc00::/7
         1     11  fec0::/10
         1     12  3ffe::/16
         1      3  ::/96

but in this case 2001:470::/32 still falls under the same precedence of ::/0, so the facts that the ISP prefix is preferred is not due to this). The clients prefer the ISP prefix, and on the router, the default route is the ISP's route, so it works "instantly".

* If you now disable pppoe-out1, the fact that the ISP prefix has now been removed is not announced immediately on bridge0. Clients still use the ISP's prefix, but on the router, the route with distance=2 is now the active one. Packets are now sent through this route but with source addresses still having the ISP's prefix. Reply-packets will of course not be routed back by HE (not in their prefix range). Ping would not even work like you observed.

* After a while (up to the RA Interval with the default value of 200-600 seconds on the IPv6->ND entry) the router sends advertisement packets with the only prefix left as active, your OS will mark the other prefix as deprecated (probably still visible if you run ipconfig / ifconfig / ip addr depending on the OS, but marked as deprecated). Now connections will use the addresses with the HE prefix as source addresses and are routed correctly.

CGGXANNX · Thu May 16, 2024 11:54 am

A possible workaround might be to have a script on the DHCPv6 client entry, that checks whether the $"pd-valid" is no longer 1, and in that case disable the IPv6 ND entry for bridge0 for 1 second and re-enable it. That might trigger a RA immediately on bridge0 when pppoe-out1 is down. Description for the script property is here (script in my screenshot is unrelated, yours should check for (1!=$"pd-valid")):

https://help.mikrotik.com/docs/display/ ... CPv6Client

dhcpv6-client-script.png

Or a script that performs the same action can also be attached to the on-down event of the PPP profile that pppoe-out1 uses (you can add a separate PPP profile, assign the script to on-down, and specify pppoe-out1 to use this profile)

https://help.mikrotik.com/docs/display/ ... erProfiles

pppoe-profile.png

ppp-script.png

mkx · Thu May 16, 2024 11:58 am

I think that some recent ROS version added support for deprecation RAs ... which are sent out when IPv6 address disappears from LAN interface. It's meant to happen if delegated prefix changes which makes change in DHCPv6 pool and consequently change in LAN interface IPv6 address. May be this solves the problem of deprecating addresses also in this (WAN link failover), but one would have to handle the LAN interface IPv6 addresses ...

CGGXANNX · Thu May 16, 2024 12:15 pm

I just tested, and unfortunately what I wrote above won't work. It appears that when the PPPoE out interface is disabled, the DHCPv6 client entry is shown as red, as expected. But the pool is still there and the addresses assigned to the bridges/vlans are still all active. On the clients, the address is still marked as "Preferred". It will be the case until the prefix expires in DHCPv6 Client

pppoe-off-ipv6-prefix.png

What needs to be done is to disable the DHCPv6 client entry with the script too. Then the addresses will become invalid (red) and deprecation messages are correctly sent, on the clients the affected addresses are marked as deprecated immediately.

Strykar · Thu May 16, 2024 1:39 pm

Does this look okay:

# Script to check pd-valid and trigger RA
:if (1 != $"pd-valid") do={
    /ipv6 nd set [find interface=bridge0] disabled=yes;
    delay 1;
    /ipv6 nd set [find interface=bridge0] disabled=no;
}

and..

# Script for on-down event of PPP profile
/system script add name=ppp-on-down source={
    :local dhcpClientId [/ipv6 dhcp-client find];
    /ipv6 dhcp-client set $dhcpClientId disabled=yes;
    delay 1;
    /ipv6 dhcp-client set $dhcpClientId disabled=no;
}
# Assign the script to the PPP profile
/ppp profile set [find name=default] on-down=ppp-on-down

I'll test using different values for RA interval first.

CGGXANNX · Thu May 16, 2024 1:45 pm

I've never test to see if setting the "on-down" value directly with the name of the script like you wrote would work. Usually, I put something like

/system script run "ppp-on-down"

in the box. See my screenshot where I put the call in the "On Up" event.

Also according to my test above, it's not necessary to disable-then-enable the ND entry, it was sufficient to just disable-then-enable the DHCPv6 client entry.

And instead of changing the on-down event of the "default" PPP profile (that profile might be used by other PPP configurations, including SSTP for instance), you can try to make a copy of that "default' profile, change the on-down event on that profile, then edit pppoe-out1 to use that newly created profile instead of default (again in my screenshots above I've made a pppoe-wan profile and used it in pppoe-out1).

Strykar · Thu May 16, 2024 1:52 pm

Here's what my /64 looks like:

> /ipv6 dhcp-client/print                        
Columns: INTERFACE, STATUS, REQUEST, PREFIX
# INTERFACE   STATUS  REQUEST  PREFIX                             
0 pppoe-out1  bound   prefix   2401:4900:1c02:5769::/64, 23h23m44s

The DHCPv6 client disabling would be on the pppoe-out1 interface:

# Script to check pd-valid and trigger RA
:if (1 != $"pd-valid") do={
    /ipv6 dhcp-client set [find interface=pppoe-out1] disabled=yes;
    delay 1;
    /ipv6 dhcp-client set [find interface=pppoe-out1] disabled=no;
}

This better?

# Create the script for on-down event of PPP profile
/system script add name=ppp-on-down source={
    :local dhcpClientId [/ipv6 dhcp-client find];
    /ipv6 dhcp-client set $dhcpClientId disabled=yes;
    delay 1;
    /ipv6 dhcp-client set $dhcpClientId disabled=no;
}
# Assign the script to the PPP profile using the run command
/ppp profile set [find name=default] on-down="/system script run ppp-on-down"

CGGXANNX · Thu May 16, 2024 3:11 pm

You don't need the script under the DHCPv6 client entry anymore (the script that check $"pd-valid"). Previously the script there was supposed to disable-enable the ND entry, and that, as tested, is not needed. Currently you try to disable-enable the DHCPv6 client entry within the script of the DHCPv6 client entry, that may even cause a loop in case of failure.

You only need to run the script from the /ppp profile on down event. That script block (including ppp-on-down) looks good. Although, as I wrote above, it might be better to create a separate /ppp profile entry, instead of modifying the "default" one.

Strykar · Thu May 16, 2024 5:32 pm

Bingo, this does the job and I now have sub-second (~ 50ms - 100ms) fail over to either IPv6 uplink:

    :local dhcpClientId [/ipv6 dhcp-client find];
    /ipv6 dhcp-client set $dhcpClientId disabled=yes;
    delay 10ms;
    /ipv6 dhcp-client set $dhcpClientId disabled=no;

Thank you very much @CGGXANNX and @mkx

Secondary IPv6 route weirdness with Hurricane Electric tunnel [SOLVED]

Secondary IPv6 route weirdness with Hurricane Electric tunnel

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness [SOLVED]

Re: Secondary IPv6 route weirdness

Re: Secondary IPv6 route weirdness with Hurricane Electric tunnel

Re: Secondary IPv6 route weirdness with Hurricane Electric tunnel

Re: Secondary IPv6 route weirdness with Hurricane Electric tunnel

Re: Secondary IPv6 route weirdness with Hurricane Electric tunnel

Who is online