Problems with RouterOS 5.7

RouterOS General
1

Dear Mikrotik support,

As you may remember, our RouterOS project was cancelled, but since we got a promise that key regeneration will be fixed, I managed to reopen it again. So here I am again :wink:

I created a simple test scenario, where clients connects to the corporate network and internet via ipsec:

internet-----(linux with dhcp and openswan)==+==(mikrotik node#1)--+---clients
corporate------/                    \        \==(mikrotik node#2)--/
                                     \
                                      \===... (more mikrotik HA pairs, currently not configured for test)

(where ‘=’ represents ipsec tunnel, only active on vrrp master side).

This should work like:

  1. client generates a dhcp request
  2. the active mikrotik node receives it on it’s back side, and relays to the dhcp-server on the front side in the tunnel
  3. the dhcp server replies and client gets address
  4. the client will reach everything (0.0.0.0/0) through the ipsec tunnel.
  5. if a mikrotik node fails, the tunnel and the back side sip address (client’s defgw) should move.

Without vrrp it works as expected.
Without ipsec the vrrp works as expected.

But together they make weird things:

  1. first of all, the back side vrrp’s adverts (the side that’s NOT terminating the ipsec) caught by ipsec policy, but not send through the tunnel, seems to disappear, and in lack of advertisements both nodes back sides became running master. Fail.
  2. on the backup node, if vrrp interfaces are down, the dhcp-relay eats up cpu and generates 100% load. After HA takeover, when vrrp interface became available, it remains the same, and on the new backup node (which lost the vrrp interface) cpu load goes up to 100%, so both nodes slows down dramatically. The node that won mastership after turning them on utilize only 0-1% cpu before takeover takes place, as expected. If load would drop on new master to this level, load won’t be an issue.
  3. the dhcp relay does not use the specified local address, use interface address under any circumstances. I could workaround this by specifing a host route with pref-src set up. (Source nat by routing, heh?)
  4. vrrp interfaces of the same resource group does not switch together. I’m not sure they should, but I could workaround the problem by increasing the opposite vrrp’s priority in “on master” script, and decreasing in “on backup” script.
  5. there’s a bug in packet sniffer, it does not show incoming packets if ipsec policy matches them, but racoon can’t send it through. By mirroring client’s port on the switch you can see that packages reach mikrotik box. (for eaxmple if a client pings it’s def gw)

Some more information: if I divide the policy into 3 (same source, different destination, one for class A, one for class B, one for class C addresses) the vrrp adverts goes out from the box on the appropriate interface.

Topology to reproduce the issue:
linux:
eth0 - internet
eth1 - 10.1.0.254/24 (corporate network server lan)
eth2 - 192.168.0.254/24

mikrotik node#1:
ether1 - 192.168.1.1/24
ether2 - 192.168.0.1/24
vrrp1 - 10.0.0.254/24 (sip address on ether1, corporate network client lan)
vrrp2 - 192.168.0.3 (sip on ether2)

mikrotik node#2:
ether1 - 192.168.1.2/24
ether2 - 192.168.0.2/24
vrrp1 - 10.0.0.254/24 (sip on ether1)
vrrp2 - 192.168.0.3 (sip on ether2)

dhcp-relay on both:
interface=vrrp1 dhcp-server=10.1.0.254 local-address=10.0.0.254

special static routes on both:
dst-address=0.0.0.0/0 gateway=vrrp2
dst-address=10.1.0.254 pref-src=10.0.0.254 gateway=vrrp2

ipsec policy that does not work:
src-address=10.0.0.0/24 src-port=any dst-address=0.0.0.0/0 dst-port=any protocol=all action=encrypt level=unique ipsec protocols=esp tunnel=yes sa-src-address=192.168.0.3 sa-dst-address=192.168.0.254 proposal=default priority=100

ipsec policies that work with vrrp (does not include multicast addresses):
src-address=10.0.0.0/24 src-port=any dst-address=0.0.0.0/1 dst-port=any protocol=all action=encrypt level=unique ipsec protocols=esp tunnel=yes sa-src-address=192.168.0.3 sa-dst-address=192.168.0.254 proposal=default priority=100
src-address=10.0.0.0/24 src-port=any dst-address=128.0.0.0/2 dst-port=any protocol=all action=encrypt level=unique ipsec protocols=esp tunnel=yes sa-src-address=192.168.0.3 sa-dst-address=192.168.0.254 proposal=default priority=100
src-address=10.0.0.0/24 src-port=any dst-address=192.0.0.0/3 dst-port=any protocol=all action=encrypt level=unique ipsec protocols=esp tunnel=yes sa-src-address=192.168.0.3 sa-dst-address=192.168.0.254 proposal=default priority=100

As you can see, ether1’s address differs from policy’s source (192.168.1.0/24 vs. 10.0.0.0/24), but it matches anyway, if destination is 0.0.0.0/0. Although there’s a workaround, it’s more than uncomfortable.

Conclusion:

  1. default route in ipsec + vrrp does not work, workaround exists
  2. default route in ipsec, client pings sip address does not work, no workaround
  3. vrrp in the same group does not move together, workaround exists
  4. dhcp-relay does not use local-address, workaround exists
  5. dhcp-relay utilize 100% cpu on backup and on new master, no workaround
  6. packet sniffer does not show some packets (tourch does not show them either), no workaround

Please let me know if you need more information to reproduce the issue. Cheers