Long time lurker, first time posting. Please let me know if there are any ways I can improve the post or information provided.
I am having a bit of trouble with multiple routing tables and could definitely use a more experienced eye.
Specifically, I am running ROS 7.1 on a CCR2004-1G-12S+2XS and trying to use Wireguard to connect to an external VPN provider. To this end, I have used the built-in scripting language to automate registration of my local public key with the provider using their REST APIs. This is working and I am able to receive the appropriate connection details (local IP, remote gateway, etc.) from their servers without issue. I am then able to take these details and set up the wireguard interface more or less without issue.
The problem arises when I try to route traffic to this interface. I am using a separate routing table and routing marks to send traffic to this interface. The central idea behind this is that the separate table allows me to blindly nuke and recreate the routes as appropriate during periodic re-registration without having to maintain persistent state information across multiple invocations.
Unfortunately, it doesn’t seem that my setup of this second table is correct as I get various Destination Host Unreachable errors when trying to access the wider internet. I am able to ping the remote gateway from the router without issue:
[admin@MikroTik] > /ping 10.25.128.1
SEQ HOST SIZE TTL TIME STATUS
0 10.25.128.1 56 64 11ms244us
1 10.25.128.1 56 64 12ms837us
2 10.25.128.1 56 64 11ms646us
3 10.25.128.1 56 64 12ms832us
4 10.25.128.1 56 64 11ms934us
sent=5 received=5 packet-loss=0% min-rtt=11ms244us avg-rtt=12ms98us max-rtt=12ms837us
[admin@MikroTik] > /ping interface=wireguard-pia 10.25.128.1
SEQ HOST SIZE TTL TIME STATUS
0 10.25.128.1 56 64 12ms984us
1 10.25.128.1 56 64 12ms934us
2 10.25.128.1 56 64 13ms573us
3 10.25.128.1 56 64 12ms759us
sent=4 received=4 packet-loss=0% min-rtt=12ms759us avg-rtt=13ms62us max-rtt=13ms573us
I am also able to ping the remote gateway from a downstream Linux host:
$ ping 10.25.128.1
PING 10.25.128.1 (10.25.128.1) 56(84) bytes of data.
64 bytes from 10.25.128.1: icmp_seq=2 ttl=63 time=12.0 ms
64 bytes from 10.25.128.1: icmp_seq=3 ttl=63 time=12.6 ms
^C
--- 10.25.128.1 ping statistics ---
3 packets transmitted, 2 received, 33.3333% packet loss, time 2055ms
rtt min/avg/max/mdev = 11.976/12.306/12.636/0.330 ms
Things break down when I try to reach beyond the first remote hop on either the router or any other host:
[admin@MikroTik] > /ping interface=wireguard-pia 1.1.1.1
SEQ HOST SIZE TTL TIME STATUS
0 126 (No error information)
0 10.25.184.192 84 64 113us host unreachable
1 126 (No error information)
1 10.25.184.192 84 64 94us host unreachable
2 126 (No error information)
2 10.25.184.192 84 64 89us host unreachable
3 126 (No error information)
3 10.25.184.192 84 64 84us host unreachable
sent=4 received=0 packet-loss=100%
###############################################
$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
From 192.168.30.1 icmp_seq=1 Destination Host Unreachable
From 192.168.30.1 icmp_seq=2 Destination Host Unreachable
From 192.168.30.1 icmp_seq=3 Destination Host Unreachable
From 192.168.30.1 icmp_seq=4 Destination Host Unreachable
^C
--- 1.1.1.1 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3102ms
And, finally, the (current) routing table and firewall rules:
[admin@MikroTik] > /ip/route/print
Flags: D - DYNAMIC; I, A - ACTIVE; c, s, d, y - COPY; H - HW-OFFLOADED
Columns: DST-ADDRESS, GATEWAY, DISTANCE
# DST-ADDRESS GATEWAY DISTANCE
DAd 0.0.0.0/0 xx.xx.xx.1 1
DAc 10.25.128.0/17 wireguard-pia 0
DAc xx.xx.xx.0/19 sfp28-1 0
DIcH 192.168.10.0/24 sfp-sfpplus1 0
DAc 192.168.20.0/24 sfp-sfpplus2 0
DAc 192.168.30.0/24 sfp-sfpplus3 0
0 As 0.0.0.0/0 10.25.128.1 1
1 As 10.25.128.1/32 wireguard-pia 1
[admin@MikroTik] > /ip/firewall/mangle/print
Flags: X - disabled, I - invalid; D - dynamic
0 D ;;; special dummy rule to show fasttrack counters
chain=prerouting action=passthrough
1 D ;;; special dummy rule to show fasttrack counters
chain=forward action=passthrough
2 D ;;; special dummy rule to show fasttrack counters
chain=postrouting action=passthrough
3 chain=prerouting action=mark-connection new-connection-mark=pia_wireguard_conn src-address=192.168.0.0/16 dst-address=!192.168.0.0/16 connection-mark=no-mark
4 chain=prerouting action=mark-routing new-routing-mark=routes-pia src-address=192.168.0.0/16 connection-mark=pia_wireguard_conn
I am (almost) certain that I have the second routing table setup incorrectly but am not sure what else to change (I usually work at layer 1 & 2). I can post my VPN registration script but, at this point, it seems to be working and sets whatever I tell it to; presumably the settings themselves are incorrect.
Apologies for the wall of text and thanks in advance for any assistance.