Thanks for responding. Ok, let me try:
Interfaces: br-wan (=main uplink), gre-vultr (=second uplink, via GRE tunnel), br-lan (local RF1912 network).
br-wan's address is 233.252.102.170 and has it's default gateway 233.252.102.169 (via NAT).
/ip address
add address=192.168.200.254/24 interface=br-lan network=192.168.200.0
add address=233.252.102.170/29 interface=br-wan network=233.252.102.168
add address=203.0.113.193/28 interface=br-wan network=203.0.113.192
add address=203.0.113.255/31 interface=gre-vultr network=203.0.113.254
/ip firewall address-list
add address=203.0.113.0/24 list=own-public-prefixes
add address=233.252.102.170 list=own-public-prefixes
add address=233.252.102.171 list=own-public-prefixes
add address=233.252.102.172 list=own-public-prefixes
add address=233.252.102.173 list=own-public-prefixes
add address=233.252.102.174 list=own-public-prefixes
add address=233.252.102.175 list=own-public-prefixes
/ip firewall nat
add action=masquerade chain=srcnat dst-address-list=!own-public-prefixes out-interface=br-wan
/ip route
add distance=1 gateway=203.0.113.254%gre-vultr pref-src=203.0.113.255 routing-mark=default_net44
add distance=1 gateway=233.252.102.169%br-wan routing-mark=default_lan
/ip route rule
add routing-mark=incoming_net44 table=main
add routing-mark=incoming_net44 table=default_net44
add dst-address=0.0.0.0/0 src-address=203.0.113.0/24 table=main
add dst-address=0.0.0.0/0 src-address=203.0.113.0/24 table=default_net44
add dst-address=0.0.0.0/0 table=main
add dst-address=0.0.0.0/0 table=default_lan
/ip firewall mangle
add action=mark-connection chain=input in-interface=gre-vultr new-connection-mark=incoming-net44 passthrough=yes
add action=mark-connection chain=input new-connection-mark=no-mark passthrough=yes protocol=gre routing-mark=incoming_net44
add action=mark-connection chain=prerouting in-interface=gre-vultr new-connection-mark=incoming-net44 passthrough=yes
add action=mark-routing chain=output connection-mark=incoming-net44 new-routing-mark=incoming_net44 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=incoming-net44 new-routing-mark=incoming_net44 passthrough=yes
From this config we can see that:
- The default gateway is on br-wan (with NAT)
- All packets coming from gre-vultr and being forwarded are tagged with incoming-net44
- All local incoming packets from gre-vultr (which are not GRE packets themselves) are tagged with incoming-net44
- All packets leaving the router (either OUTPUT or PREROUTING) and a connection is tagged with incoming-net44, it is assigned routing tag incoming_net44
- From the routing rules, all packets with routing tag incoming_net44 are first sent to table main (does not include the default gateway) and then to table default_net44
- Table default_net44 has the default gateway set for br-vultr and pref-src=203.0.113.255
Now I perform a traceroute to 203.0.113.194 (assigned to a station connected to br-wan to which the Mikrotik forwards).
Ping to 203.0.113.194 works and traceroute too. However, the 2nd last station (the Mikrotik router) shows "* * *" indicating that ICMP packets from the Mikrotik itself are not received.
This is the packet sniffer while the traceroute is going on:
[admin@ugate] > /tool sniffer quick ip-address=192.0.2.86 ip-protocol=icmp
INTERFACE TIME NUM DI SRC-MAC DST-MAC VLAN SRC-ADDRESS
gre-vultr 39.407 1 <- 192.0.2.86
gre-vultr 39.407 2 -> 233.252.102.170
gre-vultr 44.377 3 <- 192.0.2.86
gre-vultr 44.377 4 -> 233.252.102.170
gre-vultr 49.382 5 <- 192.0.2.86
gre-vultr 49.383 6 -> 233.252.102.170
gre-vultr 54.388 7 <- 192.0.2.86
It can be clearly seen that the package from 192.0.2.86 is received via gr-vultr.
Then it can be clearly seen that the Mikrotik correctly routes the response (ICMP TTL Exceeded) correctly to gre-vultr.
However, it can be seen that the source address is 233.252.102.170 and not 203.0.113.255 which I specified via pref-src!
As a result, this packet is filtered upstream (because only packets from 203.0.113.0/24 can pass gre-vultr transit) and never arrives at 192.0.2.86!
So what could go wrong here?
EDIT: For testing, I have tried setting pref-source in the default route of table "default_lan":
/ip route add distance=1 gateway=233.252.102.169%br-wan routing-mark=default_lan pref-src=203.0.113.255
This works but this route is definitely not used at all for the traceroute example.
This must be a BUG!
Would you agree on this?
