@ werdarrfr
congratulations!
hmm, in case you interested in - what i thought - more simple way...
let us say - MT with 4 ethers :
- mtcp router, LAN : 192.168.100.253/24, WAN : 172.16.1.2/24 (masqueraded)
- lte router, LAN : 192.168.2.2/24, WAN : 172.16.1.3/24 (masqueraded)
- MT,
to-mtcp : 192.168.100.1/24
to-lte : 192.168.2.1/24
to-lan192 : 192.168.1.1/24
to-lan10 : 10.10.2.1/24
requirements :
- lan192 needs to go out to the internet via lte router.
- lan10 needs to go out to the internet via mtcp router.
- both lan192 and lan10 can contact each other.
source-based routing using pbr/dst-nat
# show interfaces
[admin@MT] > interface/print
Flags: R - RUNNING
Columns: NAME, TYPE, ACTUAL-MTU, L2MTU, MAC-ADDRESS
# NAME TYPE ACTUAL-MTU L2MTU MAC-ADDRESS
0 R ether1-lan192 ether 1500 50:00:00:01:00:00
1 R ether2-lan10 ether 1500 50:00:00:01:00:01
2 R ether3-mtcp ether 1500 50:00:00:01:00:02
3 R ether4-lte ether 1500 50:00:00:01:00:03
# ip addresses
[admin@MT] > /ip address/print
Columns: ADDRESS, NETWORK, INTERFACE
# ADDRESS NETWORK INTERFACE
0 192.168.1.1/24 192.168.1.0 ether1-lan192
1 10.10.2.1/24 10.10.2.0 ether2-lan10
2 192.168.100.1/24 192.168.100.0 ether3-mtcp
3 192.168.2.1/24 192.168.2.0 ether4-lte
# default route via mtcp and lte.
# note those ECMP path on mtcp and lte gateway
[admin@MT] > ip route/print
Flags: D - DYNAMIC; A - ACTIVE; c, s, y - BGP-MPLS-VPN; + - ECMP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
# DST-ADDRESS GATEWAY DISTANCE
0 As+ 0.0.0.0/0 192.168.2.2 1
1 As+ 0.0.0.0/0 192.168.100.253 1
DAc 10.10.2.0/24 ether2-lan10 0
DAc 192.168.1.0/24 ether1-lan192 0
DAc 192.168.2.0/24 ether4-lte 0
DAc 192.168.100.0/24 ether3-mtcp 0
# masquerade on both wan interface (mtcp and lte) - for simplicity.
[admin@MT] > ip firewall/nat/add chain=srcnat out-interface=ether3-mtcp action=masquerade
[admin@MT] > ip firewall/nat/add chain=srcnat out-interface=ether4-lte action=masquerade
# pre-pbr ping and traceroute test
# from lan 192.168.1.0/24 to lan 10.10.2.0/24 and to inet (172.16.1.1/32)
lan192> ping 10.10.2.2
84 bytes from 10.10.2.2 icmp_seq=1 ttl=63 time=62.541 ms
84 bytes from 10.10.2.2 icmp_seq=2 ttl=63 time=6.323 ms
84 bytes from 10.10.2.2 icmp_seq=3 ttl=63 time=1.965 ms
84 bytes from 10.10.2.2 icmp_seq=4 ttl=63 time=2.068 ms
84 bytes from 10.10.2.2 icmp_seq=5 ttl=63 time=3.341 ms
lan192> ping 172.16.1.1
84 bytes from 172.16.1.1 icmp_seq=1 ttl=62 time=10.958 ms
84 bytes from 172.16.1.1 icmp_seq=2 ttl=62 time=7.210 ms
84 bytes from 172.16.1.1 icmp_seq=3 ttl=62 time=10.256 ms
84 bytes from 172.16.1.1 icmp_seq=4 ttl=62 time=5.275 ms
84 bytes from 172.16.1.1 icmp_seq=5 ttl=62 time=4.448 ms
lan192> trace 172.16.1.1
trace to 172.16.1.1, 8 hops max, press Ctrl+C to stop
1 192.168.1.1 2.161 ms 1.249 ms 1.716 ms
2 192.168.100.253 14.839 ms 4.361 ms 3.569 ms
3 *172.16.1.1 4.335 ms (ICMP type:3, code:3, Destination port unreachable)
# pre-pbr ping and traceroute test from lan 10.10.2.0/24 to lan 192.168.1.0/24 and to inet (172.16.1.1/32)
lan10> ping 192.168.2.2
84 bytes from 192.168.2.2 icmp_seq=1 ttl=63 time=3.009 ms
84 bytes from 192.168.2.2 icmp_seq=2 ttl=63 time=5.473 ms
84 bytes from 192.168.2.2 icmp_seq=3 ttl=63 time=5.074 ms
84 bytes from 192.168.2.2 icmp_seq=4 ttl=63 time=5.349 ms
84 bytes from 192.168.2.2 icmp_seq=5 ttl=63 time=4.085 ms
lan10> ping 172.16.1.1
84 bytes from 172.16.1.1 icmp_seq=1 ttl=62 time=6.169 ms
84 bytes from 172.16.1.1 icmp_seq=2 ttl=62 time=10.623 ms
84 bytes from 172.16.1.1 icmp_seq=3 ttl=62 time=4.208 ms
84 bytes from 172.16.1.1 icmp_seq=4 ttl=62 time=7.421 ms
84 bytes from 172.16.1.1 icmp_seq=5 ttl=62 time=9.439 ms
lan10> trace 172.16.1.1
trace to 172.16.1.1, 8 hops max, press Ctrl+C to stop
1 10.10.2.1 2.170 ms 0.985 ms 1.074 ms
2 192.168.2.2 3.946 ms 11.984 ms 2.309 ms
3 *172.16.1.1 5.542 ms (ICMP type:3, code:3, Destination port unreachable)
# pbr classifiers on MT : 10.10.2.0/24 via mtcp, 192.168.1.0/24 via lte
# creating address-list for local LAN :
[admin@MT] > ip firewall/address-list/print
Columns: LIST, ADDRESS, CREATION-TIME
# LIST ADDRESS CREATION-TIME
0 local 192.168.1.0/24 jul/14/2023 16:28:43
1 local 10.10.2.0/24 jul/14/2023 16:28:50
# dst-nat based pbr
# lan10 to-mtcp
# lan192 to-lte
[admin@MT] > ip firewall/nat/print
Flags: X - disabled, I - invalid; D - dynamic
0 chain=srcnat action=masquerade out-interface=ether3-mtcp
1 chain=srcnat action=masquerade out-interface=ether4-lte
2 chain=dstnat action=dst-nat to-addresses=192.168.100.253 dst-address-list=!local in-interface=ether2-lan10
3 chain=dstnat action=dst-nat to-addresses=192.168.2.2 dst-address-list=!local in-interface=ether1-lan192
# post-pbr traceroute test.
# from lan 192.168.1.0/24 to internet
lan192> trace 172.16.1.1
trace to 172.16.1.1, 8 hops max, press Ctrl+C to stop
1 172.16.1.1 1.683 ms 1.401 ms 1.993 ms
2 *172.16.1.1 5.192 ms (ICMP type:3, code:3, Destination port unreachable)
# from lan 192.168.1.0/24 to lan 10.10.2.0/24
lan192> trace 10.10.2.2
trace to 10.10.2.2, 8 hops max, press Ctrl+C to stop
1 192.168.1.1 2.530 ms 1.710 ms 1.277 ms
2 *10.10.2.2 3.322 ms (ICMP type:3, code:3, Destination port unreachable)
# from lan 10.10.2.0/24 to internet
lan10> trace 172.16.1.1
trace to 172.16.1.1, 8 hops max, press Ctrl+C to stop
1 172.16.1.1 1.475 ms 0.960 ms 0.973 ms
2 *172.16.1.1 6.249 ms (ICMP type:3, code:3, Destination port unreachable)
# from lan 10.10.2.0/24 to lan 192.168.1.0/24
lan10> trace 192.168.1.2
trace to 192.168.1.2, 8 hops max, press Ctrl+C to stop
1 10.10.2.1 1.306 ms 1.413 ms 1.755 ms
2 *192.168.1.2 7.687 ms (ICMP type:3, code:3, Destination port unreachable)
# notes :
# - the actual masquerade to the internet handled by those mtcp (192.168.100.253) and lte (192.168.2.2/24) routers.
# - those mtcp and lte uses fixed internal IP address.
hope this helps.