I’m updated my configuration in nearly days, post it here. if anyone find my words is not English, please help my translate, thanks. 
My example is 2 of ISP line access to internet with two pairs IP addresses, a debian server dircetly link to lan bridge port, router runs several services, like a OVPN TUN server, a PPTP server, and the debian server runs another OVPN service in TAP mode use TCP, and iperf3 service.
I use address-list, not routing rule, because I need clients traffic who have a source IP in ISP-1 access services, but destination address is ISP-2 port, gets reply from ISP-2 port. Both WAN ports accept clients from internet, and masquerade traffic from LAN.
- Set ethernet interface name, I think it’s more clearly than comment:
/interface ethernet
set [ find default-name=ether1 ] name=ether1-ISP-1
set [ find default-name=ether2 ] name=ether2-ISP-2
In my example, set 3 ports to bridge, as LAN:
/interface bridge port
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
and I’m set up a list of interface for NAT.
/interface list
add name=WAN-ports
/interface list member
add interface=ether1-ISP-1 list=WAN-ports
add interface=ether2-ISP-2 list=WAN-ports
- set IP address for each interfaces.
/ip address
add address=192.168.88.1/24 comment=LAN interface=bridge network=192.168.88.0
add address=124.127.1.2/30 interface=ether1-ISP-1 network=124.127.1.0
add address=114.240.2.2/30 interface=ether2-ISP-2 network=114.240.2.0
- set address-list, in my example, I’m set up IPv4, this list actually in my router is very long, about hundreds CIDR, I pick 3 of them shows here.
/ip firewall address-list
add address=1.2.4.0/24 list=list_ISP-2
add address=8.129.0.0/16 list=list_ISP-2
add address=14.204.0.0/15 list=list_ISP-2
also you can set a IPv6 address-list, but mangle should set in /ipv6/firewall.
/ipv6 firewall address-list
add address=2408:8000::/20 list=list_ISP-2
- set NAT, skip masquerade part if you are setting up in IPv6.
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1-ISP-1
add action=masquerade chain=srcnat out-interface=ether2-ISP-2
add action=dst-nat chain=dstnat comment="OpenVPN TAP server" dst-port=11194 in-interface-list=\
WAN-ports protocol=tcp to-addresses=192.168.88.10 to-ports=1194
add action=dst-nat chain=dstnat comment="iperf3 server" dst-port=15201 in-interface-list=WAN-ports \
protocol=tcp to-addresses=192.168.88.10 to-ports=5201
5.set mangle of firewall, the ether1-ISP-1 use main routing table, it’s no need set accept and mark connection, and my suggestion is only “Accept” the traffic forward from/to LAN, don’t set up for input/output traffic. reason is in my reply early.
/ip firewall mangle
add action=accept chain=prerouting comment="Accept dst-nat traffic from ISP-2" \
connection-nat-state=dstnat in-interface=ether2-ISP-2
add action=accept chain=prerouting comment="Accept masquerade reply traffic from ISP-2" \
connection-nat-state=srcnat in-interface=ether2-ISP-2
and mark connection when traffic arrive, here I changed configure, different from early post. Input/Output and Forward traffic needs two mark like before. but if you’re set up a dst-NAT, mapping port for a service outside your local LAN, PREROUTING seems not work. “connection-nat-state=!dstnat” not work well in FORWARD chain, my idea is use dst-port make router match traffic, this value should be original port in your local LAN, NOT mapping outside you config nat in WAN ports.
/ip firewall mangle
add action=mark-connection chain=input comment="input ISP-2 port traffic mark connection" \
connection-mark=no-mark connection-nat-state=!srcnat,dstnat connection-state=new \
dst-address-type=local in-interface=ether2-ISP-2 new-connection-mark=\
ISP-2-in-out-connection passthrough=yes
add action=mark-connection chain=forward comment="from ISP-2 port dst-nat traffic mark connection" \
connection-mark=no-mark dst-port=1194,5201 in-interface=ether2-ISP-2 \
new-connection-mark=ISP-2-PBR-connection passthrough=yes protocol=tcp
and base on destination ip address-list, or PCC, what ever you want, mark connection forward from LAN to ISP-2. dst-nat in traffic and forward from lan traffic can use same connection mark name.
/ip firewall mangle
add action=mark-connection chain=prerouting comment=\
"from local subnet base on dst-ip address-list ISP-2-PBR-connection mark" connection-mark=\
no-mark connection-nat-state=!dstnat dst-address-list=List_ISP-2 dst-address-type=\
!local in-interface=brideg new-connection-mark=ISP-2-PBR-connection passthrough=yes
if you want use PCC, only set up PCC=2/1 is alright, 2/0 should go to main connection mark, and main routing table. don’t use “src-address and port” in the situation which is two of your ISP line are rent from different provider, it’s will cause safety failure for bank web site, and some SSL problem you are not expecting.
/ip firewall mangle
add action=mark-connection chain=prerouting comment=\
"from local subnet base on PCC ISP-2-PBR-connection mark" connection-mark=\
no-mark connection-nat-state=!dstnat dst-address-type=!local in-interface=bridge \
new-connection-mark=ISP-2-PBR-connection passthrough=yes \
per-connection-classifier=src-address:2/1
if you want base on source vlan or source IP address, just change condition make router find your traffic.
/ip firewall mangle
add action=mark-connection chain=prerouting comment=\
"from local subnet base on src-ip traffic ISP-2-PBR-connection mark" connection-mark=\
no-mark connection-nat-state=!dstnat in-interface=vlan2 dst-address-type=\
!local in-interface=brideg new-connection-mark=ISP-2-PBR-connection passthrough=yes
and you can set up router itself output traffic base destination ip address-list use ISP-2 port, if you like.
add action=mark-connection chain=output comment=\
"router itself base on dst-ip address-list ISP-2-PBR-connection mark" connection-nat-state=\
!dstnat connection-state=new dst-address-list=List_ISP-2 new-connection-mark=\
ISP-2-in-out-connection passthrough=yes src-address-type=local
mark routing.
add action=mark-routing chain=output comment="ISP-2 In/Out mark routing" connection-mark=\
ISP-2-in-connection new-routing-mark=ISP-2-PBR passthrough=yes
add action=mark-routing chain=prerouting comment="ISP-2-forwarding mark routing" \
connection-mark=ISP-2-PBR-connection new-routing-mark=ISP-2-PBR passthrough=yes
- Set up routing table and routing rule, about ISP-1, it’s use main table, should in configure already.
/routing table
add disabled=no fib name=ISP-2-PBR
/routing rule
add action=lookup-only-in-table disabled=no routing-mark=ISP-2-PBR table=ISP-2-PBR
- Set up static route, don’t forget main route table need access to internet.
/ip route
add check-gateway=arp disabled=no distance=1 dst-address=0.0.0.0/0 gateway=124.127.1.1 \
pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=arp disabled=no distance=2 dst-address=0.0.0.0/0 gateway=114.240.2.1 \
pref-src="" routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=arp disabled=no distance=1 dst-address=0.0.0.0/0 gateway=114.240.2.1 \
pref-src="" routing-table=ISP-2-PBR scope=30 suppress-hw-offload=no target-scope=10
add check-gateway=arp disabled=no distance=2 dst-address=0.0.0.0/0 gateway=124.127.1.1 \
pref-src="" routing-table=ISP-2-PBR scope=30 suppress-hw-offload=no target-scope=10
I hope everything is fine this time. 