Hi all; I’m desperately trying to get my load balance-fail-over system working here but I just can;t get it to work.
It’s slightly complicated:
Hex router
WAN1: PPOE client Wimax system. Static IP.
WAN2 hooks up to a LHG LTE, receives an IP from the LTE DHCP server.
WAN3 hooks up to a LHG LTE receives an IP from the LTE DHCP server.
Issue: with only WAN running and other WAN’s disabled I have proper internet access. The moment I turn on a second or second+third WAN interface all stops.
Funny thing is that on a local network connected PC: speedtest works fine. Ping action doesn’t work. Firefox browsing works OK. Mail can’t connect to any account.
I have disbled fast tracks. Default routes on all interfaces are on NO.
Overlooking any examples on how-to set this up I believe I did it right … Apparantly not 
Filter or routing issues ..?
Where am I going wrong…
Suggestions wellcome. Script attached.
Cheers.
backup160519_edited.txt (7.26 KB)
The three rules with per-connection-classifier need passthrough=yes. When it’s passthrough=no, processing stops and it doesn’t get to following rules to mark routing.
Great ! That was the tweak needed. Works fine now.
Hi again; I was a bit too enthousiastic. I’ve changed the specific rules mentioned above with passthough = yes. Strangely enough I see traffic flow over all 3 WAN’s so mangling runs however on my workstations on the LAN I am not able to browse anything if more than 1 of the 3 WAN’s is enabled. I am able to ping names and/or IP adresses from the workstations however browsing or connecting mail doesn’t function. Any protocolfilters I am overseeing not setup right ?
Script as above.
Please help.
I don’t see anything else clearly wrong. Fasttrack rule is still disabled, right?
well; it’s as weird as it gets… At present WAN1 and WAN2 both can remain on with full fucntionality on LAN. The minute I switch on WAN3 traffic ceases. WAN1 and WAN2 work on failover but I doubt if they load balance (hard to test with little load). WAN3 is functional btw.
I’ve also tried everything else like switching off all filter rules but issue persists. I’ve slightly modified the rules enclosed again.
Fasttrack removed. Default routes from interfaces NOT added, static routes entered manually.
WAN2 and WAN3 are getting their data from an LHG LTE unit which NAT’s the internet signal. DHCP in those LTE’s is switched off, HEX uses fixed addresses 192.168.77.254 and 192.168.99.254.
is the sequence of the mangle rules correct?
Any new suggestions to play with ? Are you seeing anything which could fail ? I’ve noticed in the “check-gateway, ping” command it makes a difference between adding the nickname of the interface or the real IP address. Is that true ?
may/25/2019 20:38:28 by RouterOS 6.44.3
software id = QJLF-6VJI
model = RouterBOARD 750G r3
serial number = 6F3808ED6B3F
/interface bridge
add admin-mac=CC:2D:E0:39:18:95 auto-mac=no comment=defconf fast-forward=no
name=bridge
/interface ethernet
set [ find default-name=ether1 ] name=WAN1_Eolo speed=100Mbps
set [ find default-name=ether2 ] name=WAN2-Vodafone speed=100Mbps
set [ find default-name=ether3 ] arp=disabled disabled=yes name=WAN3-Wind
speed=100Mbps
set [ find default-name=ether4 ] name=ether4-LAN speed=100Mbps
set [ find default-name=ether5 ] advertise=“10M-half,10M-full,100M-half,100M-f
ull,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full” name=
ether5-LAN speed=100Mbps
/interface pppoe-client
add comment=“Eolo Internet” dial-on-demand=yes disabled=no interface=
WAN1_Eolo keepalive-timeout=4 max-mru=1500 max-mtu=1500 name=WAN1-Eolo
password=xxxxxx user=xxxxxxxx
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] apn=mobile.vodafone.it authentication=pap name=
4G-Vodafone
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=dhcp-pool-LAN ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=dhcp-pool-LAN disabled=no interface=bridge name=dhcp-LAN
/port
set 0 baud-rate=9600 data-bits=8 flow-control=none name=usb1 parity=none
stop-bits=1
/interface bridge port
add bridge=bridge comment=defconf interface=ether5-LAN
add bridge=bridge interface=ether4-LAN
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface detect-internet
set internet-interface-list=WAN wan-interface-list=WAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=“Eolo internet” interface=WAN1-Eolo list=WAN
add comment=“Vodafone internet” interface=WAN2-Vodafone list=WAN
add comment=“Wind internet” interface=WAN3-Wind list=WAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=
192.168.88.0
add address=88.134.90.25/24 interface=WAN1-Eolo network=88.134.90.0
add address=192.168.99.254/24 interface=WAN3-Wind network=192.168.99.0
add address=192.168.77.254/24 interface=WAN2-Vodafone network=192.168.77.0
/ip dhcp-client
add add-default-route=no dhcp-options=hostname,clientid interface=
WAN2-Vodafone use-peer-dns=no
add add-default-route=no dhcp-options=hostname,clientid interface=WAN3-Wind
use-peer-dns=no
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=
8.8.8.8,8.8.4.4,1.1.1.1,1.0.0.1 gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=8.8.8.8 name=Google
add address=8.8.4.4 name=“Google 2”
add address=1.0.0.1 name=Cloudflare
add address=1.1.1.1 name=Cloudflare
/ip firewall filter
add action=accept chain=input comment=
“defconf: accept established,related,untracked” connection-state=
established,related,untracked
add action=drop chain=input comment=“defconf: drop invalid” connection-state=
invalid
add action=accept chain=input comment=“defconf: accept ICMP” protocol=icmp
add action=drop chain=input comment=“defconf: drop all not coming from LAN”
in-interface-list=!LAN
add action=accept chain=forward comment=“defconf: accept in ipsec policy”
ipsec-policy=in,ipsec
add action=accept chain=forward comment=“defconf: accept out ipsec policy”
ipsec-policy=out,ipsec
add action=accept chain=forward comment=
“defconf: accept established,related, untracked” connection-state=
established,related,untracked
add action=drop chain=forward comment=“defconf: drop invalid”
connection-state=invalid
add action=drop chain=forward comment=
“defconf: drop all from WAN not DSTNATed” connection-nat-state=!dstnat
connection-state=new in-interface-list=WAN
add action=accept chain=output dst-port=37 protocol=tcp
/ip firewall mangle
add action=accept chain=prerouting comment=“Load balance” dst-address=
192.168.77.0/24 in-interface=bridge
add action=accept chain=prerouting comment=“Load balance” dst-address=
192.168.99.0/24 in-interface=bridge
add action=accept chain=prerouting comment=“Load balance” dst-address=
85.174.0.0/24 in-interface=bridge
add action=mark-connection chain=prerouting connection-mark=no-mark
in-interface=WAN1-Eolo new-connection-mark=WAN1_Conn passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark
in-interface=WAN2-Vodafone new-connection-mark=WAN2_conn passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark
in-interface=WAN3-Wind new-connection-mark=WAN3_conn passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark
dst-address-type=!local in-interface=bridge new-connection-mark=WAN1_Conn
passthrough=yes per-connection-classifier=both-addresses:3/0
add action=mark-connection chain=prerouting connection-mark=no-mark
dst-address-type=!local in-interface=bridge new-connection-mark=WAN2_conn
passthrough=yes per-connection-classifier=both-addresses:3/1
add action=mark-connection chain=prerouting connection-mark=no-mark
dst-address-type=!local in-interface=bridge new-connection-mark=WAN3_conn
passthrough=yes per-connection-classifier=both-addresses:3/2
add action=mark-routing chain=prerouting connection-mark=WAN1_Conn
in-interface=bridge new-routing-mark=to_WAN1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=WAN2_conn
in-interface=bridge new-routing-mark=to_WAN2 passthrough=no
add action=mark-routing chain=prerouting connection-mark=WAN3_conn
in-interface=bridge new-routing-mark=to_WAN3 passthrough=no
add action=mark-routing chain=output connection-mark=WAN1_Conn
new-routing-mark=to_WAN1 passthrough=no
add action=mark-routing chain=output connection-mark=WAN2_conn
new-routing-mark=to_WAN2 passthrough=no
add action=mark-routing chain=output connection-mark=WAN3_conn
new-routing-mark=to_WAN3 passthrough=no
/ip firewall nat
add action=dst-nat chain=dstnat dst-address=78.134.90.31 dst-port=80
protocol=tcp to-addresses=192.168.88.200 to-ports=80
add action=masquerade chain=srcnat out-interface=WAN1-Eolo
add action=masquerade chain=srcnat out-interface=WAN2-Vodafone
add action=masquerade chain=srcnat out-interface=WAN3-Wind
/ip route
add check-gateway=ping distance=1 gateway=WAN1-Eolo routing-mark=to_WAN1
add check-gateway=ping distance=1 gateway=WAN1-Eolo routing-mark=to_WAN2
add check-gateway=ping distance=1 gateway=WAN3-Wind routing-mark=to_WAN3
add check-gateway=ping distance=1 gateway=WAN1-Eolo
add check-gateway=ping distance=2 gateway=WAN2-Vodafone
add check-gateway=ping distance=3 gateway=WAN3-Wind
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/port firmware
set directory=flash
/system clock
set time-zone-name=Europe/Rome
/system identity
set name=“MikroTik hex”
/system resource irq rps
set WAN1_Eolo disabled=no
set WAN2-Vodafone disabled=no
set WAN3-Wind disabled=no
set ether4-LAN disabled=no
set ether5-LAN disabled=no
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
Cheers.
maybe confusing the remark : WAN3 is functional With that I meant that the LHG LTE attached to WAN3 provides good working signal and data.
Anyone having a clue ?
- arp=disabled on WAN3-Wind is not good idea
- both routing tables to_WAN1 and to_WAN2 have gateway=WAN1-Eolo, so no, WAN1 and WAN2 don’t really work together, only WAN1 is used
- gateway= generally doesn’t work with ethernet interfaces, gateway= is the right choice
- I see there’s PPPoE client on WAN1_Eolo that I missed before, so that’s your only working WAN right now
Tks Sob; accidentally I managed to cut myself off from the router by disabling my login interface 
Other WAN’s are dynamic so were not setup to get into the router). First thing now is when I physically get to it is install dyndns !
I noticed the mistake in the routing table thats resolved. ARP is enable too on WAN3. sadly can’t test any further right now.
Open question I’ve seen many examples of PCC routing now but it is yet unclear when to use any input or output policy rules. Could you clarify which are really necessary and which are optional. (mangling is clear to me).
You shouldn’t need to do any marking in input, because both input and forward are covered by common rules in prerouting. Marking routing in output is important, if you want router reachable from internet using any WAN, because you need to send replies where the requests came from.