Bandwidth Load Balancing - LTE modem slooow Ping and slooow connections

[wanton]

Hello,

I have a strange situation and need any help available.

My location is very unfortunate with regards to ISP providers I have to stick with a ADSL provider which is not optimal because it gets cut off fairly often and my ADSL modem goes crazy.
We analyzed the problem and what we have is a bandwidth problem - too many computers for too little bandwidth.

Unfortuneately there are no fibre-optic internet providers available in the area so I am stuck with ADSL.
To make matters worse the LTE providers aren’t very good either and do not provide a static IP address for me which I need.

So the idea was to use 2 ISP providers.
ADSL to have the static IP address and main connection route.
LTE as a fail over and bandwidth backup - whenever the ADSL fails the LTE “helps out” or “takes over”.

For my setup please check the image below.

ADSL Modem is simple TP-Link ADSL Modem
LTE Modem is simple Huawei LTE modem (working as only LAN LTE provider)
Router is Mikrotik RB3011UiASm

Both ISP are connected through ethernet ports to the router. The router to a switch, then to all the computers in the company.

My configuration is “Bandwidth based load balancing” as shown by mr Tomas Kirnak : https://mum.mikrotik.com/presentations/US12/tomas.pdf

In summary:
The idea is that whenever my ADSL ISP is overflowing its bandwidth or is cut off, the RouterOs traffic monitor implements a script that changes a routing mark to the LTE ISP.
New connections get marked so as to be routed through the LTE modem.
Mangle rules keep the connections rooted to their respective ISP until they’re closed.
Once the traffic on the ADSL ISP provider is diminished the script change the routing mark back to the ADSL ISP.

I like this configuration and it works well as in it does what it is supposed to do.

But ..

The problem that arises is that the LTE modem often times seems to fight with the router.
This shows up as a veeeeery slow connection.
I connect to the LTE modem through the LAN where it shows up as connected to the internet et al, but when I ping the ISP ip address to check my dl/up speed, I get timeouts and/or 2500ms pings with veeeeeery slow dl speeds.

If immediately afterwards I connect directly to the LTE modem using a twisted pair and ping the same ISP ip address and/or check my speed I get 25ms ping and an acceptable speed (for my location).

The only recourse is to switch off the LTE modem and force all traffic through the ADSL modem, which as I mentioned before is not very reliable.
Its a shame because when I connect directly to the LTE modem everything seems fine.

It seems as though my Router influences how the LTE connects to the internet and slows access.

Can you help me ?

[sebastia]

I would try to focus on ping (icmp) initially due to low complexity of the protocol: there isn’t much that’s “moving”.

Pinging from network is slow, pinging directly attached is fast. So what is different between these cases?
TTL (time to live) for example will be lower from network (more hops). Try pinging with higher TTL value from network to see if that resolves it.

Basically “resolution by elimination”, one after another.

[wanton]

Thanks Sebastia for the hint.

I’m sorry to say I am not a guru and although Your answer is very welcome, to me it is a bit cryptic
I’m (re)searching where I can change the TTL to check my pings. I will let You know what I find.

When I try to change TTL pinging to my ISP it tells me:
ping: -I, -L, -T flags cannot be used with unicast destination

Can You elaborate what is meant by

Try pinging with higher TTL value from network to see if that resolves it.

In the mean time this is the export from my Mangle rules :

add action=accept chain=prerouting dst-address-list=Connected src-address-list=Connected
add action=accept chain=prerouting dst-address=192.168.1.0/24
add action=accept chain=prerouting dst-address=192.168.2.0/24

add action=mark-connection chain=input comment=“Bandwidth-based-Load Balancing – WAN - RouterOs - Per Tomas Kirnak” connection-mark=no-mark in-interface=ether1-adsl
new-connection-mark=ADSL passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark in-interface=ether3-lte new-connection-mark=LTE passthrough=yes
add action=mark-routing chain=output connection-mark=ADSL new-routing-mark=ADSL-Route passthrough=yes
add action=mark-routing chain=output connection-mark=LTE new-routing-mark=LTE-Route passthrough=yes

add action=mark-connection chain=forward comment=“WAN → LAN” connection-mark=no-mark in-interface=ether1-adsl new-connection-mark=ADSL->LAN passthrough=yes
add action=mark-connection chain=forward connection-mark=no-mark in-interface=ether3-lte new-connection-mark=LTE->LAN passthrough=yes
add action=mark-routing chain=prerouting connection-mark=ADSL->LAN new-routing-mark=ADSL-Route passthrough=yes src-address-list=LAN_VPN
add action=mark-routing chain=prerouting connection-mark=LTE->LAN new-routing-mark=LTE-Route passthrough=yes src-address-list=LAN_VPN

add action=mark-connection chain=prerouting comment=“LAN → WAN” connection-mark=no-mark dst-address-list=!Connected dst-address-type=!local new-connection-mark=
LAN->WAN passthrough=yes per-connection-classifier=both-addresses:2/0 src-address-list=LAN_VPN

add action=mark-routing chain=prerouting comment=Bandwidth-Load-Balancing connection-mark=LAN->WAN new-routing-mark=ADSL-Route passthrough=yes src-address-list=
LAN_VPN

add action=mark-connection chain=prerouting comment=“Sticky Connections” connection-mark=LAN->WAN new-connection-mark=Sticky-ADSL passthrough=yes routing-mark=
ether1-mark
add action=mark-connection chain=prerouting connection-mark=LAN->WAN new-connection-mark=Sticky-LTE passthrough=yes routing-mark=ether3-mark
add action=mark-routing chain=prerouting connection-mark=Sticky-ADSL new-routing-mark=ADSL-Route passthrough=yes src-address-list=LAN_VPN
add action=mark-routing chain=prerouting connection-mark=Sticky-LTE new-routing-mark=LTE-Route passthrough=yes src-address-list=LAN_VPN

add action=mark-routing chain=prerouting new-routing-mark=LTE-Route passthrough=no src-address=192.168.0.161

The last routing marking action was added by me to allow me to work without having the connection issues interfere with the other computers in the company.
I added it so that the routing marking would apply only to my computer.

[wanton]

I also logged the last mangle rule for my connection and this came out

Any ideas ?

Regarditos and big thx for any help.

[sebastia]

With regards to TTL

on windows

ping -i 65 <address>

on linux

ping -t 65 <address>

Try pinging with higher TTL value from network to see if that resolves it.

you said pinging from behind Tik is slow. Tik is one more hop. iI’ve heard of some isp’s which limit traffic not matching specific criteria.
So, try ping with TTL +1 or higher than default, which is usually 64. (just one of the possible causes)

[wanton]

Hello again and thx for Your support..

In the mean time I tried a few things one of them was to traceroute 8.8.8.8 - and it seems I’m onto something.

This is using the traceroute tool when I tell the router to use the correct interface and the correct routing table:

This is using the traceroute tool when I tell the router to use the correct interface but do not tell it which routing table:

Can it be that my router doesn’t “know” which routing table to use with which ISP interface ?

Thx a million

[sebastia]

And how does your routing look like?

/ip route print detail

[wanton]

Here You go

Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit 
 0 A S  dst-address=0.0.0.0/0 gateway=192.168.2.1 gateway-status=192.168.2.1 reachable via  ether3-lte check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=LTE-Route 

 1 A S  dst-address=0.0.0.0/0 gateway=192.168.1.1 gateway-status=192.168.1.1 reachable via  ether1-adsl check-gateway=ping distance=1 scope=30 target-scope=10 
        routing-mark=ADSL-Route 

 2 X S  dst-address=192.168.0.197/32 gateway=192.168.1.1 gateway-status=192.168.1.1 inactive distance=1 scope=30 target-scope=10 routing-mark=ADSL-Route 

 3 A S  dst-address=0.0.0.0/0 pref-src=192.168.1.2 gateway=192.168.1.1 gateway-status=192.168.1.1 reachable via  ether1-adsl check-gateway=ping distance=1 scope=30 
        target-scope=10 

 4   S  dst-address=0.0.0.0/0 gateway=192.168.2.1 gateway-status=192.168.2.1 reachable via  ether3-lte distance=2 scope=30 target-scope=10 

 5 X S  dst-address=0.0.0.0/0 gateway=192.168.2.1 gateway-status=192.168.2.1 inactive check-gateway=ping distance=10 scope=30 target-scope=10 

 6 X S  dst-address=8.8.4.4/32 gateway=192.168.2.1 gateway-status=192.168.2.1 inactive distance=1 scope=30 target-scope=10 

 7 X S  dst-address=8.8.8.8/32 gateway=192.168.1.1 gateway-status=192.168.1.1 inactive distance=1 scope=30 target-scope=10 

 8 ADC  dst-address=192.168.0.0/24 pref-src=192.168.0.1 gateway=bridge-lan gateway-status=bridge-lan reachable distance=0 scope=10 

 9 ADC  dst-address=192.168.1.0/24 pref-src=192.168.1.2 gateway=ether1-adsl gateway-status=ether1-adsl reachable distance=0 scope=10 

10 ADC  dst-address=192.168.2.0/24 pref-src=192.168.2.2 gateway=ether3-lte gateway-status=ether3-lte reachable distance=0 scope=10

thx a million

[sebastia]

Your routing table is fine.
Wrt traceroute I could replicate your issue. When only outgoing interface is specified no routing takes place, and system expects target ip to be local → it asks through ARP for MAC of target ip. Documentation is missing for that tool…

if you want an ip to test lte, just hardcode some ip to go over LTE route, ex:
/ip route add dst-address=8.8.4.4 gateway=192.168.2.1