2 LTE ROUTERS - 1 hEX

RouterOS General
1

Alright so, long story short… hEX running on v6.49.1

I have two LTE ISPs - the Main is Afrihost Uncapped and the backup is Rain unlimited off-peak (5GB).
ether1 192.168.10.1 Afrihost LTE Uncapped @ 50-75Mbps
ether2 192.168. 9.1 Rain 4G - 5GB Off-Peak @ 15Mbps if you’re lucky
ether3 192.168. 8.1 Main Network - 100% bandwidth
ether4 192.168. 8.1 Will be dedicated to wifi APs - need to be able to limit to 15% download bandwidth or less when I want (Show me how??)
ether5 192.168. 8.1 Will be dedicated CCTV for all 4 DVRs - 100% Bandwidth

Yes, I would like to know how to bridge ether3 4 and 5 but control bandwidth on each. I run my DVRs on dedicated PCs to combine the DVRs and display all cameras around the property (separate buildings) on one screen in my house so I need to be able to see everything on my PC (connected to ether3) from ether4 and ether5 as you would with a standard network switch.

I want Afrihost to pull all the data all the time and Rain to just sit there disabled till just in case the sheet hits the fan then it can take over till Afrihot is back online.
I’m guessing I need to post some script to watch 8.8.8.8 and 8.8.4.4 to constantly make sure Afrihost is A-ok, Rain I’m not worried about in this case as it only needs to be there when Afrihost is down.

I seem to sort of have this running but looks like Rain still receiving some of the data when I look in the interface list.

Secondly, if Rain just so happens to need to run for a full day or more I need a schedule only during its active time to kill it at 17:55 and restore it at 23:05
as this is their peak hrs (when they take more money from you than they should).

thirdly I would like to be able to remotely access my hEX via the Mikrotik iOS/Android app while I’m away to make sure it’s all running smooth (I’m aware of the security breach implications)- I tried by turning on DDNS under IP>Cloud and it spat out a DNS name and pub address but my app doesn’t seem to connect when I’m off my network, assuming this has something to do with NAT or Firewall? what do I do???

Lastly, I have some stupid old DVR running on a No-IP address via HTTP Port 8220 with server port 6036 on 192.168.8.14 and again I assume it’s something to do with NAT, Firewall,
or Port Forward that I can’t seem to connect. Fortunately, this paragraph will become null and void once I upgrade this one in the future but for now its working so I want to see it remotely with the rest.

I would really appreciate your help as I have tried and failed with many different methods over months now and right now as per my /export I have internet running and that’s about it. and yes @anav I’m darn well sure my code is a complete disaster of a mess so if you could do the honors in cleaning it up and throwing out the trash then I would be most grateful to have all this running like a pro.

Sadly I do not have time to do any online courses due to workload in the security industry and worst off I have a mental span of 15min thanks to ADHD so I struggle to do courses of any sort, I learn best from my mistakes and with the help/guidance from others.

/export hide-sensitive 
# dec/04/2021 18:10:14 by RouterOS 6.49.1
# software id = Y1G6-ULK3
#
# model = RB750Gr3
# serial number = CC210B84FB20
/interface ethernet
set [ find default-name=ether2 ] disabled=yes
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool0 ranges=192.168.8.120-192.168.8.150
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=ether3 lease-time=8h name=dhcp1
/queue simple
add disabled=yes max-limit=10M/10M name="Main Queue" target=192.168.8.0/24
/user group
set full policy=local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add disabled=yes interface=ether3 trusted=yes
add disabled=yes interface=ether4 trusted=yes
add disabled=yes interface=ether5 trusted=yes
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ip address
add address=192.168.10.2/27 interface=ether1 network=192.168.10.0
add address=192.168.9.2/27 interface=ether2 network=192.168.9.0
add address=192.168.8.1/24 interface=ether3 network=192.168.8.0
/ip cloud
set ddns-enabled=yes
/ip dhcp-server network
add address=192.168.8.0/24 dns-server=192.168.9.1,192.168.10.1 gateway=192.168.8.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip firewall mangle
add action=mark-connection chain=input in-interface=ether1 new-connection-mark=ISP1_Conn passthrough=yes
add action=mark-connection chain=input in-interface=ether2 new-connection-mark=ISP2_Conn passthrough=yes
add action=mark-routing chain=output connection-mark=ISP1_Conn new-routing-mark=To_ISP1 passthrough=yes
add action=mark-routing chain=output connection-mark=ISP2_Conn new-routing-mark=To_ISP2 passthrough=yes
add action=accept chain=prerouting in-interface=ether3
add action=mark-connection chain=output connection-mark=no-mark connection-state=new new-connection-mark=ISP1_conn out-interface=ether1
add action=mark-routing chain=output connection-mark=ISP1_conn new-routing-mark=to_ISP1 out-interface=ether1
add action=mark-connection chain=output connection-mark=no-mark connection-state=new new-connection-mark=ISP2_conn out-interface=ether2
add action=mark-routing chain=output connection-mark=ISP2_conn new-routing-mark=to_ISP2 out-interface=ether2
/ip firewall nat
add action=dst-nat chain=dstnat comment=Factory-DVR dst-address=192.168.10.2 dst-port=80 in-interface=ether1 protocol=udp to-addresses=192.168.8.14 to-ports=8220
add action=dst-nat chain=dstnat dst-address=192.168.9.2 dst-port=80 in-interface=ether2 protocol=udp to-addresses=192.168.8.14 to-ports=8220
add action=masquerade chain=srcnat comment=RAIN out-interface=ether1
add action=masquerade chain=srcnat comment=AFRIHOST out-interface=ether2
/ip route
add check-gateway=ping distance=1 gateway=8.8.4.4 routing-mark=to_ISP1
add check-gateway=ping distance=2 gateway=8.8.8.8 routing-mark=to_ISP1
add check-gateway=ping distance=2 gateway=8.8.8.8 routing-mark=to_ISP2
add check-gateway=ping distance=1 gateway=8.8.4.4 routing-mark=to-ISP2
add check-gateway=ping distance=1 gateway=192.168.10.1,192.168.9.1
add check-gateway=ping distance=1 gateway=192.168.10.1
add check-gateway=ping distance=2 gateway=192.168.9.1
add distance=1 dst-address=8.8.4.4/32 gateway=192.168.9.1 scope=10
add distance=1 dst-address=8.8.8.8/32 gateway=192.168.10.1 scope=10
add distance=1 dst-address=192.168.8.0/24 gateway=ether4 pref-src=192.168.8.1 scope=10
add distance=1 dst-address=192.168.8.0/24 gateway=ether5 pref-src=192.168.8.1 scope=10
/system clock
set time-zone-name=Africa/Johannesburg
/system identity
set name="Cam hEX Router"
/system scheduler
add interval=1d name="Disable RAIN" on-event="/interface disable [find name=ether2]" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=\
    oct/24/2021 start-time=17:55:00
add interval=1d name="Enable RAIN" on-event="/interface enable [find name=ether2]" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=\
    oct/24/2021 start-time=23:05:00
/tool sniffer
set file-name=arp.pcap filter-interface=ether1

Cameron_hEX-Routerboard.rsc (4.17 KB)

2

Assistance on this one would be appreciated, please? :smiley:

3

Slow assistance then as its the weekend LOL.

(1) /ip dhcp-server (should be your bridge)
add address-pool=dhcp_pool0 disabled=no interface=ether3 lease-time=8h name=
dhcp1

(2) all bridge ports disabled ??
/interface bridge port
add disabled=yes interface=ether3 trusted=yes
add disabled=yes interface=ether4 trusted=yes
add disabled=yes interface=ether5 trusted=yes

(3) IP Address should be the bridge…
add address=192.168.8.1/24 interface=ether3 network=192.168.8.0

(4) Typo error?
add action=dst-nat chain=dstnat dst-address=192.168**.9**.2 dst-port=80 \

(5) Why the mangle rules, what is the intent here?

(6) Assuming you are using queing rules to ???

4

I’ll try to be brief to accommodate to your attention span.

I assume you want to control only the bandwidth the gear at those ports occupies on the uplink; if so, you can use different QoS rules for different IP address ranges in the same subnet - I guess you don’t insist on any particular address for each individual device. If so, you’d jusr use multiple rows in /queue simple, each with a different target (e.g. 192.168.8.0/25, 192.168.8.128/26, 192.168.8.192/26).


You actually don’t need a script, your set of routes looks like an attempt to use the scriptless failover based on recursive nexthop search, but you haven’t combed through the pre-existing routes, causing them to (likely) interfere with those implementing the failover. Likely because you have multiple equivalent routes (same dst-address, same distance) and without seeing /ip route print I can’t guess which ones won.

So add two routes:
check-gateway=ping distance=1 gateway=8.8.8.8
check-gateway=ping distance=2 gateway=8.8.4.4

Then disable the red ones on the list below taken from your export, and keep the green ones enabled; I’m not sure about the purpose of the gray ones.

There is a typo in the routing-mark value in the 4th route, a dash intead of an underscore.

/ip route
add check-gateway=ping distance=1 gateway=8.8.4.4 routing-mark=to_ISP1
add check-gateway=ping distance=2 gateway=8.8.8.8 routing-mark=to_ISP1
add check-gateway=ping distance=2 gateway=8.8.8.8 routing-mark=to_ISP2
add check-gateway=ping distance=1 gateway=8.8.4.4 routing-mark=to-ISP2
add check-gateway=ping distance=1 gateway=192.168.10.1,192.168.9.1
add check-gateway=ping distance=1 gateway=192.168.10.1
add check-gateway=ping distance=2 gateway=192.168.9.1
add distance=1 dst-address=8.8.4.4/32 gateway=192.168.9.1 scope=10
add distance=1 dst-address=8.8.8.8/32 gateway=192.168.10.1 scope=10
add distance=1 dst-address=192.168.8.0/24 gateway=ether4 pref-src=192.168.8.1 scope=10
add distance=1 dst-address=192.168.8.0/24 gateway=ether5 pref-src=192.168.8.1 scope=10


In /ip firewall mangle, you have set up everything to make the own traffic of the router to send responses via the WAN through which the request got in (in chains input and output), which only makes sense if you’ve got public IPs on the LTE devices, but there are no rules in chain prerouting to send traffic from LAN via the preferred WAN. So it looks to me like a blind copy-paste without really understanding the purpose.


I’d say the best would be to use filter rules with time parameter:
Time ::= [!]Start-End,Day
Start – 0s..1d (time interval)
End – 0s..1d (time interval)
Day ::= sun|mon|tue|wed|thu|fri|sat[,Day*]

So you need to place a chain=forward time=17h55m-23h5m,sun,mon,tue,wed,thu,fri,sat out-interface=ether2 action=drop rule to /ip firewall filter.


There’s a thing in common to both the above - the DDNS thingy allows to translate a domain name to the public IP from behind which the router accesses other public addresses in the internet, but unless you can use UDP hole punching, which is usually impossible with mobile ISPs, it doesn’t help you at all, as there is no way to set up port forwarding rules on the ISP’s equipment providing the NAT. So one possibility may be to ask the ISP for a public address (which will cost some extra ZAR), another possibility is to run a CHR in a datacenter (where a public IP is automatically part of the offer), create a VPN setup between the hEX and that CHR, and connect a VPN client on your mobile to the CHR as well.

Before venturing into any VPN setup, make sure you understand how the firewall works and make a robust one on all your routers directly connected to the internet.

5

Hi @anav and @sandy

Good to hear from you guys!

Like I said my code is a mess yet somehow it’s working right now…
Anav:
1) Done and wokring
2) Enabled and working now
3) Done and wokring
4) Uh, yeah I’m not sure what I was trying to do there, I’m assuming its meant to be the same as the to-address… dst-address=192.168.8.14 ?
I’ve deleted it for now so you Pros can point me in the right direction.
5) Disabled and deleted all - sorry it came from some youtube/forum
6) nope, was tryng to but my setup didn’t work so disabled it - now deleted

Sandy:
I appreciate that thank thanks!
1)

I assume you want to control only the bandwidth

Yes, so basically to throttle ether3 download bandwidth specifically (all wifi APs in my house) when needed for instance when the little ones are on Netflix/YouTube and I need to work. I dont need to disable them, just slow them down but their devices are on the dhcp pool. I’d like to be able to set it up and then login and enable/disable the rule when I need it.

2)

without seeing /ip route print I can’t guess which ones won.

Ok so through the process of elimination (enable/disable) I’ve managed to clean up /ip route and find ones that work and they also prioritise Afrihost.
But, these only seem to work with the LTE Routers themselves!? when I login to the Afrihost router and disable the internet/SIM the hEX doesnt seem to pick that up, but when I switch off the router then only does hEX switch over to RAIN… also noticed that row 4 and 5 are doing the work on /ip route and not the others… 6 7 and 8 seem to be "automatically added and removed with ether1-5 themselves… also, please check that my distance is correct?

 /ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  8.8.4.4/32                         192.168.10.1              3
 1 A S  8.8.8.8/32                         192.168.10.1              5
 2 A S  8.8.4.4/32                         192.168.9.1               4
 3 A S  8.8.8.8/32                         192.168.9.1               6
 4 A S  0.0.0.0/0                          192.168.10.1              1
 5   S  0.0.0.0/0                          192.168.9.1               2
 6 ADC  192.168.8.0/24     192.168.8.1     bridge1                   0
 7 ADC  192.168.9.0/27     192.168.9.2     ether2                    0
 8 ADC  192.168.10.0/27    192.168.10.2    ether1                    0

.

You actually don’t need a script, your set of routes looks like an attempt to use the scriptless failover based on recursive nexthop search, but you haven’t combed through the pre-existing routes, causing them to (likely) interfere with those implementing the failover. Likely because you have multiple equivalent routes

Yes found this in a forum, cant remember if it was mikrotik or another forum but thought I should try it. I know there were a lot of recursive tags which I have now removed so I’m hoping this simplifies things for me and the hEX

3)
Yes, /ip firewall mangle was a copy paste from a forum too which I have now removed. like you said “without really understanding the purpose.” because right now I do not, I’m used to a standard router where you just hit “port forwarding and set it up”

4)

I’d say the best would be to use filter rules with time parameter:

Done this, hoping it’s correct, it’s red and says “-inactive time”

5)

Lastly, I have some stupid old DVR running on a No-IP address via HTTP Port 8220 with server port 6036 on 192.168.8.14 and again I assume it’s something to do with NAT, Firewall,
or Port Forward that I can’t seem to connect.

I had managed to get this right on my ZTE router with the NAT function on it prior to getting rain and my network expanding rapidly. It worked really well through the ZTE router but wasnt sure if the hEX had some sort of similar NAT setup… otherwise if I setup the NAT on both routers specifically for the “stupid old DVR running on a No-IP address via HTTP Port 8220 with server port 6036 on 192.168.8.14” and then point the DVR to those routers via the hEX depending on which router is active?

Reason I got this hEX was a mate of mine recommended it to me after his tearful laughter over me physically unplugging, swopping and plugging in and the two routers from my switch and also unplugging wifi APs when I required 100% Bandwidth… and so here I am now, I like the idea of, and capability of the hEX - just dont fully understand how to use it right now.
.

[Cameron@Cam hEX Router] > /export hide-sensitive
# dec/13/2021 00:18:01 by RouterOS 6.49.1
# software id = Y1G6-ULK3
#
# model = RB750Gr3
# serial number = CC210B84FB20
/interface bridge
add name=bridge1
/interface ethernet
set [ find default-name=ether1 ] comment="Afrihost LTE"
set [ find default-name=ether2 ] comment="Rain 4G"
set [ find default-name=ether3 ] comment="Main Network"
set [ find default-name=ether4 ] comment="Bandwidth Controlled wifi"
set [ find default-name=ether5 ] comment="Dedicated Cameras"
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool0 ranges=192.168.8.120-192.168.8.150
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=bridge1 lease-time=8h name=dhcp1
/user group
set full policy=\
    local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ip address
add address=192.168.10.2/27 interface=ether1 network=192.168.10.0
add address=192.168.9.2/27 interface=ether2 network=192.168.9.0
add address=192.168.8.1/24 interface=bridge1 network=192.168.8.0
/ip cloud
set ddns-enabled=yes
/ip dhcp-server network
add address=192.168.8.0/24 dns-server=192.168.9.1,192.168.10.1 gateway=192.168.8.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8
/ip firewall filter
# inactive time
add action=drop chain=forward out-interface=ether2 time=17h55m-23h5m,sun,mon,tue,wed,thu,fri,sat
/ip firewall nat
add action=masquerade chain=srcnat comment=AFRIHOST out-interface=ether1
add action=masquerade chain=srcnat comment=RAIN out-interface=ether2
/ip route
add check-gateway=ping distance=3 dst-address=8.8.4.4/32 gateway=192.168.10.1 routing-mark=to_ether1
add check-gateway=ping distance=5 dst-address=8.8.8.8/32 gateway=192.168.10.1 routing-mark=to_ether1
add check-gateway=ping distance=4 dst-address=8.8.4.4/32 gateway=192.168.9.1 routing-mark=to_ether2
add check-gateway=ping distance=6 dst-address=8.8.8.8/32 gateway=192.168.9.1 routing-mark=to_ether2
add check-gateway=ping distance=1 gateway=192.168.10.1
add check-gateway=ping distance=2 gateway=192.168.9.1
/system clock
set time-zone-name=Africa/Johannesburg
/system identity
set name="Cam hEX Router"
/system scheduler
add interval=1d name="Disable RAIN" on-event="/interface disable [find name=ether2]" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=oct/24/2021 start-time=17:55:00
add interval=1d name="Enable RAIN" on-event="/interface enable [find name=ether2]" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=oct/24/2021 start-time=23:05:00
/tool sniffer
set file-name=arp.pcap filter-interface=ether1