Mikrotik hAP3 AC3-RouterOS config with external LTE modem

Good evening,
I am trying to set up an Internet access using a decent router and an Acer M3 5G LTE access point.

For some reasons it did not work out well with an HexS (did not recognise the Acer M3-support is looking into it. So I did buy an hAP AC3 which sees immediately the M3 obtains an IP address and install a default route. I can ping google DNS and google.com. I did install a mangle chain rule with a masquerade action and the internal hosts get out on the Internet.
So it all appears to be fine but ...

Performance are not making me particularly enthusiast.

  1. when I set up a fast connection action for the established related connection in the forwarding chain the ping doubles up in time and the connectivity breaks down ....
  2. Using a 5g AP that ranges between 120 to 240 Mbps and going through the Mikrotik I get no more than 18-20 Mbps. This happens bot if I connect the AP via USB or via one of the WIFI interfaces.

So it appears that the Mikrotik Router is the bottleneck. The processors stays low (less than 3%) memory stays 30% used 60% free. Anybody knows if this router has fasttrack/hardware offload? Even if the processor does not appear to be the problem.

The test is done locally (no tunnel involved, only NAT) but the router is originating a IPSEC IKEv2 tunnel and an overlaying GRE tunnel using the IPSEC endpoints as source/destination.

Is there any thing I should try to improve performance? Any setting I should enable?
Thank you in advance.
Fabio

This is the Acer Modem:
https://www.acer.com/us-en/networking/5g/acer-connect-enduro-m3-5g-mobile-wifi

And this is the other thread:

Post your full configuration for review, instructions here:

@jaclaz thank you for your interest.
I usually do not post complete configs in forums because of security concerns.
Beside it is about 160 lines.
Any area in particular I should be looking int performance wise?
The setup works and it is functional, only slower than expected.
Regards
Fabio

Post your configuration as instructed by @jaclaz.

But this question can be answered without seeing your configuration:

The hAP ac³ has full fasttrack support because it's a software feature. It only has hardware offload for switching (L2) not for routing (L3). The problems you see, all the issues when you enabled fasttrack, are due to:

Connections marked with fasttrack-connection are not compatible with mangle rules. If you use mangle mark-connection with mark-routing and let those connections be fasttrack-ed, you'll end up with part of the connection's packets that go through the correct routing table, while the majority of the packets use the wrong routing table. This usually results in massive packet loss (the routers on the route don't accept packets with wrong source/destination address for example), causing the delays and abysmal performance (only the few packets that are not fasttrack-ed get through).

See Connection tracking - RouterOS - MikroTik Documentation

including the warning:

:warning: Queues (except Queue Trees parented to interfaces), firewall filter, and mangle rules will not be applied for FastTracked traffic.

@CGGXANNX Thank you very much, you reply prompted me to look into this a bit further and I found:
Packet Flow in RouterOS - RouterOS - MikroTik Documentation which I need to dig into a bit further.

In any case my config is as folllows (removed sensitive IPs and IPSEC identities) hope nothing sensitive is still in ...

Cheers
Fabio

/interface bridge
add name=bridge-ipsec
add name=bridge1
add name=bridge2
add name=bridge3
/interface gre
add comment="To tunnel 57" local-address=192.168.255.10 mtu=1300 name=gre-tunnel1 remote-address=192.168.255.9
/interface wifi security
add authentication-types=wpa2-psk,wpa2-eap disabled=no name=secSarlan
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
add authentication-types=wpa2-psk comment=SL_security mode=dynamic-keys name=SL_sec supplicant-identity=""
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-Ce comment="Locally Routed 2Ghz" country=italy disabled=\
    no frequency=2437 installation=indoor mode=ap-bridge security-profile=SL_sec ssid=Mkt_hAP2
set [ find default-name=wlan2 ] comment="Locally Routed 5 Ghz" country=italy disabled=no frequency=auto installation=indoor \
    l2mtu=1500 mode=ap-bridge mtu=1300 security-profile=SL_sec ssid=Mkt_hAP5
add comment="Routed to Ita 2 Ghz" disabled=no l2mtu=1500 mac-address=F6:1E:57:B5:0E:2E master-interface=wlan1 mtu=1300 name=\
    wlan3 security-profile=SL_sec ssid=SL_hAP_ITA2 wps-mode=disabled
add comment="Routed to ITA 5 Ghz" disabled=no l2mtu=1500 mac-address=F6:1E:57:B5:0E:2F master-interface=wlan2 mtu=1300 name=\
    wlan4 security-profile=SL_sec ssid=SL_hAP_ITA5 wps-mode=disabled
add comment="Routed to USA 2 Ghz" disabled=no l2mtu=1500 mac-address=F6:1E:57:B5:0E:30 master-interface=wlan1 mtu=1300 name=\
    wlan5 security-profile=SL_sec ssid=SL_hAP_USA2 wps-mode=disabled
add comment="Routed to USA 5 Ghz" disabled=no l2mtu=1500 mac-address=F6:1E:57:B5:0E:31 master-interface=wlan2 mtu=1300 name=\
    wlan6 security-profile=SL_sec ssid=SL_hAP_USA5 wps-mode=disabled
/interface wireless manual-tx-power-table
set wlan1 comment="Locally Routed 2Ghz"
set wlan2 comment="Locally Routed 5 Ghz"
set wlan3 comment="Routed to Ita 2 Ghz"
set wlan4 comment="Routed to ITA 5 Ghz"
set wlan5 comment="Routed to USA 2 Ghz"
set wlan6 comment="Routed to USA 5 Ghz"
/interface wireless nstreme
set wlan1 comment="Locally Routed 2Ghz"
set wlan2 comment="Locally Routed 5 Ghz"
set *15 comment="Routed to Ita 2 Ghz"
set *17 comment="Routed to ITA 5 Ghz"
set *18 comment="Routed to USA 2 Ghz"
set *19 comment="Routed to USA 5 Ghz"
/ip dhcp-server option
add code=6 comment=GoogleDNS name=GoogleDNS value="'8.8.8.8''8.8.4.4'"
add code=3 name=GW42 value="'192.168.42.1'"
add code=3 name=GW36 value="'192.168.36.1'"
add code=3 name=GW37 value="'192.168.37.1'"
add code=6 name=US_dns value="'192.168.2.1'"
add code=42 name=NTP value="'192.168.128.5'"
add code=1 name=Netmask value="'255.255.255.0'"
/ip dhcp-server option sets
add name=Net42 options=GoogleDNS,GW42,NTP,Netmask
add name=Net36 options=GoogleDNS,GW36,NTP,Netmask
add name=net37 options=US_dns,GW37,NTP,Netmask
/ip ipsec profile
add dh-group=modp2048 enc-algorithm=aes-256 hash-algorithm=sha256 name=TS-3 prf-algorithm=sha256
/ip ipsec peer
add address=xxxxxxxxxx/32 exchange-mode=ike2 name=Home profile=TS-3
/ip ipsec proposal
add auth-algorithms=sha256 enc-algorithms=aes-256-cbc name=proposal1 pfs-group=modp2048
/ip pool
add name=Pool42 ranges=192.168.42.50-192.168.42.199
add name=pool37 ranges=192.168.37.50-192.168.37.199
add name=pool36 ranges=192.168.36.50-192.168.36.199
/ip dhcp-server
add address-pool=Pool42 comment="Routed Local" dhcp-option-set=Net42 interface=bridge1 lease-time=4h name=Server42
add address-pool=pool36 comment="Routed to ITA" dhcp-option-set=Net36 interface=bridge2 lease-time=4h name=Server36
add address-pool=pool37 comment="Routing to USA" dhcp-option-set=net37 interface=bridge3 lease-time=4h name=server37
/routing ospf instance
add disabled=no name=ospf-instance-1
/routing ospf area
add area-id=192.168.255.1 comment=Area0 disabled=no instance=ospf-instance-1 name=ospf-area-0
/interface bridge port
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
add bridge=bridge1 interface=wlan1
add bridge=bridge1 interface=wlan2
add bridge=bridge2 interface=wlan3
add bridge=bridge2 interface=wlan4
add bridge=bridge3 interface=wlan5
add bridge=bridge3 interface=wlan6
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/interface detect-internet
set internet-interface-list=static
/ip address
add address=192.168.42.1/24 interface=bridge1 network=192.168.42.0
add address=192.168.36.1/24 interface=bridge2 network=192.168.36.0
add address=192.168.37.1/24 interface=bridge3 network=192.168.37.0
add address=192.168.255.10/30 interface=bridge-ipsec network=192.168.255.8
add address=192.168.254.10/30 interface=gre-tunnel1 network=192.168.254.8
/ip dhcp-client
# Interface not active
add interface=wlan1
/ip firewall address-list
add address=0.0.0.0/8 list=not_in_internet
add address=172.16.0.0/12 list=not_in_internet
add address=192.168.0.0/16 disabled=yes list=not_in_internet
add address=10.0.0.0/8 list=not_in_internet
add address=169.254.0.0/16 list=not_in_internet
add address=127.0.0.0/8 list=not_in_internet
add address=224.0.0.0/4 list=not_in_internet
add address=198.18.0.0/15 list=not_in_internet
add address=192.0.0.0/24 list=not_in_internet
add address=192.0.2.0/24 list=not_in_internet
add address=198.51.100.0/24 list=not_in_internet
add address=203.0.113.0/24 list=not_in_internet
add address=100.64.0.0/10 list=not_in_internet
add address=240.0.0.0/4 list=not_in_internet
add address=192.88.99.0/24 comment="6to4 relay Anycast [RFC 3068]" list=not_in_internet
add address=192.168.32.0/20 list=Local
add address=192.168.32.0/20 list=allowed_to_router
add address=192.168.128.0/20 list=allowed_to_router
add address=192.168.0.0/22 list=Local
add address=192.168.128.0/20 comment="Local RF1918 IP in use" list=Local
add address=192.168.254.0/23 comment="Tunnel and IPSEC" list=Local
/ip firewall filter
add action=accept chain=input comment="Allowing Stablished and related" connection-state=established,related
add action=accept chain=input comment="allowing Local Nets" src-address-list=allowed_to_router
add action=accept chain=input comment="OSPF from Cisco" protocol=ospf src-address=192.168.254.9
add action=accept chain=input comment="Allow ICMP to router" protocol=icmp
add action=drop chain=input comment="Deny to router not in internet" in-interface=lte1 log=yes log-prefix=N_I_I_torouter \
    src-address-list=not_in_internet
add action=drop chain=input comment="Deny everything else to router" log=yes log-prefix=Router_access_
add action=accept chain=forward comment="Lan Firewall Rules" dst-address-list=Local src-address-list=Local
add action=drop chain=forward comment="Deny Forward non internet" disabled=yes in-interface=lte1 log=yes log-prefix=\
    Not_inInternet src-address-list=not_in_internet
add action=accept chain=forward comment="Established, Related" connection-state=established,related
add action=drop chain=forward comment="Drop invalid" connection-state=invalid log=yes log-prefix=invalid
add action=jump chain=forward comment="jump to ICMP filters" jump-target=icmp protocol=icmp
add action=accept chain=icmp comment="echo reply" icmp-options=0:0 protocol=icmp
add action=accept chain=icmp comment="net unreachable" icmp-options=3:0 protocol=icmp
add action=accept chain=icmp comment="host unreachable" icmp-options=3:1 protocol=icmp
add action=accept chain=icmp comment="host unreachable fragmentation required" icmp-options=3:4 protocol=icmp
add action=accept chain=icmp comment="allow echo request" icmp-options=8:0 protocol=icmp
add action=accept chain=icmp comment="allow time exceed" icmp-options=11:0 protocol=icmp
add action=accept chain=icmp comment="allow parameter bad" icmp-options=12:0 protocol=icmp
add action=drop chain=icmp comment="deny all other types"
/ip firewall mangle
add action=route chain=prerouting dst-address-list=!Local log-prefix=TO_CISCO route-dst=192.168.254.9 src-address=\
    192.168.36.0/23
add action=change-mss chain=forward new-mss=1260 out-interface=gre-tunnel1 protocol=tcp src-address=192.168.36.0/23 tcp-flags=\
    syn
/ip firewall nat
add action=masquerade chain=srcnat out-interface=lte1 src-address=192.168.42.0/24
add action=masquerade chain=srcnat disabled=yes out-interface=wlan1
/ip ipsec identity
add my-id=xxxxxx peer=Home remote-id=fqdn:xxxxxx
/ip ipsec policy
add dst-address=192.168.255.9/32 peer=Home proposal=proposal1 src-address=192.168.255.10/32 tunnel=yes
/routing ospf interface-template
add area=ospf-area-0 disabled=no interfaces=gre-tunnel1 networks=192.168.254.8/30,192.168.36.0/23,192.168.42.0/24 type=ptp
add area=ospf-area-0 disabled=no interfaces=bridge1,bridge2,bridge3 passive
/system clock
set time-zone-name=Europe/Rome

Sorry forgot to add that in making my FW rules I referre to:

and the offending statement (which is not in the config now) was:
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related

Cheers
Fabio

You are in good hands, with CGG, all I will say is that go with vlans and one bridge if you can.
Multiple bridges will not be efficient.
Not sure why you have any ICMP rules on the forward chain, new one to me.
You have the required simple and effective allow ICMP on the input chain, done................

@anav thank you,
I wish you could elaborate a bit further on this.
What is the difference in using VLAN and Bridges interfaces on Mikrotik?
I did ask this question some time ago and I had the impression that if you do not need VLANs don't bother with them.

In this installation I do not need 803.1q VLANs/tagging and that is the reason why I used bridges.
I am more familiar with Cisco environments (been doing easy things on Cisco since 1994 nothing complicated) and in that environment you just add interfaces on the WAN port or ports on the Bridge Virtual Interface/bridge group .... (exactly what I did here ...).

Could you please elaborate why VLAN would be more efficient? I see the one bridge statement been based on L3 Hardware Offloading - RouterOS - MikroTik Documentation

But again the docs say L3 Hardware Offloading - RouterOS - MikroTik Documentation
I need to add the VLAN to the bridge for routing.

So interfaces wlan1/2 and all ethernet ports they belong to the same broadcast domain.
Interfaces wlan 3/4 to same broadcast domain (separate from the previous)
and so wlan 5/6.
They are in separate /24 and are routed differently.

The first broadcast domain is NATted and sent directly out of the default route from the locally connected Internet interface (LTE1).

The other two (which are contiguous and constitute a /23 network in order to be efficiently matched) are routed without translation to gre tunnel1 which is laying over an IPSEC SA (out from LT1 to the central router). Further routing to the local network, firewalling Internet forwarding and NAT is handled centrally by the Cisco Router HUB in this topology.

These separate broadcast domains need to be able to communicate locally at layer 3.

Finally the ICMP forwarding chain is there to filter unwanted ICMP traffic inside the network. Not entirely my idea (see Building Advanced Firewall - RouterOS - MikroTik Documentation).

Cheers
Fabio

If the configuration you posted is what you call simple, I don't want to know what you call complex. :confused:

I believe you will have to adapt it to the various Mikrotik peculiarities, if you want it to work efficiently and flawlessly.

I see at first glance in your configuration (not that I understand all of it) several violations of the Twelve Rules of the Mikrotik Club (and of GP & CSA), just in case:

Most are of course not really relevant/important, but the multiple bridges and the use of the "old" wireless (as opposed to the "new" wifi) drivers may - I believe - actually affect negatively performance. (but the nstreme is not an option with wifi, so if you actually need it, the wireless packet is needed).

BTW which RouterOS version are you running?

:slight_smile: externall BGP, LACP, load balancing between circuits, multiple redundancies, multiple external routes …. multiple internal routing protocols redistributing between them, VRF …. you name it.
This is just a router with 3 networks, one tunnel protected by ipsec and one default route. I am running OSPF to propagate the internal networks because I wont bother writing the static routes. But I could do it with static because the routes will not change are there or they are not…

I have read the rules, they make sense , I hope I am not breaking 11 … I am here to understand Mikrotik way of doing things.

I am on routeros 7.19.4, the latest stable (or release candidate according to the ‘rule book’).
Usually when I get a router, I check what it is on it, read the release note of the last release train, and if there are no issues with the technologies I need to implement update and leave it until it breaks ….

I am reading the forum to understand the quirks of Mikrotik, can you point me to discussions/docs covering the bridges/wifi issues you mentioned above?

Re: multiple bridges:
One of the things is that for Mikrotik VLAN can be HW offloaded to bridge for some devices if you use the standard VLAN way of setting up VLAN.
But only to 1 bridge.
If you have multiple, it is not always guaranteed things will work for this aspect (and you might not even know which one will get the HW offload. Usually the first, I think).

You can use switch rules (which is also needed for devices having multiple bridges like RB4011) but it is a bit different. The standard VLAN approach has the advantage it always works (but can be with or without HW offload then).

Usage of VLAN is also more or less brand agnostic. VLAN is VLAN.
Use of bridges is rather brand specific.
My view.

Thanks @holvoetn,
very useful.I did spend sometime studying today the architecture and inner working of the Mikrotiks, an tomorrow (hopefully) I will do some trials with the unsuitable hEX S.
Apparently the best way of doing this is to create multiple VLANs interfaces and a single switch, and offload the switching of the VLANs in the vlan table to the HW by adding the VLANs to the switch and making the physical interfaces members of the VLAN. This, apparently, can be done by the single switch chip for multiple VLANs/VLANs interfaces. InterVLAN communication is then sent to the CPU(routing). The switch chip inthe hAP ac^3 also support rules and traffic can be manually sent to the CPU switching port at will.
the page Switch Chip Features - RouterOS - MikroTik Documentation has alot of interesting info.

Digging I found this:
It is possible to use the built-in switch chip and the CPU at the same time to create a Switch-Router setup, where a device acts as a switch and as a router at the same time. You can find a configuration example in the Switch-Router guide.

The wiki page exists butit is empty.
Anybody knows how is it? Never created, removed?

Too many "wiki" in the url (sometimes in the past the whole stuff has been moved around causing any kind of bad links), use this (middle wiki removed)
https://wiki.mikrotik.com/Manual:Switch_Router

BUT you are mixing liberally contents of the wiki (essentially related to routerOS 6.x) with content of the help pages (essentially related to routerOs 7.x), just like you are using wireless on 7.x, something coming from 6.x works, some works worse, some doesn't work at all (anymore).

besides you will have to go through BOTH of these threads: