Routerboard 4011 - Interface stops responding ( SRC-NAT MASQUERADE )

ccorrientes · April 2, 2020, 9:04pm

Dear MikroTik Community,
I am challenged..
Until today I had only the best experience with MikroTik, because of that I recommended a client of ours to invest in two 4011 RouterBoards as a backbone for a temporary DMZ he needs to build.

The setup of the DMZ is rather simple.. We have multiple internet connections, forwarding traffic to multiple SSL VPN servers, which then route client traffic to the MikroTik Router, which then masquerades the inbound traffic with its own local / internal IPs to reach multiple internal subnets.

HTTPs traffic hits an ISP router,
The ISP Router forwards the HTTPs traffic to a terminating SSL VPN Server
The VPN Server has multiple routes configured ( our internal private subnets ) pointing to a RouterBoard 4011 as next hop.
The VPN Server routes the decrypted VPN Client traffic ( i.e. SMTP, LDAP, DNS ) to the RouterBoard, in order to reach an internal application servers within different subnets.
The RouterBoard accepts the traffic and does an SRC-NAT masquerade into this subnet, to hide the original source IP.
The destination server responds back to the MikroTik internal interface IP, instead of answering via its default gateway ( a Cisco ASA 5508-x ).

We needed to masquerade the traffic as we wanted to avoided to add any additional route and firewall configs to our existing servers and gateway.
For some reason, we could not simply add an SRC NAT - MASQUERADE to get it work, we had to add a dedicated filter rule as well.

We used Filter-Rule ( access - accept ) matching the src-address ( DMZ ) and dst-address ( LAN )
and an SRC-NAT Rule with an src-address ( DMZ ), dst-address ( LAN ), dst-interface ( LAN ) and masquerade as action.

The config file and the layout of the DMZ are attached and hopefully, makes it easier to vizualize.

The issue is: The WAN or DMZ side interface of the RouterBoard does work for a couple of minutes, but then simply stops to forward any traffic.

It does not respond to ping from external, nor does it route any traffic.
It does respond though via the winbox terminal to its IP.
Either after a while, or many ping attempts and/or a reboot of the RouterBoard the interface comes finally back again and does its job for a ca 15-20 minutes.

Dear community.. What could be the cause? The strange thing is, it works for a certain amount of time and then simply stops working.
We can rule out the load of the system, as this happens even with just one single client connected to the DMZ.
Could it be some kind of connection time-out or is the attempt to SRC-NAT from a WAN side wrong in the first-place ?

If anyone could point me in the right direction i would be really grateful.

Thank you so much.

…Constantin

//////////////////////////////////////////////////////////////////////////////////////////////////////

# apr/02/2020 23:02:13 by RouterOS 6.46.4
# software id = 7529-G3CC
#
# model = RB4011iGS+5HacQ2HnD
# serial number = D1460BB4E374
/interface bridge
add admin-mac=C4:AD:34:D3:2A:BD auto-mac=no comment=defconf name=bridge
/interface wireless
set [ find default-name=wlan1 ] antenna-gain=0 band=5ghz-a/n/ac \
    channel-width=20/40/80mhz-XXXX country=no_country_set disabled=no \
    distance=indoors frequency=auto frequency-mode=manual-txpower \
    installation=indoor mode=ap-bridge secondary-channel=auto ssid=\
    MikroTik-D32AC7 wireless-protocol=802.11
set [ find default-name=wlan2 ] antenna-gain=0 band=2ghz-b/g/n channel-width=\
    20/40mhz-XX country=no_country_set disabled=no distance=indoors \
    frequency=auto frequency-mode=manual-txpower installation=indoor mode=\
    ap-bridge ssid=MikroTik-CC036B wireless-protocol=802.11
/interface ethernet
set [ find default-name=ether1 ] name="ether1 ( Router C )"
set [ find default-name=ether2 ] name="ether2 ( Router D )"
set [ find default-name=ether3 ] name="ether3 ( Router E )"
set [ find default-name=ether4 ] name="ether4 ( Router F )"
set [ find default-name=ether5 ] name="ether5 ( Router G )"
set [ find default-name=ether6 ] name="ether6 ( VLAN 0 )"
set [ find default-name=ether7 ] name="ether7 ( VLAN 80 )"
set [ find default-name=ether8 ] name="ether8 ( VLAN 93 )"
set [ find default-name=ether9 ] name="ether9 ( VLAN 95 )"
set [ find default-name=ether10 ] name="ether10 ( VLAN 100 )"
/interface ethernet switch port
set 0 default-vlan-id=0
set 1 default-vlan-id=0
set 2 default-vlan-id=0
set 3 default-vlan-id=0
set 4 default-vlan-id=0
set 5 default-vlan-id=0
set 6 default-vlan-id=0
set 7 default-vlan-id=0
set 8 default-vlan-id=0
set 9 default-vlan-id=0
set 10 default-vlan-id=0
set 11 default-vlan-id=0
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=bridge name=defconf
/interface bridge port
add bridge=bridge comment=defconf interface=sfp-sfpplus1
add bridge=bridge comment=defconf interface=wlan1
add bridge=bridge comment=defconf interface=wlan2
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface="ether1 ( Router C )" list=WAN
add interface="ether2 ( Router D )" list=WAN
add interface="ether3 ( Router E )" list=WAN
add interface="ether4 ( Router F )" list=WAN
add interface="ether5 ( Router G )" list=WAN
add interface="ether6 ( VLAN 0 )" list=LAN
add interface="ether7 ( VLAN 80 )" list=LAN
add interface="ether8 ( VLAN 93 )" list=LAN
add interface="ether9 ( VLAN 95 )" list=LAN
add interface="ether10 ( VLAN 100 )" list=LAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=\
    192.168.88.0
add address=192.168.70.253/24 interface="ether1 ( Router C )" network=\
    192.168.70.0
add address=192.168.71.253/24 interface="ether2 ( Router D )" network=\
    192.168.71.0
add address=192.168.72.253/24 interface="ether3 ( Router E )" network=\
    192.168.72.0
add address=192.168.73.253/24 interface="ether5 ( Router G )" network=\
    192.168.73.0
add address=192.168.0.227/24 interface="ether6 ( VLAN 0 )" network=\
    192.168.0.0
add address=192.168.80.250/24 interface="ether7 ( VLAN 80 )" network=\
    192.168.80.0
add address=192.168.93.250/24 interface="ether8 ( VLAN 93 )" network=\
    192.168.93.0
add address=192.168.95.250/24 interface="ether9 ( VLAN 95 )" network=\
    192.168.95.0
add address=192.168.100.250/24 interface="ether10 ( VLAN 100 )" network=\
    192.168.100.0
add address=192.168.74.253/24 interface="ether5 ( Router G )" network=\
    192.168.74.0
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes servers=192.168.0.180
/ip dns static
add address=192.168.88.1 name=router.lan
/ip firewall filter
add action=accept chain=forward disabled=yes dst-address=192.168.0.0/24 \
    out-interface="ether6 ( VLAN 0 )"
add action=accept chain=forward disabled=yes dst-address=192.168.80.0/24 \
    out-interface="ether7 ( VLAN 80 )"
add action=accept chain=forward dst-address=192.168.93.0/24 out-interface=\
    "ether8 ( VLAN 93 )"
add action=accept chain=forward disabled=yes dst-address=192.168.95.0/24 \
    out-interface="ether9 ( VLAN 95 )"
add action=accept chain=forward disabled=yes dst-address=192.168.100.0/24 \
    out-interface="ether10 ( VLAN 100 )"
add action=accept chain=input dst-port=8291 in-interface=all-ethernet \
    protocol=tcp src-address=0.0.0.0
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new disabled=yes in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="From OpenVPN to VLAN 1" \
    dst-address=192.168.0.0/24 log=yes out-interface="ether6 ( VLAN 0 )"
add action=masquerade chain=srcnat comment="From OpenVPN to VLAN 80" \
    dst-address=192.168.80.0/24 out-interface="ether7 ( VLAN 80 )"
add action=masquerade chain=srcnat comment="From OpenVPN to VLAN 93" \
    dst-address=192.168.93.0/24 out-interface="ether8 ( VLAN 93 )"
add action=masquerade chain=srcnat comment="From OpenVPN to VLAN 95" \
    dst-address=192.168.95.0/24 out-interface="ether9 ( VLAN 95 )"
add action=masquerade chain=srcnat comment="From OpenVPN to VLAN 100" \
    dst-address=192.168.100.0/24 out-interface="ether10 ( VLAN 100 )"
add action=masquerade chain=srcnat comment="From VLAN 0 to 192.168.70.0" \
    out-interface="ether1 ( Router C )" src-address=192.168.0.0/24
add action=masquerade chain=srcnat comment="From VLAN 0 to 192.168.71.0" \
    out-interface="ether2 ( Router D )" src-address=192.168.0.0/24
add action=masquerade chain=srcnat comment="From VLAN 0 to 192.168.72.0" \
    out-interface="ether3 ( Router E )" src-address=192.168.0.0/24
add action=masquerade chain=srcnat comment="From VLAN 0 to 192.168.73.0" \
    out-interface="ether4 ( Router F )" src-address=192.168.0.0/24
add action=masquerade chain=srcnat comment="From VLAN 0 to 192.168.74.0" \
    out-interface="ether5 ( Router G )" src-address=192.168.0.0/24
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
add action=dst-nat chain=dstnat disabled=yes dst-address=192.168.73.1 \
    protocol=tcp to-addresses=192.168.1.1
/ip route
add disabled=yes distance=1 gateway=192.168.0.2
/ip route rule
add disabled=yes dst-address=0.0.0.0/0 interface="ether1 ( Router C )" \
    routing-mark=main src-address=192.168.70.0/24
/system clock
set time-zone-name=Europe/Madrid
/system leds
add interface=wlan2 leds="wlan2_signal1-led,wlan2_signal2-led,wlan2_signal3-le\
    d,wlan2_signal4-led,wlan2_signal5-led" type=wireless-signal-strength
add interface=wlan2 leds=wlan2_tx-led type=interface-transmit
add interface=wlan2 leds=wlan2_rx-led type=interface-receive
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

Zacharias · April 3, 2020, 10:02am

I am lost inside your configuration, so many disabled rules… Why dont you clear that a bit and put your code inside code tags ?

ccorrientes · April 3, 2020, 11:05am

I am sorry for the circumstances.. I removed the disabled rules and put everything into the code tag.

Thank you very much for having a look at it

..Constantin

Zacharias · April 3, 2020, 5:19pm

I do not see anything in your config that would cause such a problem…
Since as you say the 4011 is accessible when this happens, do the following:

Check the log
Check the Resources, CPU, Ram
Check the Temperature

Also upgrade the firmware though System → Routerboard → Upgrade…

ccorrientes · April 5, 2020, 1:46pm

After investigating much further i found the culprit.

Another device was running on the same IP. This explains why the traffic is passed through and after a certain time it sporadically times out.. ( as we hit a arp timeout on either the host or on the firewall )

..Constantin