Page 1 of 1

Strange issue with a IPsec issue

Posted: Sun Apr 18, 2021 12:44 pm
by dnordenberg

For a short story see post #2

I used IPsec many times with mikrotik and this time the setup was no different in it's setup but it is acting really strange. The tunnel uses where .177 is the mikrotiks IP which is then default gateway for the rest of the devices (.178-.181) on this network. Now my servers trying to talk to these IPs are on another subnet which is on a VLAN into the Cisco ASA which is the other end of the IPsec tunnel.
This has worked perfect so many times before but now the problem begins. My primary server can not talk to or .179 but the mikrotik at .177 works and .180 and 181 too. My backup server at can talk to all IP .14.177-.14.181. ARP issues I thougt so I checked the mikrotik ARP table and it shows correct for the .178 and .179 IPs. Many other computers on other subnets can also talk to .178 and .179 but not the primary server. I checked its routing table if something has sneaked in there but no, looks ok.
I used mikrotik packet sniffer to watch the ethernet ports accosiated with these IPs and no packets from this servers IP exits the interface when pinging or trying other protocols against .178 and .179 like it does when the secondary server does the same. I did a traceroute from primary server to .178 and the last IP to respond is the mikrotik at .177 then timeout. Traceroute to .180 and .181 goes through all the way. .178+.180 is on the same physical port on the mikrotik and .179+.181 is on another. It is like if the mikrotik dislikes the primary server and thows its packets away for these two IPs.
Using latest firmware 6.48.2 on the RB450G. No firewall rules.
My temporary solution to this was to point the primary server to the mikrotik IP .177 instead and add dst-nat on the mikrotiks IP .177 for my specific TCP ports and let the mikrotik redirect the packets dst IPs to .178 and .179 instead. I did not have to use src-nat/masqurade so packets find its way back without altering src IP. I don't have access to the device today so can't post a config but is basically what I showed in this other thread but only using a single tunnel on this device : viewtopic.php?f=2&t=173704&p=849681#p849681
What the hell is going on here? This should "just work" so I don't have many ideas where to start looking :(

Any tips?

Re: Strange issue with a IPsec issue

Posted: Mon Apr 19, 2021 3:07 am
by dnordenberg
Short story:
IPsec tunnel where .177 is the mikrotik RB450G (and default GW for all the other IPs on the subnet).
A specific host on another remote subnet (other side of the IPsec tunnel) can not reach .178 and .179 IPs in the .14.176/29 subnet but other IPs work like .180 and .181.
Traceroute ends at at the mikrotik .177 then timeout after that which seems to point to a problem at the mikrotik.
The mikrotik itself and many other hosts on the remote subnet for example the host with the IP after the first one can reach all IPs on the mikrotik's subnet, even .178 and .179.
Packet sniffer shows that no packet exits the ethernet port on the mikrotik when the first host tries to ping or communicate on TCP ports on .178 and .179, packets seems to stop inside the mikrotik while there are valid ARP entries for these IPs.
No firewall rules, firmware 6.48.2.
Temporary solution, add a few dst-nat entries on the mikrotik for .177 for some used TCP ports which then points to .178 and .179.

What could cause only those two IPs not to be reachable from a specific host on the remote subnet?

Re: Strange issue with a IPsec issue

Posted: Mon Apr 19, 2021 8:34 am
by dnordenberg
Now the config: (Non IP related lines removed)
/interface bridge
add name=WAN_4G
add name=WAN_kontor
add comment="created from master port" fast-forward=no name=bridge_scada protocol-mode=none
/ip ipsec peer
add address= exchange-mode=ike2 name=ipsec_1
/ip ipsec profile
set [ find default=yes ] dh-group=modp1536 enc-algorithm=aes-256 hash-algorithm=sha256
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=1h pfs-group=modp1536
/interface bridge port
add bridge=bridge_scada interface=ether3
add bridge=bridge_scada interface=ether4
add bridge=WAN_4G interface=ether5
add bridge=bridge_scada hw=no interface=ether2
add bridge=WAN_kontor interface=ether1
/ip address
add address= comment=defconf interface=bridge_scada network=
add interface=WAN_kontor
/ip dhcp-client
add default-route-distance=2 disabled=no interface=WAN_4G
/ip firewall nat
add action=dst-nat chain=dstnat dst-address= dst-port=9010 \
    protocol=tcp to-addresses= to-ports=9010
/ip ipsec identity
add my-id=key-id:xxxx peer=ipsec_1 secret= "xxxxx"
/ip ipsec policy
add dst-address= level=unique peer=ipsec_1 sa-dst-address=\ src-address= \
add dst-address= level=unique peer=ipsec_1 sa-dst-address=\ src-address= \
/ip route
add distance=1 gateway=WAN_kontor
add distance=1 dst-address= gateway=bridge_scada pref-src=
add distance=1 dst-address= gateway=bridge_scada pref-src=

Re: Strange issue with a IPsec issue

Posted: Mon Apr 19, 2021 12:08 pm
by dnordenberg
Strange, today .178 works but still not .179.
I also did another packet sniffer run and now I see packets do exit the mikrotik routers ethernet port when pinging but nothing is heard back when pinging from the host it didn't work from. Don't know if behavior is changed or if I was to blind to see the packets going out previously :(
But can this be a TTL issue? When pinging from mikrotik, tx ICMP has 255 and the ICMP received back has 64. When looking at the ping from this remote host the ICMP is sent out as TTL 249 so I guess there should be plenty of TTL left so that is not the cause?