IPSEC Tunnel Established but not able to ping hosts

Hi
I have researched and researched on this topic and got it to work the last time, but this time around, a few things have changed and I am not able to get this work!

I have an existing setup where the HQ router connects to the branch routers to access resources via IPSEC Tunnels. Currently the setup was working fine with the HQ router which was a old Cisco VPN Router. However, the company has made the decision to completely switch to Mikrotik and also to connect to AWS VPS through CHR.

So this time around there would be 2 tunnels simultaneously created from the branches. One would be to HQ & the other to CHR (AWS).

In this scenario, I am unable to ping the host on the AWS side, as well as unable to ping the hosts in the branch from the CHR side. HQ to branch tunnels are working fine. CHR to branch tunnels are able to ping between the routers, but not able to ping the hosts.

posting the output of export for your reference:
IP References
CHR Public IP: 10.10.10.10
CHR LAN IP: 10.79.15.100

HQ Public IP: 9.9.9.9
HQ LAN IP: 192.168.1.0/24


Branch1 public IP: 1.1.1.1
Branch LAN IP: 192.168.9.0/24

Branch2 public IP: 2.2.2.2
Branch LAN IP: 192.168.7.0/24




AWS CHR Config

# nov/07/2023 21:58:42 by RouterOS 6.49.10
# software id = 
#
#
#
/interface ethernet
set [ find default-name=ether2 ] name=LAN
set [ find default-name=ether1 ] name=WAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec peer
add address=1.1.1.1/32 exchange-mode=ike2 local-address=10.10.10.10 \
    name=peer2-AWSto1
add address=2.2.2.2/32 disabled=yes exchange-mode=ike2 local-address=\
    10.10.10.10 name=peer1AWSto2
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 dpd-interval=10s enc-algorithm=\
    aes-256 lifetime=8h name=AWSipsecprofile
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc lifetime=1h pfs-group=\
    none
/user group
set full policy="local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,pas\
    sword,web,sniff,sensitive,api,romon,dude,tikapp"
/dude
set enabled=yes
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ip address
add address=10.79.15.100/24 interface=LAN network=10.79.15.0
add address=10.10.10.10 interface=WAN network=10.10.10.10
/ip dhcp-client
add !dhcp-options disabled=no interface=WAN
/ip firewall nat
add action=accept chain=srcnat disabled=yes dst-address=192.168.7.0/24 \
    src-address=10.79.15.0/24
add action=accept chain=srcnat dst-address=192.168.9.0/24 src-address=\
    10.79.15.0/24
add action=masquerade chain=srcnat ipsec-policy=out,none out-interface=WAN
/ip firewall raw
add action=accept chain=prerouting disabled=yes dst-address=192.168.7.0/24 \
    src-address=10.79.15.0/24
add action=accept chain=prerouting disabled=yes dst-address=10.79.15.0/24 \
    src-address=192.168.7.0/24
add action=accept chain=prerouting dst-address=192.168.9.0/24 src-address=\
    10.79.15.0/24
add action=accept chain=prerouting dst-address=10.79.15.0/24 src-address=\
    192.168.9.0/24
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set pptp disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip ipsec identity
add disabled=yes peer=peer1AWSto2 secret=xxxxxx
add peer=peer2-AWSto1 secret=xxxxxx
/ip ipsec policy
set 0 disabled=yes
add disabled=yes dst-address=192.168.7.0/24 peer=peer1AWSto2 \
    src-address=10.79.15.0/24 tunnel=yes
add dst-address=192.168.9.0/24 peer=peer2-AWSto1 src-address=\
    10.79.15.0/24 tunnel=yes
/ip service
set ftp disabled=yes
set www disabled=yes
set api disabled=yes
set winbox port=8292
set api-ssl disabled=yes
/system clock
set time-zone-name=Asia/Kolkata
/system identity
set name=ip-10-79-0-100.ap-north-7.compute.internal
/system logging
set 0 action=disk
set 1 action=disk
set 2 action=disk
set 3 action=disk
add disabled=yes topics=ipsec
/system package update
set channel=long-term

Branch 1 config:

# nov/07/2023 22:25:00 by RouterOS 6.48.6
# software id = SHMF-RSUZ
#
# model = RB750Gr3
# 
/interface bridge
add admin-mac=78:XX:XX:XX:XX:XX auto-mac=no comment=defconf name=\
    LANbridge1
/interface ethernet
set [ find default-name=ether1 ] name="ether1[WAN]"
/interface pppoe-client
add add-default-route=yes disabled=no interface="ether1[WAN]" name=pppoe-out1 \
    service-name=intpppoe use-peer-dns=yes user=intpppoe
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec peer
add address=9.9.9.9/32 disabled=yes exchange-mode=aggressive \
    local-address=1.1.1.1 name=peer2-1toHO
add address=10.10.10.10/32 exchange-mode=ike2 local-address=1.1.1.1 \
    name=peer1-1toAWSCHR
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024 dpd-interval=10s enc-algorithm=\
    aes-256 lifetime=8h name=kgmspl-1 nat-traversal=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc lifetime=1h pfs-group=\
    none
/ip pool
add name=dhcp ranges=192.168.9.111-192.168.9.254
/ip dhcp-server
add address-pool=dhcp disabled=no interface=LANbridge1 name=defconf
/interface bridge port
add bridge=LANbridge1 comment=defconf interface=ether2
add bridge=LANbridge1 comment=defconf interface=ether3
add bridge=LANbridge1 comment=defconf interface=ether4
add bridge=LANbridge1 comment=defconf interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=LANbridge1 list=LAN
add comment=defconf interface="ether1[WAN]" list=WAN
add interface=pppoe-out1 list=WAN
/ip address
add address=192.168.9.110/24 comment=defconf interface=LANbridge1 \
    network=192.168.9.0
/ip dhcp-client
add comment=defconf interface="ether1[WAN]"
/ip dhcp-server lease
add address=192.168.9.252 client-id=1:a:f5:df:1:88:4e mac-address=\
    0A:F5:DF:01:88:4E server=defconf
/ip dhcp-server network
add address=192.168.9.0/24 comment=defconf dns-server=192.168.9.110,8.8.8.8 \
    gateway=192.168.9.110 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.9.110 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=input protocol=ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked disabled=yes
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid disabled=yes
add action=accept chain=input comment="defconf: accept ICMP" disabled=yes \
    protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" disabled=yes \
    dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    disabled=yes in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    disabled=yes ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    disabled=yes ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related disabled=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked disabled=yes
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid disabled=yes
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new disabled=yes in-interface-list=WAN
/ip firewall nat
add action=accept chain=srcnat dst-address=10.79.15.0/24 src-address=\
    192.168.9.0/24
add action=accept chain=srcnat dst-address=192.168.1.0/24 src-address=\
    192.168.9.0/24
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface=pppoe-out1
/ip firewall raw
add action=accept chain=prerouting dst-address=10.79.15.0/24 src-address=\
    192.168.9.0/24
add action=accept chain=prerouting dst-address=192.168.9.0/24 src-address=\
    10.79.15.0/24
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set pptp disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip ipsec identity
add peer=peer1-1toAWSCHR
add disabled=yes peer=peer2-1toHO
/ip ipsec policy
add dst-address=10.79.15.0/24 peer=peer1-1toAWSCHR src-address=\
    192.168.9.0/24 tunnel=yes
add disabled=yes dst-address=192.168.1.0/24 peer=peer2-1toHO src-address=\
    192.168.9.0/24 tunnel=yes
set 2 disabled=yes
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox port=8292
set api-ssl disabled=yes
/system clock
set time-zone-name=Asia/Kolkata
/system identity
set name=KG1
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

I want to be able to ping the host 192.168.9.252 from the AWS CHR Side.

using command ping src-address=10.79.15.100 192.168.9.110 = i am able to ping the mikrotik branch1 router. but unable to ping the host connected to the same router.

Hello,

I am curious if you will have a solution, I have almost the same problem in my topic "GRE over IPSEC - cannot reach clients ", so I will keep my eye on this thread as well. Very frustrating.

Gergely

The one thing that is currently pending to be done on our side, is that the Cisco VPN Router is to be replaced by the Mikrotik routers. When that is done, I am interested to see if I am able to ping the hosts between the mikrotik to mikrotik hardware routers.. so I can isolate the problem to be with CHR.

If you go all the way Mikrotik, why not take the opportunity to move to Wireguard ?
Faster then IPSEC.

IPSEC is the customer requirement. I already had a hard time getting them switch to Mikrotik from Cisco. Will check about Wireguard and see if there are enough resources out there to support me to get it working on my own…

Your routing or firewall is probably incorrect. Add logging to your NAT and drop rules. Sniff traffic to make sure packets go where you expect them to go.

All filtering rules on the branch, HQ and CHR…are disabled.

SRCnat rules with src and dst addresses have been added and the SRCnat rules are above the masquerade rule…branch to HQ works fine, but not CHR.

I even tried disabling the HQ tunnel, thinking that two tunnels might have a conflict, but to no avail.

Reading this statement.

using command ping src-address=10.79.15.100 192.168.9.110 = i am able to ping the mikrotik branch1 router. but unable to ping the host connected to the same router.

We can guess the remote VPN router IS NOT THE NETWORK GATEWAY ROUTER to the remote network :slight_smile:

Explaination:
Machine 192.168.9.252 will send all unknown traffic back to the NETWORK GATEWAY probably 192.168.9.1
Machine 192.168.9.252 has NFI about the 10.79.15.100 traffic

So you need a static route either on each machine you want to see or for all a static route on the NETWORK GATEWAY router.

Lets assume you want the later goto the NETWORK GATEWAY router which I have assumed is 192.168.9.1 and place a static route
static route 10.79.15.0/24 next-hop 192.168.9.110

Now the main gateway knows to forward all VPN traffic back to the VPN router.

Hi
Thats a good suggestion, but can you tell me the command as to how to add this next-hop thing… i found the option for it in WINBOX, but can’t see any option to add a hop..

The static route needs to go into main gateway router >>> NOT <<< the VPN router and so question is that also a mikrotik?

On a mikrotik command is
/ip route add dst-address=10.79.15.0/24 gateway=192.168.9.110

If the main router is something else you will need to work out how to add a static route.

I finally found an absolutely mind boggling solution to my problem… i restarted the router and restarted everything, and it still WAS NOT pinging, but suddenly it started working and then stopped and won’t come online again!! Until I found it!!

SOLUTION:
Untitled.png



Just removed the connection from here and the ping started working!!! Seems like the router was stuck and releasing this connection just made it try again and it worked!!

Had the same wierd issue this morning. Tunnel is ok, shows established, but traffic didn’t go throug.

I had restarted router 3 times, updatef firmware, triggered tunnel re-establishment form both sides.

Nothing helped, untill i removed everything from the connection tab.

It was working fine for 3 months, and today other side of our tunnel had some routing changed by ISP

It worked fine with other tunnel (non-mikrotik), but mikrotik side suddenly stopped all traffic going inside the tunnel.

The main question here for me is how the fuck connections persist through restart, this is really strange behavior.