Cannot access webservers through vpn that are on a different gateway

RouterOS General
1

Hi to all,
I cannot figure out this, furthermore I’m not sure which side is the culprit.
I have in my central offices two routers:

  • the first one is a mikrotik CCR1009-7G-1C-1S+ connected to internet through a very fast optical fiber provider(pppoe interface with static IP), this router has local IP 192.168.25.62. This router is an IPSEC/l2tp server for another office that is using a mikrotik 951G-2HnD and a very good VDSL(here due the provider limitations, I have a double NAT setup, so I’m using the providers router 192.168.1.1 as a gateway for the mikrotik 192.168.26.1). Everything works and I can ping and reach everything on the other side like webservers, filshares etc and I have a very good performance


  • the second one is virtualized pfsense connected to internet through another very fast optical fiber provider(pppoe interface with static IP), this router is a gateway only for a specific debian web sever 192.168.25.136 and a freepbx/asterisk distro 192.168.25.200. This router has local ip 192.168.25.76.

I have setup all routes and returning routes on both mikrotiks and pfsense and I can ping from remote mikrotik 192.168.26.1 the debian webserver and freepbx, I can even register the phones to the freepbx server and I can reach the pfsense interface.. but for some reason I cannot open with my browser both the debian web server and the freepbx distro. Sometimes randomly it works but after some minutes again I cannot reach any web server. I was thinking about mtu on the remote mikrotik side, but nothing seems to change lowering down the mtu values even if I change the values directly on the ethernet interface of a workstation. I have tried to add pass input/forward rules to see if something changes but with no results.

So the problem is that I can ping from both sides through the vpn, but I cannnot reach web servers that are using a diffent gw, only randomly it works.

Any idea how to fix this?
many thanks

2

this is what I got on the webserver, apache logs are empty but tcpdump shows this:

16:10:26.764026 IP 192.168.26.20.51372 > freepbx.sangoma.local.http: Flags [S], seq 976993469, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:26.764084 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:27.014422 IP 192.168.26.20.51373 > freepbx.sangoma.local.http: Flags [S], seq 2419313814, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:27.014452 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:27.764852 IP 192.168.26.20.51372 > freepbx.sangoma.local.http: Flags [S], seq 976993469, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:27.764895 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:28.014830 IP 192.168.26.20.51373 > freepbx.sangoma.local.http: Flags [S], seq 2419313814, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:28.014855 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:28.766953 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:29.166956 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:29.765511 IP 192.168.26.20.51372 > freepbx.sangoma.local.http: Flags [S], seq 976993469, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:29.765537 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:30.015698 IP 192.168.26.20.51373 > freepbx.sangoma.local.http: Flags [S], seq 2419313814, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:30.015742 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:31.766918 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:32.166957 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:33.765739 IP 192.168.26.20.51372 > freepbx.sangoma.local.http: Flags [S], seq 976993469, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:33.765783 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:34.016993 IP 192.168.26.20.51373 > freepbx.sangoma.local.http: Flags [S], seq 2419313814, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:34.017020 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:37.766944 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:38.166943 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:41.765829 IP 192.168.26.20.51372 > freepbx.sangoma.local.http: Flags [S], seq 976993469, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:41.765859 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:42.017096 IP 192.168.26.20.51373 > freepbx.sangoma.local.http: Flags [S], seq 2419313814, win 64240, options [mss 1380,nop,wscale 8,nop,nop,sackOK], length 0
16:10:42.017117 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:49.966944 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:10:50.366943 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:11:05.966987 IP freepbx.sangoma.local.http > 192.168.26.20.51372: Flags [S.], seq 1214405327, ack 976993470, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:11:06.366951 IP freepbx.sangoma.local.http > 192.168.26.20.51373: Flags [S.], seq 1721642903, ack 2419313815, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
3

a very strange think is that if I run a ping or a tracert from a workstation to the remote web server, I will have access for some minutes to that specific webserver.. really strange look like something with opened states

4

What I noticed now is that if I masquerade the remote networks on the central mikrotik behind the lan interface, things works. But honestly I’m not sure that masquerading the remote lan is a good practice.