How to realise NAT redirect to LAN client from VPN?

MatoZ · Sun Oct 14, 2018 2:32 am

Hello I use RouterOS 6.43 and I have this problem:

My router is connected to 3 networks:
- WAN - public dynamic IP
- LAN 192.168.3.0/24
- VPN (OpenVPN), router IP is 10.1.1.31

I need access to one LAN client with IP 192.168.3.25 via public IP of my VPN network (PUBLIC_VPN_IP) - PUBLIC_VPN_IP:PORT. So I have created iptables DNAT rule on my VPN server:

iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 3641 -j DNAT --to-destination 10.1.1.31

And on my Mikrotik router I created these rules:
table MANGLE:

add action=mark-connection chain=prerouting disabled=no dst-address=10.1.1.31 dst-port=3641 new-connection-mark=int_to_3641 passthrough=no protocol=tcp

table NAT:

add action=dst-nat chain=dstnat disabled=no dst-address=10.1.1.31 dst-port=3641 protocol=tcp to-addresses=192.168.3.25 to-ports=80
add action=masquerade chain=srcnat disabled=no out-interface=ether1
add action=masquerade chain=srcnat connection-mark=int_to_3641 disabled=no

But it does not work. Could you help me please?
Thank you.

Sob · Sun Oct 14, 2018 3:17 am

You have the masquerade for incoming connections in wrong place, it's useless on VPN client, it needs to be on VPN server. The problem is when client is e.g. 1.2.3.4, router won't send replies back via VPN, because as it sees it, route to 1.2.3.4 leads via default route. If you masquerade connections on VPN server, they will all look as from 10.1.1.x and that's reachable over tunnel.

Problem with the above is that you lose source addresses. If you need your server to see them, you need to skip masquerade completely. Mark new connections coming from tunnel, add another default route in different routing table, and finally mark routing for replies to use the new routing table.

MatoZ · Sun Oct 14, 2018 12:41 pm

@Sob, many thanks for reply. So
I created masquerade on VPN server.
I changed new-connection-mark=int_to_3641 on mark routing in mangle table.
And I added this ip route:

/ip route add dst-address=0.0.0.0/0 gateway=10.1.1.1 routing-mark=int_to_3641

But it still not work.
Could you help me please what change/add yet?
Thanks.

sindy · Sun Oct 14, 2018 5:31 pm

What @Sob tried to tell you was that the Mikrotik (openvpn client) must "note down" through which interface (WAN or VPN) the request to the dst-nated application server has arrived, and must use that information to assign the routing-mark to the responses of the server. So a mangle rule in chain prerouting must assign a connection-mark to packets towards the server address coming from the VPN interface; as connections are tracked, any further packet belonging to the same connection, regardless the direction, has the connection-mark as a matchable attribute. So another mangle rule in chain prerouting rule then translates the connection-mark into a routing-mark for the response packets sent by the server. Beware - it either must not assign it to further packets from client to server, or you must undo the assignment using /ip route rule, because routing-mark has priority even over connected routes. More details and an example here.

Sob · Sun Oct 14, 2018 7:23 pm

You need only one, masquerade OR routing. And if you choose routing, check PCC example. It's primarily about load balancing, so ignore that and only focus on the connection and route marking part. I really have to find some better example, but all I keep remembering is this one. And I'm too lazy to write same config again and again.

sindy · Sun Oct 14, 2018 7:35 pm

all I keep remembering is this one. And I'm too lazy to write same config again and again.

That's why I keep links to my posts on ever recurring themes in my browser's bookmarks and/or in bookmarks of my forum profile.

MatoZ · Sun Oct 14, 2018 9:50 pm

Thanks a lot. I am sorry, but I am only beginner with Mikrotik. So my config is:

nat:

chain=dstnat action=dst-nat to-addresses=192.168.3.25 to-ports=80 protocol=tcp dst-address=10.1.1.31 in-interface=ovpn-out1 dst-port=3641 log=yes log-prefix="X1"

mangle:

chain=prerouting action=mark-connection new-connection-mark=int_to_3641 passthrough=no protocol=tcp dst-address=10.1.1.31 in-interface=ovpn-out1 dst-port=3641 log=no log-prefix=""
chain=output action=mark-routing new-routing-mark=to_VPN1 connection-mark=int_to_3641

iproute:

/ip route add dst-address=0.0.0.0/0 gateway=10.1.1.1 routing-mark=to_VPN1 check-gateway=ping

But it still does not work. Where is mistake please?

Sob · Sun Oct 14, 2018 10:14 pm

Almost there, you want prerouting instead of output:

/ip firewall mangle
chain=prerouting in-interface=<server LAN> action=mark-routing new-routing-mark=to_VPN1 connection-mark=int_to_3641

MatoZ · Sun Oct 14, 2018 11:02 pm

Almost there, you want prerouting instead of output:
Code: Select all
/ip firewall mangle
chain=prerouting in-interface=<server LAN> action=mark-routing new-routing-mark=to_VPN1 connection-mark=int_to_3641

@Sob many thanks, but now I have interesting error on side of web browser (PUBLIC_IP_VPN:3641):

The connection has been reset. (ERR_CONNECTION_RESET).

In Mikrotik log I see that firewall rules are using.

sindy · Sun Oct 14, 2018 11:07 pm

Maťo, do you really need to dstnat twice? I mean, can't you add a route to 192.168.3.25 already at the openvpn server, and dstnat to that address already there?

MatoZ · Mon Oct 15, 2018 1:34 am

Maťo, do you really need to dstnat twice? I mean, can't you add a route to 192.168.3.25 already at the openvpn server, and dstnat to that address already there?

@sindy VPN server is outside LAN where is 192.168.3.25 and on this client is not possible directly connection to my VPN. Only Mikrotik router is directly connected to my VPN network. So it seems internet -> VPN server -> Mikrotik router -> client. Btw How do you know my name with accents?

Sob · Mon Oct 15, 2018 2:47 am

He's psychic, he knows everything! (Or maybe it could have something to do with your writing, the order of words, ...

)

But back to main topic, rules seem to be ok, so find out what exactly happens. You can add some logging rules, e.g:

/ip firewall mangle
add chain=prerouting protocol=tcp dst-address=10.1.1.31 in-interface=ovpn-out1 dst-port=3641 action=log log-prefix="0"
add chain=prerouting connection-mark=int_to_3641 action=log log-prefix="1"
add chain=forward connection-mark=int_to_3641 action=log log-prefix="2"
add chain=postrouting connection-mark=int_to_3641 action=log log-prefix="3"

Move them to the top and you should see incoming SYN, then SYN,ACK going back, etc.

MatoZ · Mon Oct 15, 2018 12:17 pm

@Sob thanks, log:
@sindy Are you from Central/Eastern Europe?

11:09:48 firewall,info 2 forward: in:ovpn-out1 out:bridge, proto TCP (SYN), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641->192.
168.3.25:80), len 52 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:bridge, proto TCP (SYN), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641
->192.168.3.25:80), len 52 
11:09:48 firewall,info 1 prerouting: in:bridge out:(unknown 0), src-mac MAC, proto TCP (SYN,ACK), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.
168.3.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 52 
11:09:48 firewall,info 2 forward: in:bridge out:ovpn-out1, src-mac MAC, proto TCP (SYN,ACK), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.168.3
.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 52 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:ovpn-out1, src-mac MAC, proto TCP (SYN,ACK), 192.168.3.25:80->PUBLIC_IP:2659, NAT (
192.168.3.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 52 
11:09:48 firewall,info 2 forward: in:ovpn-out1 out:bridge, proto TCP (ACK), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641->192.
168.3.25:80), len 40 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:bridge, proto TCP (ACK), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641
->192.168.3.25:80), len 40 
11:09:48 firewall,info 2 forward: in:ovpn-out1 out:bridge, proto TCP (ACK,PSH), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641->
192.168.3.25:80), len 489 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:bridge, proto TCP (ACK,PSH), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:
3641->192.168.3.25:80), len 489 
11:09:48 firewall,info 1 prerouting: in:bridge out:(unknown 0), src-mac MAC, proto TCP (ACK), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.168.
3.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 40 
11:09:48 firewall,info 2 forward: in:bridge out:ovpn-out1, src-mac MAC, proto TCP (ACK), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.168.3.25:
80->10.1.1.31:3641)->PUBLIC_IP:2659, len 40 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:ovpn-out1, src-mac MAC, proto TCP (ACK), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.
168.3.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 40 
11:09:48 firewall,info 2 forward: in:ovpn-out1 out:bridge, proto TCP (RST), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641->192.
168.3.25:80), len 40 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:bridge, proto TCP (RST), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641
->192.168.3.25:80), len 40 
11:09:48 firewall,info 2 forward: in:ovpn-out1 out:bridge, proto TCP (ACK,PSH), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641->
192.168.3.25:80), len 489 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:bridge, proto TCP (ACK,PSH), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:
3641->192.168.3.25:80), len 489 
11:09:48 firewall,info 1 prerouting: in:bridge out:(unknown 0), src-mac MAC, proto TCP (RST), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.168.
3.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 40 
11:09:48 firewall,info 2 forward: in:bridge out:ovpn-out1, src-mac MAC, proto TCP (RST), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.168.3.25:
80->10.1.1.31:3641)->PUBLIC_IP:2659, len 40 
11:09:48 firewall,info 3 postrouting: in:(unknown 0) out:ovpn-out1, src-mac MAC, proto TCP (RST), 192.168.3.25:80->PUBLIC_IP:2659, NAT (192.
168.3.25:80->10.1.1.31:3641)->PUBLIC_IP:2659, len 40

sindy · Mon Oct 15, 2018 12:58 pm

The log shows that the first RST came from client side:

11:09:48 firewall,info 2 forward: in:ovpn-out1 out:bridge, proto TCP (RST), PUBLIC_IP:2659->192.168.3.25:80, NAT PUBLIC_IP:2659->(10.1.1.31:3641->192.168.3.25:80), len 40

Other than that, there is nothing in the log which would give a clue what has made the client side reset the connection. Be aware that it may not be an activity of the client itself, it can as well be some firewall on the path between the client and the Tik, so the first place I'd sniff next would be the OpenVPN server (tcpdump on both the internet-facing interface and the OpenVPN TAP).

And yes, you're right about the reason why I could guess the native spelling of your nickname. Sme obaja, ty i ja, made in... (well, sure in my case, not sure in yours

)

MatoZ · Mon Oct 15, 2018 3:11 pm

@sindy thanks, so I have pcap files, what way is better for share with you?
@sindy in Slovakia

Poznáme sa?

sindy · Mon Oct 15, 2018 4:17 pm

There is little point in sharing the files, you should be able to see on your own from them whether first RST came from the client side or whether it was the OpenVPN server's firewall which disliked the connection and sent the RST actively to both sides. If you don't have Wireshark, tcpdump itself can be used to filter the files:
tcpdump -r file-name.pcap tcp port 3641
should show you something like
18:03:45.219750 IP 192.168.32.26.57013 > 10.155.3.76.5900: Flags [P.], ......
You are interested in the source IP address on the very first line where R can be found in the list following the Flags keyword.
The further analysis steps, as always, are to identify the box which has sent the RST (so the next capture point is the client PC), and once you identify it, to find out why it disliked the connection.
A useful hint: Wireshark (tcpdump) always tells you what has happened. Much more rarely it tells you why it has happened. This is a question which only application logs can answer (but yes, sometimes there are none). My personal guess is that some anti-virus software on your PC doesn't like a TCP connection to a weird port number, but it is nothing more than a guess.

You seem not to know the same song. So I'm a tiny bit further to the west. And none of the three Matos I know is Z.

MatoZ · Mon Oct 15, 2018 8:52 pm

@sindy Thanks. Of course I have Wireshark. So during sniffing on VPN server tun0 interface:

MY_PUBLIC_IP send TCP SYN to -> 10.1.1.31 
10.1.1.31 -> MY_PUBLIC_IP  (SYN, ACK)
MY_PUBLIC_IP  -> 10.1.1.31 (ACK)
TCP connection is established.
MY_PUBLIC_IP  -> 10.1.1.31 (HTTP GET) still to port 3641
10.1.1.31 -> MY_PUBLIC_IP  (ACK)
And now
MY_PUBLIC_IP  -> 10.1.1.31 send RST. 
MY_PUBLIC_IP  -> 10.1.1.31 (TCP Spurious Retransmission).
10.1.1.31 -> MY_PUBLIC_IP  (RST).

During snffing on VPN server eth0
I see the same only with public IP of my VPN instead of 10.1.1.31. And I see more TCP PSH packets within a connection.
I disabled all security tools on my PC and my colleague tried it too from your PC with the same result. On VPN server is configured iptables, fail2ban and PSAD (both use only iptables rules) I can post here export of my iptables rules.
Any idea please?

I know this song, it occurred to me, but phrase "Sme obaja, ty i ja" is really specific only for Slovak language... So you are from Czech republic

But the song is true because I was born in last year existence of Czechoslovakia, you maybe too

Z is the first letter of my surname. I would send you my full name, but this forum does not support private messages...

sindy · Mon Oct 15, 2018 9:40 pm

From your sniffs and the log taken previously on your 'Tik, it comes out that the client sends its GET, receives an ACK for it (which it may or may not like, there's still a chance that something is wrong with the contents of that ACK but that's still what Wireshark should show you (it would be too much for tcpdump, it already needs protocol dissection only available in Wireshark) and then a reset arrives from the client side. But the browser at client PC gives a message which suggests that it has also received the reset, not sent it.

As the reset came from the client side quite fast, prior to any retransmission of the GET, it means that either the ACK has made it to the sender of the RST (whatever it is) or the RST was triggered already by the downstream GET itself. In any case, there is nothing to look for at the OpenVPN server. If you say that you've switched off all the security software on the client PC, what remains is some device on the path between the client PC and the OpenVPN's public address. So as said before, the next capturing point should be the PC itself´to confirm this assumption.

If the "spurious retransmission" after the first RST is actually a retransmission of the GET (check raw packet data and/or the syn and ack values, Wireshark won't dissect http in a retransmitted packet), it is almost sure that the RST has been sent by something else than the client itself.

PSH packet as such is nothing unusual, TCP is a stream protocol with buffering so PSH is used to tell the receiving side "stop buffering and push what you've accumulated in the buffer so far to the application right now", i.e. in most cases to indicate an end of a PDU carrying a request which needs an immediate response.

The phrase only appears in the song in Slovak so writing it in Czech would not have been a reference to the song

Private messages on this forum are disabled on purpose, because otherwise everyone would send PMs to Normis

MatoZ · Mon Oct 15, 2018 11:34 pm

@sindy during sniffing with Wireshark on my PC and on PC of my colleague (he has other ISP) RST sometimes comes from my VPN server public IP. So it seems:

Successful TCP handshake 
HTTP GET from client
TCP SYN, ACK from VPN server
TCP ACK from client
HTTP GET /robots.txt from client
TCP retransmission from client 
TCP RST from VPN server

I can send you my pcap files.

Good move with this song.

It is a big pity with PMs.

sindy · Mon Oct 15, 2018 11:52 pm

We're getting very far from Mikrotik. The RST you can see at the client side can be both the one forwarded from the server side as well as one generated by the firewall somewhere between, but the fact that it comes instead of the ACK for the GET, which you could still see at the internet-facing interface of the OpenVPN server, suggests that there is something between the client and the OpenVPN server which sends the reset towards the server in the name of the client and has dropped the ACK in the server->client direction.

You can post the pcap anywhere and place a link to it here, but it will be accessible to everyone, so if you don't want to leak the public IP of the openvpn server, you'll have to analyse the ACK packet yourself. It should not be a big deal, though. Just check that the seq and ack numbers properly match the state of the connection after the GET; if they do, the security equipment probably doesn't like the GET itself, otherwise it may tear down the connection because there is a mistake in the ACK fields; in that case you would have to capture the same connection at all places (the two interfaces of the OpenVPN server, the Mikrotik, and the server) to find out which of those devices has messed the fields.

MatoZ · Tue Oct 16, 2018 1:14 am

@sindy RST has seq out of order previous packets and does not have ack number. So I have to capture the same connection at all places...

MatoZ · Tue Oct 16, 2018 2:06 am

@sindy after sniffing on all places I see interesting thing:
192.168.3.25 uses HTTP authorization dialog, 401 Unauthorized do not contain all captures.
Mikrotik capture is:

1. Successful TCP handshake
2. MY_IP_ADD -> 10.1.1.31 HTTP GET
3. 10.1.1.31 -> MY_IP_ADD ACK
4. 10.1.1.31 -> MY_IP_ADD HTTP 401 Unauthorized
5. 10.1.1.31 -> MY_IP_ADD FIN, ACK
6.  MY_IP_ADD -> 10.1.1.31 RST 3x
9.  10.1.1.31 -> MY_IP_ADD RST

Me - so my PC MY_IP_ADD:

1. Successful TCP handshake
2. MY_IP_ADD -> MY_VPN_PUBLIC_IP HTTP GET
3. MY_IP_ADD -> MY_VPN_PUBLIC_IP 503 TCP retransmission
4. MY_VPN_PUBLIC_IP -> MY_IP_ADD RST

tun0 capture on VPN server:

1. Successful TCP handshake
2. MY_IP_ADD -> 10.1.1.31 HTTP GET
3. 10.1.1.31 -> MY_IP_ADD ACK
4. MY_IP_ADD -> 10.1.1.31 RST
5. 10.1.1.31 -> MY_IP_ADD HTTP 401 Unauthorized
6. 10.1.1.31 -> MY_IP_ADD FIN, ACK
7. MY_IP_ADD -> 10.1.1.31 RST
8. MY_IP_ADD -> 10.1.1.31 TCP Spurious Retransmission GET
9. 10.1.1.31 -> MY_IP_ADD RST

eth0 capture on VPN server:

1. Successful TCP handshake
2. MY_IP_ADD -> MY_VPN_PUBLIC_IP HTTP GET
3. MY_IP_ADD -> MY_VPN_PUBLIC_IP 503 TCP retransmission
4. MY_VPN_PUBLIC_IP -> MY_IP_ADD RST

MatoZ · Sat Oct 20, 2018 6:25 pm

@sindy, @Sob any idea please?

sindy · Sat Oct 20, 2018 7:16 pm

Well, as you were providing captures and logs taken during different attempts, I've missed that not all packets made it from tun to eth on the openvpn server (namely, the 401 from the server didn't get through the server). So once again, tcpdump on the openvpn server at both interfaces simultaneously (either using -i any or by running two tcpdumps in parallel, each sniffing on its own port into its own file, and then merge the files which likely requires that you capture into pcapng format as packets from tun won't have Ethernet as link layer), and then check whether the RST from the client end came just in time to prevent the 401 from being forwarded or whether the openvpn server has dropped it on its own.

And one more once again, the goal is to find out which box is responsible for the RST, and then investigate that box.

MatoZ · Sat Oct 20, 2018 11:19 pm

@sindy, thanks. I exported all relevant packets from dumps to txt files and replaced IP and MAC addresses, so here are:
Mikrotik: https://www.dropbox.com/s/pe6ib0tik4wjc ... k.txt?dl=0
VPN server - all interfaces: https://www.dropbox.com/s/no5xa35laxi34 ... t.txt?dl=0
Client - my PC: https://www.dropbox.com/s/3awc34vc9uq9n ... C.txt?dl=0
In this time I am on the same local network with my PC like is 192.168.3.25 (Of course I accessed via VPN_SERVER_PUBLIC_IP:3641), 401 Unauthorized did not appear this time.
Many thanks.

sindy · Sat Oct 20, 2018 11:59 pm

Having the client at the same LAN like the server is normaly a voucher for a headache, but here it seems you have enough NATs in the path to escape that.

Your captures show that the client PC (192.168.3.2) does not send the first RST but something between the Mikrotik's WAN and the OpenVPN server's public IP sends it in its name. So I expect some security device out of your control to do that. I don't know where the OpenVPN server is located, but I'd assume it is that company's firewall.

I would try to replace 3641 by some other port in the complete chain by something like 8080 to see whether that behaviour is linked to port 3641 which is IANA-registered for another protocol than http (netplay port 2).

MatoZ · Sun Oct 21, 2018 1:34 am

@sindy VPN server is located at Czech VPS server. I tried ports 80, 8080 and one dynamic port too but result is the same.

Have you any idea for other technical solution please?

sindy · Sun Oct 21, 2018 12:46 pm

Try to change the setup in such a way that the Mikrotik doesn't redirect the incoming connections to 3641 to 192.168.3.25:80 but to its own port 22, enable ssh on the Mikrotik if not enabled, and connect from the client PC using a ssh client, to see whether the filtering device is only nervous about http or about any traffic. If that helps, try to configure https on the server. The hosting may have some anti-malware measures in place, preventing plaintext http connections from setting up in general.

MatoZ · Mon Oct 22, 2018 12:39 am

@sindy thanks. I tried change port to Mikrotik SSH but result is the same. So it is about any traffic

But I have next server with public IP located at other Czech VPS provider with different technical solution (virtualization etc.) connected to my VPN network. So tomorrow I will try DNAT via this server to Mikrotik and 192.168.3.25:80.

sindy · Mon Oct 22, 2018 12:47 am

10 years ago I was organizing server hosting in Prague for a Serbian company as nothing useful could have been found in Belgrade, but in 2018 for use from Slovakia - carrot in winter? Are the prices so different?

MatoZ · Mon Oct 22, 2018 12:56 am

@sindy I do not know what now but before 3-4 years when I bought it it was very different. On Slovakia only few companies originally offered VPS servers and prices were considerably higher... In addition many companies offer their services for both countries, they consider it as single market, but they have infrastructure in Czech republic. To be exact one from my VPS servers are originally operated by Italians, they have servers across Europe but not at Slovakia.

MatoZ · Tue Oct 23, 2018 12:16 am

@sindy I removed DNAT from MY_VPN_SERVER and set the same DNAT rule to 10.1.1.31:3641 on other server which is connected to my VPN (10.1.1.20) an has static public IP address. iptables filter chain FORWARD is set to ACCEPT I enabled ip_forward too. In iptables I see that packets are forwarded. On Mikrotik I use the same configuration like with DNAT from MY_VPN_SERVER. But traffic doest not pass to Mikrotik. During sniffing on Mikrotik I do not capture any packets from server, but sniffing on server shows outgoing packets to 10.1.1.31. Please have you any idea where can be problem?

client MyPC - https://www.dropbox.com/s/bwuu2vgd6tanj ... C.txt?dl=0
Server with public IP - https://www.dropbox.com/s/mrd7muw41eh6w ... t.txt?dl=0
Mikrtoik - nothing relevant

sindy · Tue Oct 23, 2018 12:28 am

I don't understand your network topology. The mikrotik is an opevnpn client of two servers, right? So it has a different IP address from each of them?

OpenVPN on linux is tricky in the sense that if you want to route packets to some subnet behind the client, it is not enough to set a route to that subnet via the client's IP in the kernel routing table but you also have to set it in client configuration file for openvpn, using iroute. So maybe this is your issue?

MatoZ · Tue Oct 23, 2018 12:39 am

10.1.1.1 (VPN_SERVER_PUBLIC_IP) is OpenVPN server with public IP, when I access via VPN_SERVER_PUBLIC_IP:3641 it cause RESET.
10.1.1.20 (SERVER_PUBLIC_IP) is other server with public IP connected to my VPN network (VPN client), when I access via SERVER_PUBLIC_IP:3641 traffic does not pass on Mikrotik
10.1.1.31 is Mikrotik (VPN client) which connect LAN 192.168.3.0/24 (gateway 192.168.3.254) where is located client 192.168.3.25 where I need access from internet.

MatoZ · Tue Oct 23, 2018 11:38 pm

@sindy I looked for some examples with iroute, but I can ping 10.1.1.31 from 10.1.1.20. I do not access directly form one VPN client to other client outside one network (subnet) for example 192.168.3.25. I access only to 10.1.1.31 and there I DNAT traffic to 192.168.3.25. Maybe I understand badly, could you provide example for my topology please?

sindy · Tue Oct 23, 2018 11:58 pm

That's the point, in the absence of a picture, I've taken a wrong assumption about your topology from your problem description.

So provide a picture (a photo of a handmade drawing is enough but don't forget the subnets, vpn roles etc.) and bear in mind that this is a Mikrotik forum while your issue has gradually evolved into a generic network troubleshooting one.

MatoZ · Wed Oct 24, 2018 12:46 am

@sindy, picture: https://www.dropbox.com/s/74lg09ss34l01 ... k.png?dl=0

sindy · Wed Oct 24, 2018 9:51 am

As you use tun interfaces, the fact that both the Mikrotik and the other VPN client get their own IP from 10.1.1.0/x doesn't make them visible for each other one without a route, so what routes have you configured for them in their config files on the openvpn server? The Mikrotik needs to get a route to the other client's IP via the OpenVPN tunnel and vice versa. On Mikrotik, you can try /ip route check 10.1.1.20 to see whether it is fine or not.

MirhosseiniAmir · Wed Oct 24, 2018 2:50 pm

Hi every one
I have a problem with 6.42.9 long term and I had with 6.43 too.
I have some mangles for route traffic for internet gateway, have 3 internet lines. Also using MK for dns too.
The problem is that when I update the ROS there will be a connection to all DNS IP addresses connections from every interface with no mangle, although I have a rule in route that says the gateway for dns is what interface, BUT when I downgrade it the problem is solved.
You can see in picture.

MatoZ · Wed Oct 24, 2018 3:14 pm

@sindy: from Mikrotik 10.1.1.31 to Server 10.1.1.20:

/ip route check 10.1.1.20
     status: ok
  interface: ovpn-out1
    nexthop: 10.1.1.1

@MirhosseiniAmir please create your own topic.

sindy · Wed Oct 24, 2018 3:18 pm

@MatoZ, if you ping from Mikrotik to 10.1.1.20, do you get responses? And if you ping in the opposite direction, do you also get responses? Of course assuming that ICMP is permitted on all three devices' firewalls.

MatoZ · Wed Oct 24, 2018 5:22 pm

@sindy:
Ping form server 10.1.1.20 to Mikrotik 10.1.1.31

ping 10.1.1.31
PING 10.1.1.31 (10.1.1.31) 56(84) bytes of data.
64 bytes from 10.1.1.31: icmp_seq=1 ttl=64 time=49.6 ms
64 bytes from 10.1.1.31: icmp_seq=2 ttl=64 time=49.6 ms
64 bytes from 10.1.1.31: icmp_seq=3 ttl=64 time=49.5 ms
64 bytes from 10.1.1.31: icmp_seq=4 ttl=64 time=49.8 ms
64 bytes from 10.1.1.31: icmp_seq=5 ttl=64 time=49.4 ms
64 bytes from 10.1.1.31: icmp_seq=6 ttl=64 time=49.2 ms
^C
--- 10.1.1.31 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5007ms
rtt min/avg/max/mdev = 49.270/49.555/49.809/0.311 ms

Ping form Mikrotik - 10.1.1.31 to server 10.1.1.20

[mato@MikroTik] > ping 10.1.1.20
  SEQ HOST                                     SIZE TTL TIME  STATUS             
    0 10.1.1.20                                  56  64 119ms
    1 10.1.1.20                                  56  64 48ms 
    2 10.1.1.20                                  56  64 49ms 
    3 10.1.1.20                                  56  64 49ms 
    4 10.1.1.20                                  56  64 49ms 
    5 10.1.1.20                                  56  64 48ms 
    sent=6 received=6 packet-loss=0% min-rtt=48ms avg-rtt=60ms max-rtt=119ms

sindy · Wed Oct 24, 2018 6:02 pm

In that case I have no idea what is wrong. Could you tcpdump on the tun of the OpenVPN server looking towards the Mikrotik whether the TCP packets for 10.1.1.31 are there when you try to connect to the other virtual server's public IP address?

MatoZ · Wed Oct 24, 2018 6:08 pm

@sindy I will try it again, but I posted here dumps yet and yes these packets are there. I see traffic in iptables too, but Mikrotik does not receive it.

sindy · Wed Oct 24, 2018 6:21 pm

Traffic in iptables shows nothing as the packet is counted before it is sent to the interface.

If tcpdump on openvpn server's tun interface representing the tunnel to Mikrotik shows the packet and sniffing at Mikrotik's ovpn-out1 done at the same doesn't show it, the problem is in openvpn or in sniffing, hard to say which one is true. While tcpdumping/sniffing while trying the TCP connection, do the ping as well so that you could see whether the sniffing is a problem or whether some packets get through and some don't.

The way back from the server is much more complicated - Mikrotik uses the openvpn interface to route the response thanks to either src-nat at one of the virtual servers or connection marking; if src-nat is used at the virtual server which is an openvpn client, the openvpn at the openvpn server will deliver the response packets properly, otherwise it won't as the iroute to 0.0.0.0/0 would have to be in that client's configuration which is not a good idea.

But from your description I've understood that even the SYN doesn't make it to the Mikrotik so it is too early to deal with the above.

MatoZ · Thu Oct 25, 2018 1:07 am

@sindy ping pass successfully, but traffic for port 3641 not. I see SYN TCP packets in tun0 dump, but not on Mikrotik. I do not understand.

Dumps:
Server tun0 - https://www.dropbox.com/s/ixi9njb2cmdat ... 0.txt?dl=0
Mikrotik - https://www.dropbox.com/s/ixnoi59ufh1wd ... p.txt?dl=0

sindy · Thu Oct 25, 2018 2:26 am

I can see that the SYN packets are being sent with the public IP address as source. I can easily imagine OpenVPN not to actually encrypt and send packets from source addresses for which there is no backward route for that client. So use iptables at the server acting as VPN client to src-nat connections towards 10.1.1.31 to its own 10.1.1.20 when sending them to the openvpn server and see the outcome.

MatoZ · Thu Oct 25, 2018 2:14 pm

@sindy Great! SRC-NAT resolved this. Many, many, many thanks!

Who is online