Im struggling with this for a while.. My goal is to “map” IPv4 from external network (ExternalHost) to my server “End client” (Centos7) located in different network (HomeCloud).
Both boxes are in different locations, so I had to set up an IPsec tunnel between those locations. I deployed CHR instance in each location: EXTGW2 at ExternalHost and INTGW2 at HomeCloud.
So far I managed to work it this way:
Internal IP (95.168.x.x) works both ways (incoming/outcoming packets)
External IP (185.135.x.x) works only for incomming connections. For example it is possible to connect via SSH using 185.135.8.1 and you are connected to “End client” at HomeCloud
Its NOT possible to use 185.135.x.x for outcoming connection.
To get a full picture Im pasting my complete commented configuration of both CHR boxes and “End client”. Image of network setup is also included.
The main question is what did I miss? What should I do to make 185.135.x.x work for incoming and outcoming connection on Centos7 end client?
Any help or hint is very appreciated.
## Legend:
## 154.16.x.x - EXTGW2.domain.com (ExternalHost) - Primary IP
## 185.135.x.x - EXTGW2.domain.com (ExternalHost) - Secondary IP - this one I want to route inside HomeCloud and use on "End client" as secondary IP
## 172.16.x.x - EXTGW2.domain.com (ExternalHost)
## 95.168.x.x - INTGW2.domain.com PUBLIC IPv4 (HomeCloud)
## 192.168.x.x - INTGW2.domain.com LOCAL IPv4 (HomeCloud)
## 172.16.x.x - INTGW2.domain.com LOCAL IPv4 (HomeCloud)
## 95.168.x.x - End client (HomeCloud)
## CHR config
## EXTGW2.domain.com site (ExternalHost)
## External IPv4 address resides here
/interface ipip
add allow-fast-path=no ipsec-secret=xxxxxxxyyyyyyyyzzzzzzz local-address=154.16.x.x name=ipip-tunnel remote-address=95.168.xx
/ip address
add adrdess=154.16.x.x/24 interface=ether1 # Main EXTGW2 address
add address=172.16.x.x/30 interface=ipip-tunnel # Local ipip-tunnel address on EXTGW2
/ip arp
add address=185.135.x.x interface=ether1 published=yes # Another IP - this IP we want to route in to HomeCloud
/ip route
add dst-address=185.135.x.x/32 gateway=172.16.x.x
# SRCNAT
/ip firewall nat
add action=src-nat chain=srcnat comment="ExternalHost SNAT -- 154.16.x.x via ether1 (used with ipip tunnel)" out-interface=ether1 src-address=154.16.x.x to-addresses=154.16.x.x
## CHR config
## INTGW2.domain.com site (HomeCloud)
## tunnel to EXTGW2
/interface ipip
add allow-fast-path=no ipsec-secret=xxxxxxxyyyyyyyyzzzzzzz local-address=95.168.x.x name=ipip-tunnel remote-address=154.16.x.x
/ip address
add address=172.16.x.x/30 interface=ipip-tunnel
add address=192.168.x.x/32 interface=ether1 network=185.135.x.x
/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address=185.135.x.x in-interface=ipip-tunnel new-connection-mark=server-public passthrough=no
add action=mark-connection chain=prerouting connection-mark=no-mark new-connection-mark=server-public passthrough=yes src-address=185.135.x.x
add action=mark-routing chain=prerouting connection-mark=server-public in-interface=ether1 new-routing-mark=to-extgw2 passthrough=no
/ip route
add dst-address=0.0.0.0/0 gateway=172.16.x.x routing-mark=to-extgw2
## End client (Centos7) configuration:
## Need to configure 2 NIC in Vcenter for each host
## Actual state:
## Internal IP (95.168.x.x) works both ways (incoming/outcoming packets)
## External IP (185.135.x.x) works only for incomming connections. For example it is possible to connect via SSH using 185.135.x.x and you are conected to "End client" at HomeCloud
## Its NOT possible to use 185.135.x.x for outcoming connection. For example:
## ping -I 185.135.x.x google.com
## PING google.com (172.217.x.x) from 185.135.x.x : 56(84) bytes of data.
## --- google.com ping statistics ---
## 8 packets transmitted, 0 received, 100% packet loss, time 6999ms
## Taceroute ends up at EXTG2W.domain.com:
## traceroute -i ens160 google.com
## traceroute to google.com (172.217.23.206), 30 hops max, 60 byte packets
## 1 192.168.1.1 (192.168.1.1) 0.465 ms 0.366 ms 0.400 ms
## 2 172.16.1.2 (172.16.1.2) 8.006 ms 7.963 ms 8.022 ms
## 3 * * *
## 4 * * *
## 5 * * *
## 6 * * *
## 7 * * *
## 8 * * *
## 9 * * *
What about firewall filter on ExternalHost? Is it possible that after doing all this (which looks ok) you made a stupid mistake and didn’t allow new connections from ipip-tunnel to ether1?
Sob,
you pushed me the right way! I made that stupid mistake in SRCNAT rule at EXTGW2. When I do it this way:
/ip firewall nat
add action=src-nat chain=srcnat comment="SNAT -- Redirecting traffic FROM Home cloud TO External Cloud. Ether1 outgoing interface (ie. to the internet)" out-interface=ether1 src-address=185.135.x.x to-addresses=154.16.x.x
So now Im “forwarding” 185.135.x.x to-addresses=154.16.x.x which is mapped to ether1. Now from “End client” Im able to do this:
ping -I 185.135.x.x google.com
PING google.com (172.217.23.206) from 185.135.8.1 : 56(84) bytes of data.
64 bytes from prg03s05-in-f206.1e100.net (172.217.23.206): icmp_seq=1 ttl=52 time=15.8 ms
64 bytes from prg03s05-in-f206.1e100.net (172.217.23.206): icmp_seq=2 ttl=52 time=15.9 ms
64 bytes from prg03s05-in-f206.1e100.net (172.217.23.206): icmp_seq=3 ttl=52 time=16.0 ms
64 bytes from prg03s05-in-f206.1e100.net (172.217.23.206): icmp_seq=4 ttl=52 time=16.4 ms
And that is big improvement But there is still one glitch.. When I do this from “End client”
# tail -f /var/log/auth.log |grep ssh
Jan 24 21:58:40 web1 sshd[5946]: Accepted password for user from 154.16.x.x port 54537 ssh2
And thats the issue. Packets went out from EXTGW2 via ether1 interface which have public IP address 154.16.x.X
My goal is to connect via IP 185.135.x.x. I tried to map 185.135.x.x to ether1 interface at EXTG2 but this just break things (I was unable to connect to “End Client”) via 185.135.x.x.
Do you have any tips how to bypass this?
Thank you very much for your help so far.
Regards,
Jiří
Easy, don’t use srcnat for this public address at all. If you do need some other srcnat(s) on ExternalHost and it would also affect this address, then just exlude it (place before other srcnat rules):
If the accept rule is now the only rule in srcnat chain, then you don’t need it at all. It was meant only as exception from other “global” rule(s), if there would be some.
I’m not sure if I understand this:
Anyway, I’d try to look at packets. You can use either Tools->Torch or logging rules in firewall. Keep a ping running (e.g. ping -I 185.135.8.1 8.8.8.8) and check ipip-tunnel, if you see packets from 185.135.8.1 to 8.8.8.8, then if you see them leaving out via ether1, if you see responses coming back, etc. In other words, go step by step, you know what should be where, so just try to find where exactly it breaks.
I finaly manage it to work!
Cause was really stupid (as you assumed at very beginning. All I had to to was disable that SRCNAT rule (realy not needed).. And.. here it comes … RESTART whole RouterOS..
I came on this by accident. I managed to cut off my CHR completely.. so I had to do a restore and reboot from virtual console (its quite difficult to get there at my curtrent provider) and suddenly..
Uff.. so many many hours of pain… ahhh.. I’m getting realy drunk today!
Anyway. Thank you very much again for your effort here. You taught me a lot! Can I give you a kudos or something like that?
What should I do to make 185.135.x.x work for incoming and outcoming connection on Centos7 end client?
