Community discussions

MikroTik App
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Inside server not seeing external user's IP

Sun Aug 29, 2021 2:57 am

I'm a brand new MikroTik user, so please be gentle. :-)
I have a network that is presently working perfectly, with a firewall made from a PC running Linux (Ubuntu 20.04) with IPTables.
I am trying to duplicate it with a MikroTik CRS305-1G-4S+, starting with a very simple subset of the configuration.
I have entered the rules via the WebFig interface, so it is totally possible that I did it wrong there. I would appreciate
knowing how to correct this problem using the WebFig interface, if possible. I would also appreciate an explanation
of what I did wrong, since I am trying to learn to use WebFig.

The ISP has provided us with a /29 (x.y.z.128/29) with 5 usable static IPs: x.y.z.130 to x.y.z.134 and the gateway
to their network is x.y.z.129

Our internal network is the private network 172.31.0.0/16 with NO DHCP. Everything is assigned a private static IP.
The CRS305 internal interface is assigned 172.31.1.4.

We wish to have all non-server internal PCs have their traffic masqueraded by the CRS305, and this feature appears
to be working OK.

Our problem is with incoming connections to the servers. I will simplify this by describing only our first test case:

We want certain external users to SSH to the server, using the public IP x.y.z.132 and have that NATed to the
internal IP 172.31.220.4. That works, but with one problem: the address the server sees the connection
coming from is the MikroTik box at 172.31.1.4 instead of the external user's IP.

This royally screws up the server's intrusion detection software, which counts the number of failed login attempts
in the last hour from each IP and locks it out after a certain number of failures. This works fine if it sees the
external user's (hacker's) IP, but by seeing everyone coming from 172.31.1.4, EVERYONE gets locked out.

What should I do so that the server sees the connection coming from the external user's IP?

Here is the current configuration shown by Export:

# jan/02/1970 02:22:57 by RouterOS 6.48.3
# software id = ES33-601A
#
# model = CRS305-1G-4S+
# serial number = B9EA0EEFE310
/interface bridge
add admin-mac=2C:C8:1B:58:25:75 auto-mac=no comment=defconf name=bridge
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge comment=defconf interface=ether1
add bridge=bridge comment=defconf interface=sfp-sfpplus1
add bridge=bridge comment=defconf interface=sfp-sfpplus2
add bridge=bridge comment=defconf interface=sfp-sfpplus3
add bridge=bridge comment=defconf interface=sfp-sfpplus4
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=172.31.1.4/16 comment="Inside network" interface=sfp-sfpplus1 network=172.31.0.0
add address=x.y.z.130/29 comment="Connection to ISP: Route" interface=ether1 network=x.y.z.128
add address=x.y.z.132/29 comment="Connection to ISP: Slug" interface=ether1 network=x.y.z.128
/ip firewall nat
add action=masquerade chain=srcnat out-interface=bridge
add action=dst-nat chain=dstnat dst-address=x.y.z.132 dst-port=22 protocol=tcp to-addresses=172.31.220.4 to-ports=22
/ip route
add distance=1 gateway=x.y.z.129
/system routerboard settings
set boot-os=router-os

Thank you very much for any education you can give me.
 
tdw
Forum Guru
Forum Guru
Posts: 1841
Joined: Sat May 05, 2018 11:55 am

Re: Inside server not seeing external user's IP

Sun Aug 29, 2021 2:15 pm

I'm surprised it works at all as you have bridged your 'WAN' and 'LAN' ports together - this would only be necessary if you are using some of the five public IP addresses directly on servers rather than using dst-nat, and requires careful handling to prevent leakage of your internal network back to the ISP.

Assuming you are only going to be using ether1 as the WAN and sfp-sfpplus1 as the LAN, and as the IP addresses are already directly attached to interfaces* , just remove the bridge ports and bridge. Also change the masquerade rule out-interface from bridge to ether1.

* - incorrect when an interface is a member of a bridge, sometimes works but can have odd side-effects.
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Sun Aug 29, 2021 9:40 pm

Thanks, tdw, I was clearly confused about bridging. I have made the changes you suggested
and outbound masquerading is still working OK.

However, SSH to the server is not working at all now. The server is not reporting receiving any
packets at all. Clearly, I am still doing something wrong and would appreciate more guidance.
My exported configuration is now:

# jan/07/1970 16:38:29 by RouterOS 6.48.3
# software id = ES33-601A
#
# model = CRS305-1G-4S+
# serial number = B9EA0EEFE310
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip address
add address=192.168.88.1/24 comment=defconf network=192.168.88.0
add address=172.31.1.4/16 comment="Inside network" interface=sfp-sfpplus1 network=172.31.0.0
add address=x.y.z.130/29 comment="Connection to ISP: Route" interface=ether1 network=x.y.z.128
add address=x.y.z.132/29 comment="Connection to ISP: Slug" interface=ether1 network=x.y.z.128
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
add action=dst-nat chain=dstnat dst-address=x.y.z.132 dst-port=22 in-interface=ether1 protocol=tcp to-addresses=172.31.220.4 to-ports=22
/ip route
add distance=1 gateway=x.y.z.129
/system routerboard settings
set boot-os=router-os
 
tdw
Forum Guru
Forum Guru
Posts: 1841
Joined: Sat May 05, 2018 11:55 am

Re: Inside server not seeing external user's IP  [SOLVED]

Mon Aug 30, 2021 3:02 pm

Where are you trying to SSH from? As you have changed the dst-nat rule to include in-interface=ether1 only requests arriving from the external connection will be processed, not those to x.y.z.132 from inside your network.

Also, what is the default route on the server 172.31.220.4, if it is not using 172.31.1.4 as the gateway packets cannot be returned.

Unrelated, the default configuration IP address 192.168.88.1/24 is redundant and can be removed. Having some firewall rules would be good too.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Inside server not seeing external user's IP

Mon Aug 30, 2021 4:29 pm

Without at least 2 of the 3, we can only play silly bugger guessing games.

1. /export hide-sensitive file=anynameyouwish
2. network diagram
3. Set of detailed requirements (users/devices needs (allowed and not allowed) without any mention of config.
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Tue Aug 31, 2021 10:29 pm

I apologize for not replying right away. Things are crazy-busy here right now and I only get short
time windows when I can do this testing. I'll try to cover all the questions asked by tdw and anav.

tdw wrote:

> Where are you trying to SSH from? As you have changed the dst-nat rule to include in-interface=ether1 only
> requests arriving from the external connection will be processed, not those to x.y.z.132 from inside your network.

That is the intent. The inside users have physical access to the console, but in the unlikely event that they
do need to SSH to it from inside, they can always use the inside address, 172.31.220.4. This has been working
OK for many years.

> Also, what is the default route on the server 172.31.220.4, if it is not using 172.31.1.4 as the gateway packets cannot be returned.

Yes, it is using 172.31.1.4 when we are testing with the MikroTik.

> Unrelated, the default configuration IP address 192.168.88.1/24 is redundant and can be removed.

I have done that.

> Having some firewall rules would be good too.

I agree, but wanted to keep things as simple as possible until I get the SSH issue solved. I have added
some rules to the INPUT chain and ether1 interface so I don't keep getting all the login failure messages.
I will add more later, but this cuts down on the noise considerably.

anav wrote:

> Without at least 2 of the 3, we can only play silly bugger guessing games.
> 1. /export hide-sensitive file=anynameyouwish
> 2. network diagram
> 3. Set of detailed requirements (users/devices needs (allowed and not allowed) without any mention of config.

The export file I posted earlier was complete and accurate, except that I changed the first three octets of
out static IPs to "x.y.z". Since I have made some changes, I am posting the latest version below. If there is
some reason that you need to see the full static IPs, let me know.

My test network is pretty simple, although the production one has more inside machines (email server, DNS server, etc.):
I'm trying to keep things simple until I have the basics working, then I will add the others.

Please excuse the use of a variable-width font -- I don't know how to select a fixed pitch one.
Connection to ISP with 5 usable static IPs: x.y.z.130 to x.y.z.134 The gateway to the ISP is x.y.z.129
|
| ether1
+-----------+----------+
| MikroTik | x.y.z.130 (public IP)
| CRS305 | 172.31.1.4 (inside private IP)
+--+---+---+---+---+
| | | |
| | | +------ sfp-sfpplus4 Not used yet, but reserved for WiFi AP.
| | +------------ sfp-sfpplus3 Not used.
| +----------------- sfp-sfpplus2 Not used.
|
+--+-----------------+
| HP ProCurve |
| Switch |
+---------+----------+
|
|
All inside PCs, servers, etc.
Server #1 172.31.220.4 is the one I am trying to establish an SSH connection to from the outside world.

I want an outside user at some random IP (call it a.b.c.d) to SSH to our public static IP x.y.z.132 and
be connected to the server at 172.31.220.4. The server must see the SSH connection coming from
the user's IP a.b.c.d.

In the very first configuration, which tdw rightly called an error, the server was receiving the SSH
connections, but the source IP was always 172.31.1.4.

In the current configuration (below), I think I have corrected the errors tdw pointed out, but now
the server doesn't receive any SSH connections from outside. SSH connections from the inside
network (172.31.x.x) work OK.

Here is the current exported configuration file. It is complete except where I changed the first
three octets of our public IPs to "x.y.z". Nothing else has been changed.

# jan/09/1970 16:51:58 by RouterOS 6.48.3
# software id = ES33-601A
#
# model = CRS305-1G-4S+
# serial number = B9EA0EEFE310
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip address
add address=172.31.1.4/16 comment="Inside network" interface=sfp-sfpplus1 \
network=172.31.0.0
add address=x.y.z.130/29 comment="Connection to ISP: Route" interface=\
ether1 network=x.y.z.128
add address=x.y.z.132/29 comment="Connection to ISP: Slug" interface=\
ether1 network=x.y.z.128
/ip firewall filter
add action=drop chain=input dst-address=x.y.z.128/29 dst-port=23 \
in-interface=ether1 protocol=tcp
add action=drop chain=input dst-address=x.y.z.128/29 dst-port=22 \
in-interface=ether1 protocol=tcp
add action=drop chain=input dst-address=x.y.z.128/29 dst-port=8728 \
in-interface=ether1 protocol=tcp
add action=drop chain=input dst-address=x.y.z.128/29 dst-port=21 \
in-interface=ether1 protocol=tcp
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
add action=dst-nat chain=dstnat dst-address=x.y.z.132 dst-port=22 \
in-interface=ether1 log=yes protocol=tcp to-addresses=172.31.220.4 \
to-ports=22
/ip route
add distance=1 gateway=x.y.z.129
/system logging
set 1 action=echo
/system routerboard settings
set boot-os=router-os


Please let me know if you need any additional information. I really appreciate your
efforts to solve my beginner's lack of education. Thanks!
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Tue Aug 31, 2021 10:30 pm

I see that the variable-width font really screwed up my diagram. I hope you can follow it.
If you can tell me how to select a fixed-width font, I would appreciate it.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Inside server not seeing external user's IP

Tue Aug 31, 2021 11:00 pm

You get fixed-width font if you place text inside [code] [/code] environment.

The reason that SSH server sees router's LAN IP address is some sort of SRC NAT (masquerade is just another form of SRC NAT), but in config you posted the masquerade rule only performs SRC NAT on packets leaving through ether1 (and your diagram says server is behind something else ... I guess sfp-sfpplus1?).
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Tue Aug 31, 2021 11:53 pm

mkx wrote:
The reason that SSH server sees router's LAN IP address is some sort of SRC NAT (masquerade is just another form of SRC NAT), but in config you posted the masquerade rule only performs SRC NAT on packets leaving through ether1 (and your diagram says server is behind something else ... I guess sfp-sfpplus1?).
The SRC NAT / masquerade rule is for outgoing connections from random PCs on the inside network. That feature works fine.

Yes, the server (and everything else) is on the inside network, through sfp-sfpplus1 and the HP switch.
 
tdw
Forum Guru
Forum Guru
Posts: 1841
Joined: Sat May 05, 2018 11:55 am

Re: Inside server not seeing external user's IP

Wed Sep 01, 2021 1:27 pm

Do you see the counters on the dstnat rule increasing when trying to access the server from outside, and as you have logging enabled entries in the log appearing too?
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Wed Sep 01, 2021 9:08 pm

Yes, the counters keep increasing, and there are plenty of log entries also.
I'm quite puzzled, as I *think* am doing exactly what was recommended in other threads
and tutorials. The fact that it works perfectly on the old firewall (Linux/IPTables) tells me
that it *should* work.

I'm sure that there is some simple explanation that will make me say, "Boy, I sure was stupid!"
We just have to find out what it is. :-) :-) :-)

Once we get past this issue I think it will be smooth sailing, as everything else is either very
similar or well-documented.

Thanks again for taking the time to look at my problem.
 
tdw
Forum Guru
Forum Guru
Posts: 1841
Joined: Sat May 05, 2018 11:55 am

Re: Inside server not seeing external user's IP

Wed Sep 01, 2021 9:25 pm

Does the traffic arrive at the server? Something such as tcpdump would do
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Wed Sep 01, 2021 9:55 pm

No, the traffic is not arriving at the server.

Now, let me qualify that a bit. The server is an HP AlphaServer running OpenVMS, which logs
every connection attempt on the console, along with the protocol (SSH, SMTP, etc.) and the
sender's IP address. We are not seeing any connection attempts when using the MikroTik
firewall, but always do with the old firewall. This lack of announcements is why I say that
the traffic is not arriving. If we really need to dig down into the weeds, I think I can figure
out how to hook up a network analyzer to the server's Ethernet port and look at what is
on the wire, but that might take me a while.

In my original (very incorrect) MikroTik configuration that used bridging, the connection
attempts were coming in and being logged on the server console. The problem then was
that they appeared to be coming from the MikroTik box, not the outside user.

Since the server was seeing connection attempts with the old IPTables firewall, and with
the original MikroTik bridging configuration, but NOT with the current configuration, I think
it is reasonable to conclude that the problem is with my current MikroTik configuration.
Does this seem reasonable to you?

Thanks.
 
tdw
Forum Guru
Forum Guru
Posts: 1841
Joined: Sat May 05, 2018 11:55 am

Re: Inside server not seeing external user's IP

Wed Sep 01, 2021 11:44 pm

Not necessarily. With the previous setup as far as the server was concerned packets were arriving from 172.31.1.4, now they will be arriving from their true source address - hence the question about default gateway, any internal firewalling may come into play too.

You could try using the Mikrotik ping tool and specify x.y.z.130 or x.y.z.132 as the source address to check basic reachability. The packet sniffer tool can be used to check to see if the SSH dst-nat traffic leaves the Mikrotik, and if any return traffic is seen.
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Thu Sep 02, 2021 7:21 pm

Thank you, thank you, thank you!!! TDW, you helped me solve the problem. I have marked this
thread as "Solved".

It turned out to be what you earlier expected, that the server was not using 172.31.1.4 as its gateway.

Yes, I had changed it, using the OpenVMS TCPIP$CONFIG command file (script). Subsequent "Show"
commands showed that it had, indeed, been changed. What I forgot was that the script changes
the on-disk database, but NOT the running configuration. It is necessary to shut-down and restart
the networking subsystem for it to take effect. This is NOT necessary when enabling or disabling
services such as SSH, SMTP, etc., which take effect immediately, hence my misconceptions.

A reboot would have also cured the problem, but OpenVMS systems are rarely rebooted except for
major system upgrades or power failures that outlast the UPS. We typically go three or four years
between reboots.

What led me to this was your suggestion to use the ping tool to check basic reachability. I started
making a chart of of which machines and IPs could see other, going in both directions. When I saw
that my desktop PC could ping x.y.z.132 perfectly, but the server could not, I knew I had found the
problem. As soon as I stopped and restarted networking on the server, I started seeing SSH
connection attempts, all with the user's IP which is what my original concern had been.

Again, thank you for your patient help and suggestions. I was convinced that the problem was with
my MikroTik configuration, blinding me to the actual problem. You nudged me in the correct direction.

Also, thanks for correcting my horrible errors in the original configuration. I have no idea how I
managed to screw it up so bad -- probably reading/watching the wrong "tutorials". I wish MikroTik
had better new user guides and documentation to help people avoid such mistakes.

Now that I have *that* server problem solved, I will move on to the email and DNS server, but
I think that will go smoothly now that I know that DSTNAT is working the way I want. I also need
to come up with a good set of firewall rules to keep the bad guys out and prevent any malware
that gets in from "phoning home". I may have other questions later, but I'll put those in another
thread if I can't find the answer on the forum.
 
tdw
Forum Guru
Forum Guru
Posts: 1841
Joined: Sat May 05, 2018 11:55 am

Re: Inside server not seeing external user's IP

Thu Sep 02, 2021 9:59 pm

Good that you have got to the bottom of it. In general many third-party blogs and videos about Mikrotik configuration are outdated/inaccurate/insecure, stick to the forum, help/wiki pages and MUM presentations.

I would suggest a minimum of
/interface list
add name=WAN
/interface list member
add interface=ether1 list=WAN
/ip firewall filter
add action=accept chain=input comment="Allow input 'established', 'related' and 'untracked'" connection-state=established,related,untracked
add action=drop chain=input comment="Drop input 'invalid'" connection-state=invalid log-prefix="Drop invalid"
add action=accept chain=input comment="Allow input ICMP" protocol=icmp
add action=drop chain=input comment="Drop input everything else from WAN" in-interface-list=WAN
add action=accept chain=forward comment="Allow forward 'established', 'related' and 'untracked'" connection-state=established,related,untracked
add action=drop chain=forward comment="Drop forward 'invalid'" connection-state=invalid log-prefix="Drop invalid"
add action=accept chain=forward comment="Allow forward destination NAT from WAN" connection-nat-state=dstnat in-interface-list=WAN
add action=drop chain=forward comment="Drop forward everything else from WAN" in-interface-list=WAN
This blocks all WAN input to the Mikrotik, and only permits WAN forwarding from WAN for any dst-nat to internal servers you set up
 
Frisbie
just joined
Topic Author
Posts: 9
Joined: Fri Aug 27, 2021 5:03 am

Re: Inside server not seeing external user's IP

Thu Sep 02, 2021 10:40 pm

Thanks for the recommended rules, I will start with them.
Yes, I have noted that many of the blogs and videos are quite outdated. I will stick to the forum,
and try to research any problems before posting. I've got about a dozen tabs open to various
posts that look helpful.

I wish there was something in the documentation that discusses bridging versus routing.
It might have eliminated my original mistaken configuration. It seems awfully easy for a
beginner to screw things up. I'm still somewhat confused, but will keep reading and learning.

The MikroTik software is certainly powerful, but it is like learning how to use a 100-bladed
Swiss Army knife. One mistake and it becomes a suicide kit. :-) :-) :-)
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: Inside server not seeing external user's IP

Thu Sep 02, 2021 10:47 pm

The MikroTik software is certainly powerful, but it is like learning how to use a 100-bladed
Swiss Army knife.

As much as this is a curse for newbie it's what experienced MT users wouldn't give away no matter what.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Inside server not seeing external user's IP

Fri Sep 03, 2021 1:11 am

As in the Monty Python vernacular, an MT certified IT engineer floats like a duck and therefore is a witch, What do we do?? Burn all the witches@@#

https://www.bing.com/videos/search?q=mo ... &FORM=VIRE
 
kramegroeg2021
just joined
Posts: 3
Joined: Fri Jun 25, 2021 9:48 pm

Re: Inside server not seeing external user's IP

Fri Sep 03, 2021 6:45 pm

This is great and has helped me a lot. Only I have one more problem. I couldn't ping anything on the inside LAN before and then I followed these directions and now, I get a response from the gateway stating destination unreachable. So I have made progress. Is there another step I need to do so I can reach the inside LAN?

"What also floats in water?"

Pinging 192.168.88.244 with 32 bytes of data:
Reply from 192.168.88.1: Destination host unreachable.
Reply from 192.168.88.1: Destination host unreachable.
Reply from 192.168.88.1: Destination host unreachable.
Reply from 192.168.88.1: Destination host unreachable.

Ping statistics for 192.168.88.244:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Who is online

Users browsing this forum: No registered users and 51 guests