LAN to LAN forwarding

Diresta · Sun Jun 28, 2020 10:25 pm

I am trying to set up forwarding rules to redirect traffic from one to another device within the same LAN so I can silently migrate services from one server to another without having to change IP adresses for everyone that's on the LAN network. Once all services are migrated the new server will be assigned the IP of the old one and the redirect rules will be disabled.

In a bit more detail. What I'm trying to achieve is forwarding data going to old server (192.168.10.125 port 150) to new server (192.168.10.126 port 10000).

So far I managed to set up the following rules:

add action=masquerade chain=srcnat comment="Allow internet access through WAN port" ipsec-policy=out,none out-interface-list=WAN

add action=dst-nat chain=dstnat comment="External forward does work" dst-address-list=Home dst-port=150 protocol=tcp to-addresses=192.168.10.126 to-ports=10000

add action=dst-nat chain=dstnat comment="Internal forward does not work" dst-address=192.168.10.125 dst-port=150 protocol=tcp to-addresses=192.168.10.126 to-ports=10000

add action=masquerade chain=srcnat comment="Give Lan access to local devices via WAN IP" dst-address-list=LAN out-interface=bridge src-address-list=LAN

The port forward using the WAN ip works but through the LAN forward it just times out. I've spent nearly the entire day trying different setups and reading up on Hairpin NAT but I'm at a total loss on this one. I hope someone can help me solve this problem.

xvo · Sun Jun 28, 2020 10:38 pm

You need to use bridge filter and bridge nat for the traffic inside your LAN.
And even then it will work only if the path from client to server physically lies via you router.
If, for example the client and server are connected to the switch, and not to the router directly, that won't work at all.

sindy · Sun Jun 28, 2020 10:45 pm

It may be simpler to temporarily dedicate a subnet different from 192.168.10.0/24 for the server, so that the clients would have to use the router as a gateway. Bridge NAT rules are a nightmare (ARP handling is necessary, NAT is not stateful...)

Diresta · Mon Jun 29, 2020 10:35 am

Thanks to the both of you for the reply. It seems this answer from sindy is the easiest to setup:

It may be simpler to temporarily dedicate a subnet different from 192.168.10.0/24 for the server, so that the clients would have to use the router as a gateway. Bridge NAT rules are a nightmare (ARP handling is necessary, NAT is not stateful...)

I have created a new subnet and assigned the new server and ip on it (192.168.12.25) but I still have the same forwarding problem. Would it be possible to provide me with an example of how I should redirect my traffic from 192.168.10.125:150 to 192.168.12.10000?

Zacharias · Mon Jun 29, 2020 2:48 pm

Traffic inside your LAN is a Layer 2 traffic that will not go through The Firewall !!! Firewall filters Layer 3 Traffic...
So either you enable the bridge filter so that the traffic is forced to pass through the prerouting, forward and postrouting chains, as @xvo suggested or you use a whole different subnet for your server as @sindy suggested...

Personally i would go with @sindy's suggestion...

sindy · Mon Jun 29, 2020 3:02 pm

Would it be possible to provide me with an example of how I should redirect my traffic from 192.168.10.125:150 to 192.168.12.10000?

The "internal dst-nat" rule you've posted in one of the previous posts was OK for this purpose, so it's most likely just a typo in it, or you have done something wrong when adding the dedicated subnet for the new server. So if you don't find the typo, follow the suggestion in my automatic signature below.

mutluit · Mon Jun 29, 2020 5:12 pm

Port forwarding from wanIP:port to lanIP:port works.
What the OP wants to know is how to port forward from internal lanIP1:port to internal lanIP2:port .
Me too interested in the solution.

sindy · Mon Jun 29, 2020 5:25 pm

What the OP wants to know is how to port forward from internal lanIP1:port to internal lanIP2:port .

It's not exactly like this - what the OP really wants is to migrate services from old server to new server one by one without touching the clients' configuration, and whilst it should be possible to port forward even L2 traffic, the amount of hair lost while debugging it would not be proportional to the outcome. And as the real goal can be achieved using simpler means, it is unlikely that the question above will be answered here

anav · Mon Jun 29, 2020 6:19 pm

It may be simpler to temporarily dedicate a subnet different from 192.168.10.0/24 for the server, so that the clients would have to use the router as a gateway. Bridge NAT rules are a nightmare (ARP handling is necessary, NAT is not stateful...)

Okay I will bite as its not clear to me.

By the way first of, I would introduce the server at a quiet time and do the switch period.
I dont see why one needs to do it slowly like over a few days. Set it up in the LAB and then simply deploy it, you know, plug in for power, take out ethernet cable from old server and plug cable into new server, unplug power to old server, remove old server.

The way I read it is that the OP has the following scenario.
Bunch of users on SUBNET A, and a server on SUBNET A.
THe OP wants to move everyone over to a NEW SERVER ON SUBNET A over time so its transparent to users.

I do not understand your solution as it makes too many assumptions.
What if lan users use the LANIP and port number of the server.................

Are you suggesting that users be instructed to use a DYNDNS name (IE GO through the WANIP for server access). Invoking Hairpin NAT or loopback LOL.

CHANGE METHOD OF ACCESSING SERVER (using hairpin)
If So, the dst nat rules would be on day one Prior to Server Changes
dstport=XXX (current) to IP-- currentserverip to port= XXXX original

CHANGE DESTINATION NAT RULE
Day Two Admin puts the new server in place and changes the rule.
dstport=XXX (current) to-ip --newserver to port=YYYY new IP/new port

RETURN DST NAT RULE
Once the migration is satisfactorily completed and op changes new server IP and port back to original numbers.
dstport=xxxx current to ip-currentnew to port=xxxx original

mutluit · Mon Jun 29, 2020 6:21 pm

@sindy, I'm looking for a solution for port-forwarding from lanIP1:port to lanIP2:port within the same LAN. Is there a solution available for this (simpler) problem?
Ie. connections to 192.168.88.12:8512/tcp shall be (on the router) redirected to 192.168.88.11:8511

xvo · Mon Jun 29, 2020 6:59 pm

@sindy, I'm looking for a solution for port-forwarding from lanIP1:port to lanIP2:port within the same LAN. Is there a solution available for this (simpler) problem?
Ie. connections to 192.168.88.12:8512/tcp shall be (on the router) redirected to 192.168.88.11:8511

It's not a simpler problem.
And there is no good solution for this.
Mostly because the concept itself is bad: while the idea of putting devices on the same L2 network pursues the goal to offload handling of traffic between hosts on this network to cheaper (and faster) devices - switches, what you are trying to achieve needs quite the opposite - that all the traffic in your LAN passes through your router (an not even it's internal switch, but its CPU).
So not only will you need to build your whole network with that in mind, but you are at the same time making router's CPU the whole network's bottleneck.

anav · Mon Jun 29, 2020 7:05 pm

Hi XVO you nailed the issue on the head............ the concept is bad. This points out that OP is concerned with fulfilling a config solution without revealing the true requirement. Cart before the horse and the most expensive way to implement anything.
Suggest mutluit that you describe the user case(s), in terms of what users require to do their jobs etc, without any discussion of configuration or hardware involved.
What is unique, different from other requirements.
What are the limitations? (here you may state what equipment is involved, what is available to work with on the canvass so to speak).

mutluit · Mon Jun 29, 2020 8:23 pm

ATTN everybody!
I now have found a solution. Will post it shortly. But it works only if no bridge is configured in RouterOS

Could be a ROS bug...

Zacharias · Mon Jun 29, 2020 8:28 pm

ATTN everybody!
I now have found a solution. Will post it shortly. But it works only if no bridge is configured in RouterOS Could be a ROS bug...

Then how exactly did you create a Layer 2 Broadcast Domain if you configured no Bridge ?

mutluit · Mon Jun 29, 2020 8:38 pm

Ok, here's the said solution:

Solution for port forwarding for both WAN-to-LAN as well LAN-to-LAN (incl. inside same LAN):

On my router (hAP ac^2 with RouterOS 7.0beta8) with no NAT (ie. as 2nd router) now the following solution works:

IP of WAN interface (ether1): 192.168.254.253/24
IP of ether2 (ie. the gateway IP for LAN): 192.168.127.254/17
IP of host attached to router ether2: 192.168.20.1/17 (can add more hosts into that LAN by attaching a dumb switch to ether2).

WAN port 8458 is forwarded to LAN IP 192.168.20.1:8459. There an iperf server is running for testing.

All connections to 192.168.254.253:8458 (ie. the WAN interface) are correctly redirected, even connections from inside the LAN !

So, the culprit is the bridge as with bridge this solution doesn't function.

I first tried the following step-by-step guide (with some good explanations) that says it works,
but I couldn't get it working here (as said WAN-to-LAN works, but not from inside the LAN):
https://serverfault.com/questions/97034 ... tik-router
Then I removed the bridge, and voila it works! And performance is ok (Gigabit speed, s.b.).

[admin@AP1] /ip/firewall/nat> export
/ip firewall nat
add action=dst-nat chain=dstnat dst-address=192.168.254.253 dst-port=8458 in-interface=all-ethernet log=yes protocol=tcp to-addresses=192.168.20.1 to-ports=8459
add action=src-nat chain=srcnat dst-address=192.168.20.1 dst-port=8459 log=yes protocol=tcp src-address=192.168.0.0/17 to-addresses=192.168.127.254

[admin@AP1] /ip/firewall/filter> export
/ip firewall filter
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
...

host:

# ifconfig
eth0 Link encap:Ethernet HWaddr XXXXXXXXX
inet addr:192.168.20.1 Bcast:192.168.127.255 Mask:255.255.128.0

# iperf server running on host:
$ iperf -e -s -p 8459

# iperf client from anywhere:
$ iperf -e -c 192.168.254.253 -p 8458 -t 5
------------------------------------------------------------
Client connecting to 192.168.254.253, TCP port 8458 with pid 18513
Write buffer size: 128 KByte
TCP window size: 468 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.20.1 port 12146 connected with 192.168.254.253 port 8458 (ct=0.93 ms)
[ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT NetPwr
[ 3] 0.00-5.01 sec 552 MBytes 925 Mbits/sec 4417/0 0 -1K/816 us 141735.83

Btw, without the above filter rules the performance is only about half:
[ 3] 0.00-5.02 sec 314 MBytes 525 Mbits/sec 2511/0 0 -1K/695 us 94425.95
So, the "fasttrack" and the "established,related" rules should be present in the firewall to get wire-speed performance.

P.S.: of course the log=yes in the firewall rules should be removed after testing.

mutluit · Mon Jun 29, 2020 9:02 pm

ATTN everybody!
I now have found a solution. Will post it shortly. But it works only if no bridge is configured in RouterOS Could be a ROS bug...
Then how exactly did you create a Layer 2 Broadcast Domain if you configured no Bridge ?

Just assign an IP to the router port, for example ether2: 192.168.50.254/24. That's then the gateway IP for LAN 192.168.50.0/24 at ether2.
Of course w/o a bridge one loses the switch-functionality on the router, meaning one has to attach a dumb switch to ether2 to be able to attach more than one host into that LAN. Same with the other router ports...

xvo · Mon Jun 29, 2020 9:22 pm

I don't exactly understand what is that thing, that is working, for you have only one host on your LAN in your example.
And even if there is a thing, and it is actually working, how is it supposed to continue to work after you put two hosts on one dumb switch?!
These two hosts will connect to each other via the switch, not via your router, no matter the configured rules.

mutluit · Mon Jun 29, 2020 9:34 pm

I don't exactly understand what is that thing, that is working, for you have only one host on your LAN in your example.
And even if there is a thing, and it is actually working, how is it supposed to continue to work after you put two hosts on one dumb switch?!
These two hosts will connect to each other via the switch, not via your router, no matter the configured rules.

It should work b/c one uses the WANGATEWAYIP:port as the WANGATEWAYIP is on the router (in my case ether1).
As said WANGATEWAYIP in my case means the uplink-interface on this 2nd router --> going to the 1st router.
I'll now test it also with a dumb switch, and also from the other router ports (ie. from other LANs).

CZFan · Mon Jun 29, 2020 9:49 pm

@xvo, "that thing..." gave me a good laugh.

Might be a solution, but that will mean all clients will have to be reconfigured to point to WAN address and not internal address of server

mutluit · Mon Jun 29, 2020 9:52 pm

@xvo, "that thing..." gave me a good laugh.

Might be a solution, but that will mean all clients will have to be reconfigured to point to WAN address and not internal address of server

Hmm. yes, you are right. But I think that problem is solvable too. I'll check.

Update:
I now tested using a dumb switch, as well using from another LAN at ether3: both works using the GATEWAYIP:port.

I'll now see whether it as well functions if one uses the (non-existent) hostIP:port instead of the GATEWAYIP:port.

Diresta · Mon Jun 29, 2020 9:52 pm

First of all, thanks for all the replies and the effort all of you put in.

I've been going through all of it and trying different options but I don't even really understand half the solutions stated above. I think my skill level just is not up to par for this one as I have never had any Mikrotik or networking training and all I know has been self taught. All in all I don't think I'm going to be able to get it up and working the way I wanted it to without getting myself educated better about routing in general.

Sorry y'all. I hope someone else can use this port to solve their problem but I can't seem to see the forest through the trees no more

Maybe it's best I'll let it rest for a few days before giving it another try. In the meanwhile, if someone know's a good place for a beginner to learn more about mikrotik firewall/routing it'd be much appreciated.

CZFan · Mon Jun 29, 2020 10:03 pm

...
Hmm. yes, you are right. But I think that problem is solvable too. I'll check.

Like your energy / enthusiasm / attitude towards a problem!!

anav · Mon Jun 29, 2020 10:26 pm

Unfortunately you have a scared off the original poster by subverting (for lower class folks the term is 'hijacking) the thread with what appears to be a nonsense idea. Valid or not, my apologies to the original poster and I very much would like to solve your issue as its interesting and potentially useful for others.
As I said I dont know why you dont want to make a swap change at a quiet time?

To progress your issue how do lan users access the LAN server now? Vian WANIP or LANIP?

sindy · Mon Jun 29, 2020 10:37 pm

I don't even really understand half the solutions stated above.

You can always post your current configuration (minus sensitive information, see my automatic signature below) and get a few lines in return which will modify it to fulfil the goal.

Sob · Tue Jun 30, 2020 12:06 am

Unfortunately you have a scared off the original poster by hijacking the thread ...

Says the hijacking expert.

But on topic, it doesn't seem too difficult. The main problem is that normally traffic between devices in same subnet doesn't pass through router, so it can't do anything with it. But if you connect old server in a way that traffic has to pass through router, i.e. just switch two ports together, connect old server (with same address and everything) to it and rest of LAN to other one, then you can use bridge NAT to redirect connections for old server's address and selected port to router and there process them using IP firewall (dstnat to new server + hairpin NAT). I didn't test it, but it should work. It should be also possible with bridge's use-ip-firewall, but I don't like that, because it affects everything.

sindy · Tue Jun 30, 2020 12:19 am

but I don't like that, because it affects everything.

I'd even go further, s/affects/breaks/ - especially when it comes to NAT. But I've just realized that my suggestion is also only addressing one part of the issue - in fact, both the old and new server have to be moved to another subnet (which may be the same for both, but it has to be different from the clients' subnet), and the original IP address of the old server has to be assigned to the Mikrotik itself in order that my suggestion would work.

xvo · Tue Jun 30, 2020 12:20 am

I don't exactly understand what is that thing, that is working, for you have only one host on your LAN in your example.
And even if there is a thing, and it is actually working, how is it supposed to continue to work after you put two hosts on one dumb switch?!
These two hosts will connect to each other via the switch, not via your router, no matter the configured rules.
It should work b/c one uses the WANGATEWAYIP:port as the WANGATEWAYIP is on the router (in my case ether1).
As said WANGATEWAYIP in my case means the uplink-interface on this 2nd router --> going to the 1st router.
I'll now test it also with a dumb switch, and also from the other router ports (ie. from other LANs).

Ok, so basically you just "invented" hairpin NAT on your own, right?!

And that is what you actually need?
But that is not what you've asked for initially.
And has very little to do with the original post...

Sob · Tue Jun 30, 2020 12:40 am

I tried simple test with RB working as (not so much) transparent bridge. First put two ports together:

/interface bridge
add name=bridge1
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4

Add some address (can be also static, doesn't matter):

/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=bridge1

Disconnect old server from network and connect it back through RB. At this point it's like a switch, fully transparent. Then add NAT rules (192.168.80.17 is old server, 192.168.80.3 is new server):

/ip firewall nat
add action=dst-nat chain=dstnat dst-address=192.168.80.17 dst-port=80 protocol=tcp to-addresses=192.168.80.3
add action=masquerade chain=srcnat dst-address=192.168.80.0/24 src-address=192.168.80.0/24

And as expected, nothing happens. But add one bridge NAT rule:

/interface bridge nat
add action=redirect chain=dstnat dst-address=192.168.80.17/32 dst-port=80 in-bridge=bridge1 ip-protocol=tcp mac-protocol=ip

And suddenly all clients connecting to old web server get content from new one. Integrating it with existing config could be a little more work, to make sure that it doesn't conflict with anything else (so maybe the simplest choice is to not do it, and instead just use spare RB like I did, if you have one).

mutluit · Tue Jun 30, 2020 3:29 am

@Sob, your solution is very interesting, but unfortunately in current beta8 it hangs in a loop so that the router reboots endlessly

I suspect it is the masquerade rule with src-addr and dst-addr equal.
But if it works well with stable/long-term version than it could indeed be the solution for the OP.

anav · Tue Jun 30, 2020 3:37 am

Haha too funny fellas, you should know that being MTUNA certified I am authorized to point the thread in the proper direction, hijacking is such an uncultured term, where did you go to school?
My problem is I cannot figure out which thread SOB is answering LOL, the original thread by the OP, or the usurper thread??

mutluit · Tue Jun 30, 2020 3:40 am

I don't exactly understand what is that thing, that is working, for you have only one host on your LAN in your example.
And even if there is a thing, and it is actually working, how is it supposed to continue to work after you put two hosts on one dumb switch?!
These two hosts will connect to each other via the switch, not via your router, no matter the configured rules.
It should work b/c one uses the WANGATEWAYIP:port as the WANGATEWAYIP is on the router (in my case ether1).
As said WANGATEWAYIP in my case means the uplink-interface on this 2nd router --> going to the 1st router.
I'll now test it also with a dumb switch, and also from the other router ports (ie. from other LANs).
Ok, so basically you just "invented" hairpin NAT on your own, right?!
And that is what you actually need?
But that is not what you've asked for initially.
And has very little to do with the original post...

Ok, ok, got it. I now realize it as well.
I apologize for my wrong solution and the confusion it caused. Sorry.

Sob · Tue Jun 30, 2020 4:46 am

@mutluit: I generally leave MikroTik's betas to more adventurous types, because I believe that their use of terms like beta, rc or stable is slightly shifted from the usual meaning, and not for the better. I have few CHRs to play with latest stuff, but that's it. Otherwise I'm glad for every brave user who is willing to sacrifice real hardware to help squash bugs. If current beta v7 doesn't like this config, I guess it's probably the bridge NAT rule. Masquerade is just for hairpin NAT and IP firewall is well tested, everyone uses that. Bridge NAT on the other hand, I don't thing that it's used much.

Btw, it would be also possible to do similar thing with proxy ARP instead of bridge.

@anav: How can't you tell? The thing I posted is complete working config, just few rules and with comments. If you continue like this, I'll have to revoke your uncertification.

Diresta · Tue Jun 30, 2020 10:08 am

Allright, I removed everything I setup to try this and started with a blank slate. I setup a bridge on (normally unused) ports 8 & 9 on my RB3011, setup a DHCP server and connected my new server to port 9. The router assigned it an IP address but I set it to static --> 192.168.12.125.

Here are all my Firewall rules, The address-list:

/ip firewall address-list
add address=192.168.10.0/24 list=LAN
add address=192.168.12.0/24 list=LAN_Server
add address=PUBLIC_IP list=Home
add address=8.8.8.8 list="Google DNS"
add address=8.8.4.4 list="Google DNS"
add address=208.67.222.222 list="Google DNS"
add address=209.244.0.3 list="Google DNS"
add address=0.0.0.0/8 comment=RFC6890 list=NotPublic
add address=10.0.0.0/8 comment=RFC6890 list=NotPublic
add address=100.64.0.0/10 comment=RFC6890 list=NotPublic
add address=127.0.0.0/8 comment=RFC6890 list=NotPublic
add address=169.254.0.0/16 comment=RFC6890 list=NotPublic
add address=172.16.0.0/12 comment=RFC6890 list=NotPublic
add address=192.0.0.0/24 comment=RFC6890 list=NotPublic
add address=192.0.2.0/24 comment=RFC6890 list=NotPublic
add address=192.168.0.0/16 comment=RFC6890 list=NotPublic
add address=192.88.99.0/24 comment=RFC3068 list=NotPublic
add address=198.18.0.0/15 comment=RFC6890 list=NotPublic
add address=198.51.100.0/24 comment=RFC6890 list=NotPublic
add address=203.0.113.0/24 comment=RFC6890 list=NotPublic
add address=224.0.0.0/4 comment=RFC4601 list=NotPublic
add address=240.0.0.0/4 comment=RFC6890 list=NotPublic
add address=192.168.10.0/24 comment=RFC6890 list=NotPublic
add address=192.168.11.0/24 comment=RFC6890 list=NotPublic

The Filters:

/ip firewall filter
add action=accept chain=input comment=\
    "Accept established,related & untracked connections to the router" \
    connection-state=established,related,untracked
add action=accept chain=input comment=\
    "Accept all connections from local network - LAN" in-interface-list=LAN
add action=accept chain=input comment=\
    "Accept all connections from local network - LAN_SERVER" in-interface-list=\
    LAN_SERVER
add action=accept chain=input comment="Accept established and related packets" \
    connection-state=established,related
add action=accept chain=forward comment=\
    "Accept established and related packets" connection-state=\
    established,related
add action=drop chain=input comment="Drop invalid connections to the router" \
    connection-state=invalid
add action=drop chain=input comment="Drop invalid packets" connection-state=\
    invalid
add action=drop chain=input comment=\
    "Drop all packets which are not destined to routers IP address" \
    dst-address-type=!local
add action=drop chain=input comment=\
    "Drop all packets which does not have unicast source IP address" \
    src-address-type=!unicast
add action=drop chain=input comment="Drop all packets from public internet which\
    \_should not exist in public network" in-interface-list=WAN \
    src-address-list=NotPublic
add action=drop chain=forward comment="Drop invalid packets" connection-state=\
    invalid
add action=drop chain=forward comment=\
    "defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN log-prefix=dz
add action=drop chain=forward comment="Drop all packets from public internet whi\
    ch should not exist in public network" in-interface-list=WAN \
    src-address-list=NotPublic
add action=drop chain=forward comment=\
    "Block internet but allow local network for PC" disabled=yes out-interface=\
    ether1 src-address=192.168.10.253

The nat rules:

/ip firewall nat
add action=masquerade chain=srcnat comment="WAN NAT - Grants local LAN devices a\
    ccess to the internet by masquering as the WAN IP address" ipsec-policy=\
    out,none out-interface-list=WAN src-address-list=LAN
    add action=masquerade chain=srcnat comment="WAN NAT - Grants local LAN_SERVER devices a\
    ccess to the internet by masquering as the WAN IP address" ipsec-policy=\
    out,none out-interface-list=WAN src-address-list=LAN_Server
add action=masquerade chain=srcnat comment="HAIRPIN NAT - Gives local devices ac\
    cess to other local devices using the WAN IP address" dst-address-list=LAN \
    log=yes log-prefix=HP NAT out-interface=bridge src-address-list=LAN
add action=dst-nat chain=dstnat comment="This works from WAN to LAN" \
    dst-address-list=Home dst-port=125 log=yes log-prefix=WAN2LAN protocol=tcp \
    to-addresses=192.168.12.125 to-ports=9000
add action=dst-nat chain=dstnat comment=\
    "DMZ to server - Always place below other rules" dst-address-list=Home \
    to-addresses=192.168.10.125

So far I'm back where I was when I posted this thread. If i go to WANIP:125 the traffic gets forwarded tot 192.168.12.125 (newServer). But if I add a rule redirecting traffic for 192.168.10.125 to 192.168.12.125 nothing happens. I tried Sob's approach and setup bridge nat etc but nothing seemed to happen so I removed rules for that. Any pointers on where to go with above config?

mutluit · Tue Jun 30, 2020 12:22 pm

@Diresta, which RouterOS version does your device have?
And can you post the output of this:
/interface export hide-sensitive

And: together with the new servers will also the old servers be online at the same time during the transition phase?
If yes, and if your servers do have iptables, then you could use the following accepted alternative solution on each of the the old servers:
https://serverfault.com/questions/58648 ... me-network

The problem with RouterOS seems to be that port-forwarding using DNAT/SNAT within the same LAN seems not possible.
In my experiments here so far port-forwarding in ROS works only for clients from other LANs as well from the WAN side, but not from inside the same LAN.
But of the working part this fact IMO is very interesting: the old server does not need to exist nor be online

Diresta · Tue Jun 30, 2020 6:18 pm

@Diresta, which RouterOS version does your device have?

I'm running routerOS 6.44. I try to always run the newest stable release for security reasons.

And can you post the output of this:
/interface export hide-sensitive

Here it is:

/interface bridge
add admin-mac=00:25:9C:42:C7:E8 auto-mac=no comment=defconf name=bridge
add comment="Bridge for temp server" name=bridge-server
add fast-forward=no name=brige-gast
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=sfp1 ] advertise=\
    10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
/interface vlan
add interface=ether1 name=vlan-gast use-service-tag=yes vlan-id=999
add interface=ether1 name=vlan10 vlan-id=10
add interface=ether1 name=vlan20 vlan-id=20
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=LAN_SERVER
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether6
add bridge=bridge comment=defconf interface=ether7
add bridge=bridge-server interface=ether8
add bridge=bridge comment=defconf interface=sfp1
add bridge=bridge comment=defconf interface=ether10
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge-server interface=ether9
/interface detect-internet
set detect-interface-list=all
/interface l2tp-server server
set enabled=yes use-ipsec=yes
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=bridge-server list=LAN_SERVER
/interface ovpn-server server
set auth=sha1 certificate=SOMECERT cipher=aes256 default-profile=\
    open_vpn enabled=yes require-client-certificate=yes
/interface pptp-server server
set enabled=yes
/interface sstp-server server
set certificate=SOMECERT_default-profile=default-encryption enabled=yes

The old server will remain online for the whole duration of the migration to the new server so the idea of using iptables might be the easiest solution indeed. I can't believe I didn't think of that myself

I'll go give it a try later today and will report back..

xvo · Tue Jun 30, 2020 6:42 pm

The problem with RouterOS seems to be that port-forwarding using DNAT/SNAT within the same LAN seems not possible.
In my experiments here so far port-forwarding in ROS works only for clients from other LANs as well from the WAN side, but not from inside the same LAN.

It's not a RouterOS problem.
Actually it's not a problem at all - it's just basic networking concept.

mutluit · Tue Jun 30, 2020 6:54 pm

The problem with RouterOS seems to be that port-forwarding using DNAT/SNAT within the same LAN seems not possible.
In my experiments here so far port-forwarding in ROS works only for clients from other LANs as well from the WAN side, but not from inside the same LAN.
It's not a RouterOS problem.
Actually it's not a problem at all - it's just basic networking concept.

But you have to admit that it's not satisfactorily if it works as wished/intended from other LANs and WAN, but not from inside the same LAN. One has to question why ROS can't handle that, don't you agree? I would classify that as a bug, or at least as a shortcoming or as a missing capability...

xvo · Tue Jun 30, 2020 7:22 pm

But you have to admit that it's not satisfactorily if it works as wished/intended from other LANs and WAN, but not from inside the same LAN. One has to question why ROS can't handle that, don't you agree? I would classify that as a bug, or at least as a shortcoming or as a missing capability...

And you couldn't be more wrong in you classification attempt.

Two hosts on one LAN (broadcast domain) don't communicate via router, they communicate "directly", and they will never do it the other way, unless forced to.
So this task has to be solved not on the router (L3), but on the switch(es) (L2).
And earlier in the thread @Sob presenter a working solution, how to solve this on mikrotik router's bridge (which is a software switch), but this solution is not universal, and works only if router's bridge is the only switch along the way. To make it work in more complex network, something like that might need to be done on other switches. And in some situation, when unmanaged switches are used, this task can't bу solved at all without physically reconfiguring the network.

anav · Tue Jun 30, 2020 7:22 pm

Why would it be a bug? The MT RoS allows one to program (via config) traffic flow required by the admin.
This is not a plugNplay device. I have used zyxel routers and they had the same process except they called it Loopback vice hairpin nat and one checked a box.
In the MT you dont get the luxury of the checkbox, instead you have to understand what is actually going on in traffic flow and then assign appropriate rules.

Have read through this, as the only bug is lack of education.

https://help.mikrotik.com/docs/pages/vi ... Id=3211299

anav · Tue Jun 30, 2020 7:25 pm

@Diresta
How do users access the server now, by LANIP or by WANIP (dyndns type name)?

If by WANIP.
Then all they are doing is accessing server by dyndnsname:port#

You can migrate all users over to the new server by simply changing the current dstnat rule.
chain=dstnat action=dst-nat dst-port=currentport protocol=tcp dst-address=fixedstaticWANIP to address=currentserver

TO
chain=dstnat action=dst-nat dst-port=currentport protocol=tcp dst-address=fixedstaticWANIP to address=IPofnewServer to-port=newserverport

Done, anybody attempting to reach the current server via the usual method gets redirected to the new Server.
You then have some choices to make.
a. Leave as is,
b. Decide to Change IP of new server back to IP of old server (and thus need to modify dstnat rule so that too-address is back to the original IP address)
c. Decide to Change port of new server back to port of old server (and thus need simply to remove TO-ports )

(Of course to do hairpin NAT you need to add the appropriate srcnat rule)
(add chain=srcnat src-address=192.168.10.0/24 dst-address=192.168.10.0/24 action=masquerade - use appropriate subnet for your scenario)

mutluit · Tue Jun 30, 2020 9:33 pm

But you have to admit that it's not satisfactorily if it works as wished/intended from other LANs and WAN, but not from inside the same LAN. One has to question why ROS can't handle that, don't you agree? I would classify that as a bug, or at least as a shortcoming or as a missing capability...
And you couldn't be more wrong in you classification attempt.

Two hosts on one LAN (broadcast domain) don't communicate via router, they communicate "directly", and they will never do it the other way, unless forced to.
So this task has to be solved not on the router (L3), but on the switch(es) (L2).

@xvo, FYI: here's a solution using iptables on a linux router with a bridge. It reads "Port forwarding between bridged interfaces":
https://askubuntu.com/questions/720207/ ... interfaces
It's a similar problem-case: moving services from one host to another host in same LAN / same bridge, but keeping the clients' configs untouched so that they continue connecting to the old host, which by this iptables solution on that linux router silently is redirected to the new host. All hosts are in the same broadcast domain (same LAN) under same one bridge.
Yes, bridge is L2, but the above solution is using pure iptables, ie. L3+.
Since RouterOS is based on Linux and iptables, then the same solution should be reproducible in ROS I was thinking/hoping.

xvo · Tue Jun 30, 2020 9:51 pm

@xvo, FYI: here's a solution using iptables on a linux router with a bridge. It reads "Port forwarding between bridged interfaces":
https://askubuntu.com/questions/720207/ ... interfaces
It's a similar problem-case: moving services from one host to another host in same LAN / same bridge, but keeping the clients' configs untouched so that they continue connecting to the old host, which by this iptables solution on that linux router silently is redirected to the new host. All hosts are in the same broadcast domain (same LAN) under same one bridge.
Yes, bridge is L2, but the above solution is using pure iptables, ie. L3+.
Since RouterOS is based on Linux and iptables, then the same solution should be reproducible in ROS I was thinking/hoping.

What's your point?
You already have a working ROS solution for this case earlier in the thread.
Does it work for you or not?
If not, then no solution will work without reconfiguring the rest of the network, no matter what device with what os you have in place of mikrotik.

mutluit · Tue Jun 30, 2020 10:56 pm

@xvo, FYI: here's a solution using iptables on a linux router with a bridge. It reads "Port forwarding between bridged interfaces":
https://askubuntu.com/questions/720207/ ... interfaces
It's a similar problem-case: moving services from one host to another host in same LAN / same bridge, but keeping the clients' configs untouched so that they continue connecting to the old host, which by this iptables solution on that linux router silently is redirected to the new host. All hosts are in the same broadcast domain (same LAN) under same one bridge.
Yes, bridge is L2, but the above solution is using pure iptables, ie. L3+.
Since RouterOS is based on Linux and iptables, then the same solution should be reproducible in ROS I was thinking/hoping.
What's your point?

Because you said "So this task has to be solved not on the router (L3), but on the switch(es) (L2)."
In that solution it's solved on the router without using any L2, just FYI

You already have a working ROS solution for this case earlier in the thread.
Does it work for you or not?
If not, then no solution will work without reconfiguring the rest of the network, no matter what device with what os you have in place of mikrotik.

No, it doesn't work here, though I haven't tried other ROS versions.
Such a task should be doable centrally on a router or switch with just a few firewall rules, nothing more. You OTOH seem to say one needs to reconfigure the net.
Never mind, I've seen enough and experienced enough.

xvo · Tue Jun 30, 2020 11:10 pm

No, it doesn't work here, though I haven't tried other ROS versions.
Such a task should be doable centrally on a router or switch with just a few firewall rules, nothing more. You OTOH seem to say one needs to reconfigure the net.
Never mind, I've seen enough and experienced enough.

You clearly still don't understand the nature of the problem in general.
And thus don't see what can work in a particular case and what can't.

mutluit · Tue Jun 30, 2020 11:13 pm

No, it doesn't work here, though I haven't tried other ROS versions.
Such a task should be doable centrally on a router or switch with just a few firewall rules, nothing more. You OTOH seem to say one needs to reconfigure the net.
Never mind, I've seen enough and experienced enough.
You clearly still don't understand the nature of the problem in general.
And thus don't see what can work in a particular case and what can't.

I just know this: in the field out there this important use-case has to function with no excuses.

xvo · Tue Jun 30, 2020 11:25 pm

Because you said "So this task has to be solved not on the router (L3), but on the switch(es) (L2)."
In that solution it's solved on the router without using any L2, just FYI

And FYI, for iptables to be able to filter bridged traffic you have to enable "net.bridge.bridge-nf-call-iptables" which is the same as "use ip firewall" in bridge settings in ROS.
In other words: "use L3 firewall rules for L2".

xvo · Tue Jun 30, 2020 11:28 pm

I just know this: in the field out there this important use-case has to function with no excuses.

Yeah, right. Laws of physics included?

Jotne · Wed Jul 01, 2020 12:00 am

@Diresta

Changing server may be an normal operation to do with new IP. This is why we have DNS. Just redirect DNS to new server.
But why do you need to change port? What services is this that you have on port 150 or port 10000? Not standard ports?

Sob · Wed Jul 01, 2020 1:51 am

@Diresta: Don't create completely new bridge, that was just simple example on a router with no other config. If the *old* server (192.168.10.125) is connected to LAN (192.168.10.0/24 I guess) which is on bridge called "bridge", then just make sure that the *old* server is physically directly connected to one of router's ports which is part of this bridge. The point is that it must not be further in LAN behind other switch(es), so any traffic from other 192.168.10.x devices to 192.168.10.125 has to physically pass through this router. Then all the magic lies in dstnat rule in "/interface bridge nat", which sends selected packets to IP firewall and you can use dstnat to send them to new server (otherwise they would just pass through router and IP firewall wouldn't see them).

Edit: Btw, @mutluit, trust the bearded man, he was right from the start, from first response in this thread (I missed that initially).

anav · Wed Jul 01, 2020 4:49 am

Edit: Btw, @mutluit, trust the bearded man, he was right from the start, from first response in this thread (I missed that initially).

But but but , how can you trust someone who hasnt figured out how to use a razor or electric shaver??
Seriously though I will have to award an MTUNA Sarcasm Certificate to xvo, he has learned well and is dishing it out in great fashion.

Diresta · Wed Jul 01, 2020 9:05 am

And: together with the new servers will also the old servers be online at the same time during the transition phase?
If yes, and if your servers do have iptables, then you could use the following accepted alternative solution on each of the the old servers:
https://serverfault.com/questions/58648 ... me-network

First of all I'd like to thank all of you for the time and effort you put into helping me. It seems my problem took the interest of several of you and sparked several ideas (and some discussion

). Probably half of what's been posted is way above my head but the good news is:

I managed managed to solve my problem using IP tables as stated in mutluit's post above which I marked as answer. I read several posts on different forums and eventually set up a few rules on my ubuntu (old) server. It took me 5 hours to set it up (managed to roast my entire firewall and had to start from scratch but at least I learned a lot in the proces hehe) and now I am able to forward traffic that is sent to old server to new server.

Since there were a several reply's with question since my last post I'm gonna try and answer them all here at once.

How do users access the server now, by LANIP or by WANIP (dyndns type name)?

Users access the server 95% by LAN IP. There is a DMZ on my MT router that sends all traffic that is not specifically routed in the router to the server however a firewall with a deny all is set up on the server. A few ports (like http/https) are opened up to WAN but I prefer using a VPN over WAN access when it comes to more sensitive data/services.

Changing server may be an normal operation to do with new IP. This is why we have DNS. Just redirect DNS to new server.
But why do you need to change port? What services is this that you have on port 150 or port 10000? Not standard ports?

I see where you're going but won't using DNS reroute all traffic that is sent to ip 192.168.10.125 to 192.168.12.125 instead of one port? I need port based forwards so I can migrate one service at a time to my new server instead of having to migrate them all at once. The reason for different (and non standard) ports is that I have several instances of services already running in Docker on new server for test purposes. Once I figure out the best Docker config that host will be set to the final port and the forward will be changed to only forward to different ip but same port. I hope this makes sense.

@Diresta: Don't create completely new bridge, that was just simple example on a router with no other config. If the *old* server (192.168.10.125) is connected to LAN (192.168.10.0/24 I guess) which is on bridge called "bridge", then just make sure that the *old* server is physically directly connected to one of router's ports which is part of this bridge. The point is that it must not be further in LAN behind other switch(es), so any traffic from other 192.168.10.x devices to 192.168.10.125 has to physically pass through this router. Then all the magic lies in dstnat rule in "/interface bridge nat", which sends selected packets to IP firewall and you can use dstnat to send them to new server (otherwise they would just pass through router and IP firewall wouldn't see them).

I think I see what you're getting at. I will most likely give this a try later when I have some more time to play around. Since I got it working using IPtables and I really need to get started migrating services I'm gonna have to postpone on it though. Still very interested in this so thanks for the explanation.

For anyone with this same problem also running ufw & iptables on their old server I thought the least I could do is tell them the solution to my problem:
First I changed the UFW default forward policy. To do open "/etc/default/ufw" and change the default forward policy to ACCEPT

DEFAULT_FORWARD_POLICY="ACCEPT"

Than append the following to /etc/ufw/before.rules:

*nat
:PREROUTING ACCEPT [0:0]
# forward 192.168.10.125 port 125 to 192.168.12.125:10000
-A PREROUTING -p tcp --dport 125-j DNAT --to-destination 192.168.12.125:10000
-A POSTROUTING -p tcp --dport 10000 -j MASQUERADE


# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT

Little sidenote: Place this at the end and don't worry about there already being a commit above. Also don't forget to reload ufw (sudo ufw reload).

All in all I'm really glad I've got it working and I hope that in the long run more people will be helped by this thread. Thanks again!

xvo · Wed Jul 01, 2020 10:24 am

But but but , how can you trust someone who hasnt figured out how to use a razor or electric shaver??
Seriously though I will have to award an MTUNA Sarcasm Certificate to xvo, he has learned well and is dishing it out in great fashion.

This joke seems to need a shaver itself.

Jotne · Wed Jul 01, 2020 12:14 pm

I need port based forwards so I can migrate one service at a time to my new server instead of having to migrate them all at once. The reason for different (and non standard) ports is that I have several instances of services already running in Docker on new server for test purposes.

Send out information that there will be a service windows from 01:00 to 02:00 first Tuesday next month, move all service in one go.

Diresta · Wed Jul 01, 2020 12:27 pm

Send out information that there will be a service windows from 01:00 to 02:00 first Tuesday next month, move all service in one go.

Haha, that might be an option but it all depends on how you want to move your systems. All in all I'm happy with the found solution and glad I can continue with the migration.

mutluit · Wed Jul 01, 2020 8:08 pm

Glad to see that @Diresta's problem of transparent port-forwarding within the same LAN has been solved by using iptables' port-forwarding function on the old server(s).

It would have functioned also centrally on a Linux router with iptables as shown in posting #41 viewtopic.php?f=2&t=162996#p803433

So, here's what I do get & learn from this discussion: use your own open-source switch-router with Linux

Then you can do and realize everything with it.

IMO the only advantage of such nice devices like the MikroTik CRS3xx switch-routers is IMO their special ASIC switch-chip with the stateless firewall (ACL).
But the ACL as implemented in RouterOS is very limited, one even can't check the TCP flags in ACL. So then this advantage is relative/minuscule.

I've currently these MT devices in operation here: CRS326, CRS125, CRS305, hAP ac^2.
But to be fully flexible in the future I'll build my own switch-router on a dedicated PC by adding some multi-port NICs (1G and 10G, and maybe even 40G) to it.
Of course that will eat at least twice as much power than the 24W of a CRS326 switch-router, but with all the other advantages one gets it's then worth it well, IMO.

I would recommend MikroTik to open ROS, so that the admin can access/extend/modify the underlying Linux.
Otherwise for advanced use-cases like seen in this thread (and IMO especially in corporate environments) people will use their own DIY Linux router/switch with root access.

Sob · Thu Jul 02, 2020 3:56 am

It would have functioned also centrally on a Linux router with iptables as shown in posting #41 viewtopic.php?f=2&t=162996#p803433

It depends. Normally iptables don't see bridged packets, exactly as IP firewall rules don't see them in RouterOS. You need extra option for that. Quick search suggests that it may be enabled by default in some systems, so that would explain why it can work. But it's something you generally don't want to have. RouterOS has it too (/interface bridge settings set use-ip-firewall=yes), but as I already mentioned, it does weird things. But have your fun with it if you want.

xvo · Thu Jul 02, 2020 11:16 am

Forget it, @Sob.
I already tried to explain him exactly that, but he prefers to stick to his false assumptions, that particular network layout have nothing to do with the ability (or inability) to solve the problem, and that the problem lies in ROS.
As surprising as it may sound, but @anav was right in on of his early posts in this thread saying that the only bug here is the lack of education.

anav · Thu Jul 02, 2020 4:19 pm

Did I say education, I meant cult following.
Master Sob and MKX are often referred to as Cult leaders in MTUNA literature. ;-P

Sob · Thu Jul 02, 2020 8:09 pm

I can't give up so easily. If I want my own cult, I need to build up membership. I can't have just @anav, because look at him... he's eager and everything, but he already leaks my plans to public (and I didn't even share them with him), there's no way he would become submissive enough and not question the leader all the time, etc.

But more on topic, anyone who has doubts about something can test what works and what doesn't. Don't believe what others write, do it yourself. All required tools are easily available.

xvo · Thu Jul 02, 2020 8:22 pm

I can't give up so easily. If I want my own cult, I need to build up membership. I can't have just @anav, because look at him... he's eager and everything, but he already leaks my plans to public (and I didn't even share them with him), there's no way he would become submissive enough and not question the leader all the time, etc.

That made my evening

anav · Thu Jul 02, 2020 8:27 pm

xvo, you are the perfect sycophant

mkx · Mon Jul 06, 2020 11:06 pm

Did I say education, I meant cult following.
Master Sob and MKX are often referred to as Cult leaders in MTUNA literature. ;-P

C'mon @anav, why are you dragging me into this mud? I've declared @mutluit a foe expressly to avoid reading his misconceptions and now you're forcing me in? If I was to lead a cult, I want to choose my own mud pit.

anav · Mon Jul 06, 2020 11:21 pm

Too true MKX, on thread too many although I did mention your name in another thread, but cannot remember where LOL.

leandrotsampa · Thu Nov 26, 2020 4:09 pm

Hi everybody, please check if can help me.
I want to use a fowarding port Internal/External but when I enable out-interface in SRCNAT only works External.

WAN IP: 192.168.1.2
Router IP: 192.168.10.1
Server IP: 192.168.10.10

I want can access from internet port 8443 and from LAN 192.168.10.1:8443
If I remove out-interface from nat bellow this works but I have problem with Xbox Nat, and with out-interface configured only external access works.
chain=srcnat action=masquerade out-interface=ether1-wan log=no log-prefix=""

My config.
/ip firewall nat print

chain=srcnat action=masquerade out-interface=ether1-wan log=no log-prefix=""

;;; NAS HTTPS
chain=dstnat action=dst-nat to-addresses=192.168.10.10 to-ports=443 protocol=tcp in-interface-list=all dst-port=8443 log=no log-prefix=""

;;; NAS SSH
chain=dstnat action=dst-nat to-addresses=192.168.10.10 to-ports=22 protocol=tcp in-interface-list=all dst-port=8822 log=no log-prefix=""

/ip firewall filter print

;;; special dummy rule to show fasttrack counters
chain=forward action=passthrough

chain=forward action=fasttrack-connection

chain=input action=accept protocol=tcp in-interface-list=all dst-port=8291 log=no log-prefix=""

chain=input action=accept protocol=icmp

chain=input action=accept connection-state=established

chain=input action=accept connection-state=related

Who is online