VLAN trunk - master-slave way of config on physical interfaces

halacs · Sun Apr 28, 2019 10:20 pm

Hi,

First my question, then my environment.

What is the best way to define these VLAN trunks and access point? What is the most flexible way? I will have a new switch what I want to add with the same VLANs but a new trunk cable. I'm not sure I did my configuration the best/easiest way.

So about my environment.

I have two Mikrotik device with Router OS:
1) Mikrotik 951G-2HnD what I use as a WiFi access point
2) Mikrotik CCR1009-7G-1C-1S+PC what is my core switch

There is a 3rd device as well, what is a TP-Link TL-SG1024DE smart switch.

I have two VLANs on top of these devices:
- VLAN ID 2 for internal network
- VLAN ID 3 for guest network

There is a VLAN trunk cable with the two VLANs between my WiFi AP and my core router.
There is another same VLAN trunk between my TP-Link switch and my core router.

Until now now, on my core switch (CCR1009), I just simply created two VLAN network interface per physical interface I want to use as a trunk.

There was the 'master-slave' interface configuration option before a RouterOS update months ago. Now it is already retired but it gave me an idea.

I though that I can create a bridge with 'none' protocol and use it as a 'master' interface for that two physical one I want to use as trunk with same configurations. When I made this now configuration for the VLANs, I experienced connectivity issues: wireless devices could reach wired once and vica versa but the internet. When I disabled one trunk physical interface inside the bridge, remained one could reach internet. What happened here?

I guess I did something wrong but then how I can create these kind of VLANs and then define firewall rules on the core router between them without creating separated VLAN interfaces for all the VLAN IDs per psychical trunk? I have seen bridge/VLAN to define tagged and access ports but it's unclear how I can define firewall rules for them then.

Thanks for reading and for your advice in advance!

anav · Mon Apr 29, 2019 3:40 am

Suggest reading this source if your keen to do the vlan router method.........
viewtopic.php?f=13&t=143620

halacs · Mon Apr 29, 2019 9:26 am

Thanks for the link! It has an example with proper comments what seems very useful. I tried to do my setup in a very similar way but based on this one I will try it again and come back.

Thanks & br,
Halacs

halacs · Sun May 05, 2019 5:02 pm

Suggest reading this source if your keen to do the vlan router method.........
viewtopic.php?f=13&t=143620

Thank you so much for the link! It helped a lot! Very good tutorial!

anav · Sun May 05, 2019 6:27 pm

Yup pcunite is golden!

halacs · Sun May 05, 2019 10:17 pm

One more maybe a bit more advanced question:

I have site-to-site VPN between two mikrotik router. I have a same VLAN config on them with same IDs.

With my previous VLAN settings when I had separated bridges for all VLANs, I could set a bridge in the PPP profile to create a layer 2 VPN connection. Now, with this new VLAN configuration version, L2 VPN doesn't work.

Can I configure it at all without Ethernet Over IP tunneling protocol?

I thought if I set bridge (bridge where I defined VLANs) in the PPP profile, L2TP VPN connection will forward VLAN tagged packages to the bridge at the other end of the VPN. It seems it's not true somehow.

sindy · Sun May 05, 2019 11:10 pm

It never came to my mind to try to push VLANs through a L2TP tunnel in bridge mode, but I've expected it would be enough to configure the /interface bridge port and /interface bridge vlan items also for the L2TP interfaces. However, it seems RouterOS is not ready for this (at least as of 6.44.3). Whereas it adds the L2TP interfaces dynamically to the /interface bridge port list, albeit under the .id instead of the name, even after adding it (also using the .id) to /interface bridge vlan, the frames don't get through if vlan-filtering=yes on the bridges - even the tagless ones don't. If vlan-filtering=no, both tagless and tagged frames do get tunneled, but that doesn't help you much.

So if you really need to tunnel L2 to another site via L3 network, use IPsec-encrypted EoIP rather than IPsec-encrypted L2TP with bridge mode activated.

halacs · Sun May 05, 2019 11:40 pm

Okay, thanks!

I the meanwhile, I managed EoIP on top of L2TP, but then it is secure enough if I set IPsec secret at the tunnel config. I hope I can set domain name for the remote host instead of the IP, but this is what I will check only tomorrow. WAN side firewall will be interesting, what I have to open for EoIP.

Well, my reason might be strange

I'm a dual-homed guy, meaning that I have a family house in the countryside and a flat in Budapest. Among others reasons, my DLNA server is in my house what I want to use from my other home and DLNA love to use broadcast messages. Of course, these things can be useful in the office too in my case.

sindy · Mon May 06, 2019 12:27 pm

I hope I can set domain name for the remote host instead of the IP

You can and it even auto-updates the IP number if it changes dynamically.

WAN side firewall will be interesting, what I have to open for EoIP.

Actually nothing in addition to what you've already open to permit IPsec. EoIP is a special use of GRE, and as GRE has no notion of ports, the connection tracker only uses source and destination IP addresses as connection ID. And as both ends actively send the GRE packets and the default firewall permits everything that is sent by the router itself, the tracked connection establishes at each end by the first packet sent by that end and the received packets then match connection-state=established.

halacs · Mon May 06, 2019 10:16 pm

It works now, thanks! I have set the EoIP tunnel used by the public IPs of the routers and turned off the L2TP site-to-site VPN. It seems all fine with tagged L2 connection.
Local endpoint can be only an IP at the EoIP settings. My WAN IP assigned dynamically via DHCP. You meant that this IP will be updated automatically when WAN port got a new one, right?
Remote endpoint can be a DNS name too so it's straightforward. Both have to be set because of the IPsec according to the manual.

sindy · Mon May 06, 2019 11:16 pm

Local endpoint can be only an IP at the EoIP settings. My WAN IP assigned dynamically via DHCP. You meant that this IP will be updated automatically when WAN port got a new one, right?

Unfortunately not. What I had in mind was only that if your WAN address is a dynamically changing public one and you're subscribed to one of the DDNS services (such as Mikrotik's /ip cloud, you can use the domain name as the remote-address parameter of the /interface eoip, and RouterOS will track the changes and update the remote IP accordingly.

The local-address must be specified as an IP number, the reason is that the dynamically generated IPsec policy needs to match a single /32 address as the SA works in transport mode. So at this side, the handling of the dynamically changing WAN address is a little complicated:

create an /interface bridge with protocol-mode=none and no member ports
assign some /32 private IP address which doesn't conflict with any local subnet to that bridge
insert an action=accept protocol=gre src-address=the.ip.mentioned.above rule into chain=srcnat of /ip firewall nat before the default action=masquerade out-interface-list=WAN ... one
add an action=dst-nat in-interface-list=WAN protocol=udp dst-port=500,4500 to-addresses=the.ip.mentioned.above to chain=dstnat of /ip firewall nat
set the "IP mentioned above" as the local-address of the /initerface eoip

How this whole mess works:
The local-address of the dynamically generated IPsec peer and the src-address of the dynamically generated IPsec policy are inherited from the local-address of the /interface eoip which is the static private one you've chosen. Due to this, the IPsec stack detects NAT so it encapsulates the ESP packets into UDP (which costs you some MTU). The EoIP transport packets are sent from the "IP mentioned above", matched by the dynamically generated IPsec policy and encapsulated into the ESPinUDP transport packets which are also sent from the "IP mentioned above" but as they leave the machine, they are src-nated to its WAN IP. Because the IPsec policy uses "transport mode", the "IP mentioned above" is not packed into the ESP, so when the EoIP packets get decrypted and decapsulated from it at the remote end, they seem to come from the WAN IP of the sender.
The dst-nat is there because one of the machines always sends the initial packet first, and without this rule in place, it would end on the WAN IP on the remote end where the necessary IPsec peer doesn't listen.

Instead, you could configure the IPsec peer and policy manually, leaving peer's local-address empty, and using tunnel=yes on the policy. This way you would avoid the NAT of the IPsec transport packets, so the ESP packets would not be encapsulated into UDP ones, but one more IP header would be used instead, so the "IP mentioned above" would have to be non-conflicting at both ends of the connection and used also as remote-address of the /interface eoip. Besides saving a few bytes of MTU, the advantage of this approach is that you can configure other IPsec settings freely, so you can use IKEv2 with certificate-based authentication etc.

Yet another approach is to use the script in dhcp client configuration to update the local-address of the /interface eoip at each WAN IP change, but I simply hate to wear the flash chip by rewriting configuration every now and then.

halacs · Tue May 07, 2019 10:34 pm

Sounds interesting! It's a bit much for the first glance now so give me some time to process what you wrote then I'll be back with the outcome.

halacs · Wed May 22, 2019 9:04 pm

Hi,

today I had some time to understand a bit more the configuration you suggested but it still not working somehow.
When I set local IP as my real IP on the WAN interface (ppoe now actually) my EoIP is working but in case of the private IP hack, it's not.
Here is my configuration hopefully without the sensitive part: https://gist.github.com/halacs/d7015ca4 ... 703ec32717
Can you check it please what is wrong?

Thanks!

sindy · Wed May 22, 2019 10:19 pm

Well, you've posted the configuration of only one end, and you haven't shown the runtime state using /ip ipsec peer print, /ip ipsec remote-peers print, /ip ipsec policy print. So I can't say which part of the setup fails.

Technical remark, it is better to post everything right here between [code] and [/code] tags than to link to other site.

halacs · Wed May 22, 2019 11:00 pm

You are right, sorry. Here are the both end of the tunnel.

Router A (where I have set NAT because of dynamic IP)

Code: Select all


[admin@router.local.example.com] > /ip ipsec peer print

Flags: X - disabled, D - dynamic, R - responder 

 0  D  ;;; eoip-tunnel-Zoldmali

       name="peer17" address=a.b.5.107/32 local-address=172.20.0.1 profile=default exchange-mode=main send-initial-contact=yes 



 1   R name="peer1" passive=yes profile=profile_1 exchange-mode=main send-initial-contact=yes 

[admin@router.local.example.com] > /ip ipsec remote-peers print

Flags: R - responder, N - natt-peer 

 #    ID                   STATE              REMOTE-ADDRESS                               DYNAMIC-ADDRESS      UPTIME              

 0                         message-1-sent     a.b.5.107                                

 1 R                       message-2-sent     a.b.5.107                                

[admin@router.local.example.com] > /ip ipsec policy print

Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default 

 0 T * group=default src-address=::/0 dst-address=::/0 protocol=all proposal=default template=yes 



 1  DA  ;;; eoip-tunnel-Zoldmali

       src-address=172.20.0.1/32 src-port=any dst-address=a.b.5.107/32 dst-port=any protocol=gre action=encrypt level=require 

       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

[admin@router.local.example.com] >

Router B

Code: Select all

[admin@zoldmali.intra.example.com] > /ip ipsec peer print

Flags: X - disabled, D - dynamic, R - responder 

 0  D  ;;; eoip-tunnel-Ostoros

       name="peer8" address=x.y.200.185/32 local-address=a.b.5.107 profile=default exchange-mode=main send-initial-contact=yes 

[admin@zoldmali.intra.example.com] > /ip ipsec remote-peers print

Flags: R - responder, N - natt-peer 

 #    ID                   STATE              REMOTE-ADDRESS                                                                                           DYNAMIC-ADDRESS                                                                 UPTIME              

 0                         message-3-sent     x.y.200.185                                                                                            

[admin@zoldmali.intra.example.com] > /ip ipsec policy print

Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default 

 0 T * group=default src-address=::/0 dst-address=::/0 protocol=all proposal=default template=yes 



 1  DA  ;;; eoip-tunnel-Ostoros

       src-address=a.b.5.107/32 src-port=any dst-address=x.y.200.185/32 dst-port=any protocol=gre action=encrypt level=require ipsec-protocols=esp tunnel=no proposal=default ph2-count=1

sindy · Wed May 22, 2019 11:48 pm

Okay, I forgot one thing, the peers dynamically generated by the EoIP astro clock act as both initiators and responders, so the connection may get messed up by the firewall. So in addition to the action=masquerade rule in chain=srcnat, you have to add also an action=dst-nat in-interface-list=WAN protocol=udp dst-port=500,4500 to-addresses=172.20.0.1 rule to chain=dstnat of /ip firewall nat.

The point is that when a connection attempt comes from the remote peer sooner than the local peer sends its own one, the firewall's connection tracker creates a connection with the local WAN IP as destination, and this tracked connection is constantly updated so it never expires. The peer doesn't accept packets coming to the WAN IP because it is set to accept only those coming to its local-address. It's strange, though, because you only have the two addresses at one peer, the other one should handle this normally. But start from here, let's see whether it helps.

But I've understood from what you wrote earlier that both ends have dynamic public IP so both need to be configured the other one's domain name, is that true?

CZFan · Thu May 23, 2019 12:23 am

It never came to my mind to try to push VLANs through a L2TP tunnel in bridge mode, but I've expected it would be enough to configure the /interface bridge port and /interface bridge vlan items also for the L2TP interfaces. However, it seems RouterOS is not ready for this (at least as of 6.44.3). Whereas it adds the L2TP interfaces dynamically to the /interface bridge port list, albeit under the .id instead of the name, even after adding it (also using the .id) to /interface bridge vlan, the frames don't get through if vlan-filtering=yes on the bridges - even the tagless ones don't. If vlan-filtering=no, both tagless and tagged frames do get tunneled, but that doesn't help you much.

So if you really need to tunnel L2 to another site via L3 network, use IPsec-encrypted EoIP rather than IPsec-encrypted L2TP with bridge mode activated.

@sindy,
Have you tried above with L2TP and bridging control protocol?

sindy · Thu May 23, 2019 9:54 am

Have you tried above with L2TP and bridging control protocol?

Am I missing something? To me,

L2TP tunnel in bridge mode

and

L2TP and bridging control protocol

are the same thing.

halacs · Thu May 23, 2019 5:57 pm

But I've understood from what you wrote earlier that both ends have dynamic public IP so both need to be configured the other one's domain name, is that true?

Yes, that's true. Both end are with dynamic IP changed quite rarely. Initial state is a EoIP tunnel with manually configured local IP addresses at both ends. It work fine already, but the dstnat-ed version.
First I'm trying to configure the one end. When I change local ip (let's call this endpoint as 'endpoint A') EOIP tunnel became broken and I cannot ping the other network segment.

Unfortunately you did not forget to mention that two rule. They are already there in the nat table.

Code: Select all

[admin@router.intra.example.com] > /ip firewall nat print

Flags: X - disabled, I - invalid, D - dynamic 

 0    ;;; EoIP DDNS

      chain=srcnat action=accept protocol=gre log=no log-prefix="" 



 1    ;;; default configuration

      chain=srcnat action=masquerade out-interface-list=WAN log=no log-prefix="" 



 2 I  ;;; university VPN

      ;;; sstp-UNIV not ready

      chain=srcnat action=masquerade out-interface=sstp-UNIV



 3    ;;; EoIP DDNS

      chain=dstnat action=dst-nat to-addresses=172.20.0.1 to-ports=500 protocol=udp in-interface-list=WAN log=no log-prefix="" 



 4    ;;; EoIP DDNS

      chain=dstnat action=dst-nat to-addresses=172.20.0.1 to-ports=4500 protocol=udp in-interface-list=WAN log=no log-prefix="" 



 5    ;;; HTTP

      chain=dstnat action=dst-nat to-addresses=192.168.0.6 to-ports=80 protocol=tcp in-interface-list=WAN dst-port=80 log=no log-prefix="" 



 6    ;;; HTTPS

      chain=dstnat action=dst-nat to-addresses=192.168.0.6 to-ports=443 protocol=tcp in-interface-list=WAN dst-port=443 log=no log-prefix="" 



 7    ;;; SSH

      chain=dstnat action=dst-nat to-addresses=192.168.0.6 to-ports=22 protocol=tcp in-interface-list=WAN dst-port=22 log=no log-prefix="" 



 8 X  ;;; SMTP

      chain=dstnat action=dst-nat to-addresses=192.168.0.199 to-ports=25 protocol=tcp in-interface-list=WAN dst-port=25 log=no log-prefix="" 



 9    ;;; torrent kliens

      chain=dstnat action=dst-nat to-addresses=192.168.0.1 to-ports=60995-60999 protocol=tcp in-interface-list=WAN dst-port=60995 log=no log-prefix="" 



10    ;;; szerver torrent kliens

      chain=dstnat action=dst-nat to-addresses=192.168.0.6 to-ports=51413 protocol=tcp in-interface-list=WAN dst-port=51413 log=no log-prefix="" 



11    ;;; Syncthing (Gabor1)

      chain=dstnat action=dst-nat to-addresses=192.168.0.1 to-ports=22000 protocol=tcp in-interface-list=WAN dst-port=22000 log=no log-prefix="" 





admin@router.intra.example.com] > /ip ipsec peer print

Flags: X - disabled, D - dynamic, R - responder 

 0  D  ;;; eoip-tunnel-Zoldmali

       name="peer21" address=x.x.5.107/32 local-address=172.20.0.1 profile=default exchange-mode=main send-initial-contact=yes 



 1   R name="peer1" passive=yes profile=profile_1 exchange-mode=main send-initial-contact=yes 



[admin@router.intra.example.com] > /ip ipsec remote-peers print

Flags: R - responder, N - natt-peer 

 #    ID                   STATE              REMOTE-ADDRESS                                                   DYNAMIC-ADDRESS                          UPTIME              

 0 R                       message-2-sent     x.x.5.107                                                    

 1                         message-1-sent     x.x.5.107

sindy · Thu May 23, 2019 6:38 pm

Would you mind disabling the manually created peer so that it wouldn't interfere? It's strange, I do this NAT trick quite routinely and it works.

halacs · Thu May 23, 2019 7:03 pm

Sure! I disabled the manually created peer (under IPsec/Peers in winbox) but nothing changed.

Code: Select all

[admin@router.intra.example.com] > /ip ipsec remote-peers print

Flags: R - responder, N - natt-peer 

 #    ID                   STATE              REMOTE-ADDRESS                                                   DYNAMIC-ADDRESS                          UPTIME              

 0                         message-1-sent     x.x.5.107                                                    

[admin@router.intra.example.com] > /ip ipsec peer print        

Flags: X - disabled, D - dynamic, R - responder 

 0  D  ;;; eoip-tunnel-Zoldmali

       name="peer23" address=x.x.5.107/32 local-address=172.20.0.1 profile=default exchange-mode=main send-initial-contact=yes 



 1 X R name="peer1" passive=yes profile=profile_1 exchange-mode=main send-initial-contact=yes

sindy · Thu May 23, 2019 8:54 pm

That makes no sense to me. I've replicated the setup once again and it simply works:

Client:

[me@clienTik] > ip ipsec peer print where dynamic                                                                                                                                     Flags: X - disabled, D - dynamic, R - responder
 0  D  ;;; eoip1
       name="peer8" address=192.168.5.1/32 local-address=192.168.163.1 profile=default exchange-mode=main send-initial-contact=yes
[me@clienTik] > ip ipsec policy print where (!disabled)
Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default
 0 T * group=default src-address=::/0 dst-address=::/0 protocol=all proposal=default template=yes

 1  DA  ;;; eoip1
       src-address=192.168.163.1/32 src-port=any dst-address=192.168.5.1/32 dst-port=any protocol=gre action=encrypt level=require ipsec-protocols=esp tunnel=no proposal=default
       ph2-count=2
[me@clienTik] > ip ipsec remote-peers print                                                                                                                                           Flags: R - responder, N - natt-peer
 #    ID                   STATE              REMOTE-ADDRESS                                                       DYNAMIC-ADDRESS                             UPTIME
 0 R                       established        192.168.5.1                                                                                                      5m44s
 1                         established        192.168.5.1                                                                                                      5m41s
[me@clienTik] > ip firewall connection print detail where (src-address~"192.168.5.1(\$|:)" or dst-address~"192.168.5.1(\$|:)") and !(protocol~"tcp|icmp") and !(dst-address~":524[67]|:5678")
Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying, F - fasttrack, s - srcnat, d - dstnat
 0  SAC   d protocol=udp src-address=192.168.5.1:4500 dst-address=192.168.5.100:4500 reply-src-address=192.168.163.1:4500 reply-dst-address=192.168.5.1:4500 timeout=2m59s
            orig-packets=78 orig-bytes=8 020 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=76 repl-bytes=6 411 repl-fasttrack-packets=0 repl-fasttrack-bytes=0
            orig-rate=640bps repl-rate=0bps

 1    C     protocol=gre src-address=192.168.163.1 dst-address=192.168.5.1 reply-src-address=192.168.5.1 reply-dst-address=192.168.163.1 gre-key=256 timeout=23s orig-packets=40
            orig-bytes=1 120 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=0bps
            repl-rate=0bps

 2    C     protocol=gre src-address=192.168.5.1 dst-address=192.168.163.1 reply-src-address=192.168.163.1 reply-dst-address=192.168.5.1 gre-key=256 timeout=23s orig-packets=38
            orig-bytes=1 064 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=0bps
            repl-rate=0bps

Server:

[me@serverTik] > ip ipsec peer print where dynamic
Flags: X - disabled, D - dynamic, R - responder
 0  D  ;;; eoip1
       name="peer8829" address=192.168.5.100/32 local-address=192.168.5.1 profile=default exchange-mode=main send-initial-contact=yes
[me@serverTik] > ip ipsec policy print where dynamic
Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default
 0  DA  ;;; eoip1
       src-address=192.168.5.1/32 src-port=any dst-address=192.168.5.100/32 dst-port=any protocol=gre action=encrypt level=require ipsec-protocols=esp tunnel=no proposal=default
       ph2-count=2
[me@serverTik] > ip ipsec remote-peers print
Flags: R - responder, N - natt-peer
 #    ID                   STATE              REMOTE-ADDRESS                                                        DYNAMIC-ADDRESS                              UPTIME
 0 RN                      established        192.168.5.100                                                                                                      9m4s
 1  N                      established        192.168.5.100                                                                                                      9m7s
[me@serverTik] > ip firewall connection print detail where (dst-address~"192.168.5.100" or src-address~"192.168.5.100") and !(protocol~"tcp|icmp") and !(dst-address~":524[67]")
Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying, F - fasttrack, s - srcnat, d - dstnat
 0  SAC     protocol=udp src-address=192.168.5.1:4500 dst-address=192.168.5.100:4500 reply-src-address=192.168.5.100:4500 reply-dst-address=192.168.5.1:4500 timeout=2m55s
            orig-packets=110 orig-bytes=10 297 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=109 repl-bytes=8 717 repl-fasttrack-packets=0 repl-fasttrack-bytes=0
            orig-rate=0bps repl-rate=0bps

 1    C     protocol=gre src-address=192.168.5.100 dst-address=192.168.5.1 reply-src-address=192.168.5.1 reply-dst-address=192.168.5.100 gre-key=256 timeout=25s orig-packets=57
            orig-bytes=1 596 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=0bps
            repl-rate=0bps

 2    C     protocol=gre src-address=192.168.5.1 dst-address=192.168.5.100 reply-src-address=192.168.5.100 reply-dst-address=192.168.5.1 gre-key=256 timeout=25s orig-packets=57
            orig-bytes=1 596 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=0bps
            repl-rate=0bps

halacs · Fri May 24, 2019 8:50 am

I'm just wondering loudly what could be the problem. I have PPTP and L2TP VPN servers on this end. Can it be the problem? Fast forward should be on or off on the bridge where 172.20.0.1 IP is?

sindy · Fri May 24, 2019 9:32 am

I'm just wondering loudly what could be the problem. I have PPTP and L2TP VPN servers on this end. Can it be the problem? Fast forward should be on or off on the bridge where 172.20.0.1 IP is?

The bridge should have no ports so the packets actually don't go through the bridge at all, it is just a hook to hang the local IP at.

PPTP is not related to IPsec in any way, L2TP could (or rather should, as there is no encryption in L2TP alone) be because it also dynamically creates an IPsec configuration (peer, identity, policy) if you set ipsec=yes in the L2TP configuration. But in your print outputs there was just a single dynamic IPsec peer, the other one you've disabled for the test was manually configured. When your peers are in that weird state (I've never seen "message-3-sent" so far), can you print connections at both ends the way I did? Something like /ip firewall connection print detail where src-address~":4\?500" or dst-address~":4\?500" and show me the result after anonymizing the IPs?

halacs · Fri May 24, 2019 5:54 pm

Code: Select all

[admin@zoldmali.intra.example.com] > /ip firewall connection print detail where src-address~":4\?500" or dst-address~":4\?500" 

Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying, F - fasttrack, s - srcnat, d - dstnat 

 0  SAC     protocol=udp src-address=x.y.5.107:500 dst-address=a.b.200.185:500 reply-src-address=a.b.200.185:500 reply-dst-address=x.y.5.107:500 timeout=2m55s orig-packets=6 099 orig-bytes=1 015 168 orig-fasttrack-packets=0 

            orig-fasttrack-bytes=0 repl-packets=5 013 repl-bytes=733 896 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=0bps repl-rate=0bps 



[admin@zoldmali.intra.example.com] > /ip ipsec peer print

Flags: X - disabled, D - dynamic, R - responder 

 0  D  ;;; eoip-tunnel-Ostoros

       name="peer9" address=a.b.200.185/32 local-address=x.y.5.107 profile=default exchange-mode=main send-initial-contact=yes 



[admin@zoldmali.intra.example.com] > /interface eoip print

Flags: X - disabled, R - running 

 0    name="eoip-tunnel-Ostoros" mtu=1500 actual-mtu=1500 l2mtu=65535 mac-address=02:8D:2E:10:ED:D5 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off loop-protect-send-interval=5s loop-protect-disable-time=5m 

      local-address=x.y.5.107 remote-address=home.example.com current-remote-address=a.b.200.185 tunnel-id=500 keepalive=10s,10 dscp=inherit clamp-tcp-mss=yes dont-fragment=no ipsec-secret="somethingverysecretpassword-NOT-THE-REAL" allow-fast-path=no

and the other end where I want to do the trick now:

Code: Select all

[admin@router.intra.example.com] > /ip firewall connection print detail where src-address~":4\?500" or dst-address~":4\?500"

Flags: E - expected, S - seen-reply, A - assured, C - confirmed, D - dying, F - fasttrack, s - srcnat, d - dstnat 

 0  SAC F   protocol=tcp src-address=192.168.0.249:50051 dst-address=192.168.0.248:554 reply-src-address=192.168.0.248:554 

            reply-dst-address=192.168.0.249:50051 tcp-state=established timeout=23h59m59s orig-packets=5 380 148 orig-bytes=216 617 382 

            orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=7 264 270 repl-bytes=5 164 107 870 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 

            orig-rate=17.2kbps repl-rate=213.4kbps 



 1  SAC     protocol=udp src-address=a.b.200.185:500 dst-address=x.y.5.107:500 reply-src-address=x.y.5.107:500 reply-dst-address=a.b.200.185:500 

            timeout=2m51s orig-packets=5 002 orig-bytes=731 016 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=6 085 repl-bytes=1 012 664 

            repl-fasttrack-packets=0 repl-fasttrack-bytes=0 orig-rate=0bps repl-rate=0bps 



 2  SAC F   protocol=tcp src-address=192.168.0.249:50040 dst-address=192.168.0.248:554 reply-src-address=192.168.0.248:554 

            reply-dst-address=192.168.0.249:50040 tcp-state=established timeout=4m59s orig-packets=31 337 506 orig-bytes=1 255 000 250 

            orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=63 081 209 repl-bytes=87 859 385 452 repl-fasttrack-packets=0 repl-fasttrack-bytes=0 

            orig-rate=137.2kbps repl-rate=9.3Mbps 



 3    C  s  protocol=udp src-address=172.20.0.1:500 dst-address=x.y.5.107:500 reply-src-address=x.y.5.107:500 reply-dst-address=a.b.200.185:119 

            timeout=3s orig-packets=2 orig-bytes=976 orig-fasttrack-packets=0 orig-fasttrack-bytes=0 repl-packets=0 repl-bytes=0 repl-fasttrack-packets=0 

            repl-fasttrack-bytes=0 orig-rate=0bps repl-rate=0bps 



[admin@router.intra.example.com] > /ip ipsec peer print

Flags: X - disabled, D - dynamic, R - responder 

 0  D  ;;; eoip-tunnel-Zoldmali

       name="peer30" address=x.y.5.107/32 local-address=172.20.0.1 profile=default exchange-mode=main send-initial-contact=yes 



 1 X R name="peer1" passive=yes profile=profile_1 exchange-mode=main send-initial-contact=yes 

 

 [admin@router.intra.example.com] > /interface eoip print

Flags: X - disabled, R - running 

 0  R name="eoip-tunnel-Zoldmali" mtu=1500 actual-mtu=1500 l2mtu=65535 mac-address=02:77:DB:60:AC:F6 arp=enabled arp-timeout=auto loop-protect=default 

      loop-protect-status=off loop-protect-send-interval=5s loop-protect-disable-time=5m local-address=172.20.0.1 remote-address=zoldmali.example.com 

      current-remote-address=x.y.5.107 tunnel-id=500 dscp=inherit clamp-tcp-mss=yes dont-fragment=no ipsec-secret="somethingverysecretpassword-NOT-THE-REAL" 

      allow-fast-path=no

sindy · Fri May 24, 2019 7:04 pm

Oops

I forgot to tell you one thing, to disable the EoIP tunnels at both ends for more than 180 seconds and then to try again. The connection tracker at the side where you use the 172.20.0.1 still remembers the connection between the public addresses, and any matching packet received from the remote side extends its life by another 3 minutes.

The src-nat and dst-nat rules are only consulted for packets whose source and destination sockets don't match to any existing connection. So you now have two connections at the client side, the old one which grabs all what comes from the remote side and forwards it to the public IP on which no IPsec peer listens, and the new one which does the src-nat but it can never see a reply because the other connection matches that reply and "steals" it.

Or you can remove the "wrong" connection using /ip firewall connection remove [find protocol~"udp" src-address~":500\$" dst-address~":500\$" (!srcnat) (!dstnat)].

halacs · Sat May 25, 2019 7:59 am

Magic

It started to work now. I have configured both end now at the same way. Very cool! Thanks!

One more question what I don't understand now: at both end, I can see a 'responder' and an 'initiator' in the ipsec remote peers. The strange thing is the two ends are different, meaning that at one and there are only private IP as local address, while at the others public IP is used for responder. Is this okay? If yes, why?

Code: Select all


[admin@router.intra.example.com] > /ip ipsec remote-peers print terse 

 0 RN local-address=172.20.0.1 port=4500 remote-address=x.y.5.107 port=1024 state=established side=responder uptime=4m29s last-seen=29s 

 1  N local-address=172.20.0.1 port=4500 remote-address=x.y.5.107 port=4500 state=established side=initiator uptime=4m20s last-seen=20s 



[admin@zoldmali.intra.example.com] > /ip ipsec remote-peers print terse

 0 RN local-address=x.y.5.107 port=4500 remote-address=a.b.81.15 port=4500 state=established side=responder uptime=4m13s last-seen=13s 

 1  N local-address=172.20.0.2 port=4500 remote-address=a.b.81.15 port=4500 state=established side=initiator uptime=4m21s last-seen=21s

sindy · Sat May 25, 2019 10:37 am

at both end, I can see a 'responder' and an 'initiator' in the ipsec remote peers. The strange thing is the two ends are different, meaning that at one end there are only private IP as local address, while at the others public IP is used for responder. Is this okay? If yes, why?

The mere fact that there is both an initiator and responder mode remote-peer at each end is OK - there are no "client" and "server" roles of the endpoints of EoIP (or GRE in general, or IPIP) tunnels (which is a difference as compared to L2TP), so the dynamically generated IPsec peers for these types of tunnels cannot be restricted to the responder role at any end.

As for why the responder mode remote-peer reports the public IP as its local-address, I have a strong suspicion that you have omitted the action=dst-nat rule for dst-port=4500 so the incoming packets from the opposite peer to your.pub.lic.IP:4500 are not diverted to the private address (but again, adding the rule or extending the one for dst-port=500 to dst-port=500,4500 is not enough, you have to disable the peers at both ends, remove the "wrong" tracked connection, and re-enable the peers). The assumption why it nevertheless works is a bit more complex:

What I know for sure is that is that if a given IKE(v1) connnection is initiated by reception of a packet at local port 500, and the NAT-T check finds out that there is a NAT on the path between the peers and switches over to NAT-T mode and use of port 4500, the first packet of that NAT-T mode connection (which arrives to port 4500) is accepted from any source address, not just the one from which the initial packet has arrived to port 500. This is essential for the NAT-T to work because NAT devices handling large networks often use a pool of outer addresses rather than just a single one, and the initial connection to port 500 and the subsequent connection to port 4500 are not related to each other as far as the NAT device can tell, so it has no reason to assign the same outer IP to both. So the IPsec stack definitely treats packets belonging to ongoing IKE conections differently than the initial packets. And I suspect that part of this different treatment is also that the first NAT-T mode packet is accepted not only from any source address but also at any local address, not just the one to which the initial packet has arrived to port 500.

MOBIKE goes a step further, updating the remote address with each received packet. So if remote peer's address changes, as soon as the local peer receives a packet belonging to an existing SA (according to its SPI header) from the new address, it updates the destination address of that remote peer with the source address of the packet just received. But that's outside the scope of RouterOS so far unless I've missed some announcement.

halacs · Sat May 25, 2019 6:35 pm

Thank you so much for your help and the detailed explanations!

I checked that end where the issue happened and the rules were in different order: first accept gre, then the two dst-nat and finally the default masquarade. I moved default masquarade rule at the 2nd place right after the accept gre and now the two ends works the same way with private IP.

Code: Select all

[admin@zoldmali.intra.example.com] > /ip firewall nat print terse

 0    comment=EoIP DDNS chain=srcnat action=accept protocol=gre src-address=172.20.0.2 log=no log-prefix="" 

 1    comment=default configuration chain=srcnat action=masquerade out-interface=ether1-gateway log=no log-prefix="" 

 2    comment=EoIP DDNS chain=dstnat action=dst-nat to-addresses=172.20.0.2 to-ports=500 protocol=udp in-interface=ether1-gateway dst-port=500 log=no 

log-prefix="" 

 3    comment=EoIP DDNS chain=dstnat action=dst-nat to-addresses=172.20.0.2 to-ports=4500 protocol=udp in-interface=ether1-gateway dst-port=4500 log=n

o log-prefix="" 

 4    comment=torrent chain=dstnat action=dst-nat to-addresses=192.168.2.3 to-ports=60001 protocol=tcp in-interface=ether1-gateway dst-port=60001 log=

no log-prefix="" 

 5 X  comment=Picur HTTP chain=dstnat action=dst-nat to-addresses=192.168.2.9 to-ports=80 protocol=tcp in-interface=ether1-gateway dst-port=80 log=no 

log-prefix="" 

 6    comment=Anita Docker Container SSH chain=dstnat action=dst-nat to-addresses=192.168.2.8 to-ports=4567 protocol=tcp in-interface=ether1-gateway d

st-port=4567 log=yes log-prefix=Anita Docker SSH kapcsolat

sindy · Sat May 25, 2019 7:15 pm

I checked that end where the issue happened and the rules were in different order: first accept gre, then the two dst-nat and finally the default masquarade. I moved default masquarade rule at the 2nd place right after the accept gre and now the two ends works the same way with private IP.

One more clarification - order of rules does matter, but only within the same chain (srcnat or dstnat in case of /ip firewall nat). So whether you put first all srcnat rules and then all dstnat rules or whether you interleave the rules belonging to these two chains while maintaining their mutual order in each chain only affects the amount of headache when someone (e.g. yourself three months later) has to read the rules, but not the firewall operation.

In another words, if the action=dst-nat rule for dst-port=4500 was not missing or shadowed by some wider action=dstnat rule before it, the change you've done was not the reason why the responder now behaves according to expectations.

But if you have, in the previous round, removed the "wrong" tracked connection only on the "client" side, and didn't disable the EoIP tunnel interfaces back then, the tracked connection without a dstnat may have survived on the "server" side, causing the effect you've seen.

Another clarification - since the IPsec-related action=dst-nat rules do not change the destination port, only the desination address, you may use a single rule for both with dst-port=500,4500 and no to-ports.

halacs · Sat May 25, 2019 8:22 pm

Actually the dst-nat rules were there. I disabled the eoip tunnel for a couple of minutes then enabled it back.

VLAN trunk - master-slave way of config on physical interfaces [SOLVED]

VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces [SOLVED]

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Re: VLAN trunk - master-slave way of config on physical interfaces

Who is online