PPP-BCP Bridge IP addressing [solved] DHCP clients don't activate over VPN

LaKing · Sun Jul 07, 2019 3:38 am

Dear Forum, ...

I have successfully set up a SITE-TO-SITE Layer2 VPN with L2TP/IPsec tunnel and BCP bridging with two Mikrotik routers.
Works with static IP addresses. ...

After setting up a DHCP server on the primary router, clients connected to the primary router get IP Addresses, but clients on the remote end of the VPN network don't "activate".

Once a device connects to the network on the remote end, a lease is offered so at least the mac and the hostname show up in the DHCP server's leases table, but the device does not set the offered ip up. Then the lease expires, and the offered lease disappears. ...

I've tried setting arp-proxy on both bridges but no luck so far.

Could anyone point me to where to look at?
Thank you. ...

LaKing · Thu Jul 11, 2019 2:13 am

While working with devices assigned with static IP addresses, it came to my attention that TCP/UDP communication is problematic towards the remote end of the vpn tunnel.

I have a hardware device that communicates with a software on a couple of TCP and UDP ports.
If the vpn bridge gets reset, the device signals the software it's presence, and I can work with the software.
Once the OS the software runs on reboots, the device signals it's presence, but the software can't communicate with it's ports.
A workaround seems to be to reset the vpn bridge, ...

Looking for a permanent solution for both of these problems.

LaKing · Sat Jul 13, 2019 4:52 pm

An export from the server


### VPN 'server'

# jul/13/2019 15:32:11 by RouterOS 6.45.1
# software id = ZTRG-65M4
#
# model = CCR1009-7G-1C
# serial number = 84A1078A5D6B
/interface bridge
add comment="capsman" fast-forward=no name=capsman_bridge
add admin-mac=64:D1:54:DF:E2:CF arp=proxy-arp auto-mac=no comment="SITE-TO-SITE Layer2 VPN" name=hangmaffia_vpn_bridge
add comment="MazelTov LAN and CAPSMAN" fast-forward=no name=mazel_bridge
/interface ethernet
set [ find default-name=combo1 ] comment=WAN
set [ find default-name=ether1 ] comment="AUDIOLAN " speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] comment=BELACAM speed=100Mbps
set [ find default-name=ether5 ] comment="CAPSMAN" speed=100Mbps
set [ find default-name=ether6 ] comment=MAZELTOV speed=100Mbps
set [ find default-name=ether7 ] comment=HANGMAFFIA_VPN speed=100Mbps
/caps-man configuration
add country=hungary datapath.bridge=mazel_bridge name=MazelTov ssid=MazelTov
add country=hungary datapath.bridge=capsman_bridge name=Instant-Fogas ssid=INSTANT-FOGAS
/caps-man interface
add configuration=MazelTov disabled=no l2mtu=1600 mac-address=B8:69:F4:0A:B3:82 master-interface=none name=cap42 radio-mac=B8:69:F4:0A:B3:82 radio-name=B869F40AB382
add configuration=MazelTov disabled=no l2mtu=1600 mac-address=B8:69:F4:0A:B3:81 master-interface=none name=cap43 radio-mac=B8:69:F4:0A:B3:81 radio-name=B869F40AB381
add configuration=Instant-Fogas disabled=no l2mtu=1600 mac-address=CC:2D:E0:C4:2E:46 master-interface=none name=cap44 radio-mac=CC:2D:E0:C4:2E:46 radio-name=CC2DE0C42E46
add configuration=Instant-Fogas disabled=no l2mtu=1600 mac-address=CC:2D:E0:C4:2E:45 master-interface=none name=cap45 radio-mac=CC:2D:E0:C4:2E:45 radio-name=CC2DE0C42E45
/interface list
add name=WAN
add name=LAN
add name=VPN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add comment="AUDIOLAN 117.x" name=audiolanpool ranges=192.168.117.10-192.168.117.250
add comment="TESTING 118.x" name=testing-pool ranges=192.168.118.10-192.168.118.250
add comment=MAZELTOV-POOL name=mazelpool ranges=192.168.10.1-192.168.15.250
add comment="CAPSMAN pool " name=capsmanpool ranges=172.16.1.1-172.16.255.254
add comment=BELACAM-pool name=belacampool ranges=192.168.1.10-192.168.1.250
add name=testing-vpn-pool ranges=192.168.120.10-192.168.120.20
add comment="Hangmaffia VPN" name=hangmaffia_vpn_pool ranges=10.10.0.10-10.10.0.99
/ip dhcp-server
add address-pool=audiolanpool disabled=no interface=ether1 name=audiolan
add address-pool=mazelpool disabled=no interface=mazel_bridge name=mazeltov
add address-pool=capsmanpool disabled=no interface=capsman_bridge name=instant-fogas-capsman
add address-pool=belacampool disabled=no interface=ether4 name=belacam
add address-pool=hangmaffia_vpn_pool disabled=no interface=hangmaffia_vpn_bridge name=hangmaffia_vpn_server
/ppp profile
add bridge=hangmaffia_vpn_bridge comment=SITE-TO-SITE-Layer2-VPN local-address=10.11.0.1 name=hangmaffia_vpn
/caps-man manager
set enabled=yes
/caps-man manager interface
set [ find default=yes ] forbid=yes
add disabled=no interface=capsman_bridge
add disabled=no interface=mazel_bridge
/caps-man provisioning
add action=create-dynamic-enabled comment="IF Identity starts with underscore, we use this config" identity-regexp=^_ master-configuration=MazelTov
add action=create-dynamic-enabled comment="Default config" master-configuration=Instant-Fogas
/interface bridge nat
add action=accept chain=srcnat
/interface bridge port
add bridge=capsman_bridge comment="Fogas wifi port" interface=ether5
add bridge=mazel_bridge comment=Mazel interface=ether6
add bridge=capsman_bridge comment="Hangmaffia VPN" interface=ether2
add bridge=hangmaffia_vpn_bridge comment="Dedicated hangmaffia VPN port" interface=ether7 trusted=yes
/interface l2tp-server server
set default-profile=hangmaffia_vpn enabled=yes ipsec-secret=secret mrru=1600 use-ipsec=required
/interface list member
add interface=ether1 list=LAN
add interface=combo1 list=WAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=VPN
/ip address
add address=192.168.117.1/24 comment="LAN AUDIOLAN" interface=ether1 network=192.168.117.0
add address=89.133.151.117/28 comment="WAN / f5.d250.hu" interface=combo1 network=89.133.151.112
add address=192.168.118.1/24 comment="LAN TESTING" disabled=yes interface=ether7 network=192.168.118.0
add address=89.133.151.118/28 comment="WAN / f6.d250.hu" interface=combo1 network=89.133.151.112
add address=89.133.151.119/28 comment="WAN / f7.d250.hu" interface=combo1 network=89.133.151.112
add address=89.133.151.120/28 comment="WAN / f8.d250.hu" interface=combo1 network=89.133.151.112
add address=89.133.151.121/28 comment="WAN / f9.d250.hu" interface=combo1 network=89.133.151.112
add address=89.133.151.122/28 comment="WAN / fa.d250.hu" interface=combo1 network=89.133.151.112
add address=89.133.151.123/28 comment="WAN / fb.d250.hu" interface=combo1 network=89.133.151.112
add address=192.168.119.1/24 comment="LAN ether3" disabled=yes interface=ether3 network=192.168.119.0
add address=192.168.1.1/24 comment="LAN BELACAM" interface=ether4 network=192.168.1.0
add address=172.16.0.1/16 comment="LAN ether5 CAPSMAN = 172.16.0.1 - 172.16.255.254" interface=capsman_bridge network=172.16.0.0
add address=192.168.0.1/20 comment="LAN MAZELTOV " interface=mazel_bridge network=192.168.0.0
add address=89.133.151.116/28 comment="WAN / f4-cam.d250.hu" interface=combo1 network=89.133.151.112
add address=10.10.0.1/24 interface=hangmaffia_vpn_bridge network=10.10.0.0
/ip dhcp-server lease
## some statically assigned dhcp leases ...
/ip dhcp-server network
add address=10.10.0.0/24 comment=HANGMAFFIA_VPN gateway=10.10.0.1
add address=172.16.0.0/16 comment="INSTANT-FOGAS CAPSMAN ;; 172.16.0.1 - 172.16.255.254" gateway=172.16.0.1 netmask=16
add address=192.168.0.0/20 comment="MAZELTOV ;; 192.168.0.1 - 192.168.15.254" gateway=192.168.0.1
add address=192.168.1.0/24 comment="BELACAM ;; 192.168.1.1 - 192.168.1.254" gateway=192.168.1.1 netmask=24
add address=192.168.117.0/24 comment="AUDIOLAN ;; 192.168.117.1 - 192.168.117.254" gateway=192.168.117.1 netmask=24
add address=192.168.118.0/24 comment=TESTING gateway=192.168.118.1
/ip dns
set servers=195.184.180.4,195.184.181.4
/ip firewall filter
add action=accept chain=input comment="ICMP allow PING" disabled=yes icmp-options=8:0-255 protocol=icmp
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid disabled=yes
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" disabled=yes in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid disabled=yes
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new disabled=yes in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment=default out-interface-list=WAN
add action=masquerade chain=srcnat comment="CAPSMAN MASQ" out-interface=capsman_bridge
add action=masquerade chain=srcnat comment="MAZEL MASQ" out-interface=mazel_bridge
## some additional port maps 
/ip route
add distance=1 gateway=89.133.151.126
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ppp secret
add comment="client" name=x111 password=password profile=hangmaffia_vpn service=l2tp
/system clock
set time-zone-name=Europe/Budapest

And from the client


### VPN 'client'

# jul/13/2019 15:36:52 by RouterOS 6.45.1
# software id = IL68-RR17
#
# model = 2011iL
# serial number = 8E7A0A6BFA8A
/interface bridge
add admin-mac=74:4D:28:A6:A0:58 arp=proxy-arp auto-mac=no comment=defconf name=bridge
add admin-mac=74:4D:28:A6:A0:58 arp=proxy-arp auto-mac=no name=hangmaffia_vpn_bridge
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether7 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether8 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether9 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
set [ find default-name=ether10 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=bridge name=defconf
/ppp profile
add bridge=hangmaffia_vpn_bridge comment="SITE-TO-SITE Layer2 VPN" local-address=10.11.0.111 name=hangmaffia_vpn
/interface l2tp-client
add connect-to=fx.d250.hu disabled=no ipsec-secret=secret keepalive-timeout=disabled mrru=1600 name=l2tp-hangmaffia password=password profile=hangmaffia_vpn use-ipsec=yes user=x111
/interface bridge port
add bridge=hangmaffia_vpn_bridge comment=defconf interface=ether2
add bridge=hangmaffia_vpn_bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=ether6
add bridge=bridge comment=defconf interface=ether7
add bridge=bridge comment=defconf interface=ether8
add bridge=bridge comment=defconf interface=ether9
add bridge=bridge comment=defconf interface=ether10
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface l2tp-server server
set enabled=yes ipsec-secret=secret use-ipsec=yes
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=10.10.0.111/24 interface=hangmaffia_vpn_bridge network=10.10.0.0
/ip dhcp-client
add comment=defconf dhcp-options=hostname,clientid disabled=no interface=ether1
/ip dhcp-server lease
add address=192.168.88.254 client-id=ff:c8:d:4b:f0:0:4:c:76:65:2a:4c:a0:41:2b:a6:f6:b4:b6:df:16:37:b9 comment=Fedbook mac-address=A0:CE:C8:0D:4B:F0 server=defconf
add address=192.168.88.253 comment=Neutrino mac-address=00:60:35:2A:95:29 server=defconf
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 name=router.lan
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" disabled=yes in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/system clock
set time-zone-name=Europe/Budapest
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

doneware · Wed Jul 17, 2019 4:30 pm

the problem is with the ppp profile definition:

/ppp profile
add bridge=hangmaffia_vpn_bridge comment=SITE-TO-SITE-Layer2-VPN local-address=10.11.0.1 name=hangmaffia_vpn

if you do BCP, you may not have _any_ IP address configured on any PPP interfaces. so if you want to have IP addresses, just stick them on the bridge interfaces under "/ip address"
using local or remote address on ppp sessions (l2tp, pptp, sstp, whatsoever) implies /32 addressing and it will result ip addressing on a bridge member interface, which is flawed.

server:
/ppp profile
add bridge=hangmaffia_vpn_bridge comment=SITE-TO-SITE-Layer2-VPN !local-address !remote-address name=hangmaffia_vpn

client:
/ppp profile
add bridge=hangmaffia_vpn_bridge comment="SITE-TO-SITE Layer2 VPN" !local-address !remote-address name=hangmaffia_vpn

you can also get rid of proxy arp configuration and hard coded arp entries.

sindy · Wed Jul 17, 2019 5:20 pm

if you do BCP, you may not have _any_ IP address configured on any PPP interfaces. so if you want to have IP addresses, just stick them on the bridge interfaces under "/ip address"
using local or remote address on ppp sessions (l2tp, pptp, sstp, whatsoever) implies /32 addressing and it will result ip addressing on a bridge member interface, which is flawed.

I am afraid this is a misconception. If you use BCP in PPP connections, it doesn't switch the tunnel over from L3 mode to L2 one. It creates an L2 tunnel in addition to the basic L3 one, which is totally independent from it. So the IP address indicated in secret or profile is assigned to the L3 interface of the L3 tunnel, and the L2 interface of the L2 tunnel is just dynamically added as a port of the bridge indicated in the profile, no IP address is assigned to the L2 tunnel interface or to the bridge.

doneware · Wed Jul 17, 2019 6:46 pm

I am afraid this is a misconception. If you use BCP in PPP connections, it doesn't switch the tunnel over from L3 mode to L2 one. It creates an L2 tunnel in addition to the basic L3 one, which is totally independent from it. So the IP address indicated in secret or profile is assigned to the L3 interface of the L3 tunnel, and the L2 interface of the L2 tunnel is just dynamically added as a port of the bridge indicated in the profile, no IP address is assigned to the L2 tunnel interface or to the bridge.

you might want to check out the entires for active BCP sessions under "/interface l2tp-server" with the monitor command, and you'll see that those remote/local address entries show up there instead of the 0.0.0.0 (unnumbered) setup. the problem with this is, that certain packets (better said: frames) are forwarded along the tunnels, but some aren't. because of this, your client on the remote endpoint, if it starts to do DHCP discovers, will not be able to actually acquire the address. i observed that broadcast communication goes fine, but some unicast (dhcp request) is not handled properly in this case. both examples below are configured for BCP, but the latter one is with an username that has local/remote address set:

corrent one with BCP

[admin@hgw] /interface l2tp-server> monitor 1
          status: connected
          uptime: 3d21h21m50s
            user: cambridge
       caller-id: 192.168.1.133
        encoding: 
             mtu: 1500
             mru: 1500
   local-address: 0.0.0.0
  remote-address: 0.0.0.0

the one with 'local-address' set

[admin@hgw] /interface l2tp-server> monitor 2
          status: connected
          uptime: 23s
            user: cmb2
       caller-id: 31.46.xx.xx
        encoding: 
             mtu: 1500
             mru: 1500
   local-address: 1.2.3.4
  remote-address: 2.3.4.5

anyway, the golden rule of bridging says that _all_ L3/packet related functions shall be configured on the bridge interface and never to the bridge member interfaces.
with regards to the BCP/L3 discussion:
in case of L3 over L2TP you essentially do L3 over PPP over L2TP, whereas in BCP you transmit ethernet frames with BCP over PPP over L2TP. if you have IP entries (/32 addressing) on the L2TP connection, routeros will parse the packets differently.

i included two screenshots showing the correct header stack

bcp.png

l3.png

and another one which happens to go over a BCP enabled tunnel that has local/remote address negotiated by LCP. as you see, the receiver side (since it has IP addresses configured on the tunnels) will try to parse the BCP header as IP header and the story is over.

l3bcp.png

sindy · Wed Jul 17, 2019 7:57 pm

@doneware,

I'm not the OP (at least I know only 5 or so Hungarian words, and probably cannot pronounce properly even those I know)
your packet dissections confirm what I say, that the L2 tunnel and the L3 tunnel established via L2TP are fully independent from one another
I fully agree with you that attaching IP configuration to a slave (member) interface of a bridge is a Bad Idea

But I am very interested in what you say. The OP has attached to the maffia bridge an IP address which has nothing to do with the one used on the L3 tunnel interface, and the pool used by the dhcp server is in the same subnet like the address attached to the maffia bridge, and the dhcp server itself is also properly attached to the maffia bridge. On the client side, only the two Ethernet interfaces and the L2 tunnel are member ports of the maffia bridge.

So do I get it right that you are saying that once the DHCP client at the client side sends a unicast packet to the bridge (DHCPREQUEST), something (like the fact that there is an IP address attached to the maffia bridge at the client end) causes the DHCPREQUEST packet to be routed instead of bridged to the server side?

LaKing · Tue Jul 23, 2019 11:56 pm

The workaround suggested by @doneware fixed the problem we had. Thank you very very much! I own you many beers! ...

The key difference is to assign ip addresses in '/ip addresses' configuration, and we left the local and remote address parts empty. I have full working control of the device.

LaKing · Thu Jul 25, 2019 3:36 am

It seems that there are still some communication issues. ... :/ ...

PPP-BCP Bridge IP addressing [solved] DHCP clients don't activate over VPN

PPP-BCP Bridge IP addressing [solved] DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: DHCP clients don't activate over VPN

Re: PPP-BCP Bridge IP addressing [solved] DHCP clients don't activate over VPN

Who is online