Basic setup advice + Video & Sip dropout issues

reetp · Mon Dec 06, 2021 7:54 pm

Hi,

Struggling with basic stuff here. I suspect personal stupidity is to blame, but for the life of me I can't see where

Migrating from an Endian distro on my own hardware.

Seeming to get dropout with video across a standard network connection, and voip across the VPNs (without much other traffic, despite a Gigabit fibre connection!). We also can't get near the raw up/down speed that we get directly connected to the ISP modem.

Similar issues appeared on the Endian box and I thought it might be resource constrained, so thought I might migrate, but have seen similar issues with the Mikrotik.

Local LAN all runs happily at 1Gb/s

I guess trying to pin down the real problem will be the first thing. PPPoE/MTU issues?

I also presume I need some form of traffic management but happy to be proved wrong.

Setup
RB2011UiAS
OS 6.49.1

Single WAN interface with PPPoE through the ISP bridged modem. 'Upto' 1 Gb/s connection
Lan Bridged across sfp and eth 2-5
Router IP 192.168.10.250/24
Local Linux server does DHCP.

Two Ipsec IKE v2 connections with certificates, one to Endian, one to Libreswan (took a while to get it working but but nice and stable it seems)

Remote Ipsec networks
10.0.0.0/24 - Remote Endian to network with PSTN exchange with SIP card
192.168.98.0/24 - Libreswan on Linux with FreePBX

Setup below with a pretty basic firewall. A few ports forwarded to my server.

Be grateful for some advice on a more professional setup

Thanks.
John

# dec/06/2021 14:27:13 by RouterOS 6.49.1
# software id = FSDE-XXW1
#
# model = 2011UiAS
# serial number = 75xxxxxxx
/interface bridge add admin-mac=6C:3B:6B:84:2A:CE auto-mac=no comment=defconf name=bridge
/interface ethernet set [ find default-name=ether6 ] disabled=yes
/interface ethernet set [ find default-name=ether7 ] disabled=yes
/interface ethernet set [ find default-name=ether8 ] disabled=yes
/interface ethernet set [ find default-name=ether9 ] disabled=yes
/interface ethernet set [ find default-name=ether10 ] disabled=yes
/interface pppoe-client add add-default-route=yes disabled=no interface=ether1 name=pppoe-client password=ppp service-name=ISP user=adsl@isp
/interface list add comment=defconf name=WAN
/interface list add comment=defconf name=LAN

/ip pool add name=default-dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server add address-pool=default-dhcp interface=bridge name=defconf

/interface bridge port add bridge=bridge comment=defconf interface=ether2
/interface bridge port add bridge=bridge comment=defconf interface=ether3
/interface bridge port add bridge=bridge comment=defconf interface=ether4
/interface bridge port add bridge=bridge comment=defconf interface=ether5
/interface bridge port add bridge=bridge comment=defconf interface=sfp1

/ip neighbor discovery-settings set discover-interface-list=none
/interface detect-internet set detect-interface-list=all

/interface list member add comment=defconf interface=bridge list=LAN
/interface list member add comment=defconf interface=ether1 list=WAN
/interface list member add interface=pppoe-client list=WAN

/ip address add address=192.168.10.250/24 comment=defconf interface=bridge network=192.168.10.0
/ip cloud set update-time=no
/ip dhcp-client add comment=defconf interface=ether1
/ip dhcp-server network address=192.168.10.0/24 comment=defconf dns-server=192.168.88.1 gateway=192.168.10.250 netmask=24
/ip dns set servers=208.67.222.222,208.67.220.220
/ip dns static add address=192.168.10.250 comment=defconf name=router.lan

/ip firewall address-list add address=192.168.10.0/24 list="Router Admins"
/ip firewall filter add action=accept chain=input comment="Router Admin Access - Local" dst-address=192.168.10.250 dst-port=22,2224,8291 protocol=tcp src-address-list="Router Admins"
/ip firewall filter add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
/ip firewall filter add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid log-prefix="Invalid Input"
/ip firewall filter add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
/ip firewall filter add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
/ip firewall filter add action=accept chain=input comment="allow IPsec NAT" dst-port=4500 protocol=udp
/ip firewall filter add action=accept chain=input comment="allow IKE" dst-port=500 protocol=udp
/ip firewall filter add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
/ip firewall filter add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
/ip firewall filter add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
/ip firewall filter add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
/ip firewall filter add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid log-prefix="Invalid forward"
/ip firewall filter add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall filter add action=drop chain=input comment="defconf: drop all not coming from LAN" disabled=yes in-interface-list=!LAN
/ip firewall nat add action=accept chain=srcnat comment="Ipsec Endian" dst-address=10.0.0.0/24 src-address=192.168.10.0/24
/ip firewall nat add action=accept chain=srcnat comment="Ipsec Libreswan.ip" dst-address=192.168.98.0/24 log=yes src-address=192.168.10.0/24
/ip firewall nat add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none log-prefix=Masquerade out-interface-list=WAN
/ip firewall nat add action=dst-nat chain=dstnat comment="SERVER SMTP" dst-address=my.local.wan.ip dst-port=25 protocol=tcp to-addresses=192.168.10.1 to-ports=25
/ip firewall nat add action=dst-nat chain=dstnat comment="SERVER SMTPS" dst-address=my.local.wan.ip dst-port=465 protocol=tcp to-addresses=192.168.10.1 to-ports=465
/ip firewall nat add action=dst-nat chain=dstnat comment="SERVER HTTPS" dst-address=my.local.wan.ip dst-port=443 protocol=tcp to-addresses=192.168.10.1 to-ports=443
/ip firewall nat add action=dst-nat chain=dstnat comment="SERVER IMAPS" dst-address=my.local.wan.ip dst-port=993 protocol=tcp to-addresses=192.168.10.1 to-ports=993
/ip firewall nat add action=dst-nat chain=dstnat comment="SERVER SSH" dst-address=my.local.wan.ip dst-port=22222 log=yes log-prefix=ssh protocol=tcp to-addresses=192.168.10.1 to-ports=22222
/ip firewall nat add action=dst-nat chain=dstnat comment="SERVER Emby https" dst-address=my.local.wan.ip dst-port=8920 protocol=tcp to-addresses=192.168.10.190 to-ports=8920
/ip firewall raw add action=notrack chain=prerouting dst-address=10.0.0.0/24 src-address=192.168.10.0/24
/ip firewall raw add action=notrack chain=prerouting dst-address=192.168.10.0/24 src-address=10.0.0.0/24

/ip ipsec identity add auth-method=digital-signature certificate=LocalCert1 peer=ike2-Endian remote-certificate=RemoteCert1
/ip ipsec identity add auth-method=digital-signature certificate=LocalCert2 peer=ike2-Libreswan remote-certificate=RemoteCert2
/ip ipsec policy add dst-address=10.0.0.0/24 peer=ike2-Endian proposal=ike2-sha256 src-address=192.168.10.0/24 tunnel=yes
/ip ipsec policy add dst-address=192.168.98.0/24 peer=ike2-Libreswan proposal=ike2-sha256 src-address=192.168.10.0/24 tunnel=yes
/ip ipsec profile set [ find default=yes ] dh-group=modp2048 enc-algorithm=aes-256 hash-algorithm=sha512
/ip ipsec profile add dh-group=modp2048 enc-algorithm=aes-256 hash-algorithm=sha256 lifetime=6h name=ike2-remote prf-algorithm=sha256
/ip ipsec peer add address=Endian.ip/32 exchange-mode=ike2 name=ike2-Endian profile=ike2-remote
/ip ipsec peer add address=Libreswan.ip/32 exchange-mode=ike2 name=ike2-Libreswan profile=ike2-remote
/ip ipsec proposal set [ find default=yes ] auth-algorithms=sha256 enc-algorithms=aes-256-cbc,aes-256-gcm pfs-group=modp2048
/ip ipsec proposal add auth-algorithms=sha256 enc-algorithms=aes-256-cbc name=ike2-sha256 pfs-group=modp2048

reetp · Mon Dec 06, 2021 8:52 pm

Thinking aloud I have a suspicion this may be to do with MTU.

This is the biggest packer we can send:

ping -M do -s 1452 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 1452(1480) bytes of data.
76 bytes from 8.8.8.8: icmp_seq=1 ttl=116 (truncated)

This breaks

ping -M do -s 1454 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 1454(1482) bytes of data.
ping: local error: Message too long, mtu=1480

I can see the MTU & MRU is set in the PPPoE client at 1480. I've seen plenty on clamping etc. Is this a possible issue?

Also note that the IPSEC MTU is lower - 1378 is high as it will go. Hmmmmm.

ping -M do -s 1378 192.168.98.1
PING 192.168.98.1 (192.168.98.1) 1378(1406) bytes of data.
1386 bytes from 192.168.98.1: icmp_seq=1 ttl=63 time=28.0 ms

Another small oddity to note.

My server pings the FreePBX box every minute and i then see this in the firewall logs.

srcnat: in:(unknown 0) out:pppoe-client, src-mac b6:5c:ce:8e:ca:fb, proto ICMP (type 8, code 0), 192.168.10.1->192.168.98.1, len 84

It isn't important in the scheme of things but I don't understand why, and what to do about it?

reetp · Wed Dec 08, 2021 3:05 pm

So I fixed this:

srcnat: in:(unknown 0) out:pppoe-client, src-mac b6:5c:ce:8e:ca:fb, proto ICMP (type 8, code 0), 192.168.10.1->192.168.98.1, len 84

I had forgotten to add these - I had done for the other VPN. Doh.

ip firewall raw add chain=prerouting action=notrack src-address=192.168.10.0/24 dst-address=192.168.98.0/24
ip firewall raw add chain=prerouting action=notrack src-address=192.168.98.0/24 dst-address=192.168.10.0/24

Would still like some advice on PPPoE speeds and traffic prioritisation.

Basic setup advice + Video & Sip dropout issues

Basic setup advice + Video & Sip dropout issues

Re: Basic setup advice + Video & Sip dropout issues

Re: Basic setup advice + Video & Sip dropout issues

Who is online