Community discussions

MikroTik App
 
mikehhhhhhh
just joined
Topic Author
Posts: 10
Joined: Tue Apr 27, 2021 10:47 am

ICMP Packet loss when WAN is saturated

Thu Apr 29, 2021 12:00 pm

Something I've not experienced on previous routers, presumably because it was something baked in and on Mikrotik I need to configure it, however;

I experience packet loss both in and out while saturating the connection - particularly running speedtests.

Connection is 900/100 over PPPoE over FTTP.

I've tried prioritising ICMP, as well as creating a simple queue with dest=pppoe and experimented with max-limit, but it seems I have to significantly reduce my max download speed before the packet loss goes away.

I assume the speed includes overheads. The currently configured 95/940 gives me around 90/800 real world.

This ICMP packet loss is observed by pinging out while running speedtests, but also with a ping monitor;

Image


[admin@MikroTik] > export compact hide-sensitive
# apr/29/2021 09:50:09 by RouterOS 6.48.2
# software id = Y7QR-K6J3
#
# model = RB4011iGS+
# serial number = D4460DEE27EA
/interface bridge
add admin-mac=00:BB:01:32:00:00 auto-mac=no comment=defconf name=bridge
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=pppoe-out1 use-peer-dns=yes user=*
/interface ethernet switch port
set 0 default-vlan-id=0
set 1 default-vlan-id=0
set 2 default-vlan-id=0
set 3 default-vlan-id=0
set 4 default-vlan-id=0
set 5 default-vlan-id=0
set 6 default-vlan-id=0
set 7 default-vlan-id=0
set 8 default-vlan-id=0
set 9 default-vlan-id=0
set 10 default-vlan-id=0
set 11 default-vlan-id=0
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip kid-control
add name=Lily
/ip pool
add name=dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=dhcp disabled=no interface=bridge lease-script="# When \"1\" all DNS entries with IP address of D\
    HCP lease are removed\r\
    \n:local dnsRemoveAllByIp \"1\"\r\
    \n# When \"1\" all DNS entries with hostname of DHCP lease are removed\r\
    \n:local dnsRemoveAllByName \"1\"\r\
    \n# When \"1\" addition and removal of DNS entries is always done also for non-FQDN hostname\r\
    \n:local dnsAlwaysNonfqdn \"1\"\r\
    \n# DNS domain to add after DHCP client hostname\r\
    \n:local dnsDomain \"lan\"\r\
    \n# DNS TTL to set for DNS entries\r\
    \n:local dnsTtl \"00:15:00\"\r\
    \n# Source of DHCP client hostname, can be \"lease-hostname\" or any other lease attribute, like \"host-name\"\
    \_or \"comment\"\r\
    \n:local leaseClientHostnameSource \"lease-hostname\"\r\
    \n\r\
    \n:local leaseComment \"dhcp-lease-script_\$leaseServerName_\$leaseClientHostnameSource\"\r\
    \n:local leaseClientHostname\r\
    \n:if (\$leaseClientHostnameSource = \"lease-hostname\") do={\r\
    \n  :set leaseClientHostname \$\"lease-hostname\"\r\
    \n} else={\r\
    \n  :set leaseClientHostname ([:pick \\\r\
    \n    [/ip dhcp-server lease print as-value where server=\"\$leaseServerName\" address=\"\$leaseActIP\" mac-ad\
    dress=\"\$leaseActMAC\"] \\\r\
    \n    0]->\"\$leaseClientHostnameSource\")\r\
    \n}\r\
    \n:local leaseClientHostnames \"\$leaseClientHostname\"\r\
    \n:if ([:len [\$dnsDomain]] > 0) do={\r\
    \n  :if (\$dnsAlwaysNonfqdn = \"1\") do={\r\
    \n    :set leaseClientHostnames \"\$leaseClientHostname.\$dnsDomain,\$leaseClientHostname\"\r\
    \n  } else={\r\
    \n    :set leaseClientHostnames \"\$leaseClientHostname.\$dnsDomain\"\r\
    \n  }\r\
    \n}\r\
    \n:if (\$dnsRemoveAllByIp = \"1\") do={\r\
    \n  /ip dns static remove [/ip dns static find comment=\"\$leaseComment\" and address=\"\$leaseActIP\"]\r\
    \n}\r\
    \n:foreach h in=[:toarray value=\"\$leaseClientHostnames\"] do={\r\
    \n  :if (\$dnsRemoveAllByName = \"1\") do={\r\
    \n    /ip dns static remove [/ip dns static find comment=\"\$leaseComment\" and name=\"\$h\"]\r\
    \n  }\r\
    \n  /ip dns static remove [/ip dns static find comment=\"\$leaseComment\" and address=\"\$leaseActIP\" and nam\
    e=\"\$h\"]\r\
    \n  :if (\$leaseBound = \"1\") do={\r\
    \n    :delay 1\r\
    \n    /ip dns static add comment=\"\$leaseComment\" address=\"\$leaseActIP\" name=\"\$h\" ttl=\"\$dnsTtl\"\r\
    \n  }\r\
    \n}" name=defconf
/queue simple
add limit-at=512k/512k max-limit=512k/512k name=ICMP packet-marks=icmp-pkt target=""
add dst=pppoe-out1 max-limit=95M/940M name=queue1 target=""
/user group
set full policy=\
    local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,password,web,sniff,sensitive,api,romon,dude,tikapp
/interface bridge port
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=ether6
add bridge=bridge comment=defconf interface=ether7
add bridge=bridge comment=defconf interface=ether8
add bridge=bridge comment=defconf interface=ether9
add bridge=bridge comment=defconf interface=ether10
add bridge=bridge comment=defconf interface=sfp-sfpplus1
add bridge=bridge interface=ether2
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=pppoe-out1 list=WAN
add interface=ether3 list=WAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether2 network=192.168.88.0
/ip dhcp-client
add comment=defconf interface=ether1
add add-default-route=no !dhcp-options disabled=no interface=ether3
/ip dhcp-server lease
add address=192.168.88.245 client-id=1:0:11:32:b4:c0:4d mac-address=00:11:32:B4:C0:4D server=defconf
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 domain=lan gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes servers=1.1.1.1,1.0.0.1
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related \
    disabled=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=mark-connection chain=prerouting new-connection-mark=icmp-con passthrough=yes protocol=icmp
add action=mark-connection chain=postrouting new-connection-mark=icmp-con passthrough=yes protocol=icmp
add action=mark-packet chain=prerouting connection-mark=icmp-con new-packet-mark=icmp-pkt passthrough=yes \
    protocol=icmp
add action=mark-packet chain=postrouting connection-mark=icmp-con new-packet-mark=icmp-pkt passthrough=yes \
    protocol=icmp
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/ip route
add check-gateway=ping disabled=yes distance=1 gateway=8.8.8.8
add check-gateway=ping disabled=yes distance=2 gateway=8.8.4.4
add disabled=yes distance=1 dst-address=8.8.4.4/32 gateway=192.168.8.1 scope=10
add disabled=yes distance=1 dst-address=8.8.8.8/32 gateway=51.148.72.22 scope=10
/system clock
set time-zone-name=Europe/London
/tool e-mail
set address=smtp-relay.gmail.com from="Home Router <router@*net>" start-tls=yes user=\
    me@*.net
/tool graphing interface
add interface=pppoe-out1
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
 
troffasky
Member
Member
Posts: 431
Joined: Wed Mar 26, 2014 4:37 pm

Re: ICMP Packet loss when WAN is saturated

Thu Apr 29, 2021 3:10 pm

Does the packet loss go away if you go back to your previous router? Are you sure the packet loss is at your end and not happening upstream at your ISP?
 
mikehhhhhhh
just joined
Topic Author
Posts: 10
Joined: Tue Apr 27, 2021 10:47 am

Re: ICMP Packet loss when WAN is saturated

Thu Apr 29, 2021 5:02 pm

Does the packet loss go away if you go back to your previous router? Are you sure the packet loss is at your end and not happening upstream at your ISP?
Yes, no packet loss on previous router;

~12pm is when I switched to the mikrotik from OpenWRT on a RPI. 2PM is me playing with WAN failover.
Image

Outbound packet loss at least seems to start at the first hop outside of my network.

I notice every other PPPoE device I've tried asks for connection speed, I wonder if this is trait of PPPoE and I need to do a better job of controlling traffic my end?
 
troffasky
Member
Member
Posts: 431
Joined: Wed Mar 26, 2014 4:37 pm

Re: ICMP Packet loss when WAN is saturated

Thu Apr 29, 2021 11:53 pm

Needing to know the speed is not some inherent trait of PPPoE, but the router can't really prioritise anything if it doesn't know how much bandwidth is available.

Try disabling discovery on the PPPoE interface:
viewtopic.php?p=767139#p767139
 
R1CH
Forum Guru
Forum Guru
Posts: 1099
Joined: Sun Oct 01, 2006 11:44 pm

Re: ICMP Packet loss when WAN is saturated

Fri Apr 30, 2021 12:10 am

OpenWRT handles saturation much better due to fq_codel / cake schedulers, not yet available on Mikrotik. You have to cap your bandwidth significantly below link saturation point to avoid buffers being flooded.
 
mikehhhhhhh
just joined
Topic Author
Posts: 10
Joined: Tue Apr 27, 2021 10:47 am

Re: ICMP Packet loss when WAN is saturated

Sat May 01, 2021 12:30 pm

OpenWRT handles saturation much better due to fq_codel / cake schedulers, not yet available on Mikrotik. You have to cap your bandwidth significantly below link saturation point to avoid buffers being flooded.
Thanks, that makes a lot of sense.

Is this normal to run a Mikrotik this way? It seems like quite a significant issue.

Who is online

Users browsing this forum: emunt6 and 58 guests