750Gr3 issues - packet loss?

RouterOS Beginner Basics
1

I bought a 750Gr3 to replace a 750Gr2 which has been working beautifully, hoping for improve throughput since it’s dual core (I needed another switch too).

I’ve set it up the new 750Gr3 identical to the 750Gr2. But I’m having problems.

Problem 1: Broken connections/

I’m having problems with some connections simply not working correctly. Most website working fine, but some don’t. For example:

> curl http://www.speedtest.net/
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8" >
	<meta name="description" content="Test your Internet connection bandwidth to locations around the world with this interactive broadband speed test from Ookla" >
	<meta name="keywords" lang="en" content="ookla, speed, test, speedtest, speed test, bandwidth speed test, internet speed test, broadband speed test, speakeasy, flash, cnet, internet, network, connection, broadband, bandwidth, latency, ping, throughput, download, upload, connection, dsl, adsl, cable, t1, voip, isp, asp, internet, ip, ip address, tcp, ds3" >

it will only get the first part of the response, then it would stop and wait indefinitely.

It’s almost like there are some packets being silently dropped.

Problem 2: Where are the stats?

If I click on an ethernet interface on the 750Gr2 (or my wAP ac), I get a list of stats “Overall Stats”, “Tx Stats” and “Rx Stats”. I can’t see this on the 750Gr3. How do I see if packets are being dropped, paused, error, etc?

Configuration

Here is my current configuration. Note that everything works - like, DNS resolves most of the time, web sites load most of the time. But there is some odd underlying problem causing some connections to fail.

# nov/02/2016 15:00:22 by RouterOS 6.37.1
# software id = RE4F-WKP7
#
/interface ethernet
set [ find default-name=ether1 ] l2mtu=1600 name=ether1-gateway rx-flow-control=on tx-flow-control=on
set [ find default-name=ether2 ] l2mtu=1600 name=ether2-master rx-flow-control=on tx-flow-control=on
set [ find default-name=ether3 ] l2mtu=1600 master-port=ether2-master rx-flow-control=on tx-flow-control=on
set [ find default-name=ether4 ] l2mtu=1600 master-port=ether2-master rx-flow-control=on tx-flow-control=on
set [ find default-name=ether5 ] l2mtu=1600 master-port=ether2-master rx-flow-control=on tx-flow-control=on
/ip neighbor discovery
set ether1-gateway discover=no
/interface vlan
add interface=ether1-gateway name=vlan10 vlan-id=10
/interface pppoe-client
add add-default-route=yes allow=pap disabled=no interface=vlan10 name=pppoe-snap password=********* use-peer-dns=yes user=************
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=default-dhcp ranges=192.168.1.20-192.168.1.254
/ip dhcp-server
add address-pool=default-dhcp disabled=no lease-time=1h name=local-dhcp
/ipv6 dhcp-server
add address-pool=snap-ipv6 disabled=yes interface=ether2-master name=server
/interface bridge port
add interface=ether2-master
/ip address
add address=192.168.1.2/24 comment=defconf interface=ether2-master network=192.168.1.0
/ip dhcp-server lease
add address=192.168.1.70 mac-address=E8:39:35:EE:21:B2
add address=192.168.1.6 always-broadcast=yes client-id=1:e4:8d:8c:72:b7:76 mac-address=E4:8D:8C:72:B7:76 server=local-dhcp
/ip dhcp-server network
add address=192.168.1.0/24 comment="default configuration" gateway=192.168.1.2 netmask=24
/ip dns
set allow-remote-requests=yes max-udp-packet-size=1400 query-server-timeout=10s query-total-timeout=20s
/ip dns static
add address=192.168.1.2 name=router
add address=192.168.1.70 name=mc.oriontransfer.co.nz
add address=192.168.1.70 name=backup.oriontransfer.co.nz
/ip firewall filter
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="drop invalid packets and log" connection-state=invalid disabled=yes log=yes log-prefix=invalid
add action=drop chain=forward comment="drop invalid packets and log" connection-state=invalid disabled=yes log=yes log-prefix=invalid
add action=accept chain=input comment="defconf: accept established,related" connection-state=established,related
add action=drop chain=input comment="defconf: drop all from WAN" in-interface=ether1-gateway log=yes
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related" connection-state=established,related
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface=\
    ether1-gateway
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" out-interface=pppoe-snap
add action=dst-nat chain=dstnat comment="Basic services: Web/SSH, etc." dst-port=22,80 in-interface=pppoe-snap protocol=tcp to-addresses=\
    192.168.1.70
add action=dst-nat chain=dstnat comment="Minecraft Server" dst-port=25565 in-interface=pppoe-snap protocol=tcp to-addresses=192.168.1.70 to-ports=\
    25565
/ip upnp interfaces
add interface=pppoe-snap type=external
add interface=ether2-master type=internal
/ipv6 address
add from-pool=snap-ipv6 interface=ether2-master
/ipv6 dhcp-client
add add-default-route=yes interface=pppoe-snap pool-name=snap-ipv6 request=prefix
/ipv6 firewall filter
add action=drop chain=input comment="drop invalid packets and log" connection-state=invalid disabled=yes log=yes log-prefix=invalid
add action=accept chain=input connection-state=established in-interface=pppoe-snap
add action=accept chain=input connection-state=related in-interface=pppoe-snap
add action=accept chain=input in-interface=pppoe-snap protocol=icmpv6
add action=accept chain=input dst-port=546 in-interface=pppoe-snap protocol=udp
add action=drop chain=input in-interface=pppoe-snap
add action=drop chain=forward comment="drop invalid packets and log" connection-state=invalid disabled=yes log=yes log-prefix=invalid
add action=accept chain=forward connection-state=established,related in-interface=pppoe-snap
add action=accept chain=forward in-interface=pppoe-snap protocol=icmpv6
add action=accept chain=forward dst-port=22,80 in-interface=pppoe-snap protocol=tcp
add action=accept chain=forward dst-port=25565 in-interface=pppoe-snap protocol=tcp
add action=accept chain=forward dst-port=25565 in-interface=pppoe-snap protocol=udp
add action=drop chain=forward in-interface=pppoe-snap
/system clock
set time-zone-name=Pacific/Auckland
/system routerboard settings
set memory-frequency=1200DDR protected-routerboot=disabled
/tool mac-server
set [ find default=yes ] disabled=yes
add interface=ether2-master
/tool mac-server mac-winbox
set [ find default=yes ] disabled=yes
add interface=ether2-master
2

Ah, I found if I SSH in rather than using the web interface I can see stats.

So, problem #2 is worked around, but it’s odd its not showing up in web interface, no?

Here are my stats for the past couple of hours:

> /interface ethernet print stats
                      name:  ether1-gateway ether2-master      ether3     ether4     ether5
            driver-rx-byte:     152 066 596    12 890 844           0          0          0
          driver-rx-packet:         128 616       106 349           0          0          0
            driver-tx-byte:      10 825 881   160 147 720           0          0          0
          driver-tx-packet:          88 221       137 119           0          0          0
                  rx-bytes:     152 581 060     3 232 890  10 849 285  2 655 669 30 682 838
                 rx-packet:         128 616        11 481      88 778     20 161     23 675
              rx-too-short:               0             0           0          0          0
                     rx-64:             967         2 317       3 622        155      1 168
                 rx-65-127:          18 268         7 811      77 142     17 689      1 220
                rx-128-255:           6 643         1 152       6 763        200        337
                rx-256-511:           3 362         1 719       3 455        120        376
               rx-512-1023:           2 421         1 356       1 274      2 168      2 094
              rx-1024-1518:          96 955           562         819          0     19 856
               rx-too-long:               0             0           0          0          0
              rx-broadcast:               0         1 379         399         46        689
                  rx-pause:               0             0           0          0          0
              rx-multicast:               0         2 057       3 898        125        687
              rx-fcs-error:               0             0           0          0          0
            rx-align-error:               0             0           0          0          0
               rx-fragment:               0             0           0          0          0
                 rx-jabber:               0             0           0          0          0
                   rx-drop:               0             0           0          0          0
                  tx-bytes:      11 183 569     6 924 841 158 770 055 30 122 582  4 791 237
                 tx-packet:          88 056        11 146     127 832     22 047     21 708
                     tx-64:             212         2 139       4 773      3 871      2 602
                 tx-65-127:          78 059         5 489      15 488      1 994     19 516
                tx-128-255:           5 102         4 931       4 538      4 059      4 097
                tx-256-511:           2 062         2 382       4 947      2 200      2 602
               tx-512-1023:           1 651           689       2 923      2 136      2 216
              tx-1024-1518:           1 135         2 716     101 600     18 333         36
              tx-broadcast:              87         2 156       3 166      3 513      2 877
                  tx-pause:               0             0           0          0          0
              tx-multicast:              78         5 044       3 271      7 033      6 484
              tx-collision:               0             0           0          0          0
    tx-excessive-collision:               0             0           0          0          0
     tx-multiple-collision:               0             0           0          0          0
       tx-single-collision:               0             0           0          0          0
               tx-deferred:               0             0           1          0          1
         tx-late-collision:               0             0           0          0          0
                   tx-drop:               0             0           0          0          0
              tx-fcs-error:               0             0           0          0          0

It doesn’t look like there are any errors.

I’ve also enabled logging in the firewall, dropping invalid connections on the input, output and forward chains for both ipv4 and ipv6, here is some sample output:

15:13:05 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33454->192.168.1.2:80, len 52 
15:13:05 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33448->192.168.1.2:80, len 52 
15:13:05 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33450->192.168.1.2:80, len 52 
15:13:05 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33446->192.168.1.2:80, len 52 
15:13:06 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33452->192.168.1.2:80, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,RST), 192.168.1.253:48342->52.65.8.9:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:48343->52.65.8.9:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:12 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:13 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:13:13 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:13 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:13 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:14 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:14 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:13:14 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:14 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:15 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:48343->52.65.8.9:443, len 52 
15:13:16 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:16 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:16 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:16 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:13:18 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:19 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:19 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:19 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:20 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:20 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:20 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:20 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:21 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:13:22 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:22 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:48343->52.65.8.9:443, len 52 
15:13:25 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:28 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:28 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:28 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:30 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:13:31 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33454->192.168.1.2:80, len 52 
15:13:31 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33448->192.168.1.2:80, len 52 
15:13:32 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:32 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33450->192.168.1.2:80, len 52 
15:13:32 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33446->192.168.1.2:80, len 52 
15:13:33 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33452->192.168.1.2:80, len 52 
15:13:35 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:48343->52.65.8.9:443, len 52 
15:13:44 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:13:45 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:13:45 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:13:45 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:13:49 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:14:00 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:00 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:00 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:00 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:01 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:02 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:48343->52.65.8.9:443, len 52 
15:14:03 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:06 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:11 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33490->192.168.1.2:80, len 52 
15:14:13 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:16 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34441->54.66.169.248:443, len 52 
15:14:18 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:34459->54.66.169.248:443, len 52 
15:14:18 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:37112->52.84.207.109:443, len 52 
15:14:26 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:27 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:58897->54.231.50.44:443, len 40 
15:14:52 firewall,info invalid input: in:ether2-master out:(none), src-mac 7c:5c:f8:50:17:9d, proto TCP (ACK,FIN), 192.168.1.15:33492->192.168.1.2:80, len 52 
15:14:54 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac f0:25:b7:d1:0f:31, proto TCP (ACK,FIN), 192.168.1.253:48343->52.65.8.9:443, len 52 
15:15:44 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.230:443->111.69.177.47:55310, len 60 
15:15:45 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.249:443->111.69.177.47:55311, len 60 
15:15:45 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.251:443->111.69.177.47:55312, len 60 
15:15:46 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55311->202.124.127.249:443, len 40 
15:15:46 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:46 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55311->202.124.127.249:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55311->202.124.127.249:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55312->202.124.127.251:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55312->202.124.127.251:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55311->202.124.127.249:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55310->202.124.127.230:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55311->202.124.127.249:443, len 40 
15:15:48 firewall,info invalid forward: in:ether2-master out:pppoe-snap, src-mac 60:03:08:8d:7d:00, proto TCP (RST), 192.168.1.20:55312->202.124.127.251:443, len 40 
15:15:52 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.230:443->111.69.177.47:55310, len 60 
15:15:53 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.249:443->111.69.177.47:55311, len 60 
15:15:53 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.251:443->111.69.177.47:55312, len 60 
15:16:08 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.230:443->111.69.177.47:55310, len 60 
15:16:09 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.249:443->111.69.177.47:55311, len 60 
15:16:09 firewall,info invalid input: in:pppoe-snap out:(none), src-mac 3c:61:04:4d:bc:a5, proto TCP (SYN,ACK), 202.124.127.251:443->111.69.177.47:55312, len 60 
15:16:58 firewall,info invalid output: in:(none) out:pppoe-snap, proto TCP (SYN,ACK), 111.69.177.47:23->115.77.107.103:48313, len 52 
15:16:58 firewall,info invalid output: in:(none) out:pppoe-snap, proto TCP (SYN,ACK), 111.69.177.47:23->115.77.107.103:17690, len 44

It seems odd that there are packets going on the output chain which are invalid. Any advice or ideas would be appreciated. Thanks.

3

The problem of connections failing occurs also for local connections, even the MikroTik webfig interface. It just stops working and I need to reload the page. So, it’s not limited to connections from my laptop to the internet, but internal connections too, from wired desktop to router, other hosts, etc.

4

Okay, I just updated to the latest rc19 and the stats page is back, that problem has been fixed. On the other hand, all the other problem remains.

5

I see a couple potential problems:

  • flow control enabled? Is that really necessary? I would hope MikroTik is not doing this by default now
  • you have no firewall rules to drop other IPv4 packets coming in pppoe-snap (ex. input chain) - this can open your router to hacking attempts if it receives a public IPv4 address on the pppoe-snap interface

I don’t see how either of those are causing your issues, but probably best to fix them anyway.

I might suggest that you start with the Quick Set tool configured properly for PPPoE and add your IPv6 stuff. I imagine the lack of firewall rules for the PPPoE tunnel is due to manual set up of PPPoE after using QuickSet to configure normal IP WAN connectivity.

I also don’t see an MSS change rule for your PPPoE tunnel for IPv4. Perhaps this is there, but just doesn’t print out. This might lead to path-MTU discovery related issues that could explain issues loading websites.

6

Hey, thanks for you reply, it was helpful.

You are right about the MTU size. Something was royally screwed in the config.

There was one mangle rule, but after doing the factory reset and Quick Set, I see two. Additionally the MTU of the PPPoE connection changed from 1596 to 1580. I think things are working a lot better now but I’ll report back soon with an update.

7

I assume you mean 1496 and 1480. 1496 is generally much too high for PPPoE MTU, unless your provider supports RFC 4638 (most don’t). So yes, this could explain your issue.

8

Yeah my bad I did mean 1496 and 1480.

However, it seems like the issue has returned, after initially working correctly, the “Actual MTU” of the PPPoE link has gone back to 1496 and things stopped working.

That’s odd.

9

After restarting the PPPoE connection the MTU is now 1492… how does it determine this number? I tried setting Max MTU to 1480 but it also doesn’t work. I noticed the Mangle rules have changed too.. from 2 rules now only one like before when it wasn’t working..

10

Your PPPoE is running over a VLAN, so you might need an additional 4 bytes for the VLAN header. Set both max mtu and max mru to 1480, but set MRRU to 1500.

11

Okay, so did a full reset again, configured it (basically just adding vlan10 to ether1 and pointing pppoe to vlan10 instead of ether1). Now it’s back to 1480 and everything is working.. FYI here are the mangle rules:

[admin@MikroTikKitchen] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic 
 0  D chain=forward action=change-mss new-mss=1440 tcp-flags=syn protocol=tcp out-interface=all-ppp tcp-mss=1441-65535 

 1  D chain=forward action=change-mss new-mss=1440 tcp-flags=syn protocol=tcp in-interface=all-ppp tcp-mss=1441-65535 

 2  D ;;; special dummy rule to show fasttrack counters
      chain=prerouting action=passthrough 

 3  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough 

 4  D ;;; special dummy rule to show fasttrack counters
      chain=postrouting action=passthrough

Also, here is the list of interfaces with the correct MTUs:

[admin@MikroTikKitchen] /ip firewall mangle> /interface print         
Flags: D - dynamic, X - disabled, R - running, S - slave 
 #     NAME                                TYPE       ACTUAL-MTU L2MTU  MAX-L2MTU MAC-ADDRESS      
 0  R  ether1                              ether            1500  1596       2026 6C:3B:6B:6E:94:6C
 1  R  ether2-master                       ether            1500  1596       2026 6C:3B:6B:6E:94:6D
 2  RS ether3                              ether            1500  1596       2026 6C:3B:6B:6E:94:6E
 3  RS ether4                              ether            1500  1596       2026 6C:3B:6B:6E:94:6F
 4   S ether5                              ether            1500  1596       2026 6C:3B:6B:6E:94:70
 5  R  pppoe-out1                          pppoe-out        1480
 6  R  vlan10                              vlan             1500  1592            6C:3B:6B:6E:94:6C

I’m going to try restarting and rebooting, see if it sticks.

12

Hmm, yeah, it might be a bug. Simply restarting the router causes the MTU to become incorrect:

[admin@MikroTikKitchen] > ip firewall mangle print
Flags: X - disabled, I - invalid, D - dynamic 
 0  D chain=forward action=change-mss new-mss=1452 tcp-flags=syn protocol=tcp in-interface=all-ppp tcp-mss=1453-65535 

 1  D ;;; special dummy rule to show fasttrack counters
      chain=prerouting action=passthrough 

 2  D ;;; special dummy rule to show fasttrack counters
      chain=forward action=passthrough 

 3  D ;;; special dummy rule to show fasttrack counters
      chain=postrouting action=passthrough 
[admin@MikroTikKitchen] > interface print         
Flags: D - dynamic, X - disabled, R - running, S - slave 
 #     NAME                                TYPE       ACTUAL-MTU L2MTU  MAX-L2MTU MAC-ADDRESS      
 0  R  ether1                              ether            1500  1596       2026 6C:3B:6B:6E:94:6C
 1  R  ether2-master                       ether            1500  1596       2026 6C:3B:6B:6E:94:6D
 2  RS ether3                              ether            1500  1596       2026 6C:3B:6B:6E:94:6E
 3  RS ether4                              ether            1500  1596       2026 6C:3B:6B:6E:94:6F
 4   S ether5                              ether            1500  1596       2026 6C:3B:6B:6E:94:70
 5  R  pppoe-out1                          pppoe-out        1492
 6  R  vlan10                              vlan             1500  1592            6C:3B:6B:6E:94:6C
13

Manually add these two rules:

0 chain=forward action=change-mss new-mss=1440 tcp-flags=syn protocol=tcp out-interface=all-ppp tcp-mss=1441-65535

1 chain=forward action=change-mss new-mss=1440 tcp-flags=syn protocol=tcp in-interface=all-ppp tcp-mss=1441-65535

14

Okay, I followed your instructions, and it works.

Max MTU = 1480
Max MRU = 1480
MMRU = 1500

Now my internet speed… 844Mbit/s down, 532Mbit/s up :slight_smile:

I think the speed is about the same as the 750Gr2 but the CPU usage is about half (which makes sense).

15

Okay, just to summarise, all I had to do was setup the PPPoE connection (Quick Set), and it was fine until first reboot. Then, MTU was auto-detected (?) incorrectly. After setup the MTU/MRU/MMRU as you suggested, it’s fine, generates correct IP firewall mangling rules, and works well.

It’s a bug probably, but this is an acceptable workaround.

16

Yes, it certainly is a bug (hopefully MikroTik is listening). Glad to hear you got it sorted.

17

Im glad to hear you got this problem figured out.

Can you please email your findings to support@mikrotik.com so they can fix the bug.