Properly stuck trying to diagnose traffic issues (RB951-2n)

RouterOS Beginner Basics
1

Good afternoon,

Yesterday I finally switched over from the ISP-provided router to the RB951-2n with the ISP one as a modem.

Almost everything seems to be working fine - except for my game servers.
Taking as an example Minecraft (since the effects of this were dramatically ovbious in-game):

-Player A does something. Server sees player doing something and so do clients local to the server. BUT remote client (through the internet) doesn’t see any changes.

I fired up WireShark and I’m seeing these sort of things in the hundreds:

26827	100.950620	81.xx.xxx.xxx     192.168.88.139	TCP	54	[TCP ACKed unseen segment] 51588 → 25565 [ACK] Seq=1534 Ack=2216925 Win=1360 Len=0
26828	100.950628	81.xx.xxx.xxx	192.168.88.139	TCP	54	[TCP ZeroWindow] [TCP ACKed unseen segment] 51588 → 25565 [ACK] Seq=1534 Ack=2218285 Win=0 Len=0

Setup is as follows:

ISP modem>RB951-2n>TP Link Archer C2 in AP mode

The computer that sees this traffic (‘server’ in question) is connected to the RB951 directly, all other LAN clients connected to the C2. Internet clients obviously through the internet :stuck_out_tongue:

Configuration below:

#
/interface bridge
add admin-mac=D4:CA:6D:F2:32:CD auto-mac=no mtu=1500 name=bridge-local
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n country=spain disabled=no \
    distance=indoors mode=ap-bridge ssid=*** wireless-protocol=802.11
/interface ethernet
set [ find default-name=ether1 ] name=ether1-gateway
set [ find default-name=ether2 ] name=ether2-master-local
set [ find default-name=ether3 ] name=ether3-voip
set [ find default-name=ether4 ] master-port=ether2-master-local name=ether4-AP
set [ find default-name=ether5 ] auto-negotiation=no master-port=\
    ether2-master-local name=ether5-Server
/interface pppoe-client
add add-default-route=yes default-route-distance=1 disabled=no interface=\
    ether1-gateway max-mru=1460 max-mtu=1460 name=pppoe-out1 password=adslppp \
    service-name=Telefonica use-peer-dns=yes user=***
/ip neighbor discovery
set ether1-gateway discover=no
/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa2-psk mode=dynamic-keys \
    wpa-pre-shared-key="***" wpa2-pre-shared-key=\
    "***"
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-128-cbc
/ip pool
add name=dhcp ranges=192.168.88.130-192.168.88.254
/ip dhcp-server
add address-pool=dhcp disabled=no interface=bridge-local lease-time=12h name=\
    default
/interface bridge port
add bridge=bridge-local interface=ether2-master-local
add bridge=bridge-local interface=wlan1
/ip settings
set accept-source-route=yes
/ip address
add address=192.168.88.1/24 interface=ether2-master-local network=192.168.88.0
/ip dhcp-server config
set store-leases-disk=immediately
/ip dhcp-server lease
add address=192.168.88.130 client-id=1:78:d3:8d:3:fa:2e mac-address=\
    78:D3:8D:03:FA:2E server=default
add address=192.168.88.139 client-id=1:94:de:80:2:55:5c mac-address=\
    94:DE:80:02:55:5C server=default
/ip dhcp-server network
add address=192.168.10.0/24 dns-server=192.168.88.1 gateway=192.168.10.1 \
    netmask=24
add address=192.168.88.0/24 comment="default configuration" dns-server=\
    192.168.88.1 gateway=192.168.88.1 netmask=24
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,208.67.222.222
/ip dns static
add address=192.168.88.1 name=router
/ip firewall filter
add action=drop chain=input comment="Input drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="Allow state new" connection-state=new \
    in-interface=all-ethernet
add action=accept chain=input comment="Allow state established" \
    connection-state=established
add action=accept chain=input comment="Allow state related" connection-state=\
    related
add action=accept chain=input comment="Allow pings" protocol=icmp
add action=drop chain=input comment="Input drop catchall" dst-address=\
    !192.168.88.1 src-address=!192.168.88.130
add action=drop chain=forward comment="Forward drop invalid" connection-state=\
    invalid
add action=accept chain=forward in-interface=pppoe-out1 protocol=udp src-port=\
    25565
add action=accept chain=forward in-interface=pppoe-out1 protocol=tcp src-port=\
    25565
add action=accept chain=forward comment="Forward new" connection-state=new
add action=accept chain=forward comment="Forward established" connection-state=\
    established
add action=accept chain=forward comment="Forward related" connection-state=\
    related
add action=drop chain=forward comment="Drop catchall"
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" \
    out-interface=pppoe-out1 to-addresses=0.0.0.0
add action=dst-nat chain=dstnat dst-port=25565 in-interface=pppoe-out1 \
    protocol=tcp to-addresses=192.168.88.139 to-ports=25565
add action=dst-nat chain=dstnat dst-port=25565 in-interface=pppoe-out1 \
    protocol=udp to-addresses=192.168.88.139 to-ports=25565
/ip ipsec policy
set 0 dst-address=0.0.0.0/0 src-address=0.0.0.0/0
/ip proxy
set cache-path=web-proxy1
/ip route
add disabled=yes distance=1 gateway=192.168.1.1
/ip service
set telnet disabled=yes
/ip upnp
set enabled=yes
/system clock
set time-zone-autodetect=no
/system clock manual
set time-zone=+01:00
/system leds
set 0 interface=wlan1
/tool mac-server
set [ find default=yes ] disabled=yes
add interface=ether2-master-local
add interface=ether3-voip
add interface=ether4-AP
add interface=ether5-Server
add interface=wlan1
add interface=bridge-local
/tool mac-server mac-winbox
set [ find default=yes ] disabled=yes
add interface=ether2-master-local
add interface=ether3-voip
add interface=ether4-AP
add interface=ether5-Server
add interface=wlan1
add interface=bridge-local
/tool traffic-monitor
add disabled=yes interface=ether1-gateway name=tmon1 threshold=0

I’ve exhausted my ideas now and I was hoping someone could have a look!

Thanks in advance for your time :slight_smile:

2

unseen segment is normal if you started the capture while a TCP connection was already established. Or if the packet capture couldn’t keep up and packets weren’t logged.

The ZeroWindow means the receive buffer is full on one of the end points OR the end points have detected many dropped packets and continually lower the TCP window size to slow down the TCP connection to a point where packets aren’t dropped. This value does NOT refer to a buffer on the router. When a buffer on the router becomes full, it begins dropping packets.

Other troubleshooting tools in Mikrotik are Tools > Profile, check for high CPU usage. Packets will drop if the CPU can’t keep up. Check your wan, lan and bridge interfaces for dropped packets. Do a speed test, getting the speeds you expect? Watch Profile CPU usage while you do a speed test. If everything checks out, I’d start looking at the end points.

3

Thanks for looking into it Van9018 :slight_smile:

Perhaps it is noteworthy to say none of this happened when I was running exclusively off the ISP router (however unreliable that pile of rubbish could be). The setup is identical to what it was in terms of what goes where.

Anyway - WAN speeds are fine, I see the CPU usage shoot up to 18% on the download bit, almost zero usage on the upload bits.

I don’t see any substantial (or any at all) packet loss when pinging out to a random site on the Internet (in fact browsing the internet is flawless as far as I can tell?). It only happens on outgoing traffic of the previously mentioned characteristics (traffic for my game servers that is matched against NAT).

Things I’ve tried:
-Disable the input chain catchall. No changes.
-Add a mangle rule to increase TTL. No changes.
-Disable the invalid packet rules. No changes.
-Add prerouting rules for packets that match incoming traffic on those ports. No changes.
-Set a 100Mb full duplex link for the server (in case it was establishing as half?) No changes.
-Changed the port the server sits on. No changes.
-Moved the game servers to another machine. No changes - performance is identically bad, exact same symptoms.

The amount of traffic generated by the server is minimal, it barely reaches 1Mb/s the outgoing way and 5-6Mb inbound is about as much as it ever sees. I’ve used this box with much higher throughput and it had no issues keeping up. And again - it didn’t happen with the ISP router, which leads me to assume I’ve bodged something in the config. If only I knew what! :stuck_out_tongue:

4

The TCP ZeroWindow looks like your best lead. Only next step I can think of is to do a packet capture, you willing to share the capture file? Tools > Packet Sniffer. Kill the connections to the wan, start the capture then start the connections.

If capture file shows nothing interesting, then you’d have to start replacing equipment. Change the routerboard. Then change the modem (have you rebooted the modem??). Those are the two changes you’ve made since your previous modem/router combo.

Your wan interface should say what link speed it’s connected at (full duplex, half, 10/100). It’ll also say what the modem advertised.

5

Van,

Sorry for the late reply, life got in the way of this. And since internet browsing worked fine… you can imagine the rest :stuck_out_tongue:

Anyway. I built a couple of spare Netgears I had around for testing purposes. I’ve got something partially interesting. Those routers both seemingly work perfectly fine if the game servers are on my workstation. Not on the server though! Displays the exact same behaviour as it did with the MikroTik.

Which leads me to believe that the server is somehow at fault here, regardless of the MikroTik having a part in this play.

I guess I’ll carry on digging server-side and let you know.

Many thanks!