Router reseting some connections from known IPs

Lately we have been having several routers that appear to be reseting our HTTP or SSH requests to connect to them from certain IP addresses. These services have all worked in the past, but one day with nothing changing on the router or on our end, they stop working. What makes this weird is that there are no block rules at all set up for these IP addresses, and in fact they these IP addresses are explicitly allowed to connect to the router for these services. When we look at the firewall, the accept rules are hit, and none of the reject rules are when the request happens, but it appears the router resets the connection, causing the service not to work. We have tried to reboot these routers, but they still continue to do this. They work fine from other IP addresses that we control as well, that are set up with the same address list to accept requests from.

We are running 4.5 on most of our boxes at the moment. Has anyone ever run into something like this, and if so how did you fix it?

Any chance that other rules further above match packets further down the connection, and can you post your entire ruleset?
Have you tried taking such a problematic IP address and adding explicit accept rules (two, one for src-address, one for dst-address) at the very top of the ruleset?

Nope, there’s nothing else there that would match the connection further down. I’ve looked for that, and only the explicit accept rules increment. Here are the filter rules.

/ip firewall filter
add action=accept chain=forward comment="Allow/Block access to equipment subnet depending on if IP is in the address list or not." disabled=no dst-address=10.0.0.0/24 src-address=\
    192.168.50.0/23 src-address-list="Allowed IPs"
add action=reject chain=forward comment="" disabled=no dst-address=10.0.0.0/24 reject-with=icmp-network-unreachable src-address=192.168.50.0/23
add action=accept chain=forward comment="Block communication between guests on the network." disabled=no dst-address=192.168.50.1 in-interface="Guest Network" src-address=\
    192.168.50.0/23 src-address-list="Allowed IPs"
add action=drop chain=forward comment="Block communication between guests on the network." disabled=no dst-address=192.168.50.1 in-interface="Guest Network" src-address=\
    192.168.50.0/23
add action=drop chain=input comment="Block guests form talking to public IP of Router" disabled=no dst-address=x.x.x.10 src-address=192.168.50.0/23
add action=accept chain=input comment="Allow/Block SSH access if not on address list." disabled=no dst-port=22 in-interface="Guest Network" protocol=tcp src-address=\
    192.168.50.0/23 src-address-list="Allowed IPs"
add action=reject chain=input comment="" disabled=no dst-port=22 in-interface="Guest Network" protocol=tcp reject-with=icmp-network-unreachable src-address=192.168.50.0/23 \
    src-address-list="!Allowed IPs"
add action=accept chain=forward comment="Allow ICMP from un-authed guests." disabled=no protocol=icmp
add action=accept chain=input comment="" disabled=no protocol=icmp
add action=add-src-to-address-list address-list="Block Spam" address-list-timeout=1h chain=forward comment="Detect Possible Spamer" connection-limit=10,32 disabled=no dst-port=25 \
    limit=30,5 protocol=tcp
add action=drop chain=forward comment="" disabled=no dst-port=25 protocol=tcp src-address-list="Block Spam"
add action=drop chain=input comment="drop ftp brute forcers" disabled=no dst-port=21 protocol=tcp src-address-list=ftp_blacklist
add action=accept chain=output comment="" content="530 Login incorrect" disabled=no dst-limit=1/1m,9,dst-address/1m protocol=tcp
add action=add-dst-to-address-list address-list=ftp_blacklist address-list-timeout=3h chain=output comment="" content="530 Login incorrect" disabled=no protocol=tcp
add action=drop chain=input comment="drop ssh brute forcers" disabled=no dst-port=22 protocol=tcp src-address-list=ssh_blacklist
add action=add-src-to-address-list address-list=ssh_blacklist address-list-timeout=1w3d chain=input comment="" connection-state=new disabled=no dst-port=22 protocol=tcp \
    src-address-list=ssh_stage3
add action=add-src-to-address-list address-list=ssh_stage3 address-list-timeout=1m chain=input comment="" connection-state=new disabled=no dst-port=22 protocol=tcp \
    src-address-list=ssh_stage2
add action=add-src-to-address-list address-list=ssh_stage2 address-list-timeout=1m chain=input comment="" connection-state=new disabled=no dst-port=22 protocol=tcp \
    src-address-list=ssh_stage1
add action=add-src-to-address-list address-list=ssh_stage1 address-list-timeout=1m chain=input comment="" connection-state=new disabled=no dst-port=22 protocol=tcp
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes
add action=accept chain=forward comment="Detect abuse on network" disabled=no dst-port=80 protocol=tcp src-address-list=abuse
add action=accept chain=forward comment="" disabled=no dst-port=53 protocol=tcp src-address-list=abuse
add action=accept chain=forward comment="" disabled=no dst-port=53 protocol=udp src-address-list=abuse
add action=drop chain=forward comment="" disabled=no in-interface="Guest Network" src-address-list=abuse
add action=add-src-to-address-list address-list=abuse address-list-timeout=2h chain=forward comment="" connection-limit=400,32 disabled=no in-interface="Guest Network" protocol=\
    tcp src-address=192.168.50.0/23 src-address-list=!Do-Not-Block
add action=log chain=forward comment="" disabled=no log-prefix="Abuse Limit Reached" src-address-list=abuse
add action=log chain=input comment="" disabled=no dst-port=68 in-interface="Guest Network" log-prefix="DHCP Server Detected" protocol=udp src-port=67
add action=log chain=input comment="" disabled=no dst-port=68 in-interface="Guest Network" log-prefix="DHCP Server Detected" protocol=tcp src-port=67
add action=accept chain=input comment="HTTP Rules" disabled=no dst-port=80 protocol=tcp src-address-list=known
add action=drop chain=input comment="" disabled=no dst-port=80 protocol=tcp
add action=accept chain=input comment="SNMP Rules" disabled=no dst-port=161 protocol=udp src-address-list=known
add action=drop chain=input comment="" disabled=no dst-port=161 protocol=udp

I just tried putting an accept rule on the input chain for that IP as the src-address, as well as an accept rule on the output chain with that as the dst-address with the same result as before. When I do wget from a server that is tring to grab some graphs off of the router, it reports back that the connection has been reset by peer. When I try SSH to a router that is doing this to our office IP, PuTTY just says server closed the connection. So I’m guessing it is doing the same thing.

Huh. Based on that, a couple more questions: what IP address do example devices that get blocked have? Can you post your address lists, particularly the “Allowed IPs” list?
Have you checked that the IPs of the blocked devices didn’t somehow make it on to one of the dynamically populated address lists, such as “ssh_blacklist” or “abuse”? They might not be in there now, so check within the list timeout limits after an observed failure.

Beyond that, maybe contact support. I don’t see anything at all in that ruleset that would prevent access once you’ve ruled out the above.

I’ve checked and our IP addresses are not in any of the dynamic lists at all. I have a subnet defined for the Known IP addresses and all of our IP addresses are within those ranges. It’s something that I have been scratching my head about lately, it’s not just one box doing it but a couple of them, mostly with ssh. Just a client noticed an issue today when they were trying to grap one of the graphs off of the router.

I’ll get something sent up to support to see what they have to say.

Just to verify - all IP addresses on the “Allowed IPs” address list are within 192.168.50.0/23?

Beyond that I have no idea what would be wrong.

All IPs in “known” are public IP addresses. They are the two subnets we have at the colo location and the one we have here at the office.