SIP client fails to register all of a sudden

Hi!

I understand this might be not really a tricky problem but this is one of a problem that I needed to fix today and customer is pushing.
So my customer decided to move to another office in the same building. No big deal, replug cables and we’re good to go? Nope.
After reconnecting the router (they are using hAP AC@ROS 6.41) at the new place, we found out that it reboots almost instantly when being connected to a main switch. Logs indicated only “router was rebooted without proper shutdown”. No other changes were done at this point, and no devices were even rebooted - it was only router powered off and back on in a new office.
So after a while, I figured out that having a Cisco SPA8000 VoIP gateway in the network now causes hAP AC to reset. It worked just fine before for a long time, and I have no idea WHAT should it transmit to cause the router to reboot. Okay I thought, let’s try to reproduce the issue on a latest firmware and submit a support ticket if that fails.
After upgrading to 6.41.1, random reboots have gone, but SPA8000 is no longer registering on a SIP server.

Source ports on SPA8000 are in range of 50000-53000, destination port is always 5060.
SPA8000 sends out a REGISTER message (for example, from port 50601), SIP server responds back to port 50601 with an authorization request.
This reply doesn’t pass through router anymore and is not forwarded to SPA8000. This is where I’m stuck - I have no idea why could it fail.

What I already tried:

  • disabled SIP service ports (didn’t help)
  • set a NAT rule to forward all UDP 50000-53000 ports to SPA8000 (shouldn’t be neccesary but I should’ve tried… didn’t help)
  • disabled Mangle rules that I’m using to mark traffic for further QoS (didn’t help either)
  • replaced hAP AC with a crappy ASUS router (this one worked, but that’s not a solution)

Nothing.

(verbose) export of firewall settings for a good measure:

/ip firewall filter
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept established,related" connection-state=established,related
add action=accept chain=input comment="Allow remote access to settings from whitelisted IP" !connection-bytes !connection-limit !connection-mark !connection-nat-state !connection-rate !connection-state !connection-type !content \
    disabled=no !dscp !dst-address !dst-address-list !dst-address-type !dst-limit dst-port=21-23,80,1723,8291 !fragment !hotspot !icmp-options !in-bridge-port !in-bridge-port-list in-interface=ether1 !in-interface-list \
    !ingress-priority !ipsec-policy !ipv4-options !layer7-protocol !limit log=no log-prefix="[remote]" !nth !out-bridge-port !out-bridge-port-list !out-interface !out-interface-list !packet-mark !packet-size !per-connection-classifier \
    !port !priority protocol=tcp !psd !random !routing-mark !routing-table !src-address src-address-list=allowed_remote !src-address-type !src-mac-address !src-port !tcp-flags !tcp-mss !time !ttl
add action=accept chain=input comment="Allow L2TP VPN access" !connection-bytes !connection-limit !connection-mark !connection-nat-state !connection-rate !connection-state !connection-type !content disabled=no !dscp !dst-address \
    !dst-address-list !dst-address-type !dst-limit dst-port=1701,500,4500 !fragment !hotspot !icmp-options !in-bridge-port !in-bridge-port-list in-interface=ether1 !in-interface-list !ingress-priority !ipsec-policy !ipv4-options \
    !layer7-protocol !limit log=no log-prefix="" !nth !out-bridge-port !out-bridge-port-list !out-interface !out-interface-list !packet-mark !packet-size !per-connection-classifier !port !priority protocol=udp !psd !random \
    !routing-mark !routing-table !src-address !src-address-list !src-address-type !src-mac-address !src-port !tcp-flags !tcp-mss !time !ttl
add action=drop chain=input comment="defconf: drop all from WAN" disabled=no in-interface=ether1
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related" connection-state=established,related
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface=ether1
add action=reject chain=forward comment="Block MS telemetry (CEIP) data - transmit" in-interface=br-lan layer7-protocol=telemetry protocol=tcp reject-with=tcp-reset
add action=reject chain=forward comment="Block MS telemetry (CEIP) data - receive" layer7-protocol=telemetry out-interface=br-lan protocol=tcp reject-with=tcp-reset
/ip firewall mangle
add action=mark-packet chain=forward comment="QoS implementation - mark non-VoIP packets - receive" dst-address=192.168.1.0/24 new-packet-mark=default_in passthrough=yes
add action=mark-packet chain=forward comment="QoS implementation - mark non-VoIP packets - transmit" new-packet-mark=default_out passthrough=yes src-address=192.168.1.0/24
add action=mark-packet chain=forward comment="QoS implementation - mark VoIP packets - receive" new-packet-mark=voip_in passthrough=yes protocol=udp src-address-list=voip
add action=mark-packet chain=forward comment="QoS implementation - mark VoIP packets - transmit" dst-address-list=voip new-packet-mark=voip_out passthrough=yes protocol=udp
add action=mark-routing chain=prerouting comment="Policy-based routing - route blocked websites through VPN" !connection-bytes !connection-limit !connection-mark !connection-nat-state !connection-rate !connection-state \
    !connection-type !content disabled=no !dscp !dst-address dst-address-list=through_vpn !dst-address-type !dst-limit !fragment !hotspot !icmp-options !in-bridge-port !in-bridge-port-list !in-interface !in-interface-list \
    !ingress-priority !ipsec-policy !ipv4-options !layer7-protocol !limit log=no log-prefix="" new-routing-mark=through_vpn !nth !out-bridge-port !out-bridge-port-list !out-interface !out-interface-list !packet-mark !packet-size \
    passthrough=yes !per-connection-classifier !priority !protocol !psd !random !routing-mark !routing-table src-address=192.168.1.0/24 !src-address-list !src-address-type !src-mac-address !tcp-flags !time !ttl
/ip firewall nat
add action=masquerade chain=srcnat comment="Masquerade - allow address translations to WAN group" !connection-bytes !connection-limit !connection-mark !connection-rate !connection-type !content disabled=no !dscp !dst-address \
    !dst-address-list !dst-address-type !dst-limit !fragment !hotspot !icmp-options !in-bridge-port !in-bridge-port-list !in-interface !in-interface-list !ingress-priority !ipsec-policy !ipv4-options !layer7-protocol !limit log=no \
    log-prefix="" !nth !out-bridge-port !out-bridge-port-list !out-interface out-interface-list=wan !packet-mark !packet-size !per-connection-classifier !priority !protocol !psd !random !routing-mark !routing-table !src-address \
    !src-address-list !src-address-type !src-mac-address !time !to-addresses !to-ports !ttl
add action=dst-nat chain=dstnat comment="Allow remote access to Switch settings" !connection-bytes !connection-limit !connection-mark !connection-rate !connection-type !content disabled=no !dscp !dst-address !dst-address-list \
    !dst-address-type !dst-limit dst-port=81 !fragment !hotspot !icmp-options !in-bridge-port !in-bridge-port-list in-interface=ether1 !in-interface-list !ingress-priority !ipsec-policy !ipv4-options !layer7-protocol !limit log=no \
    log-prefix="[switch]" !nth !out-bridge-port !out-bridge-port-list !out-interface !out-interface-list !packet-mark !packet-size !per-connection-classifier !port !priority protocol=tcp !psd !random !routing-mark !routing-table \
    !src-address src-address-list=allowed_remote !src-address-type !src-mac-address !src-port !tcp-mss !time to-addresses=192.168.1.3 to-ports=80 !ttl
add action=dst-nat chain=dstnat comment="Allow remote access to printer settings" !connection-bytes !connection-limit !connection-mark !connection-rate !connection-type !content disabled=no !dscp !dst-address !dst-address-list \
    !dst-address-type !dst-limit dst-port=82 !fragment !hotspot !icmp-options !in-bridge-port !in-bridge-port-list in-interface=ether1 !in-interface-list !ingress-priority !ipsec-policy !ipv4-options !layer7-protocol !limit log=no \
    log-prefix="" !nth !out-bridge-port !out-bridge-port-list !out-interface !out-interface-list !packet-mark !packet-size !per-connection-classifier !port !priority protocol=tcp !psd !random !routing-mark !routing-table !src-address \
    src-address-list=allowed_remote !src-address-type !src-mac-address !src-port !tcp-mss !time to-addresses=192.168.1.135 to-ports=80 !ttl
add action=dst-nat chain=dstnat comment="RDP to server" dst-port=43398 in-interface=ether1 log=yes log-prefix="[server remote]" protocol=tcp src-address-list=!blacklist to-addresses=192.168.1.2 to-ports=3389
add action=dst-nat chain=dstnat !connection-bytes !connection-limit !connection-mark !connection-rate !connection-type !content disabled=yes !dscp !dst-address !dst-address-list !dst-address-type !dst-limit dst-port=8888 !fragment \
    !hotspot !icmp-options !in-bridge-port !in-bridge-port-list !in-interface !in-interface-list !ingress-priority !ipsec-policy !ipv4-options !layer7-protocol !limit log=yes log-prefix=voip !nth !out-bridge-port !out-bridge-port-list \
    !out-interface !out-interface-list !packet-mark !packet-size !per-connection-classifier !port !priority protocol=tcp !psd !random !routing-mark !routing-table !src-address !src-address-list !src-address-type !src-mac-address \
    !src-port !tcp-mss !time to-addresses=192.168.1.10 to-ports=80 !ttl

That’s pretty much a default config with only rules added to allow remote access.

I’m really lost now. Any thoughts on how can I get this fixed?
Another question is how did that work for months before that…

Your ip firewall filter chain=forward action=drop connection-nat-state=!dstnat connection-state=new rule refers to in-interface=ether1, while your ip firewall nat action=masquerade rule refers to out-interface-list=wan. This indicates some inconsistency. Is ether1 on interface list named “wan”?

Other than that, which tool have you used to find out that the ATA sends the registration request but the response doesn’t come? Can you run packet sniffer at the internet-facing interface and the ATA-facing interface simultaneously, filtering by IP address of the remote SIP server, to find out using Wireshark at what point the registration conversation breaks?

Hm yes, thanks for pointing that out. An inconsistency indeed, but ether1 is on “wan” interface list - so that shouldn’t really be a problem…
To check the issue, I used Packet Sniffer with Streaming option on a router, and set it to ether1 interface. It gets “401 Unauthorized” message from the SIP server on ether1, but this message is never delivered to the VoIP gateway (checked that with another Sniffer/Wireshark instance on ether2).
Upgrading to 6.42rc didn’t fix the issue so firmware version doesn’t seem to be a culprit - it’s “just” NAT now being broken for this specific device (as “normal” internet browsing works just fine)
Temporarily fixed the issue by installing an RB951Ui (with similar config) instead of crappy Asus. Will try to reset the hAP AC and redo the configuration from scratch after I’m back from my business trip.
If there’s anything else you want to see from the configuration please let me know.