Mikrotik Router SIP Connection Blocked.

canuno · Thu Sep 24, 2009 4:44 am

Sometimes my SIP device's don't register until I delete the connection. The connection to delete is UDP 5060.

Is there are any timer I can change to prevent this happening or renewing the connection from time to time to avoid this problem? Is there a script I can run?

I already changed the: Firewall|Connections|Tracking: UDP Timeout:1d 00:00:00 to 00:01:00. And is not solving the problem.

If I go to Firewall|Connections| I can delete manually that connection and it works, but I need something automatic, instead of doing this for each device in every router....

THX.

Chupaka · Thu Sep 24, 2009 2:12 pm

have you tried playing with 'Firewall->Service Ports->sip' state?

canuno · Thu Sep 24, 2009 6:47 pm

Yes, it is enabled the SIP service on ports 5060 ad 5061.

I have this problem with my SIP connections when one of my gateways fails, lost Internet Connection for a moment, change of IP, etc. and then I need to move the connected SIP connections over to the other interface, or deleting that specific connection and it will reconnect immediately after deleting that "freezed and not Real Connection". I even can disconnect my SIP device and it still appears that the connection still working... until I do the "IP - Connections - Connection Remove" It will work again as fast as I connect the device again, or if it´s already connected It will work immediately.

THX, for your help.

Chupaka · Thu Sep 24, 2009 9:02 pm

oh, I see... it's NAT problem. you may delete connections by something like

/ip firewall connection remove [find where src-address~"\^client's_ip:"]

canuno · Thu Sep 24, 2009 9:24 pm

Thats correct, but the problem is that this is happening very often, and some Routers are in a Remote area, so I want to write a Script in each device, to do this automatically, is there a way instead of doing it manually every time?

Thx.

Chupaka · Thu Sep 24, 2009 10:15 pm

what if you remove connection while SIP device is in work?..

canuno · Thu Sep 24, 2009 10:18 pm

It will Reconnect automatically, and it will work again in 2 seconds.

Chupaka · Thu Sep 24, 2009 10:45 pm

2 seconds is too long for periodic action... so you need to determine link reconnection in some way...

canuno · Thu Sep 24, 2009 10:54 pm

I was planning to run this script every 30 minutes or every hour, thats enough for the service they need, and It will keep them connected. But I´m not so sure how to write that script. The time I checked with them right now, and They say it is automatic, they removed the connection and immediately they got the green light on the device, meaning that they are connected again.

Script:

:put [/ip firewall connection remove [/find where src-address~"1.1.2.6"] ]

Where 1.1.2.6 is the src. address and 192.168.2.200 is the Dst. Address.
is that correct?

Chupaka · Fri Sep 25, 2009 9:14 pm

well, if 192.168.2.200 is the address of your SIP server, then it's better to use

/ip firewall connection remove [/find where dst-address~"192.168.2.200"]

, I believe

changeip · Fri Sep 25, 2009 9:50 pm

i think there is more going on here that we need to investigate. We have the same issue periodically and a router reboot seems to help it for a few weeks. im wondering if there is a problem with the connection tracking somehow. you shouldnt need to be removing connections manually, especially UDP, they should just flow thru.

Chupaka · Fri Sep 25, 2009 9:57 pm

changeip, are you using NAT in that config?

changeip · Fri Sep 25, 2009 11:01 pm

yes. worked fine for many years in same config setup, until just recently i replaced this offices x86 with a routerboard. have seen it in multiple locations with routerboards so far, all using nat. typically i assign a sip control port and a range of rtp ports for each phone behind nat so i dont need helpers, etc. same configs, just later versions of MT i am seeing more of this. sip helper disabled cuz it always seems to break things : )

changeip · Fri Sep 25, 2009 11:01 pm

i forgot to mention, it seems to only happen 1 min into the conversation, probably matching the udp timeout setting maybe. will packet sniff and investigate more now that i know others are having same issue.

Chupaka · Sat Sep 26, 2009 12:03 am

I thought, canuno's problem is that when router's IP changes, it still does NAT to previous IP...

canuno · Sat Sep 26, 2009 12:47 am

THX for your interest and help. I really appreciate your advice for this issue.

Chupaka · Sat Sep 26, 2009 12:53 am

you are welcome =)

burnit · Sun Oct 18, 2009 12:28 pm

There is definately an issue with the Mikrotik maintaining and tracking a SIP connection. Obviously becomes more of an issue with multiple NAT's. For some reason, different SIP kit seems to manage/handle this better, but we have found that specifically with Quintum kit, everything breaks.

This was my solution, and I run it every 5 minutes through the scheduler... Workaround works and clients satisfied.

:foreach i in=[/ip firewall connection find dst-address~":5060" protocol~"udp"] do={
/ip firewall connection remove $i
}

dolf · Fri Oct 30, 2009 6:42 pm

HI

We have the same problems and it is only fixed by deleting the connection. I will try this script.

Dolf

dolf · Fri Oct 30, 2009 6:58 pm

I tried the script but it doesn't work. I did include an ip address. I don't see anything in my logs te help me find the fault.

Thanks

Dolf

canuno · Mon Nov 02, 2009 6:03 pm

Yes, I Got the same result, I hope they can fix the bug in a new version, instead of trying to delete connections or restarting the router every week or something like that....

THX.

I tried the script but it doesn't work. I did include an ip address. I don't see anything in my logs te help me find the fault.

Thanks

Dolf

FIPTech · Sat Dec 18, 2010 6:56 pm

This problem is an old one in the Linux world. I reported it many years ago but it seems there is still no solution inside kernel 2.6.

It is related to the way connection tracking and maskerading does work. It was present with Linux 2.4 but is still here with Linux 2.6.

It does show up mainly when there is NAT masquerading.

- if there is multiple gateways used for redundancy (connection tracking keep the old public IP when the gateway change)

- the gateway is a PPPoe connection (in this case sometimes Linux can forget to masquerade and you send a private IP to the world. Does exhibit if you are sending SIP or IAX frames through the PPP connection during PPP disconnects).

The solutions are (from easier to harder) :

- do not use multiple gateways with NAT masquerading

- avoid using PPPoe. Prefer IPoA, MER, MPLS or native Ethernet xDSL links.

- use a firewall who is friendly with NAT and multiple gateways support (usually found in a session border controller)

- use static NAT instead of masquerading and change the static NAT IP address according to the used gateway

- reset the ghost connection after gateway change using for example netwatch and a script

- ask mikrotik to do something for you

- change your VoIP system for a traditionnal TDM system

tristan.bolton · Fri Oct 21, 2011 7:59 pm

Has there been any progress with this?

We are running RouterOS 5.7 and have had this problem since 3.30.

Its not just NATing, we have the same issue with routing.

We have SIP clients trying to connect to the SIP server over a VPN. However, if the SIP clients try to connect before the VPN is brought up, then they will go out the Default Gateway and not the static route down the VPN (obviously because the route is not valid until the interface comes up).

Once the interface does come up (seconds after the WAN is online), Mikrotik will not route the SIP client down the VPN, but continue it out the WAN.

We remove the connection. Works

Simple Network Diagram:

(SIP Client - 192.168.1.50) -> Mikrotik -> INTERNET -> Mikrotik -> (SIP Server - 192.168.2.50)
The 192.168.1.0/24 and 192.168.2.0/24 are routable across the VPN (I've tried OVPN & PPTP)

Let me know if you have any thoughts.

Currently this is fixed with a simple script.

Chupaka · Sun Oct 23, 2011 3:56 pm

Once the interface does come up (seconds after the WAN is online), Mikrotik will not route the SIP client down the VPN, but continue it out the WAN.

sounds a bit like 'RouterOS routes the SIP client through the VPN, but still NATted with WAN address - because it's imppossible to re-NAT established connection'...

p.s. or, it depends on your mangle setup

karina · Thu Jul 18, 2013 1:26 am

just to keep this thread up to date. same problem in ver 6.1 SIP UDP sessions need to be refreshed with a script every few hours to maintain registration when gateway is a pppoe interface.

Chupaka · Fri Jul 19, 2013 6:05 pm

SIP UDP sessions need to be refreshed with a script every few hours to maintain registration when gateway is a pppoe interface.

and all that time pppoe is up, without disconnects?

karina · Fri Jul 19, 2013 7:04 pm

yes.

still see the strange behaviour where a UDP connection time out will count down to 0 the start to count back up. The upwards count just continues forever till the connection is manually closed.

I am running a script to close all udp connections every 24hrs early in the morning, All SIP endpoints remain registered for the whole 24hr period ok.

I only see this behaviour when the gateway is pppoe

dendzo · Mon Aug 05, 2013 11:49 pm

Same problem here.
I am using Comcast cable in US at few locations, and VPN over PPPoE in Serbia at the other locations.
In the case of Cable, it is dynamic IP, but it hasn't changed when it happened. Although, at is happening more often at some locations, and some of them works for 6 months.
In the other case, it is complex network diagram and lot of things can cause this. But it is static IP.

I did my own experiment, and I haven't had a case that I needed to wait for 2 sec until connection is reestablished. I even didn't noticed any disruption at all while in a call!

THG · Wed Aug 07, 2013 4:46 pm

Have you tried to disable the service ports for SIP in the firewall settings?

dendzo · Wed Aug 07, 2013 4:52 pm

No. How do you think it is going to help? I was playing with that befire and only thing I saw as a difference is what it shows up in firewall/connection... I will try, but I will have to wait for some time to test it. It is not showing up that often.

Inssomniak · Wed Aug 07, 2013 7:33 pm

Just to add, I have about 50 PPPoE clients with SIP boxes, no nat, only routing. I dont have any complaints of any issues and I dont run any scripts to refresh any connections.
I have the SIP registration to 60 seconds.

BUT I have seen this issue in the linux world, and in some legacy CPE by ubiquiti. These CPE I cant use SIP and dynamic IP.
BUT I have also seen this with mikrotik and NAT, SIP breaks

canuno · Fri Sep 27, 2013 11:29 pm

Is there any progress on this issue? Are you working on it or Mikrotik developers don't want to spend time on this?

samsung172 · Sat Sep 28, 2013 1:06 am

In my case it help to give the udp timeout in connection tracking a higher value. I have some setup to 3 minutes. I don't work to mutch with sip, but if a customer reports any issue with ip thelephony, I change the setting, and it use to be OK.

sviin · Sun Apr 27, 2014 11:11 pm

Hi everyone.
This is sad, but the problem still exists in 6.12.
My router works as
- internet router with NAT
- PPTP client with another NAT
I have 2 SIP connections - first one to internet service, second to private IP switch on the other side of PPTP.
PPTP connection takes some time to establish, so routerboard first tries to route the connection over internet NAT and remembers this wrong route for an hour! I have to kill the connection manually to get the phone working.
And I can't turn off SIP helper, because this way second connection works, but not the first.

I did a workaround by making the private switch listen on 5068 instead of 5060. Now it works fine, but it won't work for everyone. Mikrotik, why not lower the SIP timeout or (better) make it changeable by user?

falestiny · Mon Apr 28, 2014 8:40 am

it's working with me without any problems.

sviin · Mon Apr 28, 2014 1:35 pm

it's working with me without any problems.

What exactly is working? Do you have the same setup?

Chupaka · Mon Apr 28, 2014 3:45 pm

PPTP connection takes some time to establish, so routerboard first tries to route the connection over internet NAT and remembers this wrong route for an hour! I have to kill the connection manually to get the phone working.

can't you just forbid (in Firewall Filter) that connection over Internet, and allow it only via PPTP?

sviin · Tue May 06, 2014 11:43 am

Thanks for the advice.
Tried this, but the problem is more complex then I thought. Plainly, sometimes it works, sometimes it doesn't, and I can't understand why.
Can you tell me, what exactly does SIP helper do?

Ferrograph · Sat May 17, 2014 2:40 pm

Im seeing this problem too but with cabled WAN on dhcp. After some time sip connections cant register. Its very frustrating, and Im getting it in the neck from users!

Im on v6.12, but I just updated to v6.13. Likely its still an issue in v6.13, as I dont see any fixes related to sip or UDP connection tracking.

Yes please, Mikrotik, do tell, what does IP-->Firewall-->Service Ports-->sip setting actually do?

Ferrograph · Sat May 17, 2014 3:12 pm

Im just watching the connection list with sip setting in service ports turned off and a filter in the connection table where dst-port is 5060.

Should I be seeing connections with the timeout counting up?

Ernstm · Tue Jun 03, 2014 12:51 pm

Just wanted to add my comment as I had the same problem.

I run the following script every 10 minutes.

/ip firewall connection remove [/ip firewall connection find where connection-type=sip and assured=no]

works for me at the moment. hope it helps others

sanitycheck · Wed Jul 30, 2014 12:04 am

I've been watching this topic because it's about the same problem, or a similar problem, to the one in this post:

http://forum.mikrotik.com/viewtopic.php ... 74#p439474

I listed a fix or work-around there that worked for me.

dominicbatty · Fri Nov 14, 2014 1:46 pm

I am also having the same problem with a multiple routing scenario and I can see what is happening but cannot explain why, I'm not sure if it's a bug but perhaps someone could see if they think it should be logged.

2 interfaces with public IP addresses 1.1.1.1 and 2.2.2.2
Local SIP server on private LAN - 192.168.1.1
main routing is to the line with IP 1.1.1.1
Connection is made outbound to the IP provider on the internet at 123.123.123.123:5060.

I see a NAT connection come up as follows ...

SRC address - 192.168.1.1:5060
DST address - 123.123.123.123:5060
Reply SRC address - 123.123.123.123:5060
Reply DST address - 1.1.1.1:n (random port)
Protocol - 17 (udp)

If the routing now changes to the line with 2.2.2.2, then this NAT connection never times out and continues to NAT the same traffic from my 192.168.1.1 server. It's as though it is examining the outgoing packet and as the source and destination IP addresses match the rule then it NAT's the packet based on the information in the existing rule, regardless of the fact that 2.2.2.2 is an invalid source IP on the link that has public IP 1.1.1.1

I suspect however, this is perfectly valid and is the whole point of the connection rule to match to source and destination IP addresses and apply the NAT rule accordingly.

However, the issue as I see it is that it should only do this for as long as the timeout is valid. After which it should drop the rule and re-establish it when required. What it does instead is that the timeout starts counting up instead of down and never disappears.

As people have already stated, if I remove the rule manually then as soon as the local SIP server refreshes it's registration from it's refresh parameters the rule come back correctly on the 2.2.2.2 interface. However, it's just ugly and horrible and does not appear to be working correctly.

This is the connection that comes up as soon as I delete the old wrong connection.

SRC address - 192.168.1.1:5060
DST address - 123.123.123.123:5060
Reply SRC address - 123.123.123.123:5060
Reply DST address - 2.2.2.2:n (random port)
Protocol - 17 (udp)

I suppose one option is to have a script that roams through the connections list constantly searching entries where the Reply DST Address field is based on an IP that is considered invalid and dropping them out the table but this again is another cludge.

I'll try getting this logged if people feel I'm on the right track so I'd appreciate anyone's thoughts.

Thanks, Dominic.

Ferrograph · Fri Nov 14, 2014 1:56 pm

domonic. Thanks for taking the time to post this, I figured it was something like this but not had time to investigate fully and log it as a bug.

Important detail is the timeout counting up which I dont believe should ever happen. You should totally log it.

Its been plaguing my customers with VOIP for some time now and although I can fix it by deleting all the "sip" connections in the firewall connections tab when it happens, its not good for customer confidence in the product.

Fri Nov 14, 2014 2:43 pm

However, the issue as I see it is that it should only do this for as long as the timeout is valid. After which it should drop the rule and re-establish it when required. What it does instead is that the timeout starts counting up instead of down and never disappears.

Timeout is being reset each time a packet matching the state hits the router, no matter what direction this packet is passing in.
Not sure if it helps, but have you tried specifying out-interface in your NAT rules? Please note though that in this case you might need to have several NAT rules (one for each out-interface).

bajodel · Mon Nov 17, 2014 9:33 am

Timeout is being reset each time a packet matching the state hits the router, no matter what direction this packet is passing in.

I'm not really sure of it ..I've just made a test monitoring 'sip' conn tracking: making a call (sip invite) I've seen no update on timeout.

Not sure if it helps, but have you tried specifying out-interface in your NAT rules? Please note though that in this case you might need to have several NAT rules (one for each out-interface).

I've different 'masquerade/src-nat' rules for every outbound wan connections (so I presume the rule must go 'invalid' changing wan conn); I've just made a simple test and the 'assured contrack item' never invalidates on failover.

I've just written some code lines and I'm going to test ..let me show you the idea:

wan1 ip: 1.1.1.1 ( wan1 connection-mark: w1c )
wan2 ip: 2.2.2.2 ( wan2 connection-mark: w2c )

/ip firewall connection remove [/ip firewall connection find where connection-type="sip" and connection-mark="w1c" and reply-dst-address~"2.2.2.2"]
/ip firewall connection remove [/ip firewall connection find where connection-type="sip" and connection-mark="w2c" and reply-dst-address~"1.1.1.1"]

Simple test show me this script is cleaning 'wrong' sip conntracks correctly.

..maybe better to include also the..

/ip firewall connection remove [/ip firewall connection find where connection-type="sip" and assured="no"]

NathanA · Mon Nov 17, 2014 9:59 am

Guys,

Can your SIP application work without the Linux SIP ALG/NAT helper, and if so, have you tried just turning it off...

[admin@MikroTik] > /ip firewall service-port disable sip

...to see if that fixes the problem for you?

It sounds like the Linux SIP NAT helper disregards the UDP connection timeouts you specify for connection tracking. From http://www.dslreports.com/forum/r26935307-: "The reason I suggest disabling it is that one of its 'features' is that it causes SIP connections to disrespect the UDP Timeouts and time out after one hour."

-- Nathan

bajodel · Mon Nov 17, 2014 10:15 am

Can your SIP application work without the Linux SIP ALG/NAT helper, and if so, have you tried just turning it off...

Nathan, if I've a Sip-PBX in lan I've usually disable SipAlg/helper and I make manual/specific rules for SIP and RTP, but if I've several Sip Phones which have to register to external it's really difficult to set up things (not all sip phones permit sip parameters fine tuning) ..you must do sip/rtp rules for each one's. Sometimes impossible, sometimes only .. boring

It sounds like the Linux SIP NAT helper disregards the UDP connection timeouts you specify for connection tracking. From http://www.dslreports.com/forum/r26935307-: "The reason I suggest disabling it is that one of its 'features' is that it causes SIP connections to disrespect the UDP Timeouts and time out after one hour."

interesting .. but not really promising

NathanA · Mon Nov 17, 2014 11:55 am

interesting .. but not really promising

Well, no, it's both interesting and promising. If we can pinpoint where the problem is, then that makes it easier to get specific when filing tickets with MikroTik, and perhaps even makes it easier to devise a workaround that can be used in the meantime. The better your understanding of the problem is, the more effective (and creative) you can be when you go to tackle it.

For example, the reason I asked about whether you can get away with disabling the SIP ALG in your specific case is because I have found that if my external SIP proxy is Asterisk, and I configure Asterisk to also locally bridge/proxy the RTP audio (directmedia=no, directrtpsetup=no) and then force-enable "rport"/RFC3581 behavior and symmetric RTP port response for every remote peer (nat=yes), then it solves the NAT problem. I can turn off the MikroTik SIP ALG and have multiple SIP phones behind the NAT, and even without a STUN server, all phones work fine. No manual SIP or RTP rules per phone required. It places a little bit of an extra burden on Asterisk, but it works.

Of course, if you are not running the SIP proxy that the phones are registering to and the proxy is not under your control, then you might not be able to do this.

-- Nathan

bajodel · Mon Nov 17, 2014 12:23 pm

Well, no, it's both interesting and promising. If we can pinpoint where the problem is .. (cut)

..from this point of view you are absolutely right!

(cut) .. RTP audio (directmedia=no, directrtpsetup=no) and then force-enable "rport"/RFC3581 behavior and symmetric RTP port response for every remote peer (nat=yes) .. (cut) .. multiple SIP phones .. (cut) .. all phones work fine. No manual SIP or RTP rules per phone require ..(cut)

This is really interesting and worth a try! ..time to setup an Asterisk test bed

base16 · Thu Jun 30, 2016 8:47 am

I know I'm bumping a very old post. But reporting in that this is still an issue.
We are running the suggested script every minute to make this run.
Thankfully it works, however it would be nice to have a long term fix.

canuno · Fri Jul 01, 2016 11:47 pm

All these years I´m still using the script, there has not been a real solution for this problem.

pnc · Sun Jul 03, 2016 11:43 pm

Having the same issue when upgraded to the latest version.

PPPOE session gets bumped from our ISP and phones cannot re-register. Need to clear out connections and it works. Disabled SIP in Firewall Ports etc.

We do not see this issue on a /32 /30 IP address.

fmodolo · Tue Feb 21, 2017 10:54 am

The only solution I found is to use tcp transport for sip signaling. Worked immediately without any further issues. I don't understand why Mikrotik is not taking care about this.

Ferrograph · Mon Jan 29, 2018 7:10 pm

Still got this issue in 6.41.

I've tried deleting connections but this doesnt always work. In this case the only thing to do is to reboot it. But it gals me to do that - this is about the only thing I have to reboot Mikrotik for. How sad thats its something so basic.

Ferrograph · Mon Jan 29, 2018 7:26 pm

Actually, I have a theory.

I've seen the connections timeout period count upwards which shows they are getting in some weird condition. Maybe the delete doesn't fully delete connections in this state and it gets orphaned and hidden from the list. I theorised that loosing the WAN might do some blanket clear-out of connections associated with the WAN. This is just a theory I came up with after several days observing this issue and being unwilling to reboot as the solution.

Dropping the WAN worked in my case.

aoakeley · Thu May 31, 2018 4:49 pm

Still got this issue in 6.41.

I've tried deleting connections but this doesnt always work. In this case the only thing to do is to reboot it. But it gals me to do that - this is about the only thing I have to reboot Mikrotik for. How sad thats its something so basic.

Still in 6.42.3

I thought I was going crazy today, until I came across this thread. How can something so basic still be an issue for so long?
Though to be fair I cant see any reference in this thread to anybody presenting diagnostics to Mikrotik to get them to resolve it....

At the site that I experienced this at; I have put a script in place to drop connections as a bandaid, and I will change the WAN from PPPoE to a /30 once I can get some IP's allocated. If it was not a live site I would be the one to properly document it and report, but alas it wont be.

NathanA · Wed Oct 03, 2018 7:19 am

Since I see people have still been posting in this thead, and some of the more recent responses have mentioned that they are using PPPoE, I thought I'd stop by to let people here know that as of RouterOS 6.33, if you are using PPPoE on your WAN and suffering from this problem, you can use the "on-up" PPP event to trigger automatic, surgical removal of only the problematic NAT conntrack entry whenever your PPPoE connection bounces. No more manual clearing out the entry (or all entries, if you're in a hurry) or rebooting, or scheduling a script to run every few seconds to check if it needs to be done. See this post.

-- Nathan

mixig · Thu Oct 04, 2018 7:43 pm

I can confirm that from version 4.x till now 6.4x same thing if PPP interface is in use so I use this one as a script and no more reports from customer:
/ip firewall connection remove [/ip firewall connection find where connection-type=sip and assured=no]

Shane77 · Sat Nov 24, 2018 5:37 pm

Good afternoon gurus and gentleman,

Ive followed this thread till attaining a successful result with failover operational via dual LTE connections and the script running every minute to clear out the NAT connections so thank you to all those who have ventured through frustrating circumstances so that others may learn. This configuration works perfectly with my sip phones communicating directly to the voip server on port 5060 but ive now implemented cloud pbx solutions to my clients which is currently outsourced and the port numbers to the cloud pbx (FreePBX) are nowhere near 5060. So im dealing with a new IP range which changes from client to client and different ports eg 3333 which works to an extent whereby the phone is able to make a call after switching gateways but doesnt receive a call even though its registered. Id appreciate any advice on the matter with humble thanks

Chupaka · Sun Nov 25, 2018 1:54 pm

Have you tried adding your custom ports to SIP Helper?

Shane77 · Mon Dec 03, 2018 8:20 am

Have you tried adding your custom ports to SIP Helper?

HI,

Please excuse the late reply as this exercise has my neurons in a twist. Ive tried the SIP helper by adding the port 3333, also running scripts to delete UDP connections matching the Cloud PBX server and many other permutations found on the net with no success all relating to clearing out the old NAT rules. I found that a reboot on the router OR de-registering the sip phone and registering it again brings the extension back online after failover has taken place onto the new connection.

Any suggestions?

These are the scripts ive tried:

/ip firewall connection remove [find where dst-address~"Cloud PBX IP"]
/ip firewall connection remove [find where connection-type=sip or connection-type=sip-2 or connection-type=sip-1]

I found that a reboot on the router OR de-registering the sip phone and registering it again brings the extension back online after failover has taken place onto the new connection.

Seems i was incorrect for the above statement. Rebooting the router does not work, only re-registering the extension does. So this may not be a problem with NAT?

sindy · Mon Dec 03, 2018 11:15 am

Reboot of router removes any and all tracked connections, so any packet coming in from the WAN side, except those matching dst-nat or accept rules, is dropped as it does not match any existing tracked connection. Only by re-registering the phone you create a tracked connection so further packets coming from the WAN side are matched to that connection and forwarded in.

All the scripts removing existing connections are there only to prevent LAN->WAN packets from being src-nated to the IP address of the WAN through which their connections have been previously established, so that new registration or outgoing call could establish a new connection which gets be src-nated to the IP address of the WAN active at that moment. So even if you remove those connections with inadequate src-nat address, no incoming calls can get in until the phone re-registers as a) the telephony exchange needs to learn the new public IP to which it should send incoming INVITEs and b) the connection tracker in Mikrotik's firewall has to expect the incoming INVITE as part of the UDP (or SIP if the helper is active) connection established by the REGISTER.

A plain UDP tracked connection survives 3 minutes since the last packet seen, but tracked connections with protocol~"sip" survive 60 minutes following a successful registration no matter whether any packets are seen or not. So if the registration expires before those 60 minutes and the phone attempts to re-register, it still hits an existing connection with a wrong src-nat address, and by reattempting to register over and over again it keeps it up although a failure to register shrinks the connection life back to 3 minutes. Telephony exchanges which can deal with customer side NAT send keepalive packets (either valid SIP packets or just UDP packets with the proper source and destination socket addresses) to keep the 3-minute plain UDP connections up, so they don't need the SIP helper to be active. They also ignore the RTP destination socket which the CPE sends using SDP and learn it from incoming RTP.

Shane77 · Mon Dec 03, 2018 1:27 pm

Hi Sindy,

Ive been following your response on another thread and trying to figure out a solutions since ive been sitting with this issue for over 3 weeks now and im now closer to resolving. Would u mind taking a look at my export config to tell point out possibly where im going wrong?

sindy · Mon Dec 03, 2018 1:49 pm

It's not only a matter of the Mirotik configuration but also of how the exchange behaves and of the network topology (which part of the voice solution is on the public side of the firewall and which is on the private one). This should tell whether you actually need the SIP helper in Mikrotik to be active or you can do without it, or, in other cases (i.e. other than yours because for you it does work unless WAN failover happens), whether the SIP helper is even able to help.

So I don't mind looking at the config, but I'd like to know also the topology and you should check the behaviour of the exchange, maybe it does send the keepalives which would inidicate that it is also capable of dealing with the NAT without the SIP helper.

Shane77 · Mon Dec 03, 2018 2:36 pm

HI,

As discussed.

Config
# model = RouterBOARD 931-2nD r2

/interface bridge
add fast-forward=no name=bridge1
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n disabled=no frequency=auto mode=ap-bridge ssid=********
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] authentication-types=w****** *******="*****" mode=****** supplicant-identity=*******
/ip pool
add name=dhcp_pool0 ranges=192.168.137.2-192.168.137.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=bridge1 name=dhcp1
/interface bridge port
add bridge=bridge1 interface=wlan1
add bridge=bridge1 interface=ether3
/interface bridge settings
set allow-fast-path=no
/ip firewall connection tracking
set enabled=yes
/interface list member
add interface=ether1 list=WAN
add interface=ether2 list=WAN
add interface=bridge1 list=LAN
/ip address
add address=192.168.137.1/24 interface=bridge1 network=192.168.137.0
/ip dhcp-client
add add-default-route=no dhcp-options=hostname,clientid disabled=no interface=ether1 use-peer-dns=no
add add-default-route=no dhcp-options=hostname,clientid disabled=no interface=ether2 use-peer-dns=no
/ip dhcp-server network
add address=192.168.137.0/24 gateway=192.168.137.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4,10.0.0.254
/ip firewall filter
add action=accept chain=input connection-state=established,related
add action=accept chain=input protocol=icmp
add action=drop chain=input in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN src-address=192.168.137.0/24
/ip firewall service-port
set sip ports=5060,5161 sip-timeout=10s
/ip route
add check-gateway=ping distance=1 gateway=8.8.8.8
add check-gateway=ping distance=2 gateway=8.8.4.4
add distance=1 dst-address=8.8.4.4/32 gateway=10.0.0.254 scope=10
add distance=1 dst-address=8.8.8.8/32 gateway=192.168.0.1 scope=10
/system clock
set time-zone-name=Africa/Johannesburg
/system routerboard settings
set silent-boot=no
/system script
add dont-require-permissions=no name=script1 owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
source="/ip firewall connection remove [find where connection-type=sip]"

The setup communicates with a Cloud PBX running on port 3333

sindy · Mon Dec 03, 2018 3:14 pm

The i-thing to the right of the Yealink phone symbolizes what? A softphone running on a PC, a generic PC not related to the voice service but using the same internet connectivity?

Samot · Mon Dec 03, 2018 3:54 pm

OK, so I'm reading this thread and I have yet to see any real SIP debugs or information that is proving the MT's are the actual cause here. It looks like a bunch of assumptions and guessing.

I am 100% a ITSP (Internet Telephone Service Provider) and I use MT's 100% for my end users. I have a wide variety of setups from a simple legacy PBX (FXS/PRI/T1/E1) that have one or more gateways that convert SIP to their handoff interface. I have IP-PBX's on their own, IP-PBX+IP phones connected to my hosted platform and locations with purely hosted solutions (just IP phones). In all these cases I have MT's sitting there as the core router.

I have a hotel right now with 100+ rooms that is completely _hosted in the cloud_. That is over 100+ devices registering and making calls over the Internet. Not to mention their Front Desk and other phones around the hotel. So it's more like 110+ devices.

I have offices that have 15+ users and each of their phones have BLF monitoring of the other 14 users plus Voicemail boxes, 9 Park Orbit Slots and various other things. That is basically 25+ accounts per phone that is sending SUBCRIBEs or REGISERs and getting MWI and other BLF NOTIFY messages.

So again, I'm seeing the blame being laid on MT as the core of the issue for people but I have yet to experience any of this. With amount of SIP devices/endpoints that I have out there sitting behind MTs I'm pretty sure I would have seen this and have people calling about how their voice is just not working right. I'm not.

So for the people actually having this issue, forget the MT part of it (telling me how it's this and that in the MT) and just tell me what the _actual SIP issues are_. Like most things SIP can present the same symptoms for various problems and not understanding what those problems are and how they share symptoms can lead someone to the wrong troubleshooting path and end up blaming the router when it's not (or vice versa).

Shane77 · Mon Dec 03, 2018 4:33 pm

Samot, WHOH there tiger! I swear if someone doesn't contact you from that little bit of subtle advertising then this world is gone all crazy....Nobody blamed the MK, its a basic understanding of how the NAT and UDP protocols apply in this scenario which till now has differed from all most other listed solutions deploying the failover which is also for my own understanding and learning (Caution: may slightly defer from yours)

, We all begin somewhere so consider this my entrance into the realm.

Sindy, yes those are standard Windows PC users on the same network.

I realised my description is a little vague. My deployments currently consist of sip devices and pc users on the same network mask running off two gateways on different IP ranges. The SIP devices are set to static gateway along with the most reliable LTE connection which serves as the dedicated connection this way. Now with the dire need for failover, this was the most viable solution since ive been using Mikrotik in basic setups for some time.

sindy · Mon Dec 03, 2018 11:00 pm

In your overall topology, it makes no difference whether the SIP helper is active on the 931 because the LTE router also does NAT and there is little chance that it would include a SIP ALG, and the LTE operator is quite likely to do NAT as well. So if your cloud exchange successfully handles clients connected via LTE, it definitely doesn't need the SIP helper to be running at the 931.

Looking at your configuration, I cannot see anything unusual there, except that there are no signs of automatic triggering of the connection clenaup script at the failover event.

You've chosen the scriptless failover method monitoring external reference IP addresses, which is very convenient but as it requires no script to track the state of the uplink path, that non-existent script cannot be extended with the connection cleanup part.

So what you need is to schedule a script for a periodical run (once in 10 seconds is enough as that's how often the check-gateway process pings the reference addresses) which will check the primary path availability and if it changes as compared to the previous state (in any direction, i.e. both from available to unavailable and vice versa), it cleans up the SIP connections, but rather than choosing them up to protocol~"sip", it should choose them up to dst-address~"^the.cloud.pbx.ip.address:3333\$". If the cloud PBX address is eventually specified as FQDN and it sometimes changes or multiple IP numbers are returned in the DNS response, you need to let the script deal with that because the connection's dst-address is always numeric.

But even if you automate the connection cleanup this way, you have to bear in mind that removing the old connections makes only one half of the task. The other half is that the phone has to re-register itself because the exchange must learn the new public IP through which the phone can be reached. So unless you can actively force the phones to re-register, which is unlikely, you have to set them to register as frequently as the exchange permits so that the time from failover to re-registration was as short as possible.

If the unavailability periods of up to minutes between WAN change and phone re-registration are unacceptable for your clients, you may consider a more advanced redundancy scheme. For that one, you'd have to run a virtualized Mikrotik (CHR or x86) somewhere in a data center, and create two VPN tunnels (such as GRE over IPsec) to it from the 931, each forced via one of the WANs, and route the VoIP traffic between the CHR and the 931 through these tunnels without NAT. So the failover from one of the VPN tunnels to the other one would not change the public IP address of the phone from the point of view of the cloud exchange, as it would always see the CHR's public address. Which means that the service interruptions associated to the failover events would be much shorter and even active calls wouldn't drop, there would just be a short period of silence in them. Of course, the CHR is another Single Point of Failure in the topology, the first one being the 931, but as you've seen the reliability of the connections is much lower than the reliability of the matchbox 931, so if you choose a decent data center, you should be fine too.

There are some limitations regarding the number of tunnels per CHR, so you might have to deploy more than one depending on the number of client sites. Also, you might need to NAT the LANs of the client sites when connecting via the VPN tunnels to individual addresses from some public subnet unrelated to the VoIP service to permit overlapping client subnets (although two clients use the same subnet on their LANs, the CHR can see each of them as another individual IP address from 4.4.0.0/16, but this address is accessible for it via both VPN tunnels so there is no need to remove any connections at either the 931 or the CHR if one of the tunnels goes down).

Ferrograph · Mon Dec 03, 2018 11:11 pm

I think we're forgetting that deleting the connections manually or by a script only works some if the time. If the connections get in that wierd state where the timeout starts counting up even deleting them doesn't work - even though they are gone from the connection list I think the that fact they have a timeout value means they still in effect somewhere. It's only thing that makes sense.. Only a reboot or disabling the wan interface clears them out properly.

sindy · Mon Dec 03, 2018 11:28 pm

If the connections get in that wierd state where the timeout starts counting up even deleting them doesn't work - even though they are gone from the connection list I think the that fact they have a timeout value means they still in effect somewhere. ... Only a reboot or disabling the wan interface clears them out properly.

Good point. However, I've obtained an impression that this "zombie" behaviour is specific to connections handled by the SIP helper, and what I've pointed out was that in @Shane77's specific case, the SIP helper can be disabled completely with no impact on the service.

Shane77 · Tue Dec 04, 2018 10:59 am

HI all,

Sindy, the explanation is very detailed and thorough so i respect your time taken out for this and went through the options carefully. As Ferrogragh point out, even manually deleting the connections in this case doesnt work. Only rebooting the SIP phone (or even logging into the phone setup menu and clicking the confirm button to accept a blank setting works too) brings the extension back online. May i ask one more thing of you and thats to point out a similar setup on the client side which i can follow in detail to check if i have accommodated for all that i can possible from the modem side right down to the MK. Failing to do which i will have to contract someone to complete this task and broaden my knowledge. Ive never failed at an MK task before and it seems searching and reading the web solidly over three long weeks and nights has failed.

sindy · Tue Dec 04, 2018 11:22 pm

May i ask one more thing of you and thats to point out a similar setup on the client side which i can follow in detail to check if i have accommodated for all that i can possible from the modem side right down to the MK.

In all my setups where SIP phones are behind a NAT, one of which is on Mikrotik, all the phones are registered to phone exchanges which deal with client side NAT themselves, which means that the SIP helpers of any kind are just disabled. And unfortunately none of my setups uses some type of failover setup and hosts SIP phones at the same time, it's either-or.

Given that the unremoveable connections seem to be an issue specific for connections controlled by the SIP helper, and as your overall architecture clearly shows that the phone exchange in the cloud can deal with the NAT issue without the SIP helper, I've recommended you to disable the SIP helper at the 931 completely as the only change as compared to the configuration you've posted. To my best knowledge it should be enough for you to get rid of the problem of unremovable connections, and sorry if I haven't stated that clearly enough in the previous post. Just a note, some people say you have to reboot the router to make the SIP helper really off after disabling it, I don't remember whether it was necessary in my case as I did that years ago when the SIP helper was dropping messages it was unable to parse. I have never used it anywhere ever since as its disadvantages prevail in simple cases and it is incapable to deal with complex cases.

As the next step, what @Samot has written applies - sniff the SIP communication between the phone and the exchange while you power up the phone, and then sniff when the issue actually happens. Because either the phone stops trying to re-register automatically and you have to manually kick it to do so if it fails to register because the old connection is still alive and makes the registration fail (and I've seen a lot of unusual behaviour with CPEs), which would indicate a bug in the phone's firmware, or the phone may be configured for re-registration every 5 minutes or even more often but the phone exchange tells it that it doesn't accept such a short registration time and that the minimum registration time is, say, 2 hours. So if the failover takes place and the last registration thus becomes useless (because the registered contact uri indicates the now-unavailable public IP address), and the phone doesn't re-register within some minutes, you may simply lose patience and kick it before it decides to re-register on its own.

So sniffing at phone boot should show you how the initial registration goes (whether you get the "423 registration lifetime too brief" response to REGISTER or not and if you do, what minimum registration period the exchange requires), and sniffing while it happens should show you whether the phone keeps trying to re-register after the first re-registration attempt after the failover fails.

If you actually ask me for a script which will automatically clear the connections once a failover takes place, say so clearly.

Ferrograph · Wed Dec 05, 2018 12:35 am

I've tried extensively WITH and WITHOUT the SIP helper. You still get zombie connections. Common though is dest-port 5060/5061.

Perhaps there is something in the firewall/NAT that recognises 5060/1 in addition to what the SIP helper does?

sindy · Wed Dec 05, 2018 5:41 pm

@Ferrograph, are you saying you have a sequence of manipulation steps which is guaranteed to create the unremovable connections? If so, would you mind sending a description of that sequence to support@mikrotik.com along with your supout.rif? What I felt so far was that it happens at random and therefore Mikrotik is unable to identify the root cause.

@Shane77, given that @Ferrograph says disabling the interface is guaranteed to clear the zombie connections (which requires that the reply-dst-address of the connection was created using action=masquerade), there are two more things to try:

instead of using /ip firewall connection remove in the script, use /interface disable followed by /interface enable. Given that the only reason for the failover is that the path to the internet through the interface breaks down, shutting the interface down for a moment will not do any harm (which may not be that simple in case of the fallback where the secondary WAN may be used as the only one for some specific traffic)
instead of using a common chain=srcnat rule with action=masquerade, use one action=src-nat rule per each WAN and set to-addresses to the address of that WAN manually (which may require another script to copy the currently assigned IP address into the rule's to-addresses parameter if the LTE modem doesn't accept a static address on your WAN looking towards it and you have to use a DHCP client there). The rationale behind this suggestion is that if the zombie connections only occur if the reply-dst-address is assigned using action=masquerade, assigning it using action=src-nat may be a way to avoid them.

Regarding hiring someone to solve it for you - as it seems that there is an issue with removing the zombie connections, the only people able to resolve it are Mikrotik's developers. I'm not sure it is possible to hire them for a field task like this

pjm · Thu Jul 04, 2019 12:01 pm

I'm glad i read this thread from 2009, because it's the same issue i'm having in 2019, often with sip box's (ATA's) but also with an asterisk box (freepbx) working as a proxy but suffering less often.
And thinking about this also a problem at another site with mikrotik router and asterisk, maybe once every couple of months.

At least now i have a possible solution to my issue.

Who is online