Chateau LTE12 cell lock LTE link flapping with CA

Sean1 · June 23, 2020, 10:26am

I have a Chateau LTE12 with firmware v7.0beta8 and am connecting to a cell approx. 53km (33 milles) across a bay with a pair of outdoor directional LOG antennae. The cell (Three Ireland) operates on band 20 (EARFCN 6300), band 3 (EARFCN 1275 and 1700) and band 1 (EARCN 525). I can connect on band 20 only as the primary band and use bands 3 and 1 for aggregation only due to that cell distance.

When I configured the APN and chose band 20 only, the Chateau connected and the connection was stable, but very slow (<10Mbps) due to high congestion on band 20. When I tried selecting bands 20, 1 and 3, the Chateau lost the connection and endlessly showed this status:

[admin@MikroTik] > interface lte monitor
number: 0
status: searching
model: EG12-EA
revision: EG12EAPAR01A05M4G
imei:
imsi:
uicc:

I think it was trying to connect on band 3 or 1 as the primary band, which I can only use for aggregation due to the distance. I ran the following command based on SiB’s post here to lock the Chateau on cell 127 on EARFCN 6300 (band 20):

interface/lte/at-chat lte1 input=“at+qnwlock="common/4g",1,6300,127”

The Chateau connects to this cell and aggregates with a cell on both EARFCN 1700 and 525:

[admin@MikroTik] > interface lte monitor 0
status: connected
model: EG12-EA
revision: EG12EAPAR01A06M4G
current-operator: 3
current-cellid: 828423
enb-id: 3236
sector-id: 7
phy-cellid: 127
access-technology: LTE
session-uptime: 24s
imei: [Removed]
imsi: [Removed]
uicc: [Removed]
primary-band: B20@10Mhz earfcn: 6300 phy-cellid: 127
ca-band: B3@20Mhz earfcn: 1700 phy-cellid: 115,B1@15Mhz earfcn: 525 phy-cellid: 115
cqi: 8
ri: 1
mcs: 0
rssi: -63dBm
rsrp: -97dBm
rsrq: -14dB
sinr: -2dB

The speed is now very fast (100-150Mbps). However, the connection keeps interrupting every minute or two, e.g. downloads stall, VoIP voice, video calls freeze, etc. When I print the LOG, the LTE Link keeps going down and up as shown in this snip:

11:56:43 interface,info lte1 link up
11:57:13 interface,info lte1 link down
11:57:13 interface,info lte1 link up
11:57:43 interface,info lte1 link down
11:57:43 interface,info lte1 link up
11:59:13 interface,info lte1 link down
11:59:13 interface,info lte1 link up
12:00:58 interface,info lte1 link down
12:00:58 interface,info lte1 link up
12:01:13 interface,info lte1 link down
12:01:13 interface,info lte1 link up
12:01:43 interface,info lte1 link down
12:01:43 interface,info lte1 link up
12:01:58 interface,info lte1 link down
12:01:58 interface,info lte1 link up
12:02:13 interface,info lte1 link down
12:02:13 interface,info lte1 link up
12:02:29 interface,info lte1 link down
12:02:29 interface,info lte1 link up
12:03:28 interface,info lte1 link down
12:03:28 interface,info lte1 link up

When I enable LTE debugging, this is what it logged the moment the link down:

19:18:28 lte,async,raw lte1: sent AT+QENG=“servingcell”
19:18:28 lte,async,raw lte1: rcvd +QENG: “servingcell”,“NOCONN”,“LTE”,“FDD”,272,05,CA407,127,6300,20,3,3,A34A,-99,-16,-66,8,255,140,-
19:18:28 lte,async,raw lte1: sent AT+QCAINFO
19:18:28 lte,async,raw lte1: rcvd +QCAINFO: “pcc”,6300,50,“LTE BAND 20”,1,127,-99,-16,-65,8
19:18:28 lte,async,raw lte1: rcvd +QCAINFO: “scc”,1700,100,“LTE BAND 3”,1,115,-101,-12,-81,0
19:18:28 lte,async,raw lte1: rcvd DL
19:18:28 lte,async,raw lte1: rcvd +QCAINFO: “scc”,525,75,“LTE BAND 1”,1,115,-110,-10,-90,0
19:18:28 lte,async,raw lte1: rcvd DL
19:18:28 lte,async,raw lte1: sent AT+QRSRP
19:18:28 lte,async,raw lte1: rcvd +QRSRP: -100,-98,-140,-140
19:18:28 lte,async,raw lte1: sent AT+QNETINFO=“cmr”
19:18:28 lte,async,raw lte1: rcvd +QNETINFO: “cmr”,9,12,1,2
19:18:28 lte,async,event lte1: +CGEV: NW DETACH
19:18:28 lte,packet,raw lte1 mbim: wdm <<< recv #0
19:18:28 lte,packet,raw 07000080 48000000 00000000 01000000
19:18:28 lte,packet,raw 00000000 a289cc33 bcbb8b4f b6b0133e
19:18:28 lte,packet,raw c2aae6df 0a000000 1c000000 00000000
19:18:28 lte,packet,raw 01000000 00000000 00000000 00000000
19:18:28 lte,packet,raw 00000000 00000000
19:18:28 lte,debug lte1 mbim: service: connect, command: packet service, notify
19:18:28 lte,packet lte1 mbim: status data:
19:18:28 lte,packet 00000000 01000000 00000000 00000000
19:18:28 lte,packet 00000000 00000000 00000000
19:18:28 lte,debug lte1 mbim: packet service: attaching nwe: 0 class: 0
19:18:28 lte,debug lte1 mbim: state 16=>14
19:18:28 lte,account lte1 session: 90s 7358/7482 bytes 96/101 packets
19:18:28 interface,info lte1 link down
19:18:28 lte,packet,raw lte1 mbim: wdm <<< recv #0
19:18:28 lte,packet,raw 07000080 48000000 00000000 01000000
19:18:28 lte,packet,raw 00000000 a289cc33 bcbb8b4f b6b0133e
19:18:28 lte,packet,raw c2aae6df 0a000000 1c000000 00000000
19:18:28 lte,packet,raw 02000000 20000000 00e1f505 00000000
19:18:28 lte,packet,raw 0046c323 00000000
19:18:28 lte,debug lte1 mbim: service: connect, command: packet service, notify
19:18:28 lte,packet lte1 mbim: status data:
19:18:28 lte,packet 00000000 02000000 20000000 00e1f505
19:18:28 lte,packet 00000000 0046c323 00000000
19:18:28 lte,debug lte1 mbim: packet service: attached nwe: 0 class: 32
19:18:28 lte,debug lte1 mbim: state 14=>15
19:18:28 lte,debug lte1 mbim: state 15=>16
19:18:28 interface,info lte1 link up

This is what cell-monitor reports:

[admin@MikroTik] > interface lte cell-monitor 0
Columns: PHY-CELLID, BAND, EARFCN, RSRP, RSRQ, RSSI, AGE
PHY BAN EARF RSRP RSRQ RSSI AG
0 B3 1700 8s
6 B3 1275 -116dBm -20dB -87dBm 5s
6 B3 1700 -106dBm -17dB -81dBm 2s
67 B20 6300 -105dBm -20dB -73dBm 2s
115 B1 525 -114dBm -10dB -95dBm 2s
115 B3 1275 -99dBm -10dB -81dBm 5s
115 B3 1700 -98dBm -10dB -80dBm 2s
117 B1 525 -120dBm -16dB -95dBm 2s
117 B3 1275 -104dBm -14dB -81dBm 5s
117 B3 1700 -103dBm -15dB -80dBm 2s
127 B20 6300 -97dBm -13dB -67dBm 2s
195 B20 6300 -101dBm -19dB -73dBm 2s
231 B20 6300 -104dBm -20dB -73dBm 2s
344 B20 6300 -105dBm -20dB -73dBm 2s

If I set the Chateau to band 20 only (i.e. no carrier aggregation), the link stays up even hours later. However, if I enable band 1, 3 or both, the LTE link starts flapping again. So I’m not sure if this due to the modem or network still trying to hand over to a band 1 or 3 cell when CA is enabled.

Is there anything I can try to stop the link dropping with CA? This issue also occurs in the original firmware (v7.0beta6) and the previous modem firmware (EG12EAPAR01A05M4G).

SiB · June 23, 2020, 3:34pm

We know about that #bug #behaviour and we write the post to SoC team, please register and attach to that case: https://forums.quectel.com/t/eg12-and-ca-failed-after-5-minutes-or-more-on-dynamic-aggregation/4685/14?u=sib

Of course good idea is write to MikroTik Support too, that way MikroTik as bigger SoC buyer then we can do maybe more in that case.

Sean1 · June 26, 2020, 10:01pm

I posted my issue on that Quectel forums thread and got a response to e-mail their support the log. Unfortunately, I got a reply saying I’ve been put through to the wrong support channel and that the e-mail support request needs to be generated from their customer MikroTik. So I will try contacting MikroTik support.

From own testing, I found that if I enable LTE passthrough, TCP connections do not break when the link flaps. Instead it causes a brief ~400ms latency spike. I downloaded a few large (1GB) files and each one downloaded successfully. I think this is due to my PC not seeing the split-second LTE link interruptions, causing outgoing packets to buffer instead of drop. I would be happy to put up with split-second latency spikes than broken TCP connections.

I would like to figure out a way of creating a virtual Ethernet interface on the MikroTik router that I can set the LTE passthrough and firewall NAT out interface to use instead, so I can still use the MikroTik as a router instead of a modem, otherwise I need to use another router for this LTE passthrough workaround. Basically I want to configure the MikroTik router in such a way that when the link flaps, that the gateway is still appears like it is “up” during the split-second the LTE link is down, so that WAN destined packets are not dropped as unreachable

Sean1 · August 24, 2020, 3:21pm

Since contacting MikroTik support about 2 months ago, they did some remote diagnostics on my connection, but it’s now a few weeks since their last reply, which was last month to say they were working with the modem manufacturer.

I also asked MikroTik if there was any way I could get the router to ignore the brief flaps so that it doesn’t drop the route each time the link flaps, but had no response to that query. Although the LTE passthrough gets around the issue of TCP connections breaking by using a separate router, the latency spikes are still a right pain during voice calls, particularly when the link flaps multiple times in a row. Occasionally the LTE connection loses connectivity after the link flaps, requiring me to disable and re-enable the LTE modem.

As it’s looking like this issue is not going to be fixed anytime soon, I finally decided to order a prepay data SIM and will use my older Huawei router to run the VoIP, WhatsApp, etc. calls over as its LTE link is stable. I will run everything else over the separate router connected to the Chateau LTE12 where the 3CA’s much faster speed is more important than latency/jitter, such as streaming and downloading large files.

SiB · September 14, 2020, 10:53am

I can confirm that EM12-G not have that problem, this is the differ vendor but the SoC is the same… means you should report that again and again with MikroTik Support.
I ask some ppl with Chateau to testing that but even now they thinks is not exist at his chateau.
Write to support

Sean1 · September 25, 2020, 2:08pm

Unfortunately, MikroTik support would not offer any further help and said that the SoC OEM claims that my issue is due to weak signals. They also said they could not do anything for the link flapping as they do not officially support the use of cell lock.

Like your observation, I have noticed that the link flapping occurs a lot less in recent weeks. Typically I would see the link down count up a few thousand every day, sometimes flapping several times a minute as my original post above shows. Now it flaps a few times an hour and sometimes stays up for hours at a time, close to 6 hours as I write this post:

It makes me wonder if part of the issue was with the network cell’s hardware firmware that a recent update improved. I was already on the current v7.1beta2 and 06 modem firmware when I noticed the sharp drop in the daily link downs.

Even if MikroTik or Quectel fixes this issue, I will continue to run my VoIP link on the separate 4G data connection. Besides link flapping causing audio drop-outs, I was also having issues with high traffic affecting VoIP. E.g. try running a speed test while on a VoIP call. With the VoIP SIP box on a separate 4G connection all to itself, I can download, speed test and upload all I want without affecting VoIP calls.

SiB · September 25, 2020, 3:37pm

Better way to track of change at LTE is use:
Watch LTE parameters: http://forum.mikrotik.com/t/sxt-lte-4g-cat6/134468/1

and you can record that changes like:

who can show you more about migrate between sector’s antennas one Tower or flaping etc.

I remember that your case was like ros7b5 works and all newer not work, seams like problem with ros internal, not signals etc.
This means at ros7.1beta2 with better signal sip works at all ? I thinks not even register.