CAPs update logic since 7.13

I have a small setup at home which includes RB5009 acting as a router and a CAPsMAN controller and three AX2s acting as CAPs. RB5009 and CAPs connected to Netgear M4300 switch.
Prior to 7.13 my update procedure looked like this:

  1. I uploaded ROS *npk and wifiwave2 update files to RB5009 and made a reboot, then updated RouterBoard and rebooted again.
  2. CAPs saw update files and updated themselves automatically.
  3. I had to log into each CAP to update their RouterBoards and reboot them manually.
    After these steps all components became updated to the same version of RouterOS itself and wifiwave2 package.

Now 7.13 came with completely different structure. I went to System-Packages on RB5009 and initiated downloading and update to 7.13. I thought that way transition will be smooth as the manufacture knows well how to manage its own equipment. However after I did this I lost any wi-fi connection. To solve the issue I took the following steps:

  1. Connected to RB5009 via Ethernet and found RouterOS updated to 7.13, no additional packages and RouterBoard that needed update. No surprises so far.
  2. Updated RouterBoard and rebooted. Everything went OK here.

After updating I couldn’t neither ping nor log into any CAP. So I went on and did the following:
3. Uploaded QCOM package to RB5009 and rebooted. Package was successfully installed.
Nothing has changed. CAPs remained unreachable. So I:
4. Connected CAPs directly to RB5009 one at a time. This way I was able to find them in Winbox.
5. Logged into each CAP and found two installed packages: RouterOS and QCOM.
6. So I updated RouterBoard of each CAP and rebooted them.
7. Connected all CAPs back to the switch.
8. Removed QCOM package from RB5009 as unnecessary.

After all these steps my config became functional again with up to date packages on all devices.

After all these unexpected hassle several questions remained:

  1. Why CAPs became unaccessible after RB5009 update?
  2. What’s the right updating procedure? Shouldn’t I use automatic update if I have CAPsMAN-based config?
  3. How can I force automatic updates for my CAPs the way it used to be prior to 7.13? Should I install both RouterOS AND (unnecessary on RB5009) QCOM updates on the controller so the CAPs are fully updated and then remove QCOM package from RB5009?
  4. If I follow the steps described in (3), how can I be sure that my wi-fi network won’t go down again? It’s rather difficult for me to connect and reconnect devices manually every time I upgrade.

What’s your experience with update to 7.13?

You’re mentioning 7.13 and wifiwave2 packages in the same sentence. This doesn’t mix. Up until and including 7.12.1 there was indeed wifiwave2 optional package, with 7.13 this changed … drivers are now packaged in wifi-qcom and wifi-qcom-ac packages.

In RB5009 (since it’s not wireless device) none of wifi-qcom packages is necessary, wifi capsman is part of basic package. On Wireless devices, one needs one of wifi-qcom optional packages, depending on wireless chip. Your hAP ax2 devices require wifi-qcom package.
When you configure capsman to host upgrade packages for CAP devices, it’s your responsibility to include all packages, necessary for all the different CAPs you may have in your network (with wifi capsman, this only goes as fa as wifi-qcom and wifi-qcom-ac, with legacy capsman it might have been necessary to upload packages for multiple architectures, etc.). It’s safe to upload all optional packages (and basic package) to configured capsman storage location, CAP will use whichever packages are needed. If you failed to include wifi-qcom package initially, then this might explain incomplete upgrade of CAP devices.

Another issue: 713 release notes include bolded warning, that upgrade to 7.13 needs to go via 7.12. Indeed older versions don’t see 7.13 as possible upgrade target. But this might not be enforced when upgrading CAPs via capsman. And that in turn might cause omission to install required wifi-qcom (or wifi-qcom-ac) package.

What of the possibilities above actually happened in your case I can not tell. You may arrive to some conclusion as you know what exactly you did to resolve the issue.

Thank you, now I can see things more clearly. Indeed as RB5009 doesn’t require QCOM package, it wasn’t installed and CAPs weren’t able to upload them from the controller even though they were on 7.12.1. So if I understand correctly, I have two ways to update without breaking my wi-fi network:

  1. Log into each CAP and update them through System-Packages. This will work fine but defeats the purpose of centrally updating all CAPs so two reboots for each CAP have to be made.
  2. Upload both RouterOS and QCOM packages to RB5009, wait for CAPs to pick them up and install then manually update RouterBoard on each CAP, reboot them, then remove QCOM package from RB5009, reboot it to update RouterOS on the controller, then update its RouterBoard and reboot again to complete the procedure.
    Am I right that no corner can be cut in procedure 2?

With the split of packages, you will have to foresee the required packages for the caps in the upgrade folder.
Once you remove them, they may even stop functioning depending on which upgrade strategy you defined in capsman.
Been there, done that, same thing seen at home with rb5009 and ax2/ax3 as caps.

Nothing which can be done about that now.

Now I see where I went wrong. So there’s no way to simplify the process I described before? Do I correctly understand that once I put some update files in the directory specified by CAPsMAN, CAPs will quickly update themselves regardless whether I update controller (in my case, RB5009) itself? So I don’t need to first reboot it with all the necessary packages (thus installing unnecessary QCOM) or the update procedure on CAPs is only triggered when controller itself is updated?
And if there is a way (may be some script) to automatically update RouterBoard on CAPs once the RouterOS is updated?

If you set update for caps to auto, once those files are in the required folder, they will do their thing.

I prefer to do it manually.

What parameters control this behaviour? What does it mean, manually? I’m not sure I understand the whole mechanism.

You can set capsman upgrade-policy to none, then CAPs will only upgrade when you trigger it in remote cap tab.

Correct.

More info:
https://help.mikrotik.com/docs/display/ROS/WiFi#WiFi-WiFiCAPsMAN

Upgrade policy options

none - do not perform upgrade
require-same-version - CAPsMAN suggest to upgrade the CAP RouterOS version and, if it fails it will not provision the CAP. (Manual provision is still possible)
suggest-same-version - CAPsMAN suggests to upgrade the CAP RouterOS version and if it fails it will still be provision

I set my config to none and will hit “Update” manually when I believe it’s time to do so.

Great! I didn’t even notice Upgrade button before. But what exactly does it do? Does it force the CAP to upgrade RouterOS and all installed packages from Mikrotik servers of does it force CAP to install available upgraded packages from the controller?

It will install available updates from capsman and reboot the selected cap.

Thank you! And the las question. Did you come across may be some script that would make CAPs (and) controller to automatically update their RouterBoards? I see Autoupgrade setting there. Is it what I want?

2 things:

RouterOS:
There are plenty of scripts available to achieve that (search is your friend) BUT it is generally advised not to do so.
What if a new version has a bug taking your complete environment down ?

Best to wait some time after a new release has been made so the first wrinkles can be ironed out.
And even for that waiting part there are scripts taking a period into account before upgrading :laughing:

(But still, I prefer do do it manually so I know what happens where and when).

Routerboard (=firmware):
I set that one to upgrade automatically. Any subsequent power-down (very rare here) or upgrade from my part (happens more often) will then take care of the upgrade of the Routerboard firmware. It’s generally not such a big deal if it’s one or two versions behind.

Sorry, you must have gotten me wrong here. What I meant was to upgrade Routerboard right after I manually activated ROS and packages upgrade. Does Autoupgrade option in Routerboard settings do exactly what I want?

Read last part.
Next reboot (or power down and up again) will take care of upgrade if you set that option.

So it won’t trigger that reboot automatically?

Nope.
It’s not needed either.

Next reboot will take care of it.
Or if you’re really so worried about (not needed), you can always manually reboot.

But again, sooner or later a reboot will happen (power loss or upgrade) so upgrade will take place.

Yes but if I got it right you have to manually log into every CAP and reboot it. It would be nice to be able to upgrade all components and make all necessary reboots with just one click of Upgrade button in Capsman.

Again … it doesn’t matter.
Sooner or later it will be done through power loss or upgrade to other version.

Stop worrying about it. Not needed.
It does not harm at all if Routerboard version is a couple of versions behind.

And if it REALLY bothers you, you need to take care of it manually. Simple.

I’ve read about it and once experienced this myself that if Routerboard is outdated strange things happen to DHCP stuff.