I decided to adapt morph1's low latency testing approach, but for best case scenario, relative stability tests. These were my results.
TL;DR. Mikrotik hAP ac2 has a stability issue with 5 GHz band at 40 MHz channel width (visible also on morph1's results), but all other widths and bands performed well. The original Mikrotik hAP ac has a surprisingly disappointing 2.4 GHz band stability at 20 MHz channel width (often required for those in high interference environment and still using 2.4 GHz devices). Asus are better than I thought, almost a decade ago.
Box-and-Whisker Plot of 56-byte data, 10 ms interval, 60k count ICMP pings with three different devices: Mikrotik hAP ac2, Mikrotik hAP ac and Asus RT-N56U as a basic comparison.
I tested 2.4 GHz (default and 20 MHz), and 5 GHz (default, 40 Mhz and 20 MHz) with hAP ac2's wired LAN-to-WAN performance as reference. Default 5 GHz channel width on Mikrotik devices was 80 MHz.
Lower is better:
Standard Box and Whisker parameters used. I don't think morph1 uses the default parameters or has a very different test environment, so cannot compare directly. Analysis tool is Matlab.
The blue box represents the traffic jitter, i.e. non-outlier traffic variation on the connection. Large box = large jitter which is worse for low latency applications.
The red crosses represent outliers that fall so far outside of the total range of the normal traffic, that they can be statistically rejected as random with a 99.3% confidence when the underlying distribution is or tends to the Normal distribution.
The ping range is carefully chosen. A maximum of 40 ms variation is normally desired for streaming applications like VoIP. A maximum of 250 ms delay for other applications like gaming. I zoomed in to 250 ms instead of 500 ms RTT because the worst offenders look the same. The actual maximums observed were sometimes much higher than that, e.g. 2000 ms (2s) for the "ac2 5GHz @ 20/40MHz" configuration and 525 ms for the "ac 2.4GHz @ 20MHz" configuration.
I could have zoomed in to show the jitter more closely for low latency differences, but given the number and distribution of outliers, that is far less important for stability.
Due to the significantly poorer performance of the "ac2 5GHz @ 20/40MHz" and "ac 2.4GHz @ 20MHz" configurations, I actually did them twice to check and took the best run of each. There was no significant difference in either standard deviation or outlier count between these alternate runs.
Test setup is the same as before:
- PC (client) -> WLAN -> NAT -> WAN (single hop) -> PC (server). No load on WAN or its router
- RTL8814AU WiFi chip USB 3.0 adapter connected to client PC via USB 3.0 ports
- Mikrotik devices: regulatory domain TX power for 2.4 GHz, default manual-tx-power for 5 GHz
- Default configuration for all devices: 2.4 GHz N-only, 5 GHz (auto)
- Testing generated only 8.4 kB/s traffic
- Stable link rates
- No walls separating devices and test client
- None of the test devices are Internet facing
- Low interference environment, see spectral scan below. Scan not taken from test device but from the hAP ac outside of testing
- Firmware: hAP ac2 (RouterOS + firmware: 6.43rc5), hAP ac (RouterOS + firmware: 6.41.3), Asus RT-N56U (30043807378)