CCR2004-PCIe drops from bus / Watchdog reboot due to massive PCIe AER storm on Proxmox VE 9.2 (Kernel 7.0)

Hardware
1

Hi everyone,

I'm experiencing a critical hardware-level disconnect with my CCR2004-1G-2XS-PCIe after a recent host OS update. The card completely drops from the host's PCIe bus, forcing a watchdog reboot on RouterOS. I suspect an incompatibility between the CCR2004 and the aggressive PCIe power management (ASPM) or IOMMU/ACS handling introduced in the recent Linux kernels.

Here is the breakdown of the issue. Has anyone else experienced this on recent Linux/Proxmox environments?

Environment

  • Host CPU / Motherboard: Intel Arrow Lake-HX (15th Gen)

  • Host OS: Proxmox VE 9.2.3 (Linux Kernel 7.0.14-4-pve)

  • MikroTik Device: CCR2004-1G-2XS-PCIe running RouterOS 7.23.1

  • Host Bootloader: GRUB / UEFI

Symptoms & Issue Description

Yesterday morning, the CCR2004-PCIe abruptly stopped routing traffic. When checking the RouterOS logs, I found a 6-hour gap followed by a watchdog reboot:

Plaintext

2026-07-08 07:28:31 system,error,critical router was rebooted without proper shutdown by watchdog timer
2026-07-08 13:30:24 system,critical,info cloud change time Jul/08/2026 07:33:50 => Jul/08/2026 13:30:24

The router didn't actually crash on its own; it lost power/data from the host PCIe slot.

When investigating the Proxmox Host logs (journalctl) at the exact time of the crash, I found a massive and continuous PCIe AER (Advanced Error Reporting) storm originating from the Intel PCH PCIe Root Port (80:1b.4), reporting an ACSViol (Access Control Services Violation).

This storm caused the host kernel to panic on that bus, failing the device recovery and effectively dropping the CCR2004-PCIe from the system (lspci showed no MikroTik card until a hard reboot).

Host Log Snippet:

Plaintext

Jul 08 13:38:08 fribourg kernel: pcieport 0000:80:1b.4: AER: Uncorrectable (Non-Fatal) error message received from 0000:80:1b.4
Jul 08 13:38:08 fribourg kernel: pcieport 0000:80:1b.4: PCIe Bus Error: severity=Uncorrectable (Non-Fatal), type=Transaction Layer, (Requester ID)
Jul 08 13:38:08 fribourg kernel: pcieport 0000:80:1b.4:    device [8086:7f44] error status/mask=00300000/00000000
Jul 08 13:38:08 fribourg kernel: pcieport 0000:80:1b.4:     [20] UnsupReq
Jul 08 13:38:08 fribourg kernel: pcieport 0000:80:1b.4:     [21] ACSViol                (First)
Jul 08 13:38:09 fribourg kernel: pcieport 0000:80:1b.4: AER: device recovery failed

Current Workaround / Troubleshooting

Since this started happening right after updating Proxmox VE to 9.2.3 (which brought the newer Linux 7.0 kernel), my working theory is that the kernel is attempting to force an ASPM power state on the PCIe slot, and the CCR2004 fails to negotiate it, resulting in the ACS violation and bus lockup.

I have currently added pcie_aspm=off to my GRUB_CMDLINE_LINUX_DEFAULT to prevent the host from sleeping the PCIe link, and I'm monitoring the stability.

Questions for the community / MikroTik Staff:

  1. Has anyone else seen ACSViol AER storms with the CCR2004-PCIe on Linux Kernel 6.x or 7.x?

  2. Are there any specific host BIOS settings (like forcing PCIe Gen3 or tweaking IOMMU) recommended for the latest Intel Arrow Lake platforms to prevent this?

  3. Could this be addressed in a future RouterBOARD firmware update for the CCR2004-PCIe?

Thanks in advance for any insights!