7.19.4 router was rebooted without proper shutdown by watchdog timer

Got this message on a week old E50UG, IPSec tunnel with permanent traffic.
Noticed the problem because tunnel was down, opening the log got a lot of red lines.
Manual reboot got it working.

/system watchdog export
"# 2025-09-06 13:22:52 by RouterOS 7.19.4

software id = Pxxx-xxxx

model = E50UG

serial number = xxx"

Netwatch is configured for simple ping on a server.

Any thoughts are greatly appreciated.

Please share your sanitized /export.

That way we can see your watchdog settings.

Watchdog and Netwatch are two different facilities or features that work independently.

Here it is, thank you for your involvement.

There is also a 'failed' tentative for a daily scheduled reboot, in the hope to avoid lockout, router being on a remote location.

Learning routerOS in small steps :slightly_smiling_face:

# 2025-09-06 19:00:34 by RouterOS 7.19.4
# software id = PX6W-23QA
#
# model = E50UG
# serial number = xxx
/interface bridge
add admin-mac=F4:1E:57:6A:B1:B2 auto-mac=no comment=defconf name=bridge
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=pppoe-out1 \
    use-peer-dns=yes user=xxx
/interface wireguard
add listen-port=13231 mtu=1420 name=Wireguard-ST188
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
add dh-group=modp2048 dpd-interval=30s dpd-maximum-failures=3 enc-algorithm=\
    aes-128 lifetime=8h name=Firma
/ip ipsec peer
add address=86.xxx/32 name=xxx profile=Firma
/ip ipsec proposal
add enc-algorithms=aes-128-cbc lifetime=8h name=xxx-proposal pfs-group=\
    modp2048
/ip pool
add name=dhcp ranges=192.168.11.150-192.168.11.200
/ip dhcp-server
add address-pool=dhcp interface=bridge name=defconf
/system logging action
set 3 remote=192.168.10.247
/disk settings
set auto-media-interface=bridge auto-media-sharing=yes auto-smb-sharing=yes
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=pppoe-out1 list=WAN
/interface ovpn-server server
add mac-address=FE:C5:F0:55:E8:38 name=ovpn-server1
/interface wireguard peers
add allowed-address=192.168.70.2/32 client-address=192.168.70.2/32 \
    client-endpoint=xxx client-listen-port=13231 interface=\
    Wireguard-ST188 name=Tel private-key=\
    "xxx" public-key=\
    "xxx"
/ip address
add address=192.168.11.250/24 comment=defconf interface=bridge network=\
    192.168.11.0
add address=192.168.70.1/24 interface=Wireguard-ST188 network=192.168.70.0
/ip dhcp-client
add comment=defconf disabled=yes interface=ether1
/ip dhcp-server network
add address=192.168.11.0/24 comment=defconf dns-server=192.168.11.250 \
    gateway=192.168.11.250 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.11.250 comment=defconf name=router.lan type=A
/ip firewall filter
add action=accept chain=input comment="Wireguard port" dst-port=13231 \
    protocol=udp
add action=accept chain=input comment="IPSec local management" \
    dst-address=192.168.11.250 in-interface-list=WAN ipsec-policy=in,ipsec \
    src-address=192.168.10.0/24
add action=accept chain=input comment="IPSec local management from T" \
    dst-address=192.168.11.250 in-interface-list=WAN ipsec-policy=in,ipsec \
    src-address=192.168.20.0/24
add action=accept chain=input comment="Wireguard allow local management" \
    dst-address=192.168.11.0/24 src-address=192.168.70.0/24
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=accept chain=srcnat comment="IPSec NAT accept" dst-address=\
    192.168.10.0/24 src-address=192.168.11.0/24
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip ipsec identity
add peer=xxx
/ip ipsec policy
add dst-address=192.168.10.0/24 peer=Dekotex src-address=192.168.11.0/24 \
    tunnel=yes
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" \
    dst-port=33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
    udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
    "defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
    500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
/system clock
set time-zone-name=Europe
/system logging
add action=remote topics=system
/system scheduler
add comment="Reboot every day at 1 am" interval=1d name=reboot on-event="\
    \n" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
    start-date=2025-09-05 start-time=01:00:00
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool netwatch
add disabled=no down-script="" host=192.168.10.250 http-codes="" interval=1m \
    name="Ping to xxx" src-address=192.168.11.250 test-script="" type=\
    simple up-script=""

First thing I see is that you have Netwatch configured with settings, but I do not see Watchdog. This is fine, as Netwatch and Watchdog are different functions.

"Watchdog" by default is enabled and "Ping Watchdog" is disabled by default.

How do you know that the system was rebooted by Watchdog?

Please see these man pages:

Watchdog - RouterOS - MikroTik Documentation.

Thank you for point explanation.

I don't, was an assumption, the topic title was the message in the routerOS Log, except firmware version.

Unfortunately, I was too baffled, also being new to routerOS, to save the log and simply manually restarted the box.
Log is gone, being in memory but now I am looking for ways to send the log to a Synology NAS. No idea if doing so will also remain visible in the routerOS Log section.

L.E. The Log message is also kind of bogus.

  1. The manual reboot did sort IPSec tunnel and overall behavior, then why the "Watchdog reboot" did not ?
  2. What is the meaning of "proper shutdown" from that message ? Might be one of a service crashed and lead to a forced reboot ?
  3. Might be possible that there was no reboot at all and system was strugling to recover ?

Presence of the many IPSec connection failure lines in the Log, seemed like something crashed and could not recover. There is a mention of firmware bug for IPSec failure, reported fixed for another type of processor. Might be a new bug for ARM processors ?

Did this happen once or is it repeating?

You can create a way to save watchdog log entries to the internal storage on the MT device, and/or you can create a script that runs whenever there is a bootup of the device to monitor these things.

Is it possible the device simply lost power this one time? I think that "router was rebooted without proper shutdown" would be the message if power was lost. Certainly could be other conditions that would result in this message, but I don't know what those might be.

All devices are powered by a 12V/200A battery+charger, in addition to an Victron Energy Storage System that powers the entire location.
A power failure is the least that I think of.

Will look to direct the logs to internal disk and wait for next hickup.

This was the first occurence.

It's not power. It's a kernel-level crash. There should be an autosupout.rif file generated in Files. You can look at it at https://mikrotik.com/client (after creating a free account) and then uploading the rif file to the website under "Supout Viewer". Once loaded, there might be a clue in the stored logs. You can also use same autosupout.rif in a support case - since it most likely a bug.

You can also try 7.20rc1 to see if that avoid the crash. But if it only happen once, you might want to see if it's a one-off thing.

It looks like the autosupout.rif file has not been generated, not present in Files.

Being a rather simple config, a bug is what I was thinking too.

As suggested, will wait for another occurrence. If it will, the beta/RC firmware is the next step.

Thank you all, for your time.
Will update after some monitoring period.

Creating a supout file, noticed that IPSec counter for out-state-mode-errors is increasing.
Could not find a helpful/noob explanation of what that counter means.

		  in-errors:    0
           in-buffer-errors:    0
           in-header-errors:    0
               in-no-states:    0
   in-state-protocol-errors:    0
       in-state-mode-errors:    0
   in-state-sequence-errors:    0
           in-state-expired:    0
        in-state-mismatches:    0
           in-state-invalid:    0
     in-template-mismatches:    0
             in-no-policies:    0
          in-policy-blocked:    0
           in-policy-errors:    0
                 out-errors:    0
          out-bundle-errors:    0
    out-bundle-check-errors:    0
              out-no-states:    2
  out-state-protocol-errors:    0
      out-state-mode-errors: 2933
  out-state-sequence-errors:    0
          out-state-expired:    0
         out-policy-blocked:    0
          out-policy-errors:    0

Maybe it's somehow related to SUP-198048. I experience kernel panics on 7.19.4. When will file your support request please refer mine too, as possibly related.

@memelchenkov
Do you have a topic related to your experience ?

No. I can't associate this with a specific event. 7.19.3 - everything is OK, 7.19.4 - it started to crash. I use several IPsec tunnels, so it may be related.

UPD: according to V7.19.4 [stable] is released! - #729 by pe1chl it may be external Internet-related issue (attack).