Community discussions

MikroTik App
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

[survive] script to restore resolved domains on restart or crash

Tue May 03, 2022 12:19 am

This script will restore resolved domains to static DNS so that IP addresses are know without a server is needed to resolve those right now.

The restored entries in static DNS are renewed not on their original TTL but on the TTL stated in the config line. The config line and the call to run the script in /system/script is placed in /system/schedule with and set it to run on start-up. This script will only restore already resolved domains.

When the script is running then you can see that in the log and any changes are DNS up/down are also logged. To stop the script from running then remove the Global line survive in /system/script/environment and the script will stop and clean all setting made and also the Global line.

The config line and call to run the script used in scheduler as startup:
## version 20220505-001
## Load the configline and start the script
:local scriptName survive
:global survive

#create the global config line and start the script
:global survive { "REMOVE LINE TO STOP SURVIVE-SERVICE";
                            true; surviveService active;
                             300; seconds before storing in static DNS will be repeated;
                              15; minutes before renewing static dns;
                  [:tostr mt.lv]; canary domain;
                            true; include canary domain (extra);
                            true; WireGuard domains;
                            true; IPSEC domains };
                            
:execute script="survive"; # execute = start and do not wait till scipt has ended
:delay 3s; # check if survive script has started and if it is till running
:if  ( [/system/script/job/find script=survive]) do={} else={:set $survive}

:if ( $survive ) do={
  # do some logging of the startup of the script and it's settings used
  :log info "Survive: service is using the following settings:"
  :log info "Survive: service active: $($survive->1)"
  :log info "Survive: storing new addresses in static DNS: $($survive->3) seconds"
  :log info "Survive: renewing resolved static dns entires (TTL): $($survive->5) minutes"
  :log info "Survive: canary domain: $($survive->7)"
  :log info "Survive: WireGuard domains added: $($survive->9)"
  :log info "Survive: IPSEC domain added: $($survive->11)"
  :log info "Survive: NTP-client domain added: $($survive->13)"
  } else={ :log info "Survive service is not running. Check if that is correct?"}
The script itself named survive:
 
 # script written by msatter
 # version: 20220505-001
 # only non-commercial usage of this script
 # store domains in address-list and in staticDNS, enabling resolving those when the upsteam DNS is down on restart/crash
 # when using manual static DNS entries, put a number in the domain: example.lan.1 to have those are ignored by this script
 # remark the TTL in static DNS is irrelevant because DNS cache resolve those, every 10 seconds or so (own observation)

 :local surviveActive false; # only start when the config line states true for surviveActive
 :global survive;     # imports the survive variable containing the config array
 :if ( $survive ) do={ # read the survive configline and put the values into the variables
       :set  $surviveActive    ($survive->1) ;  # decides if the survive service is active
       :set  $runEvery         ($survive->3) ;  # in seconds, storing in static DNS will be repeated
       :set  $renewStatic      ($survive->5) ;  # in minutes, renew through erasing static DNS entries (not when DNS resolving is not active)
       :set  $canaryDomain     ($survive->7) ;  # canary domain
       :set  $WGactive         ($survive->9) ;  # add WireGuard domains to survive
       :set  $IPSECactive      ($survive->11) ; # add IPSEC domains to survive
       :set  $NTPactive        ($survive->13) ; # add NTP domains to survive
 }; # EndIf

 :if ( $surviveActive ) do={ # do not start survive and remove any earlier stored values staticDNS/address-list/global if they are present
 
  # set to false to indicate that there was start ( tri-state value (true/flase/either) by using on-error )
  :if ( [/ip/firewall/address-list/find list=survive address=$canaryDomain] ) do={ :set $displayStart false } else={ :set $displayStart }
 
  # create the canary domain in address list
  :do { /ip/firewall/address-list/add list=survive address=$canaryDomain } on-error={}; # will only be active when DNS resolving is available

  # calculate the 'TTL' from the number of runEvery and renewStatic 
  :set $cleanStatic ($renewStatic/($runEvery/60)); # calculate a kind of fixed TTL to be used for survive domains
  :if ( $cleanStatic < 60 ) do={ :set $cleanstatic 60 }; # minimal 'TTL' is 60 seconds
  
  # retrieve the lasttime an active survive was done
  :do { :foreach i in=[/ip/firewall/address-list/find list=survive !comment] do={:set $lastRun [/ip/firewall/address-list/get $i creation-time ] } } on-error={ :set $lastRun ""}
  
  # logging the startup of
  :log info "Survive: for survive domains, has been activated. Updating those every $runEvery seconds";
  :log info "Survive: domains are renewed every $renewStatic minutes. Replacing the domains original TTL"
  :if ( $lastRun = "" ) do={ :log info "Survive: the domain used, if no resolving is not active, where resolve on: $lastRun" }

  while ( $survive )  do={ # start of the loop, only loops if Global $survive is containing values
  
   :local startBench [/system clock get time]; # adding the waiting time for resolving day for added domains
   
   # clean the address-list before refreshing. The canary Domain is not removed because as it not a endpoint domain
   :do { [/ip/firewall/address-list/remove  [find list=survive address!=$canaryDomain comment!=$canaryDomain]] } on-error={}
   # Loop to fill the address-list "survive

   ### add WireGuard peer addresses if WGactive = true
   :if (WGactive) do={ :foreach wg in=[/interface/wireguard/find disabled=no] do={
                       :local nameWG [/interface/wireguard/get $wg name]
                        :foreach peerWG in=[/in/wireguard/peers/find interface=$nameWG endpoint-address~"[a-z]\$"] do={
                          :local endPointDomain [/interface/wireguard/peers/get $peerWG endpoint-address]
                          :do { /ip/firewall/address-list/add  address=$endPointDomain list=survive  } on-error={}
                          :do { /ipv6/firewall/address-list/add address=$endPointDomain list=survive } on-error={}
                        }; # EndForEach
                     }; # EndForEach
   }; # EndIf

   ### add IPSEC peer addresses if IPSECactive = true
   :if (IPSECactive) do={ :foreach peer in=[/ip/ipsec/peer/find disabled=no address~"[a-z]\$]"]  do={
                           :local peerDomain [/ip/ipsec/peer/get $peer address]
                           :do { /ip/firewall/address-list/add  address=$peerDomain list=survive  } on-error={}
                           :do { /ipv6/firewall/address-list/add address=$peerDomain list=survive } on-error={}
                         }; # EndForEach
   }; # EndIf
   
   ### add NTP client
   :if (NTPactive) do={ :foreach server in=[/system/ntp/client/servers/find address~"[a-z]\$"] do={    
                          :local serverNTP [/system/ntp/client/servers/get $server address]         
                          :do { /ip/firewall/address-list/add  address=$serverNTP list=survive  } on-error={}
                          :do { /ipv6/firewall/address-list/add address=$serverNTP list=survive } on-error={}
                        }; # EndForEach
   }; # EndIf
   
   :delay 5s; # allow the address-list to resolve, the just added domains. time lost, is compensated in the waiting loop
   
   # check if resolving is available. if not then, no adding/removing is done to the static DNS
   # this mean that no comment should be entered to the survive address-list to keep this working
   # as soon as DNS resolving is restored the dyamic entries in the survive address-list is activated again by RouterOS
   :if ( [/ip/firewall/address-list/find list=survive comment~"[a-z]\$"] ) do={ # when domains are found in (Dynamic entries) comments then resolving is available after startup

    # Clean static DNS of survive entries ever so many cycles
    :if ($cleanCount >= $cleanStatic ) do={ :set $cleanCount 0; :do { /ip/dns/static/remove [find comment=survive] } on-error={ :log warning "Survive: no endpoint domainnames found" } }
    
    # put survive addresslist into an arrayIPv4
    :local arrayIP   [:toarray ""]
    :local arrayIPv4 [:toarray ""]
    :local arrayIPv6 [:toarray ""]
    
    :foreach k,v in=[/ip/firewall/address-list/find list=survive address!=$canaryDomain comment!=$canaryDomain] do={
      :set (($arrayIPv4->$k)->0) [/ip/firewall/address-list/get $v address]; :set (($arrayIPv4->$k)->1) [/ip/firewall/address-list/get $v comment]}
    :foreach k,v in=[/ipv6/firewall/address-list/find list=survive address!=$canaryDomain comment!=$canaryDomain] do={
      :set (($arrayIPv6->$k)->0) [/ipv6/firewall/address-list/get $v address]; :set (($arrayIPv6->$k)->1) [/ipv6/firewall/address-list/get $v comment]}
    :set $arrayIP ($arrayIPv4,$arrayIPv6)
    :set $arrayIPv4; :set $arrayIPv6; # remove IPv4 and IPv6 array's

     # Loop, to fill static DNS with resolved   
      :foreach k,v in=[($arrayIP)]  do={
         :if ( [:typeof  (($arrayIP->$k)->0)] = "str" ) do={ :set $surviveDomain (($arrayIP->$k)->0) }; # EndIf 
         :foreach ka,va in=[($arrayIP)]  do={
          :if ( $surviveDomain = (($arrayIP->$ka)->1) ) do={ #surviveComment 
            :set $surviveIPaddress (($arrayIP->$ka)->0); # EndIf IPv6    
            :put "$surviveDomain $surviveIPaddress"
          }; # EndIf
         }; # EndForEach
        # double added if there are also IPv6 entries. This causes no extra writes into flash
        :if ( [:toip  $surviveIPaddres] ) do={ :do { /ip/dns/static/add name=$surviveDomain address=$surviveIPaddress comment=survive type=A;  } on-error={} }; # EndIf
        :if ( [:toip6 $surviveIPaddres] ) do={ :do { /ip/dns/static/add name=$surviveDomain address=$surviveIPaddress comment=survive type=AAA } on-error={} }; # EndIf
        :set $surviveIPaddress; # clears previous IP address to avoid that it is set to a previous read domain
      } ; # EndForEach
      
   :set $cleanCount ( $cleanCount + 1 )
    # setting cleanCount to cleanStatic, innitiating a rebuild of static DNS survive domains, this whenever resolve DNS is restored after restart/reboot
   } else={ :set $cleanCount $cleanStatic }; # EndIf EndElse

   # do logging when DNS resolving was not available on start-up
   :do { :if ( [/ip/firewall/address-list/find list=survive comment=$canaryDomain and $displayStart]  ) do={ :log warning "DNS resolving has been restored."; :set $displayStart } } on-error={}; # log only once when DNS resolving is restored
   :do { :if ( [/ip/firewall/address-list/find list=survive comment=$canaryDomain and !$displayStart] ) do={} else={ :log error "DNS resolving not available, check your DNS!"; :set $displayStart true } } on-error={}; # log only once when DNS resolving is not available
   
   # deduct benchTime to emulate a TTL that has been defined in the configline
   :set $benchedTime ([/system clock get time]-[:totime $startBench])
   :set $correctedRE ( $runEvery-(([:pick $benchedTime 6 8])+([:pick $benchedTime 3 5]*60)) ); # correct dynamicly, runEvery with the needed processing time in seconds
   :if ($correctedRE < 0 ) do={ set $correctedRE 0 }; :put $correctedRE; # should not be negative
   
   # loop for a calculated number, to emulate a TTL
   :set $countTime 0; # reset before delay loop is started
   while ($countTime < $correctedRE) do={
      :set $countTime ($countTime + 1); :delay 995ms;
      :if ($survive) do={} else={:set $countTime $runEvery}
      :do { :if ( [/ip/firewall/address-list/find list=survive comment=$canaryDomain and $displayStart ] ) do={ :set $countTime $runEvery } } on-error={}; # while counting DNS resolving came back
    }; # delay routine based on runEvery

  }; # EndWhile

};  # IF !surviveActive

# removing entries in static DNS and the address-list survive
:do { /ip/dns/static/remove            [find comment~"survive"] } on-error={}; # incuding an present canary domain
:do { /ip/firewall/address-list/remove [find list=survive !dynamic ] } on-error={}
:set $survive; # erase the Global configline

# do some logging
:log warning "Survive service has been halted or did not start, all survive entries are removed (static DNS/address-list/Global storage)" 

It is still early version and I tested it as far as I could and if you find any problem or have improvements in the script then please use this tread for this.
Last edited by msatter on Thu May 05, 2022 9:04 pm, edited 7 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: [survive] restoring resolved domains on restart or crash

Tue May 03, 2022 3:09 am

Nice, getting too complex for my user article but It seems very complete in terms of addressing possible issues, DNS, NTP, DYNDNS (server down issues).
What it says to me that some of this should be a checkbox within the router......... the more complicated and indepth a script gets, is an indication that the Router is not addressing some key requirements...... In this Case, one could argue that VPN connections and connectivity needs to be more robustly (and proactively ) handled by RoS.
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: [survive] restoring resolved domains on restart or crash

Tue May 03, 2022 11:28 am

I have offered an early version to Miktotik and pointed them also to this published version.
They have more control over RouterOS than I have from userspace, but I did come close.

Checkboxes would be nice but till then, if Mikrotik will ever offer something like this, this could be used instead. The nice thing was that RouterOS have all the basic tools in address-list to implement this without having a kind of storage to keep track of the entries.
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: [survive] script to restore resolved domains on restart or crash

Wed May 04, 2022 3:38 pm

Script has been updated to avoid double adding domains if they have an IPv6 address. The remark lines updated so that they are move accurate and informative.

I wrote this script to only address that (active) domains not being resolved after a restart or crash. Keeping it so as simple as possible, despite it looks very complex. It is not, at least to my.

Extra stuff like restarting an service like WG or IPSEC demands other script with their own approach.

On first start-up there was never any resolving being done so the script will now do anything. Then a long time between restart due to having a device as backup then outdated IP addresses could be used and after the first TTL refresh (static DNS refresh) these entries should be accurate again.

There could be a check on this like, if previous IPv4 address-list date is more than a hour ago then do not use saved survive but wait till resolving is available again. Better seems to keep using even if outdated and on first resolve erase matching static DNS entries replace those with the current ones.
Normal cycle is to keep stored and renew when fixed TTL expired.
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: [survive] script to restore resolved domains on restart or crash

Thu May 05, 2022 11:38 am

The script is now released and I have added a log line that shows when the last time the used domains where resolved to IP addresses. Keeping it simple I just put the last date and time in there instead of calculating the the difference that would need a lot of code.

Look in the log after a reboot and you will find the information.

An start of the script without the config-line in Global is now also displayed in the log as warning: Survive service has been halted or did not start, all survive entries are removed (static DNS/address-list/Global storage)
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: [survive] script to restore resolved domains on restart or crash

Thu May 05, 2022 3:40 pm

If I can, and no one is offended ...
                            true; IPSEC domains
missing ; -------------------------------------^
  :log info "script $scriptName started, $settings
missing " ----------------------------------------^
 
msatter
Forum Guru
Forum Guru
Topic Author
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: [survive] script to restore resolved domains on restart or crash

Thu May 05, 2022 9:02 pm

Thank, I have updated the script for the scheduler.

Using now :execute instead of :run so that the script will return control to the calling script. I have adapted the log lines to state each setting separate and it does now also log if the script has crashed or was not present in scripts. Then the settings are then not written to the log.
 
NeoPhyTex360
just joined
Posts: 9
Joined: Wed Apr 04, 2018 4:10 pm

Re: [survive] script to restore resolved domains on restart or crash

Tue Jul 26, 2022 6:06 pm

Nice, works like a charm with wireguard problems reconnecting after reboot of server (+netwatch). Good work

Who is online

Users browsing this forum: rano and 20 guests