Community discussions

MikroTik App
 
mikruser
Long time Member
Long time Member
Topic Author
Posts: 578
Joined: Wed Jan 16, 2013 6:28 pm

Suggestion: Completely virtual router based on two physical routers

Fri Jul 29, 2016 3:20 pm

Hello,

Currently, with VRRP, we have manual edit config on each physical router.

Suggestion: completely virtual router, visible in Winbox as one router (like RAID1(mirror) volume based on two HDD)
 
User avatar
javajox
newbie
Posts: 44
Joined: Fri Aug 23, 2013 9:32 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Aug 14, 2016 7:34 pm

+1 I'm also interested in having this feature
 
User avatar
jspool
Member
Member
Posts: 468
Joined: Sun Oct 04, 2009 4:06 am
Location: Oregon

Re: Suggestion: Completely virtual router based on two physical routers

Mon Oct 17, 2016 12:18 am

+1 This is needed and would be very useful.
 
ezanolin
just joined
Posts: 23
Joined: Sat Feb 25, 2006 2:15 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Oct 28, 2016 3:37 pm

Just my 2 cents..

Clustering would indeed be very useful, Mikrotik essentially has no HA mode which makes it difficult to push into the enterprise environment. We have VRRP but its not hitless for anything statefull that you may be doing (firewall or tunnelling). Clustering like Junos does on the J and SRX series should be possible to achieve on the current hardware platform. Either that or start producing redundant chassis hardware designs like Cisco 6500 or Juniper MX series devices.

In either case you need to implement dual routing engines, so you need the ability to synchronise state information between devices and delegate a master routing engine. Just this work would make clustering possible, that same work can then be used to make a redundant chassis.
 
User avatar
vmiro
Frequent Visitor
Frequent Visitor
Posts: 80
Joined: Sun Jan 29, 2006 6:53 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Nov 17, 2016 9:52 am

+1 This is a absolutely necessary for use in enterprise environment.
I got several installation with two CCRs, configured with VRRP and is quite hard to maintain this installation. Every change in configuration has to be done on both routers.
I'm using Fortinet FortiGate in my company which supports HA and two physical devices acts as a single logical device.

mIRO
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Nov 20, 2016 9:20 pm

Give this a go: https://github.com/svlsResearch/ha-mikrotik
It does exactly what you are asking for, except for stateful connection synchronization. I have been using it to run 6 pairs of CCR1036 for over a year now.
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Nov 21, 2016 9:51 pm

Give this a go: https://github.com/svlsResearch/ha-mikrotik
It does exactly what you are asking for, except for stateful connection synchronization. I have been using it to run 6 pairs of CCR1036 for over a year now.
@nathan1 i was testing this on a lab using 2 450g, but for some reason on the first HA cyclce , the "B" became the active and the "A" the standby but the HA work normal only this Letter switch
do you have this problem ?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Nov 21, 2016 9:55 pm

Give this a go: https://github.com/svlsResearch/ha-mikrotik
It does exactly what you are asking for, except for stateful connection synchronization. I have been using it to run 6 pairs of CCR1036 for over a year now.
@nathan1 i was testing this on a lab using 2 450g, but for some reason on the first HA cyclce , the "B" became the active and the "A" the standby but the HA work normal only this Letter switch
do you have this problem ?
There is no affinity for a primary right now. So this works as designed. Did you want to have an affinity for one vs. the other? In my setup - they are equal in terms of choice, at some sites my B is currently the active one just due to order in which I upgraded/cycled.
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Nov 21, 2016 9:57 pm

Give this a go: https://github.com/svlsResearch/ha-mikrotik
It does exactly what you are asking for, except for stateful connection synchronization. I have been using it to run 6 pairs of CCR1036 for over a year now.
@nathan1 i was testing this on a lab using 2 450g, but for some reason on the first HA cyclce , the "B" became the active and the "A" the standby but the HA work normal only this Letter switch
do you have this problem ?
There is no affinity for a primary right now. So this works as designed. Did you want to have an affinity for one vs. the other? In my setup - they are equal in terms of choice, at some sites my B is currently the active one just due to order in which I upgraded/cycled.
i know that work

but since A and B i think that when A became online again, in my mind need to be Active again :D
but it work very well
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Nov 21, 2016 10:06 pm

Give this a go: https://github.com/svlsResearch/ha-mikrotik
It does exactly what you are asking for, except for stateful connection synchronization. I have been using it to run 6 pairs of CCR1036 for over a year now.
@nathan1 i was testing this on a lab using 2 450g, but for some reason on the first HA cyclce , the "B" became the active and the "A" the standby but the HA work normal only this Letter switch
do you have this problem ?
There is no affinity for a primary right now. So this works as designed. Did you want to have an affinity for one vs. the other? In my setup - they are equal in terms of choice, at some sites my B is currently the active one just due to order in which I upgraded/cycled.
i know that work

but since A and B i think that when A became online again, in my mind need to be Active again :D
but it work very well
In theory, the VRRP could be adjusted to force A back to the primary when it is around. However, you will probably end up with extraneous reboots/connection breaks due to this. I simply treat them like one logical device in my head and the hostname that is currently active doesn't really come in to play. The A and B simple give me a hint as to which one is currently physically active, if I happen to be on site and dealing with them physically, this is useful. Otherwise, I just ignore it entirely and address the active one with the floating IP/hostname.

If you end up really wanting to see A active, I can add the feature when I have some more time. You can always force A to become active again by rebooting B when A is back. (hint: do an $HASyncStandby until it is GOOD before doing this).
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: RE: Re: Suggestion: Completely virtual router based on two physical routers

Mon Nov 21, 2016 10:23 pm

Give this a go: https://github.com/svlsResearch/ha-mikrotik
It does exactly what you are asking for, except for stateful connection synchronization. I have been using it to run 6 pairs of CCR1036 for over a year now.
@nathan1 i was testing this on a lab using 2 450g, but for some reason on the first HA cyclce , the "B" became the active and the "A" the standby but the HA work normal only this Letter switch
do you have this problem ?
There is no affinity for a primary right now. So this works as designed. Did you want to have an affinity for one vs. the other? In my setup - they are equal in terms of choice, at some sites my B is currently the active one just due to order in which I upgraded/cycled.
i know that work

but since A and B i think that when A became online again, in my mind need to be Active again :D
but it work very well
In theory, the VRRP could be adjusted to force A back to the primary when it is around. However, you will probably end up with extraneous reboots/connection breaks due to this. I simply treat them like one logical device in my head and the hostname that is currently active doesn't really come in to play. The A and B simple give me a hint as to which one is currently physically active, if I happen to be on site and dealing with them physically, this is useful. Otherwise, I just ignore it entirely and address the active one with the floating IP/hostname.

If you end up really wanting to see A active, I can add the feature when I have some more time. You can always force A to become active again by rebooting B when A is back. (hint: do an $HASyncStandby until it is GOOD before doing this).
Oh
You are the creator of the scripts
Very good work, I like do read codes and try to understand them, but your is very complex for one that don't made it,
Very good work
[emoji106]


Enviado de meu XT1580 usando Tapatalk
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: RE: Re: Suggestion: Completely virtual router based on two physical routers

Mon Nov 21, 2016 10:30 pm

@nathan1 i was testing this on a lab using 2 450g, but for some reason on the first HA cyclce , the "B" became the active and the "A" the standby but the HA work normal only this Letter switch
do you have this problem ?
There is no affinity for a primary right now. So this works as designed. Did you want to have an affinity for one vs. the other? In my setup - they are equal in terms of choice, at some sites my B is currently the active one just due to order in which I upgraded/cycled.
i know that work

but since A and B i think that when A became online again, in my mind need to be Active again :D
but it work very well
In theory, the VRRP could be adjusted to force A back to the primary when it is around. However, you will probably end up with extraneous reboots/connection breaks due to this. I simply treat them like one logical device in my head and the hostname that is currently active doesn't really come in to play. The A and B simple give me a hint as to which one is currently physically active, if I happen to be on site and dealing with them physically, this is useful. Otherwise, I just ignore it entirely and address the active one with the floating IP/hostname.

If you end up really wanting to see A active, I can add the feature when I have some more time. You can always force A to become active again by rebooting B when A is back. (hint: do an $HASyncStandby until it is GOOD before doing this).
Oh
You are the creator of the scripts
Very good work, I like do read codes and try to understand them, but your is very complex for one that don't made it,
Very good work
[emoji106]


Enviado de meu XT1580 usando Tapatalk
Yep, created it after years of frustration with maintaining pairs of routers. Happy to see that it might work for you. It has been rock solid for us but let me know if you run into any issues.
 
ujemvi
just joined
Posts: 13
Joined: Wed May 16, 2012 9:37 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Nov 22, 2016 5:38 am

Dude, you should try to include this script you made in the Wiki.
It seems really solid and it solves one major need for enterprise needs.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Nov 22, 2016 6:10 am

Dude, you should try to include this script you made in the Wiki.
It seems really solid and it solves one major need for enterprise needs.
I don't think the Mikrotik wiki is actually community driven, unless I misunderstand something. Are you aware of a way to add an entry? The edit history also seems to suggest that it may be Mikrotik engineers only :(
 
jarda
Forum Guru
Forum Guru
Posts: 7756
Joined: Mon Oct 22, 2012 4:46 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Nov 22, 2016 7:27 am

Send it to support and ask them to put the script on the wiki.
 
ovidiu
just joined
Posts: 14
Joined: Sun Jan 15, 2017 9:28 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Feb 05, 2018 4:40 pm

Many thanks to Nathan1 for this solution. I tested first on a pair of small RB925ui-5ac2nD. Didn't succeed at first try because lack of instructions, but after 2 hours the pair was working as intended.
Then I installed the script on a pair of RB3011UiAS-RM and looks fine. It is still in my lab but next days will move them into production. The setup have 1 internet static IP Ethernet connection, 2 pppoe internet connections (static IP) one vlan connected to 2 RB925ui-5ac2nD providing guest wifi and separate LAN wifi and one wireless link to a remote connection using ubiquiti antennas.
Now I realized that I can connect antenna only to one router (I don't have redundant switch) so to avoid problems the router A must be always master. (of course I don't need fully redundant link to that ubiquiti since it is used only for nightly remote backups and anytime someone can plug the cable to router B )
Does anybody knows how to make always active the router A ?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Feb 05, 2018 6:15 pm

Many thanks to Nathan1 for this solution. I tested first on a pair of small RB925ui-5ac2nD. Didn't succeed at first try because lack of instructions, but after 2 hours the pair was working as intended.
Then I installed the script on a pair of RB3011UiAS-RM and looks fine. It is still in my lab but next days will move them into production. The setup have 1 internet static IP Ethernet connection, 2 pppoe internet connections (static IP) one vlan connected to 2 RB925ui-5ac2nD providing guest wifi and separate LAN wifi and one wireless link to a remote connection using ubiquiti antennas.
Now I realized that I can connect antenna only to one router (I don't have redundant switch) so to avoid problems the router A must be always master. (of course I don't need fully redundant link to that ubiquiti since it is used only for nightly remote backups and anytime someone can plug the cable to router B )
Does anybody knows how to make always active the router A ?
Hey Ovidiu,

This is the first setup I've seen deployed using ha-mikrotik that has a physical reason for choosing A over B. In theory, I can add a feature that would force this but it does feel a little bit odd. You can temporarily "force" it to stay on one vs. the other by rebooting the primary, which will then obviously be sticky until another event occurs. The software is designed to have an exact pair such that they are basically indistinguishable.

Just so I can understand the use case....Are the RB925ui-5ac2nD the ones that you want to force a primary? There is a physical antenna you are connecting to the RB925ui-5ac2nD for which you only have one? I'm not following how a redundant switch would come into play here, if you had it.

PS: You are also the only one that I know of that I can recall that isn't using CCRs with ha-mikrotik. Please let me know if you run into anything that feels odd.
 
ovidiu
just joined
Posts: 14
Joined: Sun Jan 15, 2017 9:28 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Feb 05, 2018 6:42 pm

Hi Nathan,
No, i used RB925ui-5ac2nD just for lab tests without activating wifi. They will be connected to the redundant RB3011UiAS-RM
Please understand that we are talking about a very small office with only about 15-16 people + some visitors quite often. There is no point to buy CCR. I agree that your script is perfect in a normal situation where everything should be redundant. As I explained, there is a wireless link for offsite copy of backups. To be fully redundant i should connect it through a redundant switch. But hey, I don't care about offsite copy of backups. Of course I can plug it to the normal switch where all computers are connected, but I was thinking to block LAN access to the backup using the router.
I know I can remotely restart the master so the slave will take over and since we have good on-line UPS they won't flip for long time.
So Nathan, if we can change something easy to your script would be excellent, but is not a must. I can live very happy the way it is.
Bty, in case of power fail, I found a solution: in System > Routerboard > Settings there is a menu "boot delay" witch seams that doesn't synchronize. I set higher time for router B and now every time the router A start first as master.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Feb 05, 2018 7:02 pm

No problem not using CCRs, they are definitely expensive for many deployments. I just wanted to let you know that you are the first one that I know of to test alternative platforms, so good for all of us. I would like to hear how well it works for you after you run for a while.

The boot delay sounds like a great solution if you just want one to always become primary when they are both booted nearly simultaneously (i.e. after power recovery). This wouldn't force A to become primary again after A was primary and then rebooted but that is the feature I could add if you really wanted it. I think this could work based on a pretty simply change that enables VRRP preemption.

It sounds like you have found a pretty workable solution though. Maybe you run it for a while and then see if you generally find it stable and if you still want this feature after a while of running, I will add it. How does that sound?
 
ovidiu
just joined
Posts: 14
Joined: Sun Jan 15, 2017 9:28 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Feb 05, 2018 7:25 pm

Yes Nathan, I'm sure will be fine for long time. I will let you know when I will put them into production (now I run them at my home). I have to implement some VPN solution and hope to find a way to allow access only from some countries, geoip. After that I will plug them into the rack.
Some other feature would be great: to receive email in case that one router is out for more than few minutes. Suppose that one of the routers have a problem. I will never know without to manually check the state.
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 11, 2018 2:43 am

Many thanks, you've saved me days! I tested this on virtualised routers first and had a problem that all interfaces would get disabled, including the VRRP parent, until I hashed out the following line in the ha_startup script:
/system routerboard settings set silent-boot=yes

It's a virtual x86, so it made sense that it failed. I additionally reduced the subnet in the ha_config from /24 to /29. The ha_switchrole script appears to have hardcoded values, which don't match the settings from ha_config, so I set the HA sync interface and then assume it should ping the slave (169.254.23.2), right?

I see no references to the scripts using telnet or ssh so I additionally stopped it restricting those protocols to the HA addresses:
Edited ha_startup script from:
:foreach service in [:toarray "ftp,telnet,ssh"] do={
to:
:foreach service in [:toarray "ftp"] do={


Excellent work, we typically implement redundancy using OSPF, BGP and/or VRRP but bridging VPLS tunnels and retrofitting redundancy on complicated routers with allot of /30 subnets is very easy using the collection of scripts you're written!

Mikrotik should really incorporate your work as a heartbeat HA function, instead of wasting time on kid control...
Last edited by bbs2web on Sun Feb 11, 2018 8:25 pm, edited 1 time in total.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 11, 2018 3:02 am

Hey bbs2web,

Nice work debugging it for your platform. We can put a an on-error around the silent-boot so it works correctly in both cases.

I assume you changed the VRRP address as well when you changed it to a /29? I'd only be reluctant to switch it to a /29 since it won't cover what I have used for the .10 VRRP address since it was created. I guess we can go with a /28 if you feel that you really want to shrink the /24.

I adjust the rules for all 3 services to make sure that the other device can always be used to manually access all of the services. It is more of a management/debugging tool when something might go wrong vs. part of ha-mikrotik automation.

Good catch on the switchrole, it is actually a script I very rarely use and wasn't intended to be committed. It needs to be changed to use $haOtherAddress and $haInterface rather than the fixed IP and interface.

Is it generally working well for you on x86? How long does it take for an ha_pushbackup to slave to boot back up?


Many thanks, you've saved me days! I tested this on virtualised routers first and had a problem that all interfaces would get disabled, including the VRRP parent, until I hashed out the following line in the ha_startup script:
/system routerboard settings set silent-boot=yes

It's a virtual x86, so it made sense that it failed. I additionally reduced the subnet in the ha_config from /24 to /29. The ha_switchrole script appears to have hardcore values which don't match the settings from ha_config so I set the HA sync interface and then assume it should ping the slave (169.254.23.2), right?

I see no references to the scripts using telnet or ssh so I additionally stopped it restricting those protocols to the HA addresses:
Edited ha_startup script from:
:foreach service in [:toarray "ftp,telnet,ssh"] do={
to:
:foreach service in [:toarray "ftp"] do={


Excellent work, we typically implement redundancy using OSPF, BGP and/or VRRP but bridging VPLS tunnels and retrofitting redundancy on complicated routers with allot of /30 subnets is very easy using the collection of scripts you're written!

Mikrotik should really incorporate your work as a heartbeat HA function, instead of wasting time on kid control...
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 11, 2018 8:46 pm

Hi Nathan,

Booting a x86 virtual takes approximately 40 seconds. I converted a customer's active backup routers that we were maintaining, with about 70 individual vrrp interfaces to your ha system. Entire process took about 30 minutes and the process is elegantly simple.

No longer have to work with /29 subnets everywhere and no longer have to do everything twice.

Yes, I made first master 169.254.23.1/29, the initial slave 169.254.23.2/29 and the floating vrrp ip 169.254.23.3.

I'm implementing this on two pairs of CCR1036 routers, at a financial institution, during their maintenance window tomorrow morning. They already have a spanning tree mess, with their Cisco stack running RPVST+ and their HyperV environment running with switches in MSTP mode. This way they have 10 seconds failover redundancy for bridged vlans using VPLS between their primary and DR site.

The client has PCI DSS and ISO compliance tests scheduled in the next 45 days. Confident that everything works!

Really, really excellent work, well done and thank you!
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 9:40 am

Would you please consider accepting the following patch, it does the following:
  • Changes '] > ' to stop rancid (configuration revision management) matching it to the RouterOS prompt.
  • Changes netmask from /24 to /29 and moved VRRP IP from .10 to .3.
  • Set schedulers' start date to Unix Epoch (Jan/01/1970).
  • Set schedulers' intervals and start time to prevent overlapping.
  • Only change FTP service, prevents SSH not being reachable on master or enabling Telnet.
  • Replaces hard coded values with variables.
  • Disables adding default route (makes loopback interfaces reachable).
  • Disables silencing Routerboard boot process by default and handle errors (eg VM)

--- HA_init.rsc 2018-02-18 08:54:22.000000000 +0200
+++ ../../HA_init.rsc   2018-02-18 09:32:25.000000000 +0200
@@ -1,7 +1,7 @@
 :do {
 /system script
 remove [find name=ha_checkchanges_new]
-add name=ha_checkchanges_new owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":if ([:len [/system script job find where script=\"ha_checkchanges\"]] > 1) do={:error \"already running checkchanges\"; } \
+add name=ha_checkchanges_new owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":if ([:len [/system script job find where script=\"ha_checkchanges\"]]  > 1) do={:error \"already running checkchanges\"; } \
        \n:global isMaster\
        \n:global isStandbyInSync\
        \n:global haPassword\
@@ -39,11 +39,11 @@
 remove [find name=ha_config_new]
 add name=ha_config_new owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source="/system script run [find name=\"ha_config_base\"]\
        \n:global haNetwork \"169.254.23.0\"\
-       \n:global haNetmask \"255.255.255.0\"\
-       \n:global haNetmaskBits \"24\"\
+       \n:global haNetmask \"255.255.255.248\"\
+       \n:global haNetmaskBits \"29\"\
        \n:global haAddressA \"169.254.23.1\"\
        \n:global haAddressB \"169.254.23.2\"\
-       \n:global haAddressVRRP \"169.254.23.10\""
+       \n:global haAddressVRRP \"169.254.23.3\""
 remove [find name=ha_functions_new]
 add name=ha_functions_new owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":global HADebug do={\
        \n   :put \$1\
@@ -103,7 +103,7 @@
        \n   :error \"Are you sure the other device is configured properly? I am unable to ping MAC \$pingMac\"\
        \n}\
        \n\
-       \n:if ([:len [/ip address find where interface=\"\$haInterface\" and comment!=\"HA_AUTO\"]] > 0) do {\
+       \n:if ([:len [/ip address find where interface=\"\$haInterface\" and comment!=\"HA_AUTO\"]]  > 0) do {\
        \n   :error \"Interface \$haInterface has IP addresses. HA should completely own the interface and it cannot be used by anything else. Please correct\"\
        \n}\
        \n\
@@ -155,7 +155,7 @@
        \n:execute \"ha_setidentity\"\
        \n:do { :local k [/system script find name=\"on_master\"]; if ([:len \$k] = 1) do={ /system script run \$k } } on-error={ :put \"on_master failed\" }"
 remove [find name=ha_pushbackup_new]
-add name=ha_pushbackup_new owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":if ([:len [/system script job find where script=\"ha_pushbackup\"]] > 1) do={:error \"already running pushbackup\"; } \
+add name=ha_pushbackup_new owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":if ([:len [/system script job find where script=\"ha_pushbackup\"]]  > 1) do={:error \"already running pushbackup\"; } \
        \n:global haPassword\
        \n:global isMaster\
        \n:global haAddressOther\
@@ -247,7 +247,7 @@
        \n}\
        \n/log warning \"ha_startup: 0.3\"\
        \n/interface ethernet disable [find]\
-       \n:global haStartupHAVersion \"0.2alpha - ea961767e45b63b81aac87eed37301d8b70bedf7\"\
+       \n:global haStartupHAVersion \"0.2alpha - 858dc62b5a9e215a5e5896137a053d01d16695c6\"\
        \n:global isStandbyInSync false\
        \n:global isMaster false\
        \n:global haPassword\
@@ -268,7 +268,7 @@
        \n/system scheduler remove [find comment=\"HA_AUTO\"]\
        \n\
        \n#Pause on-error just in case we error out before the spin loop - hope 5 seconds is enough.\
-       \n/system scheduler add comment=HA_AUTO name=ha_startup on-event=\":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disable [find default-name!=\\\"\\\$haInterface\\\"]; /log error \\\"ha_startup: FAILED - DISABLED ALL INTERFACES\\\" }\" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=startup \
+       \n/system scheduler add comment=HA_AUTO name=ha_startup on-event=\":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disable [find default-name!=\\\"\\\$haInterface\\\"]; /log error \\\"ha_startup: FAILED - DISABLED ALL INTERFACES\\\" }\" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=startup \
        \n\
        \n#/interface ethernet reset-mac-address\
        \n/ip address remove [find interface=\"\$haInterface\"]\
@@ -315,8 +315,8 @@
        \n   }\
        \n}\
        \n\
-       \n/ip route remove [find comment=\"HA_AUTO\"]   \
-       \n/ip route add gateway=\$haAddressOther distance=250 comment=HA_AUTO\
+       \n#/ip route remove [find comment=\"HA_AUTO\"]   \
+       \n#/ip route add gateway=\$haAddressOther distance=250 comment=HA_AUTO\
        \n\
        \n/log warning \"ha_startup: 4\"\
        \n\
@@ -337,10 +337,10 @@
        \n/ip address add address=\$haAddressVRRP netmask=255.255.255.255 interface=HA_VRRP comment=\"HA_AUTO\"\
        \n\
        \n/log warning \"ha_startup: 6\"\
-       \n/system scheduler add comment=HA_AUTO interval=30m name=ha_exportcurrent on-event=\"/export file=\\\"HA_current.rsc\\\"\" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/20/2000 start-time=22:37:10\
-       \n/system scheduler add interval=10m name=ha_checkchanges on-event=ha_checkchanges policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/1/2000 start-time=18:00:30 comment=HA_AUTO\
+       \n/system scheduler add comment=HA_AUTO interval=10m name=ha_exportcurrent on-event=\"/export file=\\\"HA_current.rsc\\\"\" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=00:05:00\
+       \n/system scheduler add interval=10m name=ha_checkchanges on-event=ha_checkchanges policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=00:10:00 comment=HA_AUTO\
        \n#Still need this - things like DHCP leases dont cause a system config change, we want to backup periodically.\
-       \n/system scheduler add comment=HA_AUTO interval=24h name=ha_auto_pushbackup on-event=ha_pushbackup policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/20/2000 start-time=05:00:00\
+       \n/system scheduler add comment=HA_AUTO interval=24h name=ha_auto_pushbackup on-event=ha_pushbackup policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=05:00:00\
        \n/log warning \"ha_startup: 7\"\
        \n:if ([:len [/file find name=\"HA_dsa\"]] = 1) do={\
        \n   /ip ssh import-host-key private-key-file=HA_rsa\
@@ -352,9 +352,9 @@
        \n/user add address=\"\$haNetwork/\$haNetmaskBits\" comment=HA_AUTO group=full name=ha password=\"\$haPassword\"\
        \n/log warning \"ha_startup: 8\"\
        \n#So you dont get annoyed with constant beeping\
-       \n/system routerboard settings set silent-boot=yes\
+       \n#:do {/system routerboard settings set silent-boot=yes} on-error={};\
        \n\
-       \n:foreach service in [:toarray \"ftp,telnet,ssh\"] do={\
+       \n:foreach service in [:toarray \"ftp\"] do={\
        \n   :local serviceAddresses \"\"\
        \n   :foreach k in=[/ip service get [find name=\$service] address] do={\
        \n      :if (\$k != \"\$haAddressA/32\" and \$k != \"\$haAddressB/32\" and \$k != \"\$haAddressVRRP/32\") do {\
@@ -365,7 +365,7 @@
        \n   /ip service set [find name=\$service] address=[:toarray \$serviceAddresses]\
        \n}\
        \n\
-       \n:if ([:len [/file find where name=\"HA_run-after-hastartup.rsc\"]] > 0) do {\
+       \n:if ([:len [/file find where name=\"HA_run-after-hastartup.rsc\"]]  > 0) do {\
        \n   /import HA_run-after-hastartup.rsc\
        \n}\
        \n/delay 5\
@@ -388,7 +388,7 @@
        \n   /system script run [find name=\"ha_pushbackup\"]\
        \n   :put \"delaying 60\"\
        \n   /delay 60\
-       \n   :if (\$isMaster && [/ping 169.254.23.3 count=1 interface=ether1 ttl=1] >= 1) do {\
+       \n   :if (\$isMaster && [/ping \$haAddressOther count=1 interface=\$haInterface ttl=1]  >= 1) do {\
        \n      :put \"REBOOTING MYSELF\"\
        \n      :execute \"/system reboot\"\
        \n   } else {\
diff -uNr scripts/ha_checkchanges.script ../../scripts/ha_checkchanges.script
--- scripts/ha_checkchanges.script      2018-02-17 11:58:46.000000000 +0200
+++ ../../scripts/ha_checkchanges.script        2018-02-17 12:35:29.000000000 +0200
@@ -1,4 +1,4 @@
-:if ([:len [/system script job find where script="ha_checkchanges"]] > 1) do={:error "already running checkchanges"; }
+:if ([:len [/system script job find where script="ha_checkchanges"]]  > 1) do={:error "already running checkchanges"; }
 :global isMaster
 :global isStandbyInSync
 :global haPassword
diff -uNr scripts/ha_config.script ../../scripts/ha_config.script
--- scripts/ha_config.script    2018-02-18 08:54:28.000000000 +0200
+++ ../../scripts/ha_config.script      2018-02-18 08:54:06.000000000 +0200
@@ -1,7 +1,7 @@
 /system script run [find name="ha_config_base"]
 :global haNetwork "169.254.23.0"
-:global haNetmask "255.255.255.0"
-:global haNetmaskBits "24"
+:global haNetmask "255.255.255.248"
+:global haNetmaskBits "29"
 :global haAddressA "169.254.23.1"
 :global haAddressB "169.254.23.2"
-:global haAddressVRRP "169.254.23.10"
\ No newline at end of file
+:global haAddressVRRP "169.254.23.3"
\ No newline at end of file
diff -uNr scripts/ha_install.script ../../scripts/ha_install.script
--- scripts/ha_install.script   2018-02-17 12:13:18.000000000 +0200
+++ ../../scripts/ha_install.script     2018-02-17 12:37:49.000000000 +0200
@@ -29,7 +29,7 @@
    :error "Are you sure the other device is configured properly? I am unable to ping MAC $pingMac"
 }

-:if ([:len [/ip address find where interface="$haInterface" and comment!="HA_AUTO"]] > 0) do {
+:if ([:len [/ip address find where interface="$haInterface" and comment!="HA_AUTO"]]  > 0) do {
    :error "Interface $haInterface has IP addresses. HA should completely own the interface and it cannot be used by anything else. Please correct"
 }

diff -uNr scripts/ha_pushbackup.script ../../scripts/ha_pushbackup.script
--- scripts/ha_pushbackup.script        2018-02-17 12:13:47.000000000 +0200
+++ ../../scripts/ha_pushbackup.script  2018-02-17 12:38:25.000000000 +0200
@@ -1,4 +1,4 @@
-:if ([:len [/system script job find where script="ha_pushbackup"]] > 1) do={:error "already running pushbackup"; }
+:if ([:len [/system script job find where script="ha_pushbackup"]]  > 1) do={:error "already running pushbackup"; }
 :global haPassword
 :global isMaster
 :global haAddressOther
diff -uNr scripts/ha_startup.script ../../scripts/ha_startup.script
--- scripts/ha_startup.script   2018-02-17 12:39:39.000000000 +0200
+++ ../../scripts/ha_startup.script     2018-02-18 09:32:33.000000000 +0200
@@ -35,7 +35,7 @@
 /system scheduler remove [find comment="HA_AUTO"]

 #Pause on-error just in case we error out before the spin loop - hope 5 seconds is enough.
-/system scheduler add comment=HA_AUTO name=ha_startup on-event=":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disable [find default-name!=\"\$haInterface\"]; /log error \"ha_startup: FAILED - DISABLED ALL INTERFACES\" }" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=startup
+/system scheduler add comment=HA_AUTO name=ha_startup on-event=":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disable [find default-name!=\"\$haInterface\"]; /log error \"ha_startup: FAILED - DISABLED ALL INTERFACES\" }" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=startup

 #/interface ethernet reset-mac-address
 /ip address remove [find interface="$haInterface"]
@@ -82,8 +82,8 @@
    }
 }

-/ip route remove [find comment="HA_AUTO"]
-/ip route add gateway=$haAddressOther distance=250 comment=HA_AUTO
+#/ip route remove [find comment="HA_AUTO"]
+#/ip route add gateway=$haAddressOther distance=250 comment=HA_AUTO

 /log warning "ha_startup: 4"

@@ -104,10 +104,10 @@
 /ip address add address=$haAddressVRRP netmask=255.255.255.255 interface=HA_VRRP comment="HA_AUTO"

 /log warning "ha_startup: 6"
-/system scheduler add comment=HA_AUTO interval=30m name=ha_exportcurrent on-event="/export file=\"HA_current.rsc\"" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/20/2000 start-time=22:37:10
-/system scheduler add interval=10m name=ha_checkchanges on-event=ha_checkchanges policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/1/2000 start-time=18:00:30 comment=HA_AUTO
+/system scheduler add comment=HA_AUTO interval=10m name=ha_exportcurrent on-event="/export file=\"HA_current.rsc\"" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=00:05:00
+/system scheduler add interval=10m name=ha_checkchanges on-event=ha_checkchanges policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=00:10:00 comment=HA_AUTO
 #Still need this - things like DHCP leases dont cause a system config change, we want to backup periodically.
-/system scheduler add comment=HA_AUTO interval=24h name=ha_auto_pushbackup on-event=ha_pushbackup policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/20/2000 start-time=05:00:00
+/system scheduler add comment=HA_AUTO interval=24h name=ha_auto_pushbackup on-event=ha_pushbackup policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=05:00:00
 /log warning "ha_startup: 7"
 :if ([:len [/file find name="HA_dsa"]] = 1) do={
    /ip ssh import-host-key private-key-file=HA_rsa
@@ -119,9 +119,9 @@
 /user add address="$haNetwork/$haNetmaskBits" comment=HA_AUTO group=full name=ha password="$haPassword"
 /log warning "ha_startup: 8"
 #So you dont get annoyed with constant beeping
-/system routerboard settings set silent-boot=yes
+#:do {/system routerboard settings set silent-boot=yes} on-error={};

-:foreach service in [:toarray "ftp,telnet,ssh"] do={
+:foreach service in [:toarray "ftp"] do={
    :local serviceAddresses ""
    :foreach k in=[/ip service get [find name=$service] address] do={
       :if ($k != "$haAddressA/32" and $k != "$haAddressB/32" and $k != "$haAddressVRRP/32") do {
@@ -132,7 +132,7 @@
    /ip service set [find name=$service] address=[:toarray $serviceAddresses]
 }

-:if ([:len [/file find where name="HA_run-after-hastartup.rsc"]] > 0) do {
+:if ([:len [/file find where name="HA_run-after-hastartup.rsc"]]  > 0) do {
    /import HA_run-after-hastartup.rsc
 }
 /delay 5
diff -uNr scripts/ha_switchrole.script ../../scripts/ha_switchrole.script
--- scripts/ha_switchrole.script        2018-02-17 12:14:19.000000000 +0200
+++ ../../scripts/ha_switchrole.script  2018-02-18 09:17:57.000000000 +0200
@@ -4,7 +4,7 @@
    /system script run [find name="ha_pushbackup"]
    :put "delaying 60"
    /delay 60
-   :if ($isMaster && [/ping 169.254.23.3 count=1 interface=ether1 ttl=1] >= 1) do {
+   :if ($isMaster && [/ping $haAddressOther count=1 interface=$haInterface ttl=1]  >= 1) do {
       :put "REBOOTING MYSELF"
       :execute "/system reboot"
    } else {


Good catch on the switchrole, it is actually a script I very rarely use and wasn't intended to be committed. It needs to be changed to use $haOtherAddress and $haInterface rather than the fixed IP and interface.
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 3:10 pm

The following patch keeps the HA heartbeat and configuration synchronisation interface's original MAC address on both routers. Not necessary on hardware routers with a direct point-to-point network cable but necessary when working with virtual guests or where HA interfaces connect via switch:
--- scripts/ha_startup.script   2018-02-17 12:39:39.000000000 +0200
+++ ../../scripts/ha_startup.script     2018-02-18 15:01:54.000000000 +0200
@@ -37,9 +37,9 @@
 #Pause on-error just in case we error out before the spin loop - hope 5 seconds is enough.
 /system scheduler add comment=HA_AUTO name=ha_startup on-event=":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disable [find default-name!=\"\$haInterface\"]; /log error \"ha_startup: FAILED - DISABLED ALL INTERFACES\" }" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=startup

-#/interface ethernet reset-mac-address
+/interface ethernet reset-mac-address [find default-name="$haInterface"]
 /ip address remove [find interface="$haInterface"]
 /ip address remove [find comment="HA_AUTO"]
 /interface vrrp remove [find name="HA_VRRP"]
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 3:10 pm

All sound good and I will integrate them but two questions:
Nice catch on rancid, it actually impacts me as well. I think we need to fix rancid and give it a stricter prompt for export. Even if we escape ha-mikrotik, it will still break rancid if there is any other script on the devices that use ] >.
Can you try the below patch to rancid and see how it works for you?

Can you help me understand the "Disables adding default route" and how it interacts with the loopbacks for you?
I actually use the default because I have a MASQUERADE rule in my setup that allows the ha-mikrotik network to get out to the internet. I do this to test RouterOS upgrades: I login to the standby, do a RouterOS upgrade, then do a push from the primary, check if the standby looks right, then switch roles and repeat on the new secondary (old master).

Additionally, I've been thinking about giving the secondary a stable known address in addition to the floating ones (ie: .3 is always master, .4 is always secondary). If I do this, it would allow for a NAT setup to allow easier external access to the secondary for monitoring. Additionally, maybe an simple that can be used with the Mikrotik SNMP script GET to monitor the state of the pair. Any thoughts on how you might want to monitor the secondary in general?
--- mtrancid.orig	2018-02-18 07:55:03.199828386 -0500
+++ mtrancid	2018-02-18 07:55:20.856371114 -0500
@@ -235,9 +235,13 @@
 	print STDERR "    In Export: $_" if ($debug);
 	my $buffer = "";
 
+    #Be much stricter on the quit prompt when exporting. If scripts contain ] > then it is incorrectly terminated early.
+    my $prompt_quit = "${prompt}quit\$";
+	print STDERR "    Quit prompt for export: $prompt_quit\n" if ($debug);
+
 	while (<INPUT>) {
 		tr/\015//d;
-		if (/$prompt/) { $found_end=1; $clean_run=1; return 0};
+		if (/$prompt_quit/) { $found_end=1; $clean_run=1; return 0};
 		next if(/^(\s*|\s*$cmd\s*)$/);
 		next if(/^#/);
 		return(1) if /(bad command name )/;
Would you please consider accepting the following patch, it does the following:
  • Changes '] > ' to stop rancid (configuration revision management) matching it to the RouterOS prompt.
  • Changes netmask from /24 to /29 and moved VRRP IP from .10 to .3.
  • Set schedulers' start date to Unix Epoch (Jan/01/1970).
  • Set schedulers' intervals and start time to prevent overlapping.
  • Only change FTP service, prevents SSH not being reachable on master or enabling Telnet.
  • Replaces hard coded values with variables.
  • Disables adding default route (makes loopback interfaces reachable).
  • Disables silencing Routerboard boot process by default and handle errors (eg VM)
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 4:50 pm

With regard to changing to a /29...we are going to need a better upgrade procedure. Upgrades (rather undocumented) have always consisted of basically just doing an /import HA_init.rsc, pushing, switch roles, push, done. If we change the default VRRP addressing and then use this method then this will break all existing users that use the /24. The secondary ends up taking over and they never reconcile their differences and end up in a reboot loop.

I agree that the user should be able to select their own network but I think I'd rather do it with the existence of an alternate configuration that overrides the standard configuration.
It can also be done as extra parameters to $HAInstall to make it easier to deploy clusters that are similar.

Would this work for you?

PS: Any interest in taking this to the github project so we can track the features/issues a little cleaner?
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 4:56 pm

I centralise logging and was receiving SMS messages indicating loss of BGP peers. This was due to me originating syslog messages from the loopback IPs, which would then route out:
/system logging action
set 3 remote=54.119.65.26 src-address=54.79.22.1
I prefer having the standby router exclusively accessible via the acting master, PuTTY's tunneling features really help with this...


I hear your point about having predictable master/slave IPs, but currently handle standby router monitoring by getting notified if the HA interface on the acting master is down two checks in a row (we run Zabbix and have automated discovery which notifies us of any interface which is down when it was ever up). This way I simply need to know that the HA interface is operational and it will not send notifications if it happens to get checked whilst rebooting).


I understand your more conservative approach to RouterOS updates. I had:
  • Upgraded acting master, which switches it to standby mode
  • Connected to new standby router, upgraded firmware to complete the process and rebooted
  • Validated configuration via mac telnet
  • Repeated the steps above on the current master

Can you help me understand the "Disables adding default route" and how it interacts with the loopbacks for you?
I actually use the default because I have a MASQUERADE rule in my setup that allows the ha-mikrotik network to get out to the internet. I do this to test RouterOS upgrades: I login to the standby, do a RouterOS upgrade, then do a push from the primary, check if the standby looks right, then switch roles and repeat on the new secondary (old master).

Additionally, I've been thinking about giving the secondary a stable known address in addition to the floating ones (ie: .3 is always master, .4 is always secondary). If I do this, it would allow for a NAT setup to allow easier external access to the secondary for monitoring. Additionally, maybe an simple that can be used with the Mikrotik SNMP script GET to monitor the state of the pair. Any thoughts on how you might want to monitor the secondary in general?
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 5:01 pm

Perfect, I'll have some time tomorrow to fiddle with Rancid and agree that discussing this on Github is probably better. Perhaps I should break up the patch in to separate ones, where each one handles a specific point?
I agree that the user should be able to select their own network but I think I'd rather do it with the existence of an alternate configuration that overrides the standard configuration.
It can also be done as extra parameters to $HAInstall to make it easier to deploy clusters that are similar.

PS: Any interest in taking this to the github project so we can track the features/issues a little cleaner?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Feb 18, 2018 5:20 pm

Let's pick it up from here on github.

I have integrated your changes into a test branch for us: https://github.com/svlsResearch/ha-mikr ... bs2webtest
Issues created for the exclusions: https://github.com/svlsResearch/ha-mikrotik/issues

Excluded for now:
  • No rancid escape fix here. If you still want to do this escaping, let's do it with the generate script. The rancid fix appears to be working OK for me though.
  • Kept the default gateway for now. I understand your use case though, you don't want your secondary getting out.
  • Keeps original /24 addressing until we can sort out the ha-mikrotik upgrade path.
 
ovidiu
just joined
Posts: 14
Joined: Sun Jan 15, 2017 9:28 am

Re: Suggestion: Completely virtual router based on two physical routers

Tue Mar 06, 2018 8:54 am

No problem not using CCRs, they are definitely expensive for many deployments. I just wanted to let you know that you are the first one that I know of to test alternative platforms, so good for all of us. I would like to hear how well it works for you after you run for a while.

The boot delay sounds like a great solution if you just want one to always become primary when they are both booted nearly simultaneously (i.e. after power recovery). This wouldn't force A to become primary again after A was primary and then rebooted but that is the feature I could add if you really wanted it. I think this could work based on a pretty simply change that enables VRRP preemption.

It sounds like you have found a pretty workable solution though. Maybe you run it for a while and then see if you generally find it stable and if you still want this feature after a while of running, I will add it. How does that sound?
2 week passed without any problem, the delayed startup ensure the desired router to be the active one.
So this script is working fine on smaller routers as well.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Thu Aug 30, 2018 2:57 am

Anyone tested and confirmed this works exactly as expected on 6.42.x ?
We're running this on a couple of routers in a data center and it seems to work fine. However 2 problems i've noticed and I don't know if they are an issue with the later firmware or something going on with the script
1) I can't seem to make either of them a preemptive Master. I've tried adjusting VRRP priorities but if I reboot A and then B takes over, A will never be master until B reboots. We would rather have A always be the active master if it's online
2) I noticed the VRRP instance flaps a lot. I currently have B totally disconnected because it was flapping every few hours. We've tried changing ethernet cables and the same problem still happens. This is a big problem because these routers run BGP as well as PPPoE connections, resulting in extended downtime during a change over. Fine if we have an actual router failure, but not fine during normal day to day operation. There doesn't appear to be a physical interface issue, i'm not sure if its VRRP or the script. Can I just increase the VRRP timers to start with? (Won't break anything on the script or pairs?)

I also have another question regarding firmware updates. Is there any special care that must be taken? i.e. do I need to update both routers at same time or can I do 1, bring it online, reboot the other so the one with latest firmware becomes active, check everything is working fine and then update the backup?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Aug 30, 2018 3:42 am

I have not been able to test it on 6.42.x just yet, you may be the first. It is on my todo list. VRRP should not be flapping at all - are they directly connected or are you going via a switch? anything interesting in the logs? Were you running 6.38.x before going to 6.42.x? Did you have any of this VRRP flapping before or is this new? What does the CPU load look like? My units are not very heavy on CPU load, I wonder if your timers are slipping from other loads (BGP? high PPPoE count?)

Regarding preempting, this is by design. Since ha-mikrotik is not stateful, it is rather expensive to keep switching masters (ie: VPN users disconnected 2x), so I made it this way intentionally. Others have asked about preemption but nobody seemed bothered by it enough to warrant it being implemented. See my note below on the VRRP interval on why your change may not have stuck.

As far as firmware upgrades go, I have always done it by upgrading the standby and then checking if it looks right and then doing the master, sometimes forcing sync and then doing another reboot before letting the upgraded guy takeover. Since ha-mikrotik is not supported by Mikrotik themselves, it is somewhat of a crapshoot but I have had general good success. I have many pairs running this code so I generally pick the pair that won't be catastrophic if something goes wrong for the upgrade test.

For changing the VRRP interval, you would want to edit the ha_startup script on the master (look for line after "ha_startup: 5") and then sync the standby and then reboot the master after the standby reboots. If you get the timers out of sync, I believe they will ignore each other and both become master. You can't do this via the VRRP interfaces, as they will be removed and rebuilt on every boot.

I hope this helps.
Anyone tested and confirmed this works exactly as expected on 6.42.x ?
We're running this on a couple of routers in a data center and it seems to work fine. However 2 problems i've noticed and I don't know if they are an issue with the later firmware or something going on with the script
1) I can't seem to make either of them a preemptive Master. I've tried adjusting VRRP priorities but if I reboot A and then B takes over, A will never be master until B reboots. We would rather have A always be the active master if it's online
2) I noticed the VRRP instance flaps a lot. I currently have B totally disconnected because it was flapping every few hours. We've tried changing ethernet cables and the same problem still happens. This is a big problem because these routers run BGP as well as PPPoE connections, resulting in extended downtime during a change over. Fine if we have an actual router failure, but not fine during normal day to day operation. There doesn't appear to be a physical interface issue, i'm not sure if its VRRP or the script. Can I just increase the VRRP timers to start with? (Won't break anything on the script or pairs?)

I also have another question regarding firmware updates. Is there any special care that must be taken? i.e. do I need to update both routers at same time or can I do 1, bring it online, reboot the other so the one with latest firmware becomes active, check everything is working fine and then update the backup?
 
hamster
newbie
Posts: 26
Joined: Sun Dec 11, 2016 2:46 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Oct 03, 2018 5:44 am

I've just installed this on two x86, version 6.42.9... So far, so good. Thanks for this!

Quick question, if I may: why is it neccessary to reboot the standby router once it receives new configuration?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Oct 03, 2018 3:07 pm

Quick question, if I may: why is it neccessary to reboot the standby router once it receives new configuration?
"/system backup load" is used to keep the general configuration in sync, which requires a reboot.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Thu Feb 21, 2019 9:22 am

So we have had a hardware failure on one of the routers and this script saved us a lot of downtime
However now comes the time to replace with another router. I have an identical model here

There are no instructions on what to do to bring a new standby router back into the mix (preferably without any downtime). Do I simply install the new backup router, connect the 2 via ether8 then run the ha_init script on the existing router once again and do through the same procedure?
Or is there something else I need to do only on the new backup to bring it in

Will it know to keep the existing primary config, and not override the primary with the backup?
Can this be done with little to no downtime?

Thanks
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Feb 21, 2019 12:00 pm

So we have had a hardware failure on one of the routers and this script saved us a lot of downtime
However now comes the time to replace with another router. I have an identical model here

There are no instructions on what to do to bring a new standby router back into the mix (preferably without any downtime). Do I simply install the new backup router, connect the 2 via ether8 then run the ha_init script on the existing router once again and do through the same procedure?
Or is there something else I need to do only on the new backup to bring it in

Will it know to keep the existing primary config, and not override the primary with the backup?
Can this be done with little to no downtime?

Correct, basically replace it and connect it physically like the old one. The replacement should be running the same RouterOS and reset-configuration per original docs. You will then $HAInstall like you originally did, changing the MAC of B (or A) and then following the on screen instructions for bootstrapping.

This can done live and with no downtime, the script should not do anything on the master when it discovers it is already master.

Do you have A or B alive right now? Assuming it is A, you can do something like this and follow the instructions:
$HAInstall interface=$haInterface macA=$haMacMe macB="[NEW MAC FOR B]" password=$haPassword

If it is B:
$HAInstall interface=$haInterface macB=$haMacMe macA="[NEW MAC FOR A]" password=$haPassword

This just pulls the global variables (the current config) for redeployment, you could also just populate them all again with constants like you originally did.

Try this just to see how your variables will populate (it only prints):
:put "interface=$haInterface macA=$haMacA macA=$haMacB macMe=$haMacMe password=$haPassword"
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Tue Feb 26, 2019 5:20 am

Awesome, i'll give it a go next time i'm at the DC but backup beforehand. Thanks
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Sat Mar 02, 2019 5:32 pm

Wow, this project still alive...
Good I never had a chance to put it in production..

But very nice

Sent from my XT1580 using Tapatalk

 
christopherh
newbie
Posts: 29
Joined: Sun Feb 24, 2019 7:43 am
Location: Sydney, Australia

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 18, 2019 10:10 am

Hello All,

I've followed the instructions from 1 to 8 on the GitHub page, however before $HAInstall gives me the info to bootstrap the second router, it reboots and kicks me out.

How do I bootstrap the second router?

Thanks,
Christopher H.

**EDIT: I worked it out - had to re-run the $HAInstall command to generate the commands to bootstrap the second router.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 25, 2019 1:29 am

Went to change out the dead router and noticed MikroTik has a new hardware revision of CCR series which require 6.43.5 as the minimum RouterOS version and cannot be downgraded any further. I've read on the github page there's a known bug with 6.43.x and its causing reboots and intermittent issues

Can anyone confirm if this has been fixed?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 25, 2019 1:52 am

Went to change out the dead router and noticed MikroTik has a new hardware revision of CCR series which require 6.43.5 as the minimum RouterOS version and cannot be downgraded any further. I've read on the github page there's a known bug with 6.43.x and its causing reboots and intermittent issues

Can anyone confirm if this has been fixed?
Interesting. I will begin testing some newer versions and let you know if they look stable.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 25, 2019 2:02 am

I have deployed 6.43.13 to a pair and I will report back if it appears stable.
 
User avatar
raystream
Frequent Visitor
Frequent Visitor
Posts: 67
Joined: Tue Mar 20, 2018 6:56 pm
Location: Germany
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 25, 2019 10:33 am

how can i do a software upgrade after installing your ha sytem?

Just update the primary and then the secondary goes active when the primary reboots
after that upgrade the second one

will the patch be still there after upgrade?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 25, 2019 2:23 pm

how can i do a software upgrade after installing your ha sytem?

Just update the primary and then the secondary goes active when the primary reboots
after that upgrade the second one

will the patch be still there after upgrade?
Yes, this is the easiest way to do it, if you don't mind the extra reboots of the active router. If you want to reduce the reboots of the active and test a little, you can upgrade the standby (login with /system ssh $haAddressOther from the active) and then test $HASyncStandby (and $HAPushStandby) from the active once the standby comes back from the upgrade.

Make sure you are upgrading to a version that you know to work.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Mar 25, 2019 4:24 pm

Please see this issue on github for folks looking for updates on newer RouterOS: https://github.com/svlsResearch/ha-mikrotik/issues/7

TLDR: 6.43.13 is testing well so far. See more on the github issue.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Mar 26, 2019 2:27 pm

6.43.13 is going to require that you upgrade ha-mikrotik before you upgrade to 6.43.13 to safely use. The existing code will not work reliably. The fixed code is still being tested and I expect it will be tested/working within a few days, please check here for updates: https://github.com/svlsResearch/ha-mikrotik/issues/7

I will make a new release of ha-mikrotik on github once testing is complete.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Tue Mar 26, 2019 2:43 pm

You're a legend for following up with this so quickly and in depth. Thank you very much
I'll wait for the tested update
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Mar 28, 2019 5:15 pm

This is the rc1 for 6.42.11 / 6.43.13 / 6.44.1 and I expect it to be the final release. I am now running it on 6 pairs in production.

If anyone wants to test this on their lab setup and report back, please do:
https://github.com/svlsResearch/ha-mikr ... ag/v0.6rc1
Following along from this issue:
https://github.com/svlsResearch/ha-mikrotik/issues/7

I will stamp v0.6 tomorrow.

@millenium7 You mentioned a new hardware release of the CCR model you had. I don't have any of this updated gear, is it done as the same exact model # (/system routerboard print)?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Mar 28, 2019 5:45 pm

You're a legend for following up with this so quickly and in depth. Thank you very much
I'll wait for the tested update
See prior post but specifically for you, since you are dealing with recovering a failed standby, I wanted to double check that it still works as expected and write some docs.

I just simulated a hardware failure on a standby:
/file remove [find]; /system reset-configuration keep-users=no no-defaults=yes skip-backup=yes
I then did a rebuild based on the instructions I published here and it was all set and worked well: https://github.com/svlsResearch/ha-mikr ... ed-standby

For you, I would recommend that you upgrade your new standby to v6.43.13 or v6.44.1 and then follow the above procedure. Once you confirm the standby looks good and is rebuilt, do a $HASwitchRole to have the standby takeover. You can now upgrade the standby (original active) to the same RouterOS version you have on the replaced hardware and get everything consistent.

If you have any questions, let me know. I know you are going to wait for the final release, which is fine. Instructions remain the same.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Thu Mar 28, 2019 11:51 pm


@millenium7 You mentioned a new hardware release of the CCR model you had. I don't have any of this updated gear, is it done as the same exact model # (/system routerboard print)?
CCR1036-8G-2S+ on both but the new one has a normal USB port, 2x AC input and RJ45 console port
Unsure of any other changes internally
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Thu Mar 28, 2019 11:57 pm

One thing that's not so clear in your rebuild instructions

[NEW MAC FOR A]

Because you say 'FOR' A. Do you mean the new MAC you are going to give out, or put in the existing MAC that A has?

I.e.
OldA (dead)
- Ether1: 11:11:11:11:11:11
- ....
- Ether8: 11:11:11:11:11:18
OldB
- Ether1: 22:22:22:22:22:21
- ....
- Ether8: 22:22:22:22:22:28


NewA
- Ether1: 33:33:33:33:33:31
- ....
- Ether8: 33:33:33:33:33:38

So do I put in 11:11:11:11:11:18 or 33:33:33:33:33:38 when running $HAInstall on OldB?


And would that cause NewA's MAC addresses to be used on OldB. Or would it make NewA use the MAC addresses of OldB?
If its the former wouldn't that cause downtime as all MAC addresses would change?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Mar 29, 2019 12:10 am

One thing that's not so clear in your rebuild instructions

[NEW MAC FOR A]

Because you say 'FOR' A. Do you mean the new MAC you are going to give out, or put in the existing MAC that A has?

I.e.
OldA (dead)
- Ether1: 11:11:11:11:11:11
- ....
- Ether8: 11:11:11:11:11:18
OldB
- Ether1: 22:22:22:22:22:21
- ....
- Ether8: 22:22:22:22:22:28


NewA
- Ether1: 33:33:33:33:33:31
- ....
- Ether8: 33:33:33:33:33:38

So do I put in 11:11:11:11:11:18 or 33:33:33:33:33:38 when running $HAInstall on OldB?


And would that cause NewA's MAC addresses to be used on OldB. Or would it make NewA use the MAC addresses of OldB?
If its the former wouldn't that cause downtime as all MAC addresses would change?
You would put in 33:33:33:33:33:38 (new device ether8 MAC), this is the one you would see in /ip neighbor print with ether8 connected between them.

These MACs are not used for assignment, they are only used to detect which device is which during initialization. There should be no downtime, even if you get them wrong (assuming you don't get them wrong and then reboot the current working one).

Take a look here at ha_startup to see how the MACs are used to determine A vs B at startup and only at startup:
https://github.com/svlsResearch/ha-mikr ... script#L87

Does that answer your question?

PS: Also just updated github to include different wording, similar to the install wording (MAC_OF_A_ETHER8).
Last edited by nathan1 on Fri Mar 29, 2019 12:18 am, edited 1 time in total.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Mar 29, 2019 12:13 am

It does yes. Can I suggest changing the wording though?, 'FOR A' implies the mac you are giving it
Maybe 'NEW MAC OF A' is clearer?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Mar 29, 2019 12:19 am

It does yes. Can I suggest changing the wording though?, 'FOR A' implies the mac you are giving it
Maybe 'NEW MAC OF A' is clearer?
Definitely. I just changed it to be consistent with the original installation instructions. Let me know if you think it still needs more clarification.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Mar 29, 2019 12:58 pm

It does yes. Can I suggest changing the wording though?, 'FOR A' implies the mac you are giving it
Maybe 'NEW MAC OF A' is clearer?
Do not proceed with the upgrade, hopefully you did not use rc1. There is an issue after ~24 hours of runtime with the new RouterOS that I am trying to debug.
Problem is with RouterOS (old versions still appear stable with new ha-mikrotik) but newer ones have a problem.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Mar 29, 2019 1:38 pm

Havn't updated yet. In the meantime we're waiting for our old device to get back from an RMA request, new one not going in yet and probably won't as i'm unsure of any config differences. I know for instance the new one has 2x SFP+ instead of 1x SFP+ and 1x SFP so that could cause an issue. But do still want to run the latest MikroTik firmware on this pair if possible. 6.44 does have some improvements that are useful to us

I'll wait until its confirmed working. Once again thanks for going out of your way to actually bug test this
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Mar 29, 2019 2:21 pm

Havn't updated yet. In the meantime we're waiting for our old device to get back from an RMA request, new one not going in yet and probably won't as i'm unsure of any config differences. I know for instance the new one has 2x SFP+ instead of 1x SFP+ and 1x SFP so that could cause an issue. But do still want to run the latest MikroTik firmware on this pair if possible. 6.44 does have some improvements that are useful to us

I'll wait until its confirmed working. Once again thanks for going out of your way to actually bug test this
Sounds good. Take a look at the latest update on the github issues, if you haven't. I believe the problem is an out of memory caused by another script that I run that is misbehaving with 6.44, so everything is still looking good for ha-mikrotik on the newer OS. Will definitely know more in a day or two.

I do agree, I thought it was odd that they changed up the hardware and didn't rev the model number. I always run ha-mikrotik pairs with the exact same hardware. I keep spares of my original CCR that I use to swap out when needed.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Apr 01, 2019 5:27 pm

v0.6 is stamped. Everything has been stable for multiple days now.

https://github.com/svlsResearch/ha-mikrotik/releases
 
danypd69
just joined
Posts: 14
Joined: Fri Jun 07, 2013 3:01 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 17, 2019 12:08 am

Hello, I am trying to use the scripts with two routers (1100Hx2 and 1100Dx4) with RouterOS version 6.44.2 but I am unable to make it working.
Are these models ok?
My problem is that after the initial $HAPushStandby any connection to the slave does not work anymore, if I try to do $HAPushStandby again to copy the configuration I get a connection failed error.

Here are the configuration (except scripts) of the routers

MASTER
# model = RouterBOARD 1100Dx4
# serial number = 735B078BE677
/interface vrrp
add interface=ether8 name=HA_VRRP on-backup=ha_onbackup on-master=ha_onmaster
/interface ethernet switch port
set 0 default-vlan-id=0
set 1 default-vlan-id=0
set 2 default-vlan-id=0
set 3 default-vlan-id=0
set 4 default-vlan-id=0
set 5 default-vlan-id=0
set 6 default-vlan-id=0
set 7 default-vlan-id=0
set 8 default-vlan-id=0
set 9 default-vlan-id=0
set 10 default-vlan-id=0
set 11 default-vlan-id=0
set 12 default-vlan-id=0
set 13 default-vlan-id=0
set 14 default-vlan-id=0
set 15 default-vlan-id=0
/ip address
add address=169.254.23.1/24 comment=HA_AUTO interface=ether8 network=\
    169.254.23.0
add address=169.254.23.10 comment=HA_AUTO interface=HA_VRRP network=\
    169.254.23.10
add address=192.168.0.9/24 interface=ether1 network=192.168.0.0
/ip dns
set servers=8.8.8.8
/ip firewall filter
add action=accept chain=output comment=HA_AUTO out-interface=ether8
add action=accept chain=input comment=HA_AUTO in-interface=ether8
/ip route
add distance=1 gateway=192.168.0.254
add comment=HA_AUTO distance=250 gateway=169.254.23.2
/ip service
set ftp address=169.254.23.1/32,169.254.23.2/32,169.254.23.10/32
/system clock
set time-zone-name=Europe/Rome
/system identity
set name=MikroTik_HA_A_ACTIVE
/system routerboard settings
set silent-boot=yes
/system scheduler
add comment=HA_AUTO name=ha_startup on-event=":do {:global haInterface; /syste\
    m script run [find name=ha_startup]; } on-error={ :delay 5; /interface eth\
    ernet disable [find default-name!=\"\$haInterface\"]; /log error \"ha_star\
    tup: FAILED - DISABLED ALL INTERFACES\" }" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=\
    startup
add comment=HA_AUTO name=ha_report_startup on-event=ha_report_startup policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=\
    startup
add comment=HA_AUTO interval=10m name=ha_exportcurrent on-event=\
    "/export file=\"HA_current.rsc\"" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    jan/01/1970 start-time=00:05:00
add comment=HA_AUTO interval=10m name=ha_checkchanges on-event=\
    ha_checkchanges policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    jan/01/1970 start-time=00:10:00
add comment=HA_AUTO interval=1d name=ha_auto_pushbackup on-event=\
    ha_pushbackup policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    jan/01/1970 start-time=05:00:00
add dont-require-permissions=no name=ha_config_base owner=admin policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":global\
    \_haPassword \"123451234512345\"\
    \n:global haInterface \"ether8\"\
    \n:global haMacA \"64:D1:54:FF:52:CA\"\
    \n:global haMacB \"D4:CA:6D:42:2D:86\""
SLAVE
# model = 1100Hx2
# serial number = 3E6A02BF6232
/interface ethernet
set [ find default-name=ether1 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:C4 name=ether2
set [ find default-name=ether2 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:C5 name=ether3
set [ find default-name=ether3 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:C6 name=ether4
set [ find default-name=ether4 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:C7 name=ether5
set [ find default-name=ether6 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:C9 name=ether7
set [ find default-name=ether7 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:CA name=ether8
set [ find default-name=ether8 ] l2mtu=1592 name=ether9
set [ find default-name=ether9 ] disabled=yes l2mtu=1592 mac-address=64:D1:54:FF:52:CC name=ether10
set [ find default-name=ether5 ] disabled=yes name=ether14
set [ find default-name=ether10 ] disabled=yes name=ether15
set [ find default-name=ether11 ] disabled=yes name=ether16
set [ find default-name=ether12 ] name=ether17
set [ find default-name=ether13 ] name=ether18
/interface vrrp
add interface=ether8 name=HA_VRRP on-backup=ha_onbackup on-master=ha_onmaster
/interface ethernet switch
set 2 name=switch3
/interface ethernet switch port
set 5 default-vlan-id=0 vlan-mode=fallback
set 6 default-vlan-id=0 vlan-mode=fallback
set 7 default-vlan-id=0 vlan-mode=fallback
set 8 default-vlan-id=0 vlan-mode=fallback
/ip address
add address=169.254.23.2/24 comment=HA_AUTO interface=ether8 network=169.254.23.0
add address=169.254.23.10 comment=HA_AUTO interface=HA_VRRP network=169.254.23.10
/ip firewall filter
add action=accept chain=output comment=HA_AUTO out-interface=ether8
add action=accept chain=input comment=HA_AUTO in-interface=ether8
/ip route
add comment=HA_AUTO distance=250 gateway=169.254.23.1
/ip service
set ftp address=169.254.23.1/32,169.254.23.2/32,169.254.23.10/32
/system identity
set name=MikroTik_HA__STANDBY
/system routerboard settings
set silent-boot=yes
/system scheduler
add comment=HA_AUTO name=ha_startup on-event=":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disab\
    le [find default-name!=\"\$haInterface\"]; /log error \"ha_startup: FAILED - DISABLED ALL INTERFACES\" }" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=startup
add comment=HA_AUTO name=ha_report_startup on-event=ha_report_startup policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=startup
add comment=HA_AUTO interval=10m name=ha_exportcurrent on-event="/export file=\"HA_current.rsc\"" policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=00:05:00
add comment=HA_AUTO interval=10m name=ha_checkchanges on-event=ha_checkchanges policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    jan/01/1970 start-time=00:10:00
add comment=HA_AUTO interval=1d name=ha_auto_pushbackup on-event=ha_pushbackup policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    jan/01/1970 start-time=05:00:00
add dont-require-permissions=no name=ha_config_base owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    ":global haPassword \"123451234512345\"\
    \n:global haInterface \"ether8\"\
    \n:global haMacA \"64:D1:54:FF:52:CA\"\
    \n:global haMacB \"D4:CA:6D:42:2D:86\""
Can someone tell me what I am doing wrong?
Thanks

Daniele
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 17, 2019 12:36 am

Hello, I am trying to use the scripts with two routers (1100Hx2 and 1100Dx4) with RouterOS version 6.44.2 but I am unable to make it working.
Are these models ok?
My problem is that after the initial $HAPushStandby any connection to the slave does not work anymore, if I try to do $HAPushStandby again to copy the configuration I get a connection failed error.
Do not do this. Never try to use this without exactly the same hardware, you are going to run into some serious problems.
 
danypd69
just joined
Posts: 14
Joined: Fri Jun 07, 2013 3:01 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 17, 2019 9:51 am

Hello, I am trying to use the scripts with two routers (1100Hx2 and 1100Dx4) with RouterOS version 6.44.2 but I am unable to make it working.
Are these models ok?
My problem is that after the initial $HAPushStandby any connection to the slave does not work anymore, if I try to do $HAPushStandby again to copy the configuration I get a connection failed error.
Do not do this. Never try to use this without exactly the same hardware, you are going to run into some serious problems.
Ok, thanks for the info. I will try to get another router.
 
danypd69
just joined
Posts: 14
Joined: Fri Jun 07, 2013 3:01 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 17, 2019 2:32 pm

Just a question ,should it work if i use two RB1100AHx4?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 17, 2019 3:02 pm

Just a question ,should it work if i use two RB1100AHx4?
I'm not sure I've seen anyone use RB1100 yet, most of us us the CCR line. If I remember correctly, someone did successfully run it on the RB750, which I think bodes well for you. You may be the first on the RB1100. I believe it should work and I'm willing to offer advice if something seems odd with it and the platform.

Edit: Look here viewtopic.php?t=110690#p640702 for someone that was using different RB models. I don't know if they went to production or not but did have it in the lab.
 
danypd69
just joined
Posts: 14
Joined: Fri Jun 07, 2013 3:01 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 17, 2019 3:13 pm

Ok I will let you know what happens.
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Apr 26, 2019 2:40 pm

Nathan1

It's a long time when I had played with this.
So I don't know if have this already.

It's possible to have a public management ip active on the standby router?
If this already have ignore this post.



Sent from my XT1580 using Tapatalk

 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Apr 26, 2019 2:54 pm

Nathan1

It's a long time when I had played with this.
So I don't know if have this already.

It's possible to have a public management ip active on the standby router?
If this already have ignore this post.



Sent from my XT1580 using Tapatalk

Hey Raffav,

I do this with NAT from the master to the standby, there is nothing built in to do it. May I ask why you are looking to do this? Monitoring? I did not build anything in for this because the only use case I see is monitoring, you do not want to make any configuration changes on the standby. I really don't want to have any interface up except for the $haInterface on the standby, it makes for cleaner cutovers, which makes exposing a public IP via a different interface somewhat cumbersome.

Monitoring is a valid use case though, I figure folks can configure NAT for that though, if they want it. For example, I also have masquerading setup so the standby can deliver logs to my central syslog server.

Happy to help you figure something out that works for you, if you have a specific use case in mind.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri May 03, 2019 12:33 pm

Hi nathan1, I'm trying to put vrrp interface HA_VRRP on a bride, but i'm not be able to do this. Is it possible or not? Thanks,
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri May 03, 2019 9:27 pm

Hi nathan1, I'm trying to put vrrp interface HA_VRRP on a bride, but i'm not be able to do this. Is it possible or not? Thanks,
I have never tried nor would I recommend this. May I ask what the design is to require this?
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu May 09, 2019 5:49 pm

It's a bit complicated to explain and my english is not so good.
I've made some changes to avoid that need, and all it's working fine.
Very great job Nathan1, congratulations
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu May 09, 2019 6:14 pm

It's a bit complicated to explain and my english is not so good.
I've made some changes to avoid that need, and all it's working fine.
Very great job Nathan1, congratulations
Sounds great, glad you got it to work.
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri May 10, 2019 3:49 am

Nathan1

It's a long time when I had played with this.
So I don't know if have this already.

It's possible to have a public management ip active on the standby router?
If this already have ignore this post.



Sent from my XT1580 using Tapatalk

Hey Raffav,

I do this with NAT from the master to the standby, there is nothing built in to do it. May I ask why you are looking to do this? Monitoring? I did not build anything in for this because the only use case I see is monitoring, you do not want to make any configuration changes on the standby. I really don't want to have any interface up except for the $haInterface on the standby, it makes for cleaner cutovers, which makes exposing a public IP via a different interface somewhat cumbersome.

Monitoring is a valid use case though, I figure folks can configure NAT for that though, if they want it. For example, I also have masquerading setup so the standby can deliver logs to my central syslog server.

Happy to help you figure something out that works for you, if you have a specific use case in mind.
Monitoring, and like access inspection, for helth check,
Nat is not a good because you use the mastar as a entry point.

Think something like that,
Something went wrong on master, that trigger the swap but for some reasons both became master.(vrrp stop to receive vvrp protocol and trigger the swap)
How do you access the standby if you use Nat

Sent from my XT1580 using Tapatalk

 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri May 10, 2019 4:22 am

Nathan1

It's a long time when I had played with this.
So I don't know if have this already.

It's possible to have a public management ip active on the standby router?
If this already have ignore this post.



Sent from my XT1580 using Tapatalk

Hey Raffav,

I do this with NAT from the master to the standby, there is nothing built in to do it. May I ask why you are looking to do this? Monitoring? I did not build anything in for this because the only use case I see is monitoring, you do not want to make any configuration changes on the standby. I really don't want to have any interface up except for the $haInterface on the standby, it makes for cleaner cutovers, which makes exposing a public IP via a different interface somewhat cumbersome.

Monitoring is a valid use case though, I figure folks can configure NAT for that though, if they want it. For example, I also have masquerading setup so the standby can deliver logs to my central syslog server.

Happy to help you figure something out that works for you, if you have a specific use case in mind.
Monitoring, and like access inspection, for helth check,
Nat is not a good because you use the mastar as a entry point.

Think something like that,
Something went wrong on master, that trigger the swap but for some reasons both became master.(vrrp stop to receive vvrp protocol and trigger the swap)
How do you access the standby if you use Nat

Sent from my XT1580 using Tapatalk
You should have serial out of band access to both devices. If you don’t, I really don’t suggest running this. Health inspections and monitoring are all viable via current master (NAT). If you ever have a double master situation, you really should have serial.

I’ve been running this for years and I’ve never had a situation like this, it doesn’t mean it won’t happen but it makes the entire setup more complicated. The standby is simply a slave device when it is waiting.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Thu Jun 13, 2019 10:22 am

Do you run this on any routers other than 1009's?
I also want to ask if its normal behavior for the standby to regularly reboot? I don't know the exact interval but maybe once every 2 hours?

We were running the older version on 6.42.3 and aside from the standby rebooting it did seem to work fine for months. However we've had another issue come up, details are here viewtopic.php?f=3&t=149273
I don't think your script has anything to do with it. Infact I don't think you can make an interface stop transmitting any packets at all even if you try to. But it's either some sort of configuration, script issue perhaps due to the repeated reboots which could have corrupted something, firmware bug or hardware issue

Long story short is i've now upgraded the script, updated both routers to 6.44.3, then removed one of the routers and replaced with dual PSU version (same model number), so far stable but interval between issues is unknown, can be a couple hours or a full day
If we have this issue occur again will be replacing both routers with CCR1016's, have you run the script on those before?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Jun 13, 2019 1:11 pm

Do you run this on any routers other than 1009's?
I also want to ask if its normal behavior for the standby to regularly reboot? I don't know the exact interval but maybe once every 2 hours?

We were running the older version on 6.42.3 and aside from the standby rebooting it did seem to work fine for months. However we've had another issue come up, details are here viewtopic.php?f=3&t=149273
I don't think your script has anything to do with it. Infact I don't think you can make an interface stop transmitting any packets at all even if you try to. But it's either some sort of configuration, script issue perhaps due to the repeated reboots which could have corrupted something, firmware bug or hardware issue

Long story short is i've now upgraded the script, updated both routers to 6.44.3, then removed one of the routers and replaced with dual PSU version (same model number), so far stable but interval between issues is unknown, can be a couple hours or a full day
If we have this issue occur again will be replacing both routers with CCR1016's, have you run the script on those before?
I exclusively use 1009s but I know others have tried with success using other devices. As always, make sure you have serial access and exactly matched pairs. It is normal for the standby to regularly reboot, it will happen automatically at least once a day (scheduled) and regularly if there is a configuration change on the primary. This is how the standby stays in sync.

The configuration changes are detected via: /system history print
If you look at that on the primary, you should find what configuration is changing regularly that causes the standby to reboot. The reboots are "normal reboots", it is scripted and shouldn't just be going down hard. If you find there is nothing in the system history but you are still seeing reboots, then it is something that probably needs to be looked into. The one daily reboot is a forced one regardless of the history but that should only happen once a day (you will see ha_auto_pushbackup in /system scheduler print)

Your issue with the transmitting is odd, I don't really see how ha-mikrotik could cause this. I still have 5+ pairs of CCR1009 running flawlessly for years with this process.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Jun 17, 2019 5:55 am

Ok. As I said I don't think its the script because I can't think of any way to even make an interface do that, even if intentionally trying
Good to know the reboots are a normal thing. It was happening more than once a day but knowing that its also caused by 'system history print' answers why because we have a script that removes and replaces RADIUS information regularly, and that shows up in the history log. So if I fix that script so it instead looks for a change before replacing rather than hard remove/add it should reduce the number of reboots

When there is a change detected in config, what is the procedure the standby router does to update its config?
Does it find the exact change and then input that command. Or does it do a backup/restore from the config on the active router?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Jun 17, 2019 6:13 am

When there is a change detected in config, what is the procedure the standby router does to update its config?
Does it find the exact change and then input that command. Or does it do a backup/restore from the config on the active router?
It does a backup and restore along with copying files it finds on the filesystem. You should definitely try to reduce extraneous config changes.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Jun 17, 2019 10:08 am

I noticed the copying of files to be a problem. Is it possible for you to change that in your script to exclude anything beginning with 'log.' ?
Reason is I was logging to disk any errors to try and help troubleshoot the issues we were having when we couldn't catch it in time, but when the router rebooted, the new active would override the log files that were stored.
The issue we were/are having doesn't seem to log anything but the scripts I run to detect interface issues does and then the files get overwritten
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Jun 17, 2019 2:20 pm

I noticed the copying of files to be a problem. Is it possible for you to change that in your script to exclude anything beginning with 'log.' ?
Reason is I was logging to disk any errors to try and help troubleshoot the issues we were having when we couldn't catch it in time, but when the router rebooted, the new active would override the log files that were stored.
The issue we were/are having doesn't seem to log anything but the scripts I run to detect interface issues does and then the files get overwritten
How are you logging it? Can you change the name to start with HA_? If you can, it will be excluded without any changes to the code.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Tue Jun 18, 2019 6:04 am

Just with the default 'disk' action which creates file beginning with 'log.' then the sequence number, then ends in txt i.e.
log.0.txt and log.1.txt by default
The reason for logging to disk is incase connectivity is lost i.e. interfaces locking up, at least logs would be stored if theres no other way for the router to reach the internet
If these sorts of nuances were published on the github page it would have been helpful. I didn't know HA files weren't mirrored, if I had I would have changed the disk action to start with that (or just used different names for each routers logs)

Granted MikroTik should pull their finger out and write a proper High Availability module instead of wasting time on stupid crap like 'Kid Control'
It's not your responsibility to do it for them, and i'm grateful for you publishing your scripts and being so helpful with your responses
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Suggestion: Completely virtual router based on two physical routers

Tue Aug 06, 2019 1:51 am

RouterOS 6.45+ sets the VRRP interface to standby when the associated parent interface is not running. Whilst this makes perfect sense for classic VRRP implementations it causes a problem with the use of VRRP in the context of this high availability implementation. The problem is that since the sync interface, which VRRP is bound to, is directly connected to the partner router; VRRP will stay down when the second router is restarted or failed. The master router will essentially immediately shut all its ports the moment the standby router is restarted and the act of firing scripts when the partner router flaps its ports during initialisation can result in routers having an inconsistent state (we had one pair where both had their ports enabled).

Whilst I've logged a query with MikroTik, noting the lack of reference in the change log archives, I unfortunately don't expect anything to happen.

A work around to the problem was for us to migrate VRRP on to a bridge, to which we then add the sync interface.

To change a stack running eg 6.44.5 to 6.45.3:
  • Apply the following changes to the ha_startup script
  • Run ha_pushback on master to transfer config to slave
  • Upgrade slave to 6.45.3
  • Upgrade firmware and restart slave
  • Upgrade master to 6.45.3 (initiates failover)
  • Upgrade firmware and restart slave (previous master)
  • Restart master, to switch slave back to master


Patch:
@@ -38,10 +38,10 @@
 /system scheduler add comment=HA_AUTO name=ha_startup on-event=":do {:global haInterface; /system script run [find name=ha_startup]; } on-error={ :delay 5; /interface ethernet disable [find default-name!=\"\$haInterface\"]; /log error \"ha_startup: FAILED - DISABLED ALL INTERFACES\" }" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=Jan/01/1970 start-time=startup

 /interface ethernet reset-mac-address [find default-name="$haInterface"]
-/ip address remove [find interface="$haInterface"]
 /ip address remove [find comment="HA_AUTO"]
+/interface bridge port remove [find comment="HA_AUTO"]
+/interface bridge remove [find comment="HA_AUTO"]
 /interface vrrp remove [find name="HA_VRRP"]
-/ip address remove [find interface="HA_VRRP"]
 /ip firewall filter remove [find comment="HA_AUTO"]
 /ip service set [find name="ftp"] disabled=yes

@@ -50,11 +50,14 @@
 /interface ethernet get [find default-name="$haInterface"] orig-mac-address
 /log warning "ha_startup: 2.2"
 :local mac [[/interface ethernet get [find default-name="$haInterface"] orig-mac-address]]
+/log warning "ha_startup: 2.3"
+/interface bridge add name="bridge-$haInterface" comment="HA_AUTO"
+/interface bridge port add bridge="bridge-$haInterface" interface="$haInterface" comment="HA_AUTO"
 /log warning "ha_startup: 3"
 :if ("$mac" = "$haMacA") do {
    :global haIdentity "A"
    /log warning "I AM A"
-   /ip address add interface=$haInterface address=$haAddressA netmask=$haNetmask comment="HA_AUTO"
+   /ip address add interface="bridge-$haInterface" address=$haAddressA netmask=$haNetmask comment="HA_AUTO"
    :global haAddressMe $haAddressA
    :global haAddressOther $haAddressB
    :global haMacMe $haMacA
@@ -63,7 +66,7 @@
    :if ("$mac" = "$haMacB") do {
       :global haIdentity "B"
       /log warning "I AM B"
-      /ip address add interface=$haInterface address=$haAddressB netmask=$haNetmask comment="HA_AUTO"
+      /ip address add interface="bridge-$haInterface" address=$haAddressB netmask=$haNetmask comment="HA_AUTO"
       :global haAddressMe $haAddressB
       :global haAddressOther $haAddressA
       :global haMacMe $haMacB
@@ -90,17 +93,17 @@
 #If firewall is empty, place-before=0 won't work. Add first rule.
 :if ([:len [/ip firewall filter find]] = 0) do {
    /log warning "ha_startup: 4.1"
-   /ip firewall filter add chain=output action=accept out-interface=$haInterface comment="HA_AUTO"
-   /ip firewall filter add chain=input action=accept in-interface=$haInterface comment="HA_AUTO"
+   /ip firewall filter add chain=output action=accept out-interface="bridge-$haInterface" comment="HA_AUTO"
+   /ip firewall filter add chain=input action=accept in-interface="bridge-$haInterface" comment="HA_AUTO"
 } else {
    /log warning "ha_startup: 4.2"
-   /ip firewall filter add chain=output action=accept out-interface=$haInterface comment="HA_AUTO" place-before=0
-   /ip firewall filter add chain=input action=accept in-interface=$haInterface comment="HA_AUTO" place-before=0
+   /ip firewall filter add chain=output action=accept out-interface="bridge-$haInterface" comment="HA_AUTO" place-before=0
+   /ip firewall filter add chain=input action=accept in-interface="bridge-$haInterface" comment="HA_AUTO" place-before=0
 }
 /log warning "ha_startup: 4.3"

 /log warning "ha_startup: 5"
-/interface vrrp add interface=$haInterface version=3 interval=1 name=HA_VRRP on-backup="ha_onbackup" on-master="ha_onmaster"
+/interface vrrp add interface="bridge-$haInterface" version=3 interval=1 name=HA_VRRP on-backup="ha_onbackup" on-master="ha_onmaster"
 /ip address add address=$haAddressVRRP netmask=255.255.255.255 interface=HA_VRRP comment="HA_AUTO"

 /log warning "ha_startup: 6"
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Aug 06, 2019 2:08 am

RouterOS 6.45+ sets the VRRP interface to standby when the associated parent interface is not running. Whilst this makes perfect sense for classic VRRP implementations it causes a problem with the use of VRRP in the context of this high availability implementation. The problem is that since the sync interface, which VRRP is bound to, is directly connected to the partner router; VRRP will stay down when the second router is restarted or failed. The master router will essentially immediately shut all its ports the moment the standby router is restarted and the act of firing scripts when the partner router flaps its ports during initialisation can result in routers having an inconsistent state (we had one pair where both had their ports enabled).
....
This is a disappointing behavioral change from them, thanks for investigating it. I have entered a bug on github for now: https://github.com/svlsResearch/ha-mikrotik/issues/11
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 12, 2019 9:05 pm

Hi,
I'm testing with a pair of CCR1036 with 6.44.5 software, and it's not working well. At first, i bootstrapped correctly router A, i can see it active but, once router B is synced, after reboot, B becomes active and automatically A becomes in standby mode. Then, if i try to switchrole, i get this error:

/ip smb shares remove [find comment=HA_AUTO]
end_mkDirCode
status: failed
failure: connection failed

or if i reboot router B, router A appears like active, but when B is back online again, it takes active rol and A takes standby rol. Always B is active and A inactive.
Have someone tried with this hardware? Maybe that model isn't compatible, or am i making something wrong? Thanks,
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 12, 2019 10:00 pm

Hi,
I'm testing with a pair of CCR1036 with 6.44.5 software, and it's not working well. At first, i bootstrapped correctly router A, i can see it active but, once router B is synced, after reboot, B becomes active and automatically A becomes in standby mode. Then, if i try to switchrole, i get this error:

/ip smb shares remove [find comment=HA_AUTO]
end_mkDirCode
status: failed
failure: connection failed

or if i reboot router B, router A appears like active, but when B is back online again, it takes active rol and A takes standby rol. Always B is active and A inactive.
Have someone tried with this hardware? Maybe that model isn't compatible, or am i making something wrong? Thanks,
Did you do this bootstrapping from reset routers?

This platform/version should work fine. Can you send some more logs from when you first issue the switchrole?

From the current master (in each case), does this succeed?
/tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:22 am

Thanks for the reply nathan1. Yes, both routers are reset to defaults before bootstraping. I'll try to exec that commend and i'll post it here, I have a look to log file and it seems nothing wrong, maybe the log debug is not set to a detailed level.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 3:33 pm

This is what i get when i try to sync router B at first time

Code: Select all

[admin@MikroTik_HA_A_ACTIVE] > $HASyncStandby
status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

status: finished
downloaded: 0KiBC-z pause]
duration: 1s

MASTER VERSION: ! history=.id=*b3;action=script removed;by=admin;policy=write;time=jan/02/1970 00:10:41 file=*d105d certificate= !
STANDB VERSION: ! !
NEED TO PUSH
mkdirCode: :foreach k in=[/file find type!="directory"] do={ :local xferfile [/file get $k name]; if ([:pick "$xferfile" 0 3] != "HA_") do={ :put "removing $xfe
rfile"; /file remove $k; } };
/delay 2;
:do { /ip smb shares add comment=HA_AUTO name=mkdir disabled=yes directory=/skins } on-error={}
/ip smb shares set [find comment=HA_AUTO] directory="pub"
/ip smb shares remove [find comment=HA_AUTO]
end_mkDirCode
status: finished
downloaded: 0KiB
total: 0KiB
duration: 3s

status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

status: finished
downloaded: 1KiBC-z pause]
total: 1KiB
duration: 1s

status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

Saving system configuration
Configuration backup saved
status: finished
downloaded: 31KiB-z pause]
total: 31KiB
duration: 1s

status: finished
downloaded: 44KiB-z pause]
total: 44KiB
duration: 1s

status: failed

OK - status failed is OK from last fetch, standby is rebooting.
00:15:53 echo: ssh,critical SSH host keys exported!
[admin@MikroTik_HA_A_ACTIVE] >
And the other command

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > $HASwitchRole
I am master - switching role
mkdirCode: :foreach k in=[/file find type!="directory"] do={ :local xferfile [/file get $k name]; if ([:pick "$xferfile" 0 3] != "HA_") do={ :put "removing $
xferfile"; /file remove $k; } };
/delay 2;
:do { /ip smb shares add comment=HA_AUTO name=mkdir disabled=yes directory=/skins } on-error={}
/ip smb shares set [find comment=HA_AUTO] directory="pub"
/ip smb shares set [find comment=HA_AUTO] directory="skins"
/ip smb shares remove [find comment=HA_AUTO]
end_mkDirCode
status: failed
failure: connection failed
[admin@MikroTik_HA_B_ACTIVE] > /tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
status: failed
failure: connection failed
[admin@MikroTik_HA_B_ACTIVE] >
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 3:50 pm

....
[admin@MikroTik_HA_B_ACTIVE] > /tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
status: failed
failure: connection failed
[admin@MikroTik_HA_B_ACTIVE] >[/Codebox]
Please try it again but run the test command before you do a switch role, I’m trying to figure out if the FTP server is somehow broken from B to A or if it is somehow dying during the switch.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 4:33 pm

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > /tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
status: failed
failure: connection failed
[admin@MikroTik_HA_B_ACTIVE] >
On router A, ftp server is always disabled. If I enable it manually

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > /tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
status: failed

failure: poll err
[admin@MikroTik_HA_B_ACTIVE] >
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 5:42 pm

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > /tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
status: failed
failure: connection failed
[admin@MikroTik_HA_B_ACTIVE] >
On router A, ftp server is always disabled. If I enable it manually

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > /tool fetch src-path=HA_boot_log.txt dst-path=testing.txt address=$haAddressOther user=ha password=$haPassword mode=ftp
status: failed

failure: poll err
[admin@MikroTik_HA_B_ACTIVE] >
Can you show a /log print and /file print on A after it boots and becomes standby? It seems like something is going wrong with the startup of this guy, the FTP server should definitely be enabled automatically.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 6:19 pm

/log print

Code: Select all

[admin@MikroTik_HA_A_ACTIVE] > /log print
00:25:28 system,info router rebooted
00:25:28 health,warning PSU2 entered state FAIL
00:25:33 interface,info ether8 link up (speed 100M, full duplex)
00:25:33 interface,info ether1 link up (speed 100M, full duplex)
00:26:02 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:26:03 system,info,account user admin logged in via local
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:11 system,info new script added by admin
00:26:13 smb,info created new share: pub
00:26:14 script,warning ha_startup: START
00:26:14 script,warning ha_startup: 0.1
00:26:14 script,warning ha_startup: 0.2
00:26:14 script,warning ha_startup: 0.3
00:26:14 interface,info ether1 link down
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 interface,info ether8 link down
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 script,warning ha_startup: version 0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:26:14 script,warning ha_startup: 1 ether8
00:26:14 system,info new script scheduled by admin
00:26:14 system,info new script scheduled by admin
00:26:14 script,warning ha_startup: 2
00:26:14 system,info device changed by admin
00:26:14 system,info ip service changed by admin
00:26:14 system,info device changed by admin
00:26:14 script,warning ha_startup: 2.1 1
00:26:14 script,warning ha_startup: 2.2 1
00:26:14 script,warning ha_startup: 3 74:4D:28:C7:80:BB 1
00:26:14 script,warning ha_startup: 3.1 74:4D:28:C7:80:BB 1
00:26:14 script,warning I AM A
00:26:14 system,info address added by admin
00:26:14 system,info route added by admin
00:26:14 script,warning ha_startup: 4
00:26:14 script,warning ha_startup: 4.1
00:26:14 system,info filter rule added by admin
00:26:15 system,info filter rule added by admin
00:26:15 script,warning ha_startup: 4.3
00:26:15 script,warning ha_startup: 5
00:26:15 system,info device added by admin
00:26:15 system,info address added by admin
00:26:15 script,warning ha_startup: 6
00:26:15 system,info new script scheduled by admin
00:26:15 system,info new script scheduled by admin
00:26:15 system,info new script scheduled by admin
00:26:15 script,warning ha_startup: 7
00:26:15 system,info user ha added by admin
00:26:15 script,warning ha_startup: 8
00:26:15 system,info tile rb settings changed by admin
00:26:15 system,info ip service changed by admin
00:26:15 vrrp,info HA_VRRP now BACKUP
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info system identity changed by admin
00:26:17 system,info,account user admin logged out from 20:CF:30:C1:88:C7 via winbox
00:26:17 system,info,account user admin logged out via local
00:26:19 interface,info ether8 link up (speed 100M, full duplex)
00:26:24 vrrp,info HA_VRRP now MASTER, master down timer
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:25 system,info device changed by admin
00:26:26 system,info device changed by admin
00:26:26 system,info system identity changed by admin
00:26:26 interface,info ether1 link up (speed 100M, full duplex)
00:27:14 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:27:14 system,info,account user admin logged in via local
/file print

Code: Select all

[admin@MikroTik_HA_A_ACTIVE] > /file print
# NAME TYPE SIZE CREATION-TIME
0 HA_init.rsc script 28.2KiB jan/02/1970 00:25:20
1 HA_backup_beforeHA.backup backup 37.4KiB jan/02/1970 00:26:11
2 HA_backup_beforeHA.rsc script 30.0KiB jan/02/1970 00:26:13
3 pub directory jan/02/1970 00:26:13
4 HA_boot_interface_print.txt .txt file 1909 jan/02/1970 00:26:14
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 6:53 pm

/log print

Code: Select all

[admin@MikroTik_HA_A_ACTIVE] > /log print
00:25:28 system,info router rebooted
...
00:26:14 script,warning ha_startup: START
00:26:14 script,warning ha_startup: 0.1
00:26:14 script,warning ha_startup: 0.2
00:26:14 script,warning ha_startup: 0.3
...
00:26:14 script,warning ha_startup: version 0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:26:14 script,warning ha_startup: 1 ether8
00:26:14 system,info new script scheduled by admin
00:26:14 system,info new script scheduled by admin
00:26:14 script,warning ha_startup: 2
00:26:14 system,info device changed by admin
00:26:14 system,info ip service changed by admin
00:26:14 system,info device changed by admin
00:26:14 script,warning ha_startup: 2.1 1
00:26:14 script,warning ha_startup: 2.2 1
00:26:14 script,warning ha_startup: 3 74:4D:28:C7:80:BB 1
00:26:14 script,warning ha_startup: 3.1 74:4D:28:C7:80:BB 1
00:26:14 script,warning I AM A
00:26:14 system,info address added by admin
00:26:14 system,info route added by admin
00:26:14 script,warning ha_startup: 4
00:26:14 script,warning ha_startup: 4.1
00:26:14 system,info filter rule added by admin
00:26:15 system,info filter rule added by admin
00:26:15 script,warning ha_startup: 4.3
00:26:15 script,warning ha_startup: 5
00:26:15 system,info device added by admin
00:26:15 system,info address added by admin
00:26:15 script,warning ha_startup: 6
00:26:15 system,info new script scheduled by admin
00:26:15 system,info new script scheduled by admin
00:26:15 system,info new script scheduled by admin
00:26:15 script,warning ha_startup: 7
00:26:15 system,info user ha added by admin
00:26:15 script,warning ha_startup: 8
00:26:15 system,info tile rb settings changed by admin
00:26:15 system,info ip service changed by admin
00:26:15 vrrp,info HA_VRRP now BACKUP
00:26:19 interface,info ether8 link up (speed 100M, full duplex)
00:26:24 vrrp,info HA_VRRP now MASTER, master down timer
00:26:26 interface,info ether1 link up (speed 100M, full duplex)
00:27:14 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:27:14 system,info,account user admin logged in via local
/file print

Code: Select all

[admin@MikroTik_HA_A_ACTIVE] > /file print
# NAME TYPE SIZE CREATION-TIME
0 HA_init.rsc script 28.2KiB jan/02/1970 00:25:20
1 HA_backup_beforeHA.backup backup 37.4KiB jan/02/1970 00:26:11
2 HA_backup_beforeHA.rsc script 30.0KiB jan/02/1970 00:26:13
3 pub directory jan/02/1970 00:26:13
4 HA_boot_interface_print.txt .txt file 1909 jan/02/1970 00:26:14
This log looks like it is from when A is ACTIVE? Can you do the same thing when it is STANDBY? or do you think it should be STANDBY here?
Can you get the two to the state where switch role would fail (but don't run any scripts yet) and produce a /log print from each.
It seems like this log shows it working correctly but it is hard to tell without observing the state of both.
Is there any notable difference in configuration on these vs the github ha-mikrotik code or any other global configuration?
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 7:19 pm

Thanks again Nathan1.
The init script from github is the same, i haven't added or removed any line.
Yes you're right, this is router A active. This is the same with router A when it becomes standby itself after bootstrap of router B
I've doing some tests, and if i enable ftp server on router A and i set vrrp priority of router A to 101, then A becomes active and all is working well. Also, i have to update the 60 seconds delay on Switchrole script because the router takes more time to reboot.

Log print (without exec any command)

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > /log print
00:25:28 system,info router rebooted
00:25:28 health,warning PSU2 entered state FAIL
00:25:33 interface,info ether8 link up (speed 100M, full duplex)
00:25:33 interface,info ether1 link up (speed 100M, full duplex)
00:26:02 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:26:03 system,info,account user admin logged in via local
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info new script added by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:07 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:08 system,info changed script settings by admin
00:26:11 system,info new script added by admin
00:26:13 smb,info created new share: pub
00:26:14 script,warning ha_startup: START
00:26:14 script,warning ha_startup: 0.1
00:26:14 script,warning ha_startup: 0.2
00:26:14 script,warning ha_startup: 0.3
00:26:14 interface,info ether1 link down
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 interface,info ether8 link down
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 system,info device changed by admin
00:26:14 script,warning ha_startup: version 0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:26:14 script,warning ha_startup: 1 ether8
00:26:14 system,info new script scheduled by admin
00:26:14 system,info new script scheduled by admin
00:26:14 script,warning ha_startup: 2
00:26:14 system,info device changed by admin
00:26:14 system,info ip service changed by admin
00:26:14 system,info device changed by admin
00:26:14 script,warning ha_startup: 2.1 1
00:26:14 script,warning ha_startup: 2.2 1
00:26:14 script,warning ha_startup: 3 74:4D:28:C7:80:BB 1
00:26:14 script,warning ha_startup: 3.1 74:4D:28:C7:80:BB 1
00:26:14 script,warning I AM A
00:26:14 system,info address added by admin
00:26:14 system,info route added by admin
00:26:14 script,warning ha_startup: 4
00:26:14 script,warning ha_startup: 4.1
00:26:14 system,info filter rule added by admin
00:26:15 system,info filter rule added by admin
00:26:15 script,warning ha_startup: 4.3
00:26:15 script,warning ha_startup: 5
00:26:15 system,info device added by admin
00:26:15 system,info address added by admin
00:26:15 script,warning ha_startup: 6
00:26:15 system,info new script scheduled by admin
00:26:15 system,info new script scheduled by admin
00:26:15 system,info new script scheduled by admin
00:26:15 script,warning ha_startup: 7
00:26:15 system,info user ha added by admin
00:26:15 script,warning ha_startup: 8
00:26:15 system,info tile rb settings changed by admin
00:26:15 system,info ip service changed by admin
00:26:15 vrrp,info HA_VRRP now BACKUP
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info device changed by admin
00:26:15 system,info system identity changed by admin
00:26:17 system,info,account user admin logged out from 20:CF:30:C1:88:C7 via winbox
00:26:17 system,info,account user admin logged out via local
00:26:19 interface,info ether8 link up (speed 100M, full duplex)
00:26:24 vrrp,info HA_VRRP now MASTER, master down timer
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:24 system,info device changed by admin
00:26:25 system,info device changed by admin
00:26:26 system,info device changed by admin
00:26:26 system,info system identity changed by admin
00:26:26 interface,info ether1 link up (speed 100M, full duplex)
00:27:14 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:27:14 system,info,account user admin logged in via local
01:16:27 system,info script removed by admin
01:16:27 system,info new script added by admin
01:20:01 info fetch: file "HA_standby-haConfigVer.txt" downloaded
01:21:26 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
01:21:26 interface,info ether1 link down
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info device changed by admin
01:21:26 system,info system identity changed by admin
01:21:28 system,info,account user admin logged out from 20:CF:30:C1:88:C7 via winbox
01:21:28 system,info,account user admin logged out via local
01:22:05 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
01:22:05 system,info,account user admin logged in via local
File print

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > /file print
# NAME TYPE SIZE CREATION-TIME
0 HA_init.rsc script 28.2KiB jan/02/1970 00:25:20
1 HA_backup_beforeHA.backup backup 37.4KiB jan/02/1970 00:26:11
2 HA_backup_beforeHA.rsc script 30.0KiB jan/02/1970 00:26:13
3 pub directory jan/02/1970 00:26:13
4 HA_boot_interface_print.txt .txt file 1909 jan/02/1970 00:26:14
5 HA_get-version.txt .txt file 51 jan/02/1970 01:20:00
6 HA_current.rsc script 31.9KiB jan/02/1970 01:15:01
7 HA_mkdirs.txt .txt file 393 jan/02/1970 01:20:02
8 HA_rsa file 1704 jan/02/1970 01:20:08
9 HA_rsa.pub ssh key 451 jan/02/1970 01:20:08
10 HA_dsa file 668 jan/02/1970 01:20:08
11 HA_dsa.pub ssh key 604 jan/02/1970 01:20:08
12 HA_run-after-hastartup.txt .txt file 132 jan/02/1970 01:20:10
13 HA_b2s.rsc script 31.9KiB jan/02/1970 01:20:12
14 HA_b2s.backup backup 44.1KiB jan/02/1970 01:20:12
15 HA_restore-backup.rsc.txt .txt file 49 jan/02/1970 01:20:14
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 7:36 pm

Thanks again Nathan1.
The init script from github is the same, i haven't added or removed any line.
Yes you're right, this is router A active. This is the same with router A when it becomes standby itself after bootstrap of router B
I've doing some tests, and if i enable ftp server on router A and i set vrrp priority of router A to 101, then A becomes active and all is working well. Also, i have to update the 60 seconds delay on Switchrole script because the router takes more time to reboot.
Aha, I do see something wrong. Your ha_startup is never making it beyond 8.
00:26:15 script,warning ha_startup: 8
https://github.com/svlsResearch/ha-mikr ... cript#L153

So it never makes it to where the FTP is enabled:
https://github.com/svlsResearch/ha-mikr ... cript#L174

and we never see ha_startup: DONE

Can you show me "/system resource print" and "/system routerboard print"?

There seems to be something in this code that isn't working on your unit:
#So you dont get annoyed with constant beeping - try catch because this may fail on some platforms (x86).
:do {/system routerboard settings set silent-boot=yes} on-error={};

:foreach service in=[:toarray "ftp"] do={
   :local serviceAddresses ""
   :foreach k in=[/ip service get [find name=$service] address] do={
      :if ($k != "$haAddressA/32" and $k != "$haAddressB/32" and $k != "$haAddressVRRP/32") do={
         :set serviceAddresses "$serviceAddresses,$k"
      }
   }
   :set serviceAddresses "$serviceAddresses,$haAddressA,$haAddressB,$haAddressVRRP"
   /ip service set [find name=$service] address=[:toarray $serviceAddresses]
}
Can you try to run those lines by hand and see if you can get an error produced? It might be easier to upload it to a file and then /import it interactively.
A few :put traces might also help. Let me know if you need help with that.

Edit: can you please also show "/ip service export" as well.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:07 pm

All commands run on router A as standby
system resource print

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > /system resource print
uptime: 1h44m47s
version: 6.44.5 (long-term)
build-time: Jul/04/2019 10:32:21
factory-software: 6.43.10
free-memory: 3673.3MiB
total-memory: 3968.0MiB
cpu: tilegx
cpu-count: 36
cpu-frequency: 1200MHz
cpu-load: 0%
free-hdd-space: 885.0MiB
total-hdd-space: 1024.0MiB
architecture-name: tile
board-name: CCR1036-8G-2S+
platform: MikroTik
system resource print

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > /system routerboard print
routerboard: yes
model: CCR1036-8G-2S+
revision: r2
serial-number: 968E0A064382
firmware-type: tilegx
factory-firmware: 6.44.3
current-firmware: 6.44.3
upgrade-firmware: 6.44.5
ip service export

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > /ip service export
# jan/02/1970 02:11:49 by RouterOS 6.44.5
# software id = JZU2-0TFL
#
# model = CCR1036-8G-2S+
# serial number = 968E0A064382
/ip service
set ftp address=169.254.23.1/32,169.254.23.2/32,169.254.23.10/32 disabled=yes
And the result of those lines copy&paste, no errors reported neither log file or screen, and ftp remains disabled

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > :do {/system routerboard settings set silent-boot=yes} on-error={};
[admin@MikroTik_HA_A_STANDBY] > :foreach service in=[:toarray "ftp"] do={
{... :local serviceAddresses ""
{... :foreach k in=[/ip service get [find name=$service] address] do={
{{... :if ($k != "$haAddressA/32" and $k != "$haAddressB/32" and $k != "$haAddressVRRP/32") do={
{{{... :set serviceAddresses "$serviceAddresses,$k"
{{{... }
{{... }
{... :set serviceAddresses "$serviceAddresses,$haAddressA,$haAddressB,$haAddressVRRP"
{... /ip service set [find name=$service] address=[:toarray $serviceAddresses]
{... }
[admin@MikroTik_HA_A_STANDBY] >
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:23 pm

Any errors running this?
:if ([:len [/file find where name="HA_run-after-hastartup.rsc"]] > 0) do={
   /import HA_run-after-hastartup.rsc
}
/delay 5
#We need FTP to do our HA work
/ip service set [find name="ftp"] disabled=no
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:32 pm

No errors shown again and ftp is enabled

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > :if ([:len [/file find where name="HA_run-after-hastartup.rsc"]] > 0) do={
{... /import HA_run-after-hastartup.rsc
{... }
[admin@MikroTik_HA_A_STANDBY] > /delay 5
[admin@MikroTik_HA_A_STANDBY] > #We need FTP to do our HA work
[admin@MikroTik_HA_A_STANDBY] > /ip service set [find name="ftp"] disabled=no
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:37 pm

Very odd. Everything seems to be working but for some reason the script is not completing. I think I need to put some more trace in and give you another build to try to track this down. I can put a test release on github in about 2 hours.

Just to confirm, if you enable ftp on the standby. Does a $HAPushBackup work from the active(B) to standby(A)? And it always works from A to B?
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:49 pm

$HAPushBackup seems to do nothing.
If i run on active (router B), i can see at log file how user ha has logged in and out, but if i run $HAPushBackup on standby (router A), i can't see any log on active (router B)
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:55 pm

$HAPushBackup seems to do nothing.
If i run on active (router B), i can see at log file how user ha has logged in and out, but if i run $HAPushBackup on standby (router A), i can't see any log on active (router B)
Sorry, wrong command. Try $HAPushStandby. It will only work from active to standby. Can you try it in both cases of A being master and then B being master? You will need to enable FTP by hand when A is standby.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 8:59 pm

The result of $HAPushStandby, router A (standby) reboots

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > $HAPushStandby
mkdirCode: :foreach k in=[/file find type!="directory"] do={ :local xferfile [/file get $k name]; if ([:pick "$xferfile" 0 3] != "HA_") do={ :put "removing $
xferfile"; /file remove $k; } };
/delay 2;
:do { /ip smb shares add comment=HA_AUTO name=mkdir disabled=yes directory=/skins } on-error={}
/ip smb shares set [find comment=HA_AUTO] directory="pub"
/ip smb shares set [find comment=HA_AUTO] directory="skins"
/ip smb shares remove [find comment=HA_AUTO]
end_mkDirCode
status: finished
downloaded: 0KiB
total: 0KiB
duration: 3s

status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

status: finished
downloaded: 1KiBC-z pause]
total: 1KiB
duration: 1s

status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

Saving system configuration
Configuration backup saved
status: finished
downloaded: 32KiB-z pause]
total: 32KiB
duration: 1s

status: finished
downloaded: 44KiB-z pause]
total: 44KiB
duration: 1s

status: failed

OK - status failed is OK from last fetch, standby is rebooting.
[admin@MikroTik_HA_B_ACTIVE] >
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:02 pm

The result of $HAPushStandby, router A (standby) reboots

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > $HAPushStandby
mkdirCode: :foreach k in=[/file find type!="directory"] do={ :local xferfile [/file get $k name]; if ([:pick "$xferfile" 0 3] != "HA_") do={ :put "removing $
xferfile"; /file remove $k; } };
/delay 2;
:do { /ip smb shares add comment=HA_AUTO name=mkdir disabled=yes directory=/skins } on-error={}
/ip smb shares set [find comment=HA_AUTO] directory="pub"
/ip smb shares set [find comment=HA_AUTO] directory="skins"
/ip smb shares remove [find comment=HA_AUTO]
end_mkDirCode
status: finished
downloaded: 0KiB
total: 0KiB
duration: 3s

status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

status: finished
downloaded: 1KiBC-z pause]
total: 1KiB
duration: 1s

status: finished
downloaded: 0KiBC-z pause]
total: 0KiB
duration: 1s

Saving system configuration
Configuration backup saved
status: finished
downloaded: 32KiB-z pause]
total: 32KiB
duration: 1s

status: finished
downloaded: 44KiB-z pause]
total: 44KiB
duration: 1s

status: failed

OK - status failed is OK from last fetch, standby is rebooting.
[admin@MikroTik_HA_B_ACTIVE] >
Looks good. Did you need to enable ftp by hand on A for this? When it comes back after, are you able to issue another $HAPushStandby or do you need to enable FTP again? Our debugging so far suggests you need to enable FTP but I just wanted to double check.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:10 pm

Ftp was enabled when i put that code

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > :if ([:len [/file find where name="HA_run-after-hastartup.rsc"]] > 0) do={
{... /import HA_run-after-hastartup.rsc
{... }
[admin@MikroTik_HA_A_STANDBY] > /delay 5
[admin@MikroTik_HA_A_STANDBY] > #We need FTP to do our HA work
[admin@MikroTik_HA_A_STANDBY] > /ip service set [find name="ftp"] disabled = no
And after that, ftp is always enabled
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:16 pm

Hi nathan1

This solution can work on the Chr version?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:21 pm

Ftp was enabled when i put that code

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > :if ([:len [/file find where name="HA_run-after-hastartup.rsc"]] > 0) do={
{... /import HA_run-after-hastartup.rsc
{... }
[admin@MikroTik_HA_A_STANDBY] > /delay 5
[admin@MikroTik_HA_A_STANDBY] > #We need FTP to do our HA work
[admin@MikroTik_HA_A_STANDBY] > /ip service set [find name="ftp"] disabled = no
And after that, ftp is always enabled
Pretty strange. Can you see if you can switch roles reliably now?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:22 pm

Hi nathan1

This solution can work on the Chr version?
I believe there were some folks that tried successfully but I have not personally done it.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:35 pm

$HASwitchRole now is working. Router A becomes active after run that command, but when router B reboots, router A change its role to standby and always router B becomes active router

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > log print
00:25:28 system,info router rebooted
00:25:28 health,warning PSU2 entered state FAIL
00:25:29 script,warning ha_startup: START
00:25:29 script,warning ha_startup: 0.1
00:25:29 script,warning ha_startup: 0.2
00:25:31 vrrp,info HA_VRRP now BACKUP
00:25:31 script,warning ha_startup: 0.3
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 script,warning ha_startup: version 0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:25:31 script,warning ha_startup: 1 ether8
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info system identity changed by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info new script scheduled by admin
00:25:31 system,info new script scheduled by admin
00:25:31 script,warning ha_startup: 2
00:25:31 system,info device changed by admin
00:25:31 system,info address removed by admin
00:25:31 system,info address removed by admin
00:25:31 system,info device removed by admin
00:25:31 system,info filter rule removed by admin
00:25:31 system,info filter rule removed by admin
00:25:31 system,info ip service changed by admin
00:25:31 system,info device changed by admin
00:25:31 script,warning ha_startup: 2.1 1
00:25:31 script,warning ha_startup: 2.2 1
00:25:31 script,warning ha_startup: 3 74:4D:28:C7:80:BB 1
00:25:31 script,warning ha_startup: 3.1 74:4D:28:C7:80:BB 1
00:25:31 script,warning I AM A
00:25:31 system,info address added by admin
00:25:31 system,info route removed by admin
00:25:31 system,info route added by admin
00:25:32 script,warning ha_startup: 4
00:25:32 script,warning ha_startup: 4.1
00:25:32 system,info filter rule added by admin
00:25:32 system,info filter rule added by admin
00:25:32 script,warning ha_startup: 4.3
00:25:32 script,warning ha_startup: 5
00:25:32 system,info device added by admin
00:25:32 system,info address added by admin
00:25:32 script,warning ha_startup: 6
00:25:32 system,info new script scheduled by admin
00:25:32 system,info new script scheduled by admin
00:25:32 system,info new script scheduled by admin
00:25:32 script,warning ha_startup: 7
00:25:32 vrrp,info HA_VRRP now BACKUP
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info system identity changed by admin
00:25:32 system,info user ha removed by admin
00:25:32 system,info user ha added by admin
00:25:32 script,warning ha_startup: 8
00:25:32 system,info tile rb settings changed by admin
00:25:32 system,info ip service changed by admin
00:25:33 interface,info ether8 link up (speed 100M, full duplex)
00:25:37 system,info ip service changed by admin
00:25:37 script,warning ha_startup: DONE
00:25:40 vrrp,info HA_VRRP now MASTER, master down timer
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:41 system,info device changed by admin
00:25:42 system,info device changed by admin
00:25:42 system,info system identity changed by admin
00:25:42 interface,info ether1 link up (speed 100M, full duplex)
00:26:03 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
00:26:03 interface,info ether1 link down
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:04 system,info device changed by admin
00:26:04 system,info system identity changed by admin
00:26:07 vrrp,info HA_VRRP now MASTER, master down timer
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:08 system,info device changed by admin
00:26:09 system,info device changed by admin
00:26:09 system,info system identity changed by admin
00:26:12 interface,info ether1 link up (speed 100M, full duplex)
00:26:33 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:26:34 script,info ha_startup: ha_report_startup debug version=6.44.5 (long-term) firmware=6.44.3 badC=0 goodC=1 delay1C=0 delay2C=0 uptime=00:01:28 isMaster=
true haInitTries=1 haStartupHasRun=00:00:23 haStartupHAVersion=0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:26:34 system,info,account user admin logged in via local
00:27:15 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
00:27:15 interface,info ether1 link down
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info system identity changed by admin
00:27:16 system,info,account user admin logged out from 20:CF:30:C1:88:C7 via winbox
00:27:16 system,info,account user admin logged out via local
00:28:28 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:28:28 system,info,account user admin logged in via local
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:37 pm

Hi nathan1

This solution can work on the Chr version?
Yes, i tried it on chr image 6.41 and it was working fine, now i'm trying to run it on ccr1036 and i'm having some issues with the script.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:46 pm

$HASwitchRole now is working. Router A becomes active after run that command, but when router B reboots, router A change its role to standby and always router B becomes active router

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > log print
00:25:28 system,info router rebooted
00:25:28 health,warning PSU2 entered state FAIL
00:25:29 script,warning ha_startup: START
00:25:29 script,warning ha_startup: 0.1
00:25:29 script,warning ha_startup: 0.2
00:25:31 vrrp,info HA_VRRP now BACKUP
00:25:31 script,warning ha_startup: 0.3
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 script,warning ha_startup: version 0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:25:31 script,warning ha_startup: 1 ether8
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info system identity changed by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info new script scheduled by admin
00:25:31 system,info new script scheduled by admin
00:25:31 script,warning ha_startup: 2
00:25:31 system,info device changed by admin
00:25:31 system,info address removed by admin
00:25:31 system,info address removed by admin
00:25:31 system,info device removed by admin
00:25:31 system,info filter rule removed by admin
00:25:31 system,info filter rule removed by admin
00:25:31 system,info ip service changed by admin
00:25:31 system,info device changed by admin
00:25:31 script,warning ha_startup: 2.1 1
00:25:31 script,warning ha_startup: 2.2 1
00:25:31 script,warning ha_startup: 3 74:4D:28:C7:80:BB 1
00:25:31 script,warning ha_startup: 3.1 74:4D:28:C7:80:BB 1
00:25:31 script,warning I AM A
00:25:31 system,info address added by admin
00:25:31 system,info route removed by admin
00:25:31 system,info route added by admin
00:25:32 script,warning ha_startup: 4
00:25:32 script,warning ha_startup: 4.1
00:25:32 system,info filter rule added by admin
00:25:32 system,info filter rule added by admin
00:25:32 script,warning ha_startup: 4.3
00:25:32 script,warning ha_startup: 5
00:25:32 system,info device added by admin
00:25:32 system,info address added by admin
00:25:32 script,warning ha_startup: 6
00:25:32 system,info new script scheduled by admin
00:25:32 system,info new script scheduled by admin
00:25:32 system,info new script scheduled by admin
00:25:32 script,warning ha_startup: 7
00:25:32 vrrp,info HA_VRRP now BACKUP
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info system identity changed by admin
00:25:32 system,info user ha removed by admin
00:25:32 system,info user ha added by admin
00:25:32 script,warning ha_startup: 8
00:25:32 system,info tile rb settings changed by admin
00:25:32 system,info ip service changed by admin
00:25:33 interface,info ether8 link up (speed 100M, full duplex)
00:25:37 system,info ip service changed by admin
00:25:37 script,warning ha_startup: DONE
00:25:40 vrrp,info HA_VRRP now MASTER, master down timer
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:41 system,info device changed by admin
00:25:42 system,info device changed by admin
00:25:42 system,info system identity changed by admin
00:25:42 interface,info ether1 link up (speed 100M, full duplex)
00:26:03 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
00:26:03 interface,info ether1 link down
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:04 system,info device changed by admin
00:26:04 system,info system identity changed by admin
00:26:07 vrrp,info HA_VRRP now MASTER, master down timer
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:08 system,info device changed by admin
00:26:09 system,info device changed by admin
00:26:09 system,info system identity changed by admin
00:26:12 interface,info ether1 link up (speed 100M, full duplex)
00:26:33 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:26:34 script,info ha_startup: ha_report_startup debug version=6.44.5 (long-term) firmware=6.44.3 badC=0 goodC=1 delay1C=0 delay2C=0 uptime=00:01:28 isMaster=
true haInitTries=1 haStartupHasRun=00:00:23 haStartupHAVersion=0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:26:34 system,info,account user admin logged in via local
00:27:15 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
00:27:15 interface,info ether1 link down
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info system identity changed by admin
00:27:16 system,info,account user admin logged out from 20:CF:30:C1:88:C7 via winbox
00:27:16 system,info,account user admin logged out via local
00:28:28 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:28:28 system,info,account user admin logged in via local
Do you have these in an isolated lab setup? Would it be possible for you to screen share to me so I can take a look in realtime? We can take it to private message for debugging if that works.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 9:49 pm

$HASwitchRole now is working. Router A becomes active after run that command, but when router B reboots, router A change its role to standby and always router B becomes active router

Code: Select all

[admin@MikroTik_HA_A_STANDBY] > log print
00:25:28 system,info router rebooted
00:25:28 health,warning PSU2 entered state FAIL
00:25:29 script,warning ha_startup: START
00:25:29 script,warning ha_startup: 0.1
00:25:29 script,warning ha_startup: 0.2
00:25:31 vrrp,info HA_VRRP now BACKUP
00:25:31 script,warning ha_startup: 0.3
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 system,info device changed by admin
00:25:31 script,warning ha_startup: version 0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:25:31 script,warning ha_startup: 1 ether8
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info system identity changed by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info script removed from scheduler by admin
00:25:31 system,info new script scheduled by admin
00:25:31 system,info new script scheduled by admin
00:25:31 script,warning ha_startup: 2
00:25:31 system,info device changed by admin
00:25:31 system,info address removed by admin
00:25:31 system,info address removed by admin
00:25:31 system,info device removed by admin
00:25:31 system,info filter rule removed by admin
00:25:31 system,info filter rule removed by admin
00:25:31 system,info ip service changed by admin
00:25:31 system,info device changed by admin
00:25:31 script,warning ha_startup: 2.1 1
00:25:31 script,warning ha_startup: 2.2 1
00:25:31 script,warning ha_startup: 3 74:4D:28:C7:80:BB 1
00:25:31 script,warning ha_startup: 3.1 74:4D:28:C7:80:BB 1
00:25:31 script,warning I AM A
00:25:31 system,info address added by admin
00:25:31 system,info route removed by admin
00:25:31 system,info route added by admin
00:25:32 script,warning ha_startup: 4
00:25:32 script,warning ha_startup: 4.1
00:25:32 system,info filter rule added by admin
00:25:32 system,info filter rule added by admin
00:25:32 script,warning ha_startup: 4.3
00:25:32 script,warning ha_startup: 5
00:25:32 system,info device added by admin
00:25:32 system,info address added by admin
00:25:32 script,warning ha_startup: 6
00:25:32 system,info new script scheduled by admin
00:25:32 system,info new script scheduled by admin
00:25:32 system,info new script scheduled by admin
00:25:32 script,warning ha_startup: 7
00:25:32 vrrp,info HA_VRRP now BACKUP
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info device changed by admin
00:25:32 system,info system identity changed by admin
00:25:32 system,info user ha removed by admin
00:25:32 system,info user ha added by admin
00:25:32 script,warning ha_startup: 8
00:25:32 system,info tile rb settings changed by admin
00:25:32 system,info ip service changed by admin
00:25:33 interface,info ether8 link up (speed 100M, full duplex)
00:25:37 system,info ip service changed by admin
00:25:37 script,warning ha_startup: DONE
00:25:40 vrrp,info HA_VRRP now MASTER, master down timer
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:40 system,info device changed by admin
00:25:41 system,info device changed by admin
00:25:42 system,info device changed by admin
00:25:42 system,info system identity changed by admin
00:25:42 interface,info ether1 link up (speed 100M, full duplex)
00:26:03 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
00:26:03 interface,info ether1 link down
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:03 system,info device changed by admin
00:26:04 system,info device changed by admin
00:26:04 system,info system identity changed by admin
00:26:07 vrrp,info HA_VRRP now MASTER, master down timer
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:07 system,info device changed by admin
00:26:08 system,info device changed by admin
00:26:09 system,info device changed by admin
00:26:09 system,info system identity changed by admin
00:26:12 interface,info ether1 link up (speed 100M, full duplex)
00:26:33 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:26:34 script,info ha_startup: ha_report_startup debug version=6.44.5 (long-term) firmware=6.44.3 badC=0 goodC=1 delay1C=0 delay2C=0 uptime=00:01:28 isMaster=
true haInitTries=1 haStartupHasRun=00:00:23 haStartupHAVersion=0.6 - 8b14022883a2b1e541d1579e70e11b6bd023d601
00:26:34 system,info,account user admin logged in via local
00:27:15 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
00:27:15 interface,info ether1 link down
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info device changed by admin
00:27:15 system,info system identity changed by admin
00:27:16 system,info,account user admin logged out from 20:CF:30:C1:88:C7 via winbox
00:27:16 system,info,account user admin logged out via local
00:28:28 system,info,account user admin logged in from 20:CF:30:C1:88:C7 via winbox
00:28:28 system,info,account user admin logged in via local
Do you have these in an isolated lab setup? Would it be possible for you to screen share to me so I can take a look in realtime? We can take it to private message for debugging if that works.
00:26:03 vrrp,info HA_VRRP now BACKUP, got higher priority 100 from 169.254.23.2
Actually, can you do "/interface vrrp print" on both? Did you keep that vrrp priority change that you temporarily added? They should both be 100 and it should be reset by ha_startup, ha-mikrotik does not support different VRRP priorities - it has no preference for A vs B and cannot currently support this.
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:23 pm

Hi My last time I had played with this was very time ago

I am trying to play it again but I can't find it to make it work

I got stuck after importing the file to load the scripts them I do not what to do it

I past this on the terminal but nothing happens
`$HAInstall interface="ether3" macA="00:0C:29:42:A6:67" macB="00:0C:29:D8:83:02" password="1q2w3e4r5t6y"`
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:29 pm

Hi My last time I had played with this was very time ago

I am trying to play it again but I can't find it to make it work

I got stuck after importing the file to load the scripts them I do not what to do it

I past this on the terminal but nothing happens
`$HAInstall interface="ether3" macA="00:0C:29:42:A6:67" macB="00:0C:29:D8:83:02" password="1q2w3e4r5t6y"`
Did you /import HA_init.rsc ? Were there any errors?
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:30 pm

yes I imported
no no error /no logs
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:32 pm

yes I imported
no no error /no logs
Did it say anything? like "Script file loaded and executed successfully"?
What RouterOS version?
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:39 pm

last stable one

When I load the ha_init.rsc I got this msg
that was loaded
if I go to the sys scripts I see a bunch of scripts added
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:40 pm

last stable one

When I load the ha_init.rsc I got this msg
that was loaded
if I go to the sys scripts I see a bunch of scripts added
Can you confirm the version? It does not work beyond 6.44.5 right now (see bbs2web post above).
What does " :put $HAInstall" show?
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:49 pm

6.45.6
When I run I just got a blank space like is was no value


"
[admin@MikroTik1] > :put $HAInstall

[admin@MikroTik1] >

"
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Sep 13, 2019 10:59 pm

6.45.6
When I run I just got a blank space like is was no value


"
[admin@MikroTik1] > :put $HAInstall

[admin@MikroTik1] >

"
Please try with 6.44.5 if you want to try it.
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Sep 16, 2019 8:30 am

Actually, can you do "/interface vrrp print" on both? Did you keep that vrrp priority change that you temporarily added? They should both be 100 and it should be reset by ha_startup, ha-mikrotik does not support different VRRP priorities - it has no preference for A vs B and cannot currently support this.
/interface vrrp print

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > interface vrrp print
Flags: X - disabled, I - invalid, R - running, M - master, B - backup
# NAME INTERFACE MAC-ADDRESS VRID PRIORITY INTERVAL VERSION V3-PROTOCOL
0 RM HA_VRRP ether8 00:00:5E:00:01:01 1 100 1s 3 ipv4
If I change priority of router A by hand, A becomes active router, but if I run $HASwitchRole, the change is reverted and it's 100 back again, that's correct.
I can share my screen no problem, let's talk by PM if you want, thanks a lot for your time Nathan1
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Sep 16, 2019 3:32 pm

Actually, can you do "/interface vrrp print" on both? Did you keep that vrrp priority change that you temporarily added? They should both be 100 and it should be reset by ha_startup, ha-mikrotik does not support different VRRP priorities - it has no preference for A vs B and cannot currently support this.
/interface vrrp print

Code: Select all

[admin@MikroTik_HA_B_ACTIVE] > interface vrrp print
Flags: X - disabled, I - invalid, R - running, M - master, B - backup
# NAME INTERFACE MAC-ADDRESS VRID PRIORITY INTERVAL VERSION V3-PROTOCOL
0 RM HA_VRRP ether8 00:00:5E:00:01:01 1 100 1s 3 ipv4
If I change priority of router A by hand, A becomes active router, but if I run $HASwitchRole, the change is reverted and it's 100 back again, that's correct.
I can share my screen no problem, let's talk by PM if you want, thanks a lot for your time Nathan1
So if you leave all of the priorities alone - are you able to $HASwitchRole reliably now or is there still a problem on one of them? If so, can you outline which switchrole fails and provide a log both master and standby when you get into that state?
 
jandres
just joined
Posts: 17
Joined: Thu May 02, 2019 1:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Sep 16, 2019 11:07 pm

The problem is after switchrole. When active reboots, always B becomes active router. The sequence is as follows:

Router B(active) - switchrole - router A (standby) reboots - router B reboots also after a 60s delay - router A is online again at first as active - router B is online a little bit time that router A - router B becomes active - router A becomes standby again

So, switchrole is performed, but automatically reverted after reboot. Router A is active only a few seconds.

Regards
 
dmitk
just joined
Posts: 4
Joined: Tue Sep 24, 2019 2:58 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Sep 24, 2019 3:35 pm

Hello,
trying to install ha-mikrotik on two fresh long term ROS 6.44.5 CCR1036-8G-2S+ :
On the first node:
[admin@HA-1] > /import HA_init.rsc

Script file loaded and executed successfully
[admin@HA-1] > $HAInstall interface="ether8" macA="CC:2D:E0:BD:9F:A8" macB="CC:2D:E0:BD:9F:D4" password="passw0rd"
CC:2D:E0:BD:9F:A8
CC:2D:E0:BD:9F:D4
SEQ HOST SIZE TTL TIME STATUS
0 CC:2D:E0:BD:9F:D4 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "CC:2D:E0:BD:9F:A8" and $mac = "CC:2D:E0:BD:9F:D4") do={
:error "Interface ether8 MAC $mac does not match (A=CC:2D:E0:BD:9F:A8 or B=CC:2D:E0:BD:9F:D4) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###

On second node:
Used part of config above, node restarted and nothing happens. If I run manually HA_bootstrap.rsc on second node, I'll get an error:
[admin@MikroTik] > /import HA_bootstrap.rsc
failure: local address cannot be 0.0.0.0
[admin@MikroTik] >

also user ha has not been created on the second node.

I have seen above that people successfully used ha-mikrotik on ROS 6.44.5
Any suggestions ?
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Sep 25, 2019 12:55 pm

Thanks for this great script, i try to get it working on two CCR1009-7G-1C-1S+.
After Some troubles it seems to work.

Another question, is there any possibility to use the usr-led for showing which router is active and which one is passive?

Thanks in advance.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Sep 25, 2019 2:22 pm

The problem is after switchrole. When active reboots, always B becomes active router. The sequence is as follows:

Router B(active) - switchrole - router A (standby) reboots - router B reboots also after a 60s delay - router A is online again at first as active - router B is online a little bit time that router A - router B becomes active - router A becomes standby again

So, switchrole is performed, but automatically reverted after reboot. Router A is active only a few seconds.

Regards
Email me at nathan4321 at Google’s mail service, if you want to setup a time to debug this in screen sharing.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Sep 25, 2019 2:24 pm

Hello,
trying to install ha-mikrotik on two fresh long term ROS 6.44.5 CCR1036-8G-2S+ :
On the first node:
[admin@HA-1] > /import HA_init.rsc

Script file loaded and executed successfully
[admin@HA-1] > $HAInstall interface="ether8" macA="CC:2D:E0:BD:9F:A8" macB="CC:2D:E0:BD:9F:D4" password="passw0rd"
CC:2D:E0:BD:9F:A8
CC:2D:E0:BD:9F:D4
SEQ HOST SIZE TTL TIME STATUS
0 CC:2D:E0:BD:9F:D4 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "CC:2D:E0:BD:9F:A8" and $mac = "CC:2D:E0:BD:9F:D4") do={
:error "Interface ether8 MAC $mac does not match (A=CC:2D:E0:BD:9F:A8 or B=CC:2D:E0:BD:9F:D4) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###

On second node:
Used part of config above, node restarted and nothing happens. If I run manually HA_bootstrap.rsc on second node, I'll get an error:
[admin@MikroTik] > /import HA_bootstrap.rsc
failure: local address cannot be 0.0.0.0
[admin@MikroTik] >

also user ha has not been created on the second node.

I have seen above that people successfully used ha-mikrotik on ROS 6.44.5
Any suggestions ?
Are these MACs correct? They should be the MACs of each ether8.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Sep 25, 2019 2:27 pm

Thanks for this great script, i try to get it working on two CCR1009-7G-1C-1S+.
After Some troubles it seems to work.

Another question, is there any possibility to use the usr-led for showing which router is active and which one is passive?

Thanks in advance.
You can create two scripts that will automatically be called to do this. “on_backup” and “on_master”.
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 11:44 am

Hi,
but i need to add those scripts on the vrrp (on Master and on Backup) or?
Or is there any logic included, that all scripts with *_on_backup will be executed?
Thanks
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 1:44 pm

Hi,
but i need to add those scripts on the vrrp (on Master and on Backup) or?
Or is there any logic included, that all scripts with *_on_backup will be executed?
Thanks
Only on the master, you never make changes on the standby once you have setup ha-mikrotik. Add the scripts on the master and either wait for a normal sync or $HASyncStandby on from the master to force it sooner. Note that the LEDs will only be right when the pair is in sync, there will be periods when the current standby has the LEDs of the master until it completes booting.
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 2:02 pm

Thanks!
It's working now :)

Just another question, if I execute SwitchRole i get the following output:
/delay 2;
:do { /ip smb shares add comment=HA_AUTO name=mkdir disabled=yes directory=/skins } on-error={}
/ip smb shares set [find comment=HA_AUTO] directory="pub"
/ip smb shares set [find comment=HA_AUTO] directory="skins"
/ip smb shares remove [find comment=HA_AUTO]
 end_mkDirCode
      status: finished
  downloaded: 0KiB
       total: 0KiB
    duration: 3s

      status: finished
  downloaded: 0KiBC-z pause]
       total: 0KiB
    duration: 1s

      status: finished
  downloaded: 1KiBC-z pause]
       total: 1KiB
    duration: 1s

      status: finished
  downloaded: 0KiBC-z pause]
       total: 0KiB
    duration: 1s

Saving system configuration
Configuration backup saved
      status: finished
  downloaded: 32KiB-z pause]
       total: 32KiB
    duration: 1s

      status: finished
  downloaded: 45KiB-z pause]
       total: 45KiB
    duration: 1s

  status: failed

OK - status failed is OK from last fetch, standby is rebooting.
delaying 60
Just asking because the status : failed
Is that a critical issue? Or is there any configuration error.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 2:33 pm

Thanks!
It's working now :)

Just another question, if I execute SwitchRole i get the following output:
/delay 2;
:do { /ip smb shares add comment=HA_AUTO name=mkdir disabled=yes directory=/skins } on-error={}
/ip smb shares set [find comment=HA_AUTO] directory="pub"
/ip smb shares set [find comment=HA_AUTO] directory="skins"
/ip smb shares remove [find comment=HA_AUTO]
 end_mkDirCode
      status: finished
  downloaded: 0KiB
       total: 0KiB
    duration: 3s

      status: finished
  downloaded: 0KiBC-z pause]
       total: 0KiB
    duration: 1s

      status: finished
  downloaded: 1KiBC-z pause]
       total: 1KiB
    duration: 1s

      status: finished
  downloaded: 0KiBC-z pause]
       total: 0KiB
    duration: 1s

Saving system configuration
Configuration backup saved
      status: finished
  downloaded: 32KiB-z pause]
       total: 32KiB
    duration: 1s

      status: finished
  downloaded: 45KiB-z pause]
       total: 45KiB
    duration: 1s

  status: failed

OK - status failed is OK from last fetch, standby is rebooting.
delaying 60
Just asking because the status : failed
Is that a critical issue? Or is there any configuration error.
Expected, it is fine. Note the message “ OK - status failed is OK from last fetch, standby is rebooting.”.

It happens because the service is abruptly stopped during the standby reboot while fetch is still connected.
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 8:16 pm

OK,
One Last question, would it be possible to secure the Sync Ports especually through EOIP Tunnel or something?

I need to make a Setup where the routers are not placed in the same room.

And is it possible to build IPSec Tunnels with certificates? With PSK it works great in a failover, but will the certificates also get synced between activ and passive?

Thanks!
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 8:44 pm

OK,
One Last question, would it be possible to secure the Sync Ports especually through EOIP Tunnel or something?

I need to make a Setup where the routers are not placed in the same room.

And is it possible to build IPSec Tunnels with certificates? With PSK it works great in a failover, but will the certificates also get synced between activ and passive?

Thanks!
No built in way to secure the sync ports beyond the VRRP auth. For them to be in different locations, don’t you need to extend layer2 anyway? Folks have reported extending the layer2 for the sync ports with a switch, but I have not personally done this.

For certificates, you will want to write a script to import keys from the filesystem into the certificate store. Files on the master will be synchronized and you can then have a startup script import them. I do this on all of my pairs for OpenVPN certificates.
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 9:03 pm

OK, yes the layer2 ist extented but i would use a complete different way for Sync. So If i Install a EOIP Tunnel before hainstall would this Work if i select the EOIP Interface?

May you Share your Script?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 9:17 pm

OK, yes the layer2 ist extented but i would use a complete different way for Sync. So If i Install a EOIP Tunnel before hainstall would this Work if i select the EOIP Interface?

May you Share your Script?
No, that won't work and I'd strongly advise against trying to do something like that. The sync port is critical to operation and any additional failure modes increases the chances of split brain. Adding a switch already increases these odds.
Can you help me understand what you are trying to do by "securing" the sync port and what you mean by "secure"?

For certificate imports, pretty straight forward scheduler task on startup (runs on both and the files once placed on the master filesystem will be pushed from master):
/system scheduler
add name=on_reboot_fix_certifcates on-event="/certificate remove [find]\
    \n/certificate import file-name=SOMEWHERE.com.crt passphrase=\"\"\
    \n/certificate import file-name=SOMEWHERE.com.key passphrase=PASSPHRASE" \
    policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-time=startup
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 9:58 pm

Thanks for sharing your script.

Yes, i was thinking about a man in the middle Attack on this Board. As far as i know the file Sync goes through smb or FTP? So with man in the middle you can gather information about PSK etc.

Or i am completley wrong?

I know it depends on my different Setup were the routers Not placed in the same room.

I agree with you more devices in the Sync Channel reduces the available.

Thanks for discussion!
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 10:09 pm

Thanks for sharing your script.

Yes, i was thinking about a man in the middle Attack on this Board. As far as i know the file Sync goes through smb or FTP? So with man in the middle you can gather information about PSK etc.

Or i am completley wrong?

I know it depends on my different Setup were the routers Not placed in the same room.

I agree with you more devices in the Sync Channel reduces the available.

Thanks for discussion!
Yes, it is true that the sync uses FTP. SMB is not actually used for communication but it is used as a hack to mkdir because Mikrotik has not exposed another way to create a directory (last I checked).

A MITM attack could gather quite a bit of information but a MITM attack in this scenario could also potentially attack the VRRP layer and cause all sorts of network issues. Is this not going to be on a trusted layer 2 network? Or your concerns are beyond that and simply trying to reduce the attack surface of cleartext information?

The reason why I had to use FTP is due to this feature: https://wiki.mikrotik.com/wiki/Manual:C ... tic_Import
Unless something has changed from when I first developed it, FTP is the only method that supports automatic imports. The entire ha-mikrotik setup hinges on this functionality right now.
It is a bummer that they haven't exposed more robust functionality onboard that I can leverage...but that goes for why ha-mikrotik had to be created in the first place too. It would be nice if they had this all built in.

ha-mikrotik was designed to have a point to point physical connection between the two devices.

So unfortunately, I don't know an easy way around this. You really want to make sure that the layer 2 between these devices can be trusted.

I know this isn't the solution you were looking for but hopefully it gives some clarity.
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Sep 26, 2019 10:40 pm

Yes, i would use a dark fiber for the Sync. But your right, the Problem ist the cleartext...

Thanks

Especually i get an Idea for Monitoring those Connection.
 
DMccabe
just joined
Posts: 3
Joined: Tue Apr 26, 2016 8:59 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Oct 17, 2019 5:39 pm

Hello,
trying to install ha-mikrotik on two fresh long term ROS 6.44.5 CCR1036-8G-2S+ :
On the first node:
[admin@HA-1] > /import HA_init.rsc

Script file loaded and executed successfully
[admin@HA-1] > $HAInstall interface="ether8" macA="CC:2D:E0:BD:9F:A8" macB="CC:2D:E0:BD:9F:D4" password="passw0rd"
CC:2D:E0:BD:9F:A8
CC:2D:E0:BD:9F:D4
SEQ HOST SIZE TTL TIME STATUS
0 CC:2D:E0:BD:9F:D4 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "CC:2D:E0:BD:9F:A8" and $mac = "CC:2D:E0:BD:9F:D4") do={
:error "Interface ether8 MAC $mac does not match (A=CC:2D:E0:BD:9F:A8 or B=CC:2D:E0:BD:9F:D4) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###

On second node:
Used part of config above, node restarted and nothing happens. If I run manually HA_bootstrap.rsc on second node, I'll get an error:
[admin@MikroTik] > /import HA_bootstrap.rsc
failure: local address cannot be 0.0.0.0
[admin@MikroTik] >

also user ha has not been created on the second node.

I have seen above that people successfully used ha-mikrotik on ROS 6.44.5
Any suggestions ?
Has there been any more progress on this? I'm stuck at the same position, and am about to completely disassemble the script and start from scratch if it fails on this error.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Oct 17, 2019 7:33 pm

Hello,
trying to install ha-mikrotik on two fresh long term ROS 6.44.5 CCR1036-8G-2S+ :
On the first node:
[admin@HA-1] > /import HA_init.rsc

Script file loaded and executed successfully
[admin@HA-1] > $HAInstall interface="ether8" macA="CC:2D:E0:BD:9F:A8" macB="CC:2D:E0:BD:9F:D4" password="passw0rd"
CC:2D:E0:BD:9F:A8
CC:2D:E0:BD:9F:D4
SEQ HOST SIZE TTL TIME STATUS
0 CC:2D:E0:BD:9F:D4 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "CC:2D:E0:BD:9F:A8" and $mac = "CC:2D:E0:BD:9F:D4") do={
:error "Interface ether8 MAC $mac does not match (A=CC:2D:E0:BD:9F:A8 or B=CC:2D:E0:BD:9F:D4) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###

On second node:
Used part of config above, node restarted and nothing happens. If I run manually HA_bootstrap.rsc on second node, I'll get an error:
[admin@MikroTik] > /import HA_bootstrap.rsc
failure: local address cannot be 0.0.0.0
[admin@MikroTik] >

also user ha has not been created on the second node.

I have seen above that people successfully used ha-mikrotik on ROS 6.44.5
Any suggestions ?
Has there been any more progress on this? I'm stuck at the same position, and am about to completely disassemble the script and start from scratch if it fails on this error.
The user emailed me and I attempted to help them but they never responded. Can you provide me access to your kit? I still haven't seen this in the wild and I am unable to reproduce it.
 
TeslaBMWandTheRest
just joined
Posts: 2
Joined: Wed Oct 23, 2019 9:38 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Oct 23, 2019 10:20 pm

Hi Nathan,

Nice project, I'm impressed. I'm preparing to switch from my no longer supported SonicWall NSA 2400 to two CCR1009-7G-1C-1S+ (2nd revision). I was trying out your project. As far as I can tell at this moment it works on 6.44.5. The configuration is synchronised, and manually rebooting the Active makes the Backup Active.

To make it work, I needed to change your script, I think you have hard coded ether8 as the HA connection. I had to switch this to ether7 to make the script work.

Can you advise me what the proper method is to make the failover work from the command line? If I reboot the router manually it takes slightly long for the standby to become active.

Do I understand correctly that the VRRP only monitors the connection between the two devices? So for example if WAN1 goes down on the Active it doesn't switch to the Backup if the WAN1 is up on the Backup?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Oct 25, 2019 1:35 pm

Hi Nathan,

Nice project, I'm impressed. I'm preparing to switch from my no longer supported SonicWall NSA 2400 to two CCR1009-7G-1C-1S+ (2nd revision). I was trying out your project. As far as I can tell at this moment it works on 6.44.5. The configuration is synchronised, and manually rebooting the Active makes the Backup Active.

To make it work, I needed to change your script, I think you have hard coded ether8 as the HA connection. I had to switch this to ether7 to make the script work.

Can you advise me what the proper method is to make the failover work from the command line? If I reboot the router manually it takes slightly long for the standby to become active.

Do I understand correctly that the VRRP only monitors the connection between the two devices? So for example if WAN1 goes down on the Active it doesn't switch to the Backup if the WAN1 is up on the Backup?
The interface is not hard coded, you can specify interface= during $HAInstall (shown on github page). Are you saying you had to actually change the script code to make this work? I'd like to look into this if that is the case.

You can use $HASwitchRole on the master to force a failover. Note, this shouldn't really be much faster than rebooting the master assuming they are both in sync already. It is actually slower, since it verifies they are in sync. Can you explain what you are seeing with "slightly long"? It should takeover nearly immediately after VRRP failure but it could take a while to reappear on the network depending on your upstream switch setup (spanning tree delays?). With console access, you will see the standby switch near instantly.

Correct, the only thing monitored is the VRRP between the two devices. The standby has completely shutdown interfaces so it has no idea if an uplink is up or down. In theory, you could attempt to write a health check script for the master and force it down, but I suspect this will be hard to get right for something like an upstream link. You would likely need some robust delay/debounce logic.
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Fri Oct 25, 2019 2:26 pm

Hi Nathan,

Any progress why it not working in newer version? Maybe script level permissions?
And yes this solution need to be implemented by Mt team. I totally aprove it [emoji6]
Maybe in v7 who knows.
I was playing with this in 2 vm and is very cool [emoji41]
Thanks for this hard work and time.



Sent from my Moto Z3 Play using Tapatalk

 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Oct 25, 2019 2:38 pm

Hi Nathan,

Any progress why it not working in newer version? Maybe script level permissions?
And yes this solution need to be implemented by Mt team. I totally aprove it [emoji6]
Maybe in v7 who knows.
I was playing with this in 2 vm and is very cool [emoji41]
Thanks for this hard work and time.



Sent from my Moto Z3 Play using Tapatalk
User bbs2web has outlined why it does not work with newer versions: viewtopic.php?p=756873#p743339
I am waiting to see if Mikrotik reverts this behavior before I integrate this.
 
joelwhrs
just joined
Posts: 18
Joined: Thu Mar 19, 2015 8:04 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Oct 30, 2019 10:44 pm

I'm having an odd issue with running the script on 6.44.5. The setup on the first router goes fine, but the commands to run to bootstrap router B seem incomplete which seems to cause the bootstrap to fail. Below is my output. It seems as if $HAAddressOther isn't getting populated. Any ideas on what's causing this? I'm running it in a GNS3 environment.

This is the line that is causing the issues;
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"Password123\";"
[admin@MikroTik] > $haInstall interface="ether8" macA="0C:8B:8E:52:EC:07" macB="0C:8B:8E:57:6B:07" password="Password123"
0C:8B:8E:52:EC:07
0C:8B:8E:57:6B:07
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 0C:8B:8E:57:6B:07                          70     2ms
    sent=1 received=1 packet-loss=0% min-rtt=2ms avg-rtt=2ms max-rtt=2ms

Saving system configuration
Configuration backup saved
I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!!  00:04:08
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "0C:8B:8E:52:EC:07" and $mac = "0C:8B:8E:57:6B:07") do={
   :error "Interface ether8 MAC $mac does not match (A=0C:8B:8E:52:EC:07 or B=0C:8B:8E:57:6B:07) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
   /system backup save name=HA_backup_beforeHA dont-encrypt=yes
   /export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"Password123\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
###
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Oct 30, 2019 11:38 pm

I'm having an odd issue with running the script on 6.44.5. The setup on the first router goes fine, but the commands to run to bootstrap router B seem incomplete which seems to cause the bootstrap to fail. Below is my output. It seems as if $HAAddressOther isn't getting populated. Any ideas on what's causing this? I'm running it in a GNS3 environment.

This is the line that is causing the issues;
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"Password123\";"
[admin@MikroTik] > $haInstall interface="ether8" macA="0C:8B:8E:52:EC:07" macB="0C:8B:8E:57:6B:07" password="Password123"
0C:8B:8E:52:EC:07
0C:8B:8E:57:6B:07
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 0C:8B:8E:57:6B:07                          70     2ms
    sent=1 received=1 packet-loss=0% min-rtt=2ms avg-rtt=2ms max-rtt=2ms

Saving system configuration
Configuration backup saved
I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!!  00:04:08
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "0C:8B:8E:52:EC:07" and $mac = "0C:8B:8E:57:6B:07") do={
   :error "Interface ether8 MAC $mac does not match (A=0C:8B:8E:52:EC:07 or B=0C:8B:8E:57:6B:07) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
   /system backup save name=HA_backup_beforeHA dont-encrypt=yes
   /export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"Password123\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
###
Can you help me understand what you have run? Where did you run $haInstall? did you run it on A *AND* on B?
 
joelwhrs
just joined
Posts: 18
Joined: Thu Mar 19, 2015 8:04 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Oct 31, 2019 1:09 pm

The code I copied was right after running $HAInstall on RouterA. I ran the code $haInstall exported on RouterB, but it didn't appear to work. RouterB restarted, but after the restart, it never ran HA_bootstrap.rsc. I tried importing the HA_bootstrap.rsc file on RouterB manually, but it just failed due to a blank IP address. I then tried running the commands manually, but it wouldn't allow me to run
IP address add address=\"/24\"
as the IP address was blank.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Oct 31, 2019 2:15 pm

The code I copied was right after running $HAInstall on RouterA. I ran the code $haInstall exported on RouterB, but it didn't appear to work. RouterB restarted, but after the restart, it never ran HA_bootstrap.rsc. I tried importing the HA_bootstrap.rsc file on RouterB manually, but it just failed due to a blank IP address. I then tried running the commands manually, but it wouldn't allow me to run
IP address add address=\"/24\"
as the IP address was blank.
This is GNS3? Can you easily provide me access to screen sharing so I can take a look?
 
joelwhrs
just joined
Posts: 18
Joined: Thu Mar 19, 2015 8:04 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Oct 31, 2019 2:18 pm

Correct. Absolutely. Let me know when/what works for you.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Oct 31, 2019 2:24 pm

Correct. Absolutely. Let me know when/what works for you.
Shoot me an email at nathan4321 at googles mail service and we can coordinate. I can take a look now if that works for you.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Nov 01, 2019 3:04 am

For anyone following along with the bootstrapping problem, joelwhrs and I have reproduced it on his setup and are working on fixes. One is a timing issue with run-after-reset running too early on some platforms (viewtopic.php?t=123656) and the other is an oddity that hopefully we can nail down tomorrow.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sat Nov 02, 2019 2:49 pm

For anyone that wants to test, I have just pushed a test build that works with newer RouterOS (bbs2web bridge method) and has a more robust bootstrapping.
https://raw.githubusercontent.com/svlsR ... A_init.rsc
I am testing with 6.45.7 on this.

If you are going to test with an upgrade, I'd suggest upgrading ha-mikrotik before RouterOS. You should see the VRRP interface on the bridge once both pairs are pushed and rebooted.
Please let me know if you have any questions.

Note, if you have any firewall rules (ie: forwarding from standby) that tied to the old physical interface, you will need to change these rules to the new bridge interface.

Thanks bbs2web for doing the debugging/code to work around the new RouterOS vrrp behavior.
 
joes2
just joined
Posts: 4
Joined: Tue Sep 19, 2017 4:55 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 8:05 am

Hello,
trying to install ha-mikrotik on two fresh long term ROS 6.44.5 CCR1036-8G-2S+ :
On the first node:
[admin@HA-1] > /import HA_init.rsc

Script file loaded and executed successfully
[admin@HA-1] > $HAInstall interface="ether8" macA="CC:2D:E0:BD:9F:A8" macB="CC:2D:E0:BD:9F:D4" password="passw0rd"
CC:2D:E0:BD:9F:A8
CC:2D:E0:BD:9F:D4
SEQ HOST SIZE TTL TIME STATUS
0 CC:2D:E0:BD:9F:D4 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "CC:2D:E0:BD:9F:A8" and $mac = "CC:2D:E0:BD:9F:D4") do={
:error "Interface ether8 MAC $mac does not match (A=CC:2D:E0:BD:9F:A8 or B=CC:2D:E0:BD:9F:D4) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###

On second node:
Used part of config above, node restarted and nothing happens. If I run manually HA_bootstrap.rsc on second node, I'll get an error:
[admin@MikroTik] > /import HA_bootstrap.rsc
failure: local address cannot be 0.0.0.0
[admin@MikroTik] >

also user ha has not been created on the second node.

I have seen above that people successfully used ha-mikrotik on ROS 6.44.5
Any suggestions ?
Has there been any more progress on this? I'm stuck at the same position, and am about to completely disassemble the script and start from scratch if it fails on this error.
The user emailed me and I attempted to help them but they never responded. Can you provide me access to your kit? I still haven't seen this in the wild and I am unable to reproduce it.

Issue is in line 401 in the bridge definition. Fixed by removing fast forward:

\n /interface bridge add name=\"bridge-\$haInterface\" protocol-mode=none comment=\"HA_AUTO\"\
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 2:15 pm

Hello,
trying to install ha-mikrotik on two fresh long term ROS 6.44.5 CCR1036-8G-2S+ :
On the first node:
[admin@HA-1] > /import HA_init.rsc

Script file loaded and executed successfully
[admin@HA-1] > $HAInstall interface="ether8" macA="CC:2D:E0:BD:9F:A8" macB="CC:2D:E0:BD:9F:D4" password="passw0rd"
CC:2D:E0:BD:9F:A8
CC:2D:E0:BD:9F:D4
SEQ HOST SIZE TTL TIME STATUS
0 CC:2D:E0:BD:9F:D4 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "CC:2D:E0:BD:9F:A8" and $mac = "CC:2D:E0:BD:9F:D4") do={
:error "Interface ether8 MAC $mac does not match (A=CC:2D:E0:BD:9F:A8 or B=CC:2D:E0:BD:9F:D4) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###

On second node:
Used part of config above, node restarted and nothing happens. If I run manually HA_bootstrap.rsc on second node, I'll get an error:
[admin@MikroTik] > /import HA_bootstrap.rsc
failure: local address cannot be 0.0.0.0
[admin@MikroTik] >

also user ha has not been created on the second node.

I have seen above that people successfully used ha-mikrotik on ROS 6.44.5
Any suggestions ?
Has there been any more progress on this? I'm stuck at the same position, and am about to completely disassemble the script and start from scratch if it fails on this error.
The user emailed me and I attempted to help them but they never responded. Can you provide me access to your kit? I still haven't seen this in the wild and I am unable to reproduce it.

Issue is in line 401 in the bridge definition. Fixed by removing fast forward:

\n /interface bridge add name=\"bridge-\$haInterface\" protocol-mode=none comment=\"HA_AUTO\"\
What was the error/issue you were having? It looks like you edited your original post.
 
joes2
just joined
Posts: 4
Joined: Tue Sep 19, 2017 4:55 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 4:59 pm

What was the error/issue you were having? It looks like you edited your original post.
I had the same issue as that post above.

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33


when the startup script ran, gave above error. I just noticed your latest post which already resolved it https://raw.githubusercontent.com/svlsR ... A_init.rsc. I didn't see that version merged in github.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 5:38 pm

What was the error/issue you were having? It looks like you edited your original post.
I had the same issue as that post above.

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 01:02:33


when the startup script ran, gave above error. I just noticed your latest post which already resolved it https://raw.githubusercontent.com/svlsR ... A_init.rsc. I didn't see that version merged in github.
You said "Fixed by removing fast forward" in the post earlier this morning, what was the issue with fast-forward? 72e8 is an earlier commit. Did master not work for you? If not, can you please report the issue so I can get it fixed?

Thanks
 
joes2
just joined
Posts: 4
Joined: Tue Sep 19, 2017 4:55 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 9:49 pm

You said "Fixed by removing fast forward" in the post earlier this morning, what was the issue with fast-forward? 72e8 is an earlier commit. Did master not work for you? If not, can you please report the issue so I can get it fixed?

Thanks
Upon more work on this, never mind, red herring. I was testing on older versions that did not support "fast-forward=yes" as an option. My issue was actually running $HAInstall > 2 minutes after booting. I didn't see that note anywhere.

:if ($haStartupHasRun != nil || uptime > 2m) do={
/log warning "ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! $haStartupHasRun $uptime"
:put "ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! $haStartupHasRun $uptime"
} else={
:set haStartupHasRun [/system resource get uptime]
 
joes2
just joined
Posts: 4
Joined: Tue Sep 19, 2017 4:55 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 10:22 pm

One more question if you don't mind. After running through the initialization and $HASyncStandby, B syncs and restarts as expected. But when it comes online, it is not in standby mode. B becomes the active and A is put into standby. Everything is default install. I did not set haPreferMac. Is that the expected behavior? I thought there was no affinity without setting the preferred mac.

Thank you for all the help.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 02, 2019 10:57 pm

One more question if you don't mind. After running through the initialization and $HASyncStandby, B syncs and restarts as expected. But when it comes online, it is not in standby mode. B becomes the active and A is put into standby. Everything is default install. I did not set haPreferMac. Is that the expected behavior? I thought there was no affinity without setting the preferred mac.

Thank you for all the help.

I'm responding to your two posts here in one...What did you mean by "My issue was actually running $HAInstall > 2 minutes after booting. I didn't see that note anywhere."? I'm not sure I understand what you ran into here, it sounds odd.
Edit: I see what you are saying. You are running into the safety in ha_startup that prevents a double run if the environment is cleared. You are right, it will fail on you if you try to install with >2m of uptime. I put this in as a patch some time back to deal with the CLI crashing and losing the environment (https://github.com/svlsResearch/ha-mikrotik/issues/9). This is a side effect of it, that I have never personally run into but I see how you are getting it now. Issue #14 on github has been created to fix this.

You are right, there should be no affinity without haPreferMac set. As for yours seemingly have an affinity, this definitely sounds wrong. Did you happen to run $HAInstall on both A and B? I'm just grabbing at straws here trying to guess what may have happened. Are you able to give me access to them to take a look?
If haPreferMac is what is getting you here somehow, you SHOULD be a 3.5 line in the guy that tookover and a higher vrrp priority. Can you check for this?
 
Sivics
just joined
Posts: 4
Joined: Thu May 30, 2013 5:48 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Dec 05, 2019 3:20 pm

A little correction on this code:
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "XX:XX:XX:XX:XX:XX" and $mac = "YY:YY:YY:YY:YY:YY") do={
:error "Interface ether8 MAC $mac does not match (A=XX:XX:XX:XX:XX:XX or B=YY:YY:YY:YY:YY:YY) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
Shouldn't this code
:if ($mac = "XX:XX:XX:XX:XX:XX" and $mac = "YY:YY:YY:YY:YY:YY") do={
Be written like this?
:if ($mac != "XX:XX:XX:XX:XX:XX" and $mac != "YY:YY:YY:YY:YY:YY") do={
Bye!
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Dec 05, 2019 3:54 pm

A little correction on this code:
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "XX:XX:XX:XX:XX:XX" and $mac = "YY:YY:YY:YY:YY:YY") do={
:error "Interface ether8 MAC $mac does not match (A=XX:XX:XX:XX:XX:XX or B=YY:YY:YY:YY:YY:YY) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"passw0rd\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
Shouldn't this code
:if ($mac = "XX:XX:XX:XX:XX:XX" and $mac = "YY:YY:YY:YY:YY:YY") do={
Be written like this?
:if ($mac != "XX:XX:XX:XX:XX:XX" and $mac != "YY:YY:YY:YY:YY:YY") do={
Bye!
Absolutely. I guess that check has never worked correctly. Thanks for catching that.
 
TheNetworkBerg
just joined
Posts: 15
Joined: Mon Sep 30, 2019 9:50 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 09, 2019 11:50 am

Hey there nathan1,

Firstly I want to thank you for all the effort you have been putting into this project.
I have been trying to play around with your HA script but I am not really making any headway.
I see that some people have managed to run this on CHRs where I am trying to get this live right now.
My current network emulator is EVE-NG with CHRs running 6.45.6 and 6.44.4

I am not coming right with bootstrapping the secondary device, I can successfully run the HAInstall on the MikroTik where I have imported your script on.
I get the folllowing output:
[admin@MikroTik] > $HAInstall interface="ether8" macA="50:00:00:01:00:07" macB="50:00:00:02:00:07" password="1TPower0987"
50:00:00:01:00:07
50:00:00:02:00:07
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 50:00:00:02:00:07                          70     2ms
    sent=1 received=1 packet-loss=0% min-rtt=2ms avg-rtt=2ms max-rtt=2ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!!  00:11:57
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "50:00:00:01:00:07" and $mac = "50:00:00:02:00:07") do={
   :error "Interface ether8 MAC $mac does not match (A=50:00:00:01:00:07 or B=50:00:00:02:00:07) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
   /system backup save name=HA_backup_beforeHA dont-encrypt=yes
   /export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents=":local haBootstrapOK false; :while (!\$haBootstrapOK) do={:do { /ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"1TPower0987\"; :set haBootstrapOK true;} on-error={/log warning \"ha_startup: 0.0 B bootstrap failed...waiting\"; :delay 5};}"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
###
When I copy the script to MikroTik_B that I want to bootstrap nothing happens, everything runs through without issues. I then restart the router with

/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc

The router comes back, but nothing is really happening even after I try to run $HASyncStandby on the MikroTik_A
So I don't know if there's something I can do to debug the situation as I am following your instructions to a T.
I have loaded multiple different CHRs to try and run this on, unfortunately without any luck.
I would love to be able to run some MikroTiks in HA especially 1072's and perhaps even make a video on how to configure this for others who struggle like I am so that more people can benefit from this.

If there's something I can do from my end to help out or if you can point me to another thread or additional information that would be highly appreciated.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 09, 2019 2:51 pm

Hey there nathan1,

Firstly I want to thank you for all the effort you have been putting into this project.
I have been trying to play around with your HA script but I am not really making any headway.
I see that some people have managed to run this on CHRs where I am trying to get this live right now.
My current network emulator is EVE-NG with CHRs running 6.45.6 and 6.44.4

I am not coming right with bootstrapping the secondary device, I can successfully run the HAInstall on the MikroTik where I have imported your script on.
I get the folllowing output:
[admin@MikroTik] > $HAInstall interface="ether8" macA="50:00:00:01:00:07" macB="50:00:00:02:00:07" password="1TPower0987"
50:00:00:01:00:07
50:00:00:02:00:07
  SEQ HOST                                     SIZE TTL TIME  STATUS
    0 50:00:00:02:00:07                          70     2ms
    sent=1 received=1 packet-loss=0% min-rtt=2ms avg-rtt=2ms max-rtt=2ms

I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!!  00:11:57
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether8"] orig-mac-address]]
:if ($mac = "50:00:00:01:00:07" and $mac = "50:00:00:02:00:07") do={
   :error "Interface ether8 MAC $mac does not match (A=50:00:00:01:00:07 or B=50:00:00:02:00:07) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
   /system backup save name=HA_backup_beforeHA dont-encrypt=yes
   /export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents=":local haBootstrapOK false; :while (!\$haBootstrapOK) do={:do { /ip address add address=\"/24\" interface=ether8; /user add name=ha group=full password=\"1TPower0987\"; :set haBootstrapOK true;} on-error={/log warning \"ha_startup: 0.0 B bootstrap failed...waiting\"; :delay 5};}"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
###
When I copy the script to MikroTik_B that I want to bootstrap nothing happens, everything runs through without issues. I then restart the router with

/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA_bootstrap.rsc

The router comes back, but nothing is really happening even after I try to run $HASyncStandby on the MikroTik_A
So I don't know if there's something I can do to debug the situation as I am following your instructions to a T.
I have loaded multiple different CHRs to try and run this on, unfortunately without any luck.
I would love to be able to run some MikroTiks in HA especially 1072's and perhaps even make a video on how to configure this for others who struggle like I am so that more people can benefit from this.

If there's something I can do from my end to help out or if you can point me to another thread or additional information that would be highly appreciated.
The error "ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 00:11:57" is a problem.
This is a bug if the initial bootstrap master has been up for >2m. I just committed a quick fix for this, please try it:
New init: https://raw.githubusercontent.com/svlsR ... A_init.rsc
Diff: https://github.com/svlsResearch/ha-mikr ... 6387ea4423

Please report back if this fixes your issue. If it doesn't please email me if we can debug it together on a remote session.
 
TheNetworkBerg
just joined
Posts: 15
Joined: Mon Sep 30, 2019 9:50 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 09, 2019 3:16 pm

Yes that worked perfectly, thanks so much for your prompt assistance.
Routers are synced 8)!!
Can't wait to play around with this!
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Dec 09, 2019 4:59 pm

Yes that worked perfectly, thanks so much for your prompt assistance.
Routers are synced 8)!!
Can't wait to play around with this!
Glad it worked. I will stamp a new release shortly with a bunch of fixes. Please let me know if you run into any other issues.
 
n4p
Member Candidate
Member Candidate
Posts: 115
Joined: Wed Nov 25, 2015 9:54 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Dec 10, 2019 9:44 am

Hi nathan,
could you be so fine and give me an short information what i need to know if i upgrade routeros from 6.44.5 to 6.44.6 ?
As far as i know there some significate changes in the section from vrrp or?

Thanks!
Kind regards!
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Dec 10, 2019 4:18 pm

Hi nathan,
could you be so fine and give me an short information what i need to know if i upgrade routeros from 6.44.5 to 6.44.6 ?
As far as i know there some significate changes in the section from vrrp or?

Thanks!
Kind regards!
If you want to upgrade to the latest RouterOS, you simply upgrade ha-mikrotik first on your current version of RouterOS:
Warning: the current ha-mikrotik in master is still in testing. It may cause you problems but it is working for me, I'd recommend doing this in a test environment first.
1) Upgrade ha-mikrotik on your ACTIVE (https://raw.githubusercontent.com/svlsR ... A_init.rsc) - follow directions: https://github.com/svlsResearch/ha-mikr ... a-mikrotik
2) Once both original ACTIVE and STANDBY have been rebooted to the new ha-mikrotik, you can now upgrade RouterOS like normal.

For what its worth, I just double checked this procedure and upgraded a pair of CCR1009-8G-1S-1S+ from 6.42.11 to 6.44.6.
 
TheNetworkBerg
just joined
Posts: 15
Joined: Mon Sep 30, 2019 9:50 am

Re: Suggestion: Completely virtual router based on two physical routers

Tue Dec 10, 2019 4:39 pm

Hi nathan1

I hope you do not mind, but I have made a video on configuring the HA script and have posted it on YouTube.
I have given you full credit for the script as this is completely your work, I just want to make more people aware of this.
The video is just there to give some tips on how to import the script and run it in a video format so that people can reference it form video to see how a successful sync looks.
There is a link to your Github in the video and I have also asked MikroTik to reach out to you as I really believe this should be a feature on ROS not just a script.

Thank you again for all your hard work on this project, I personally appreciate all your hard work and effort and I find the HA script to do exactly what I want in HA.

https://youtu.be/GEef9P8wwxs

If you are interested in seeing the video.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Dec 10, 2019 5:10 pm

Hi nathan1

I hope you do not mind, but I have made a video on configuring the HA script and have posted it on YouTube.
I have given you full credit for the script as this is completely your work, I just want to make more people aware of this.
The video is just there to give some tips on how to import the script and run it in a video format so that people can reference it form video to see how a successful sync looks.
There is a link to your Github in the video and I have also asked MikroTik to reach out to you as I really believe this should be a feature on ROS not just a script.

Thank you again for all your hard work on this project, I personally appreciate all your hard work and effort and I find the HA script to do exactly what I want in HA.

https://youtu.be/GEef9P8wwxs

If you are interested in seeing the video.
I don't mind at all, great work! Thanks!
I assume this is obvious to you, but it wasn't clear to me via the video so maybe not to others. The admins don't need to run $HASync/$HAPush unless they are trying to force it in realtime (like you were). It happens automatically in the scheduler (every 10m). So any change done via any tool (cli, web, client, scripting) will propagate ~10m assuming everything is functioning correctly. Just thought I should mention this.

Linked on github:
https://github.com/svlsResearch/ha-mikr ... twork-berg
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Dec 11, 2019 2:15 am

Please be careful upgrading to the latest stable (6.46 confirmed broken), it appears to be broken with auto files. With the last git master, when you $HASyncStandby, you will see something like "ha_checkchanges: unable to find xxx/yyy! is auto working on this platform? xxxOffset: yyyOffset: " if it fails to work.
The latest long-term works fine. I am debugging if there is a workaround for the auto not working.

Mikrotik continues to make it hard to keep this thing working. :(

Update: 6.46 .auto scripts are broken. The prior version (6.45.7) seems to work fine for auto scripts but I have not tested beyond that.
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: Suggestion: Completely virtual router based on two physical routers

Wed Dec 11, 2019 5:03 am

Mikrotik continues to make it hard to keep this thing working. :(
I'm somewhat surprised and also irritated that they havn't implemented this natively into RouterOS by now. If nothing else they should contact you and ask to work on native implementation (paid of course!)
This script is one of the best community made ones and fixes something that SHOULD have been implemented long ago. Instead of them wasting time on pointless crap like 'kid control' they should be working on genuinely useful features especially in the enterprise market. IS-IS routing protocol, multicore routing/BGP, 64-bit CHR, MPLS Fast Reroute etc
And importantly, device clustering like this. Ideally they'd do it better and have route and session state sharing between the routers so in the event of a failover, the backup does not need to re-establish connections, reconverge with BGP/OSPF etc. It'll just flick over seamlessly. This is a lot harder to implement, but the basic functionality of config mirroring in a simple active/backup role should absolutely be there
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Dec 11, 2019 5:11 am

Mikrotik continues to make it hard to keep this thing working. :(
I'm somewhat surprised and also irritated that they havn't implemented this natively into RouterOS by now. If nothing else they should contact you and ask to work on native implementation (paid of course!)
This script is one of the best community made ones and fixes something that SHOULD have been implemented long ago. Instead of them wasting time on pointless crap like 'kid control' they should be working on genuinely useful features especially in the enterprise market. IS-IS routing protocol, multicore routing/BGP, 64-bit CHR, MPLS Fast Reroute etc
And importantly, device clustering like this. Ideally they'd do it better and have route and session state sharing between the routers so in the event of a failover, the backup does not need to re-establish connections, reconverge with BGP/OSPF etc. It'll just flick over seamlessly. This is a lot harder to implement, but the basic functionality of config mirroring in a simple active/backup role should absolutely be there
Agree with all of this. I'd love to stop maintaining it. :)
For what it's worth, I continue to maintain it and introduce new hardware platforms into my environment. I am now running pairs of CCR1036-8G-2S+, which I believe you are also running? So at least collectively, we are making it work.
 
solatpour
just joined
Posts: 1
Joined: Sat Apr 04, 2020 7:48 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sat Apr 04, 2020 8:05 pm

@nathan1
Hello Nathan. I've just implemented your instruction and script in simulation file in GNS3 before implement it in my real Network. I could follow it until Activating first router as : "MikroTik_HA_A_ACTIVE". After that I tried to bootstrap my 2nd router and it was just done and rebooted but after rebooting nothing was happened for the 2nd. I tried HASyncStandby but it shows this message: Got Error. I don't know where is the problem. both routers connected via ether 8 as you mention and I can discover them in Winbox and also neighbours.
As I have to be very cautious in order to implement it in my Network, I have to test it in a simulation environment first. you definitely know my concern. Plz Let me know if I made a mistake or if you have a solution for me.
Thanks in advance.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Sun Apr 05, 2020 9:12 pm

@nathan1
Hello Nathan. I've just implemented your instruction and script in simulation file in GNS3 before implement it in my real Network. I could follow it until Activating first router as : "MikroTik_HA_A_ACTIVE". After that I tried to bootstrap my 2nd router and it was just done and rebooted but after rebooting nothing was happened for the 2nd. I tried HASyncStandby but it shows this message: Got Error. I don't know where is the problem. both routers connected via ether 8 as you mention and I can discover them in Winbox and also neighbours.
As I have to be very cautious in order to implement it in my Network, I have to test it in a simulation environment first. you definitely know my concern. Plz Let me know if I made a mistake or if you have a solution for me.
Thanks in advance.
Is this from the master on github? Please include some actual log outputs. I can't really diagnose anything from this.
 
joelwhrs
just joined
Posts: 18
Joined: Thu Mar 19, 2015 8:04 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Apr 06, 2020 5:25 pm

@nathan1
Hello Nathan. I've just implemented your instruction and script in simulation file in GNS3 before implement it in my real Network. I could follow it until Activating first router as : "MikroTik_HA_A_ACTIVE". After that I tried to bootstrap my 2nd router and it was just done and rebooted but after rebooting nothing was happened for the 2nd. I tried HASyncStandby but it shows this message: Got Error. I don't know where is the problem. both routers connected via ether 8 as you mention and I can discover them in Winbox and also neighbours.
As I have to be very cautious in order to implement it in my Network, I have to test it in a simulation environment first. you definitely know my concern. Plz Let me know if I made a mistake or if you have a solution for me.
Thanks in advance.
I had a similar issue but discovered that I just wasn't running the factory reset on both routers. I thought I didn't need to do this as the config looked pretty much empty. This was keeping it from bootstrapping though, and I was able to successfully bootstrap both devices every time that both were fully reset. It's step 4 on the wiki. https://github.com/svlsResearch/ha-mikrotik
 
raffav
Member
Member
Posts: 345
Joined: Wed Oct 24, 2012 4:40 am

Re: Suggestion: Completely virtual router based on two physical routers

Mon Apr 06, 2020 7:28 pm

@nathan1
I have a curiosity
how do you keep the disable interface when you sync and apply the bkp file in the standby router? it via some script that read the standby on the vrrp or in the identity ?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Mon Apr 06, 2020 8:03 pm

@nathan1
I have a curiosity
how do you keep the disable interface when you sync and apply the bkp file in the standby router? it via some script that read the standby on the vrrp or in the identity ?
The startup script disables all interfaces as soon as it can, generally well before link comes up. See ha_startup, it is one of the first things it does.
 
kical
just joined
Posts: 7
Joined: Sun Dec 23, 2018 9:47 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 08, 2020 9:54 am

Has Anybody Tried on a CCR1072?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 08, 2020 12:42 pm

Has Anybody Tried on a CCR1072?
Yes. See here: https://github.com/svlsResearch/ha-mikrotik/pull/17
 
MetUys
newbie
Posts: 31
Joined: Mon Mar 17, 2014 1:19 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 15, 2020 5:04 pm

Hi @nathan1,
Good work here, and thanks for maintaining it for so many years.

Im using our countrywide lock-down due to COVID-19 to trial the ha_mikrotik setup. (finally some time to do it)
Setup: 2x RBD52G-5HacD2HnD (aka hAP ac^2, all I can get my hands on), both on ROS v645.6 and firmware updated to same.
Port config: MK1 ether1 to LAN for config drop, ether4 for ha heartbeat between MK1 and MK2.
Both devices have all files deleted and reset. "/system reset-configuration keep-users=no no-defaults=yes skip-backup=yes"
Uploading and executing the "/import HA_init.rsc" works fine (output: Script file loaded and executed successfully)

Using github release v0.6, I run into the same problem @TheNetworkBerg did, where if the $HAInstall is run after 2minutes of uptime it presents the error:
$HAInstall interface="ether4" macA="74:4D:28:6D:39:23" macB="74:4D:28:6D:38:8F" password="12345678"
74:4D:28:6D:39:23
74:4D:28:6D:38:8F
SEQ HOST SIZE TTL TIME STATUS
0 74:4D:28:6D:38:8F 70 0ms
sent=1 received=1 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms

Saving system configuration
Configuration backup saved
I am not master - running ha_startup first
ha_startup: ERROR ATTEMPTED TO RUN AGAIN!!! 00:05:27
###
#Maybe try: /tool mac-telnet
###PASTE THIS ON THE OTHER DEVICE - YOUR CONFIG WILL BE RESET AND LOST!!!###
:global mac [[/interface ethernet get [find default-name="ether4"] orig-mac-address]]
:if ($mac = "74:4D:28:6D:39:23" and $mac = "74:4D:28:6D:38:8F") do={
:error "Interface ether4 MAC $mac does not match (A=74:4D:28:6D:39:23 or B=74:4D:28:6D:38
:8F) - please check config\r\nUse orig-mac address!"
}
:if ([:len [/file find name=HA_backup_beforeHA.backup]] = 0) do={
/system backup save name=HA_backup_beforeHA dont-encrypt=yes
/export file=HA_backup_beforeHA.rsc
}
/export file=HA_bootstrap.rsc
/delay 2
/file print file=HA_bootstrap.rsc
/file set [find name=HA_bootstrap.rsc] contents="/ip address add address=\"/24\" interface=e
ther4; /user add name=ha group=full password=\"12345678\";"
/system reset-configuration no-defaults=yes keep-users=no skip-backup=yes run-after-reset=HA
_bootstrap.rsc
###END OF PASTE FOR OTHER DEVICE###
###
If a reset both sides and am quick enough to execute before the 2min time,
or if I grab the latest master branch in Github,
or if I just make the changes mentioned here: https://github.com/svlsResearch/ha-mikr ... 6387ea4423
The result is I get disconnected from the unit after:
Saving system configuration
Configuration backup saved
I am not master - running ha_startup first
I have not done anything to MK2 other than delete all files and reset the device.
If I plug the LAN into MK1 ether4 I can see the unit and all interfaces are disabled (bar ether4).

How should I resolve this in order to move forward with the setup?
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Wed Apr 15, 2020 6:20 pm

Hi @nathan1,
Good work here, and thanks for maintaining it for so many years.

Im using our countrywide lock-down due to COVID-19 to trial the ha_mikrotik setup. (finally some time to do it)
Setup: 2x RBD52G-5HacD2HnD (aka hAP ac^2, all I can get my hands on), both on ROS v645.6 and firmware updated to same.
Port config: MK1 ether1 to LAN for config drop, ether4 for ha heartbeat between MK1 and MK2.
Both devices have all files deleted and reset. "/system reset-configuration keep-users=no no-defaults=yes skip-backup=yes"
Uploading and executing the "/import HA_init.rsc" works fine (output: Script file loaded and executed successfully)

If a reset both sides and am quick enough to execute before the 2min time,
or if I grab the latest master branch in Github,
or if I just make the changes mentioned here: https://github.com/svlsResearch/ha-mikr ... 6387ea4423
The result is I get disconnected from the unit after:
Saving system configuration
Configuration backup saved
I am not master - running ha_startup first
I have not done anything to MK2 other than delete all files and reset the device.
If I plug the LAN into MK1 ether4 I can see the unit and all interfaces are disabled (bar ether4).

How should I resolve this in order to move forward with the setup?
You should definitely be running the git master for this. Can you get back into both devices after they reboot? Can you show /log print on both devices and look for the HA messages to show me?
This is one of those platforms that I don't really support or test, if you can provide access, I can take a closer look.
 
MetUys
newbie
Posts: 31
Joined: Mon Mar 17, 2014 1:19 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Apr 16, 2020 9:38 am

Hi @nathan1,

Yes I can get back into both devices. (screenshot of logs at bottom, this is from the v0.6 release, where I executed before the 2min counter)
and 100% I can give access, should I reach out to you on your google mail service account? (I see it in the previous comments)

Its totally a POC setup so we can mess around as much as we want.

Should I reset both devices, or do you want to do something before then?

Below log is Router A where script was imported on and run, Router B has nothing on it still, no logs (other than the interface up/down from me switching cables and my login) there are no files either.
logs after v0.6 (down before 2min counter).png
You do not have the required permissions to view the files attached to this post.
 
nathan1
Member Candidate
Member Candidate
Posts: 160
Joined: Sat Jan 16, 2016 7:05 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Apr 16, 2020 3:44 pm

Hi @nathan1,

Yes I can get back into both devices. (screenshot of logs at bottom, this is from the v0.6 release, where I executed before the 2min counter)
and 100% I can give access, should I reach out to you on your google mail service account? (I see it in the previous comments)

Its totally a POC setup so we can mess around as much as we want.

Should I reset both devices, or do you want to do something before then?

Below log is Router A where script was imported on and run, Router B has nothing on it still, no logs (other than the interface up/down from me switching cables and my login) there are no files either.
logs after v0.6 (down before 2min counter).png
Yes, contact me on gmail. You want to use the master, this looks like v0.6?
 
MetUys
newbie
Posts: 31
Joined: Mon Mar 17, 2014 1:19 pm

Re: Suggestion: Completely virtual router based on two physical routers

Thu Apr 16, 2020 7:42 pm

Thanks @Nathan1,
I have reached out to you on gmail.
Correct, this is v0.6
Will reset both and use master now.
 
MetUys
newbie
Posts: 31
Joined: Mon Mar 17, 2014 1:19 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Apr 17, 2020 5:10 pm

Hi,
for those interested, we managed to make some headway on this. The problems being:
  • no serial console to execute the install via, which results in a premature disconnect on ether1. to get around this connect to the second Mikrotik and execute the $HAInstall commands via mac-telnet to the first mikrotik over the heart-beat link (note: you will still get disconnected, but it seems to be delayed long enough that the install concludes what it needs to) you will need to reconnect to the first mikrotik again and re-execute the $HAInstall to get the export code to run on the second mikrotik to bootstrap it
  • The native winbox text wrapping breaks the bootstrap output resulting in it failing to execute correctly on the second mikrotik. Simple fix, copy to text editor and adjust before executing on the second unit.
  • Hardware root storage is volatile. to get around this the scripts were adjusted to prefix the flash partition so that content would survive reboots.
  • Device is slow in comparison to CCRs. which (we guess) is resulting in some odd write to flash not completing correctly. to get around this some delays were added.

Nathan1 made a branch on GitHub just for the flash storage adjustments. (Im not sure if the delays were added to it, but that's the one to use if you have a device with volatile storage)

I have retested from scratch and this seems to work well.
 
dlemery
just joined
Posts: 1
Joined: Sun Mar 11, 2018 5:03 pm

Re: Suggestion: Completely virtual router based on two physical routers

Fri Aug 28, 2020 1:45 am

Hi!

@Nathan1, this script is fantastic! We're about to put it to good use :)

We did find a flaw with the Watchdog service and this combined. Basically that service pings an IP and reboots the MikroTik if it can't reach it. So when it's enabled, it works fine on the active router but the standby is in a constant reboot loop. All the interfaces are disabled, so naturally it can't ping the IP and considers itself down.

I was originally thinking of using the Watchdog service as a means of seeing if the switch that the active router is connected to goes down. If it did, the standby could kick in which is connected to the second switch for redundancy. Does anyone have any thoughts?

Thanks :)
-Derek
Last edited by dlemery on Fri Aug 28, 2020 1:47 am, edited 1 time in total.
 
User avatar
nichky
Forum Guru
Forum Guru
Posts: 1275
Joined: Tue Jun 23, 2015 2:35 pm

Re: Suggestion: Completely virtual router based on two physical routers

Tue Jan 26, 2021 11:41 pm

@Nathan1

i made it work, but ONE important thing.
i was getting script error!!!
You know when you adding on MASTER :

$HAInstall interface="ether8" macA="[MAC_OF_A_ETHER8]" macB="[MAC_OF_B_ETHER_8]" password="[A RANDOM PASSWORD OF YOUR CHOOSING]"

then it pops up on the terminal script and then that script gooes to slave .All this process i was doing on MikroTik terminal.

Once i started doing on putty everything was fine...are you aware of this?
 
siouxpersniper
just joined
Posts: 1
Joined: Tue Jun 08, 2021 1:11 am

Re: Suggestion: Completely virtual router based on two physical routers

Tue Jun 08, 2021 1:13 am

Has anyone been able to get this to run on a pair of 4011's? I'm running 6.46.8
 
cdman
newbie
Posts: 29
Joined: Sun Jan 01, 2006 11:47 pm
Location: Bulgaria/Sofia

Re: Suggestion: Completely virtual router based on two physical routers

Mon Sep 05, 2022 3:10 am

Anyone tested this with version 7.5 ? , or is there a better way now with v7 ?

Who is online

Users browsing this forum: alotofbacardi, baragoon, GoogleOther [Bot], Pincha3, SpOuK3 and 93 guests