How to prevent router fatal failures during configuration changes

RouterOS Beginner Basics
1

I’m new to the Mikrotik router ecosystem implementing an RB4011iGS+5HacQ2HnD running RouterOS 7.19.1. Twice over the past week, the router has gone stupid after I have made a configuration change. The first was when I changed the router identity. The second is when I change the LAN IP range. I’ve only used the WinBox GUI so far.

The first incident took some time to recover. I successfully performed a reset-reboot sequence, but the configuration was lost. Lesson learned I starting making backups. Fast forward to today for the second incident. After changing the LAN IP the router went stupid. No amount of reset-reboots would recover the router. The router is back online after a long, hard netinstall is finally completed. What a learning curve!

The catastrophic failures are giving me cause for concern. I’m wondering if this is the best replacement router.

I’m hoping someone in Mikrotik land can offer some advice on how to make configuration changes that eliminate failures. Today’s failure took me five hours to recover from. The backups I thought I had on the router were gone. Makes sense if you are rebuilding the router from scratch and didn’t cloud store the backups.

If the failure happened during office hours with the device in production, I suspect I might be unemployed:(

I’m Newbie to Mikrotik after 40 some years of the technology world. I can’t ever recall failures on Cisco enterprise routers. If they stopped, it’s time to grab your bags and leave, as dark voids may appear on the Internet.

I have not used safe mode to make changes. Is that the best practice approach? How do you know if a command is going to take the router south? Lots of questions,hoping for some answers to restore my confidence.

2

So then do.

“Best practice” is code for “stop thinking and do it my way.” Safe mode is conditionally useful. You will tend to use it less over time as you come to understand RouterOS.

Above all, realize that while there are bugs in RouterOS, and it can be random, these are edge cases. In the main, what RouterOS does with a given configuration is deterministic. Your task is to learn those predictable behaviors.

3

Mikrotiks are pretty robust and discovering a really basic error is very unlikely.

What you are saying regarding that resetting the router - well - reset it, that’s the correct behavior. This is done because the config of the router quite often contains secrets (vpn keys, cerificates, etc.) and resetting it also resets the password, so these would be disclosed.

So best practice rule no. 1: keep the backups somewhere safe but not on the router

Safe mode can come in handy, but don’t rely on it too much.

As to your second thing. You changed the lan address. Did the address make sense? (The address also implies a network/netmask.) What’s the “went stupid” part? Was it that after an address change, it wasn’t available on its old address? Was it that the dhcp server was not reconfigured automatically?

Just a best practice no. 2: Mikrotik has an application called Winbox, which can connect to the router with only L2 access, so it works regardless of the ip settings.

Mikrotiks have a configuration interface that’s a pretty close wrapper on the underlying linux, so it’s a bit harder to get used to at first. The learning curve can be a bit steep.

However once you get to know it a bit, they’re extremely reliable both in hardware and software. Pretty good in terms of security, and software upgrades basically until the heat death of the universe. They’re a common sight in server rooms and data centers in some capacity, so they can’t be that bad. (Of course in large DCs they usually do something auxiliary, not the heavy lifting.)

Feel free to come back with any definite questions.

I’m not trying to condescend, but… if you’re feeling unsure (which based on what you wrote, you probably are,) why not take the device out of production - if need be temporarily replacing the previous one - and spend a few hours of quality time together until you have it configured and are satisfied that it won’t cause issues. You’ll have a much easier time that way and the users will be much happier.

4

Added interesting material to get a better way how to behave around a 'Tik:

The Twelve Rules

and

Good Practice and Common Sense Advice

5

New entry:

6

The MikroTik router always follows the leader! When he is stupid, nasty things can happen.
Lesson learnt here: it is not a good idea to change the LAN IP of the router, unless you completely understand what you are doing and understand the consequences.
A MikroTik router is not like the typical home router that has wizards for such actions. Even QuickSet is dangerous to use for actions like this.

Next time you want to change the LAN IP, at first do not CHANGE it but ADD another LAN IP, test if that works as you intended, and only then remove the old one. And probably use safe mode.

7

After changing the LAN IP the router went stupid

You should be able to log into the device using MAC address of some port inside the bridge like ether2 (in default config at least).

I won’t write things people already mentioned, just keep in mind that Mikrotik won’t tell you “are you sure you want to change IP address? You won’t be able to connect via the 3rd network layer to the device after this operation!” - it just changes the address.

8

A related thing I sometimes do is enable a DHCP client on the router/switch in addition to a static IP. I find this practice most helpful on “mobile” devices like the hAP ax² I carry as a “utility router” in my backpack.

9

Search for OffBridge using the search function. You will find the simplest method to configure the router from a safe spot.

10

My initial post was me licking my wounds just after I recovered the router. I had to learned way more than I wanted to in order to recover the router.

Thanks to everyone who responded. I’m not going to reply individually but rather cover the touch points.

When I used the term stupid I mean I was unable to connect using WinBox or web. The router was no longer handing out IPs in the subnet. It was giving me IP’s in the range associated of the WAN, not the LAN IP range. I didn’t move my laptop to another port to see if that worked. I did try manually assigning LAN IP’s to my laptop in case dhcp was the issue.

The wash-rinse-repeat cycle of holding the reset button while apply power is what I term the reset-reboot. I literally wore out a finger and the button trying to time the release. Nothing restored access. Reset-reboot did get me to the Netinstall.

I’m confident in the changes I made. Identity is easy to find. I acknowledge I was modifying the actual name of the router. That can cause problems but it is not like I typed in a name that had special characters or inserted spaces. OfficeRouter is generic.

The LAN IP change was 192.168.200.1 to 192.168.201.1. I didn’t enter a incorrect formatted IP. I acknowledge the change impacts the whole router. I would expect a reboot to apply such a change. It rebooted and went stupid.

My inability to connect is troubling. Use Winbox is great advice, if it works. It hasn’t worked reliably for me. I will try the laptop on a different port and the config line change to see if it helps. I will read the Winbox resources. I do like the tool when it worked.

If my laptop connected doesn’t get an IP from the router, web interface doesn’t help. I contemplated the serial console as the hill I would die on but never had to get there. Spending considerable time to connect when it went stupid is concerning.

Don’t use VLAN1! SOund like don’t use IP www.xxx.yyy.0 it is the network address but you can on some devices. Wait, didn’t you say don’t use .0:) I suspect the season Mikrotik folks know these hidden gems before they cause problems.

If the router moves into production there needs to be a backup strategy. I’ve had to learn how to use backup and what can go wrong just trying to set the thing up. Way to much learning!

My post is not to trash the router but to understand it. The five hour nightmare of restoring it puts me on notice. I purchased the router because of it capabilities. I knew going in the configuration challenges. I just looking for what I am doing that causes the crash.

11

Use MAC connection for Winbox. It’s one of the best features of Mikrotik. No need for IPs, just remember to properly enable MAC-server.

According to usage of VLAN1 and x.y.z.0 or x.y.z.255 addresses … you can if you are aware of consequences when you start to mix different vendors’ devices as their understanding what you can or not could be different.

12

It is not immediate/evident that WInbox can connect in two ways.

  1. via IP (like any “normal” tool, i.e. at L3)
  2. via MAC (using a proprietary protocol, at L2)

See also here:

If you click on the IP of a device that is listed in the lower part of the opening window:

the “Connect To:” field will be filled with the IP, but if you click on the MAC of the same device, the field will be filled with the MAC.

When you press the Connect button the connection will be (hopefully) established with the chosen device using the one or the other method.

In the example screenshot on the help page you can see that a few devices show an IP address of 0.0.0.0, this means that the connection is to a port that has no IP assigned.

Still, you can connect via MAC to those devices, in theory you could keep your Mikrotik devices without IP assigned and still be able to connect via MAC.

Even if possible, it is generally speaking not a very good idea, as while fiddling you could accidentally disable the MAC-server on the router or mis-configure the allowed interface-list and thus lock you out of MAC access.

A common enough issue when fiddling with IP addresses is that you change the IP (and/or netmask) of the port of the router but the IP (and/or netmask) of the computer you are using is not changed accordingly, connection via MAC bypasses this problem.

13

From what you write it appears you are just inexperienced in managing MikroTik routers. You have changed something that has quite wide impact (for example, not that changing the router IP will NOT change all the DHCP configuration items, so a device using DHCP will fail to connect after that, when you moved the IP outside of the existing subnet).
Even when you change the IP via the “QuickSet” wizard, the DHCP config will not be changed. It can only do that on the first boot.
So you need to follow my advice and only ADD a new IP so you can still use the old IP to change the other config items.

And as written, use MAC level connect when the IP level has been messed up. When you are local to the router you can still use MAC address to connect (just click on that in the device list in winbox) and when remote and prepared you can use RoMON.

14

Well, “Don’t use VLAN1!” means simply “Don’t use VLAN1!” and has NOTHING to do with IP addresses.

Read from here:

15

I was around when the IP support for 0 & 255 was implemented. We were conditioned to not use them and then Cisco said “oh yeah you can>”

16

I duplicated what I did to make the router “stupid”. It wasn’t stupid I just didn’t have the configuration needed to communicate with it.

In Quickset (now that I have read the tweleve rules, I will apply more caution) on the Local network tab I changed IP Address: from 192.168.200.1 to 192.168.201.1 and hit apply. I failed to change the DHCP ranges below it to reflect the IP change.

I rebooted the router and got no DHCP reservations on LAN interfaces. I also got no MAC using neighbours in WinBox. Attempting to connect to the MAC of the bridge failed with a MAC Connect syn timeout.

The IP failure I can understand but the MAC failure caused confusion. The consensus is use WinBox and connect to the MAC. I am unable to get it to work. I gone back and recreated my scenario and WinBox doesn’t connect on the MAC.

I now have a router without DHCP and unable to MAC connect. I got back into the router by manually assigning 192.168.201.2 to my laptop. I could talk to the router again. Sure as I had did it, Quickset had changed the bridge IP and not the DHCP. I could clearly see that using export.

I recall when I first used QUICKset I changed both the IP and the DHCP range. In my haste to test a problem I had with OpenVPN I only changed one IP and not the range. OUCH!

I’m also learning the benefit of Safe-Mode.

17

I have tired /interface bridge set bridge admin-mac=no auto-admin=no . It kicks an error admin-mac must be a mac address.

The interface bridge export shows
add admin-mac:[mac address found on device label] auto-mac=no comment=defcon name=bridge

To me that suggests the mac is hard coded and won’t change. Am I mistaken?

18

Yes, I already tried to explain to you, that will not work with QuickSet. The best way of handling QuickSet is to NOT use it.
You may be accustomed to routers that change the DHCP configuration (and other things) when you change the LAN IP address, but RouterOS does not do that.
That is why it is better to only ADD a new IP address, then move the remainder of your config (e.g. DHCP or VPN) to that new address, and only when everything is working again remove the original address.
MAC connect requires the Tools->MAC Server (MAC Winbox server) to be active on the interface, which it is by default (interface list ALL or LAN) but that may be changed and then MAC connect possibly no longer works.

19

Yes you are.
You can change the MAC address on the bridge to whatever you like.
Usually SoHo devices have in the setup script a provision to make the bridge MAC the same as the first interface added to bridge (usually ether2).

Changing MAC on the command line is (slightly) tricky, you need a single command line like:

/interface bridge set bridge1 admin-mac=4e:a4:50:f4:e7:1C auto-mac=no

A good question is which MAC to use. JFYI:

20

Just FYI: the auto-mac implementation was updated (I don’t really know when, but for quite some time now) and these recommendations come from the shortcomings of the old behavior.

The new implementation adheres much better to the principle of least surprise, so now I would say that it’s perfectly adequate to use auto-mac for non-esoteric use cases.