Community discussions

 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Netinstall broken (start up very slow) after v6.37.3

Tue Aug 29, 2017 1:08 pm

Hi everyone

You may have gathered if you have read any of my previous posts that we deploy a lot of Mikrotik as CPE (hAP Lite and hAP AC Lite mainly). To do this we use netinstall to put a basic config onto the router which connects to our config server and pulls the rest of the data. This is done so that if a customer tries to factory reset the unit it does not break things and also because if we need to change the config we can ask users to factory reset which triggers a re-download.

We have been using ROS v6.37.3 for some time. This has worked perfectly. However, the latest batch of hAP Lite units we received insisted we use v6.38 or later. No problem I thought, let's go for the latest ROS version and tweak to make sure that no configuration commands have changed.

However somewhere between ROS v6.37.3 and v6.40.2 netinstall seems to be a bit broken. The main symptom we are seeing is extremely slow start up of the router the first time after the netinstall. When I say slow, I mean of the order of 4-5 minutes from power up to the device requesting a DHCP address on WAN1.

I have also tried v6.38.7 and that seems to be doing the same thing.

What is weird though is that it does not seem to happen every time. Maybe 2/3 routers will exhibit the problem, the other 1/3 will work straight away.

Obviously this is a very difficult thing to diagnose as there is no way to access the router until DHCP has happened at which point the problem has gone away. I did try soldering pins to what looks to be a UART header inside the hAP Lite and using a 3.3v serial adapter to try to get a console but no such luck (there is a thread on here somewhere that says it won't work that I found afterwards).

Has anyone else seen a similar issue? Anyone at Mikrotik shed any light?

Thanks, Chris
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 2946
Joined: Tue Feb 25, 2014 12:49 pm
Location: Capalbio, Tuscany, Italy

Re: Netinstall broken (start up very slow) after v6.37.3

Tue Aug 29, 2017 4:53 pm

Hi everyone

You may have gathered if you have read any of my previous posts that we deploy a lot of Mikrotik as CPE (hAP Lite and hAP AC Lite mainly). To do this we use netinstall to put a basic config onto the router which connects to our config server and pulls the rest of the data. This is done so that if a customer tries to factory reset the unit it does not break things and also because if we need to change the config we can ask users to factory reset which triggers a re-download.

We have been using ROS v6.37.3 for some time. This has worked perfectly. However, the latest batch of hAP Lite units we received insisted we use v6.38 or later. No problem I thought, let's go for the latest ROS version and tweak to make sure that no configuration commands have changed.

However somewhere between ROS v6.37.3 and v6.40.2 netinstall seems to be a bit broken. The main symptom we are seeing is extremely slow start up of the router the first time after the netinstall. When I say slow, I mean of the order of 4-5 minutes from power up to the device requesting a DHCP address on WAN1.

I have also tried v6.38.7 and that seems to be doing the same thing.

What is weird though is that it does not seem to happen every time. Maybe 2/3 routers will exhibit the problem, the other 1/3 will work straight away.

Obviously this is a very difficult thing to diagnose as there is no way to access the router until DHCP has happened at which point the problem has gone away. I did try soldering pins to what looks to be a UART header inside the hAP Lite and using a 3.3v serial adapter to try to get a console but no such luck (there is a thread on here somewhere that says it won't work that I found afterwards).

Has anyone else seen a similar issue? Anyone at Mikrotik shed any light?

Thanks, Chris
Is slow on first time because the routeros do some sanity check from some 6.37.x and above, and next format user space, generate 1 Key RSA 2048 and 1 DSA Key 1024 (used for SSH and other tings, on security package), those processes are hardware intensive and flash are slower than nand
I'm Italian, not English. Sorry for my imperfect grammar.
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: Netinstall broken (start up very slow) after v6.37.3

Wed Aug 30, 2017 9:46 pm

That makes sense. But 4 minutes? That sounds very long even on the 650MHz processor in the hAP Lite.

Also it doesn't happen if you don't use a configure script. Does that somehow bypass the key generation? (surely not)
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
User avatar
strods
MikroTik Support
MikroTik Support
Posts: 1409
Joined: Wed Jul 16, 2014 7:22 am
Location: Riga, Latvia

Re: Netinstall broken (start up very slow) after v6.37.3

Thu Aug 31, 2017 1:06 pm

Can you provide your configure script? When do you stop time tracker? Does it happen when router get IP address from server? How fast is device accessible through Layer2 after Netinstall process?
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: Netinstall broken (start up very slow) after v6.37.3

Mon Sep 04, 2017 1:26 am

Can you provide your configure script? When do you stop time tracker? Does it happen when router get IP address from server? How fast is device accessible through Layer2 after Netinstall process?
I don't really want to share the configure script in public but quite happy to send to Mikrotik support privately.

L2 link comes up pretty quick but it takes 3-4 minutes for DHCP client to request an IP. The device is not accessible until this time as there is no serial console. I stop the timer when I am able to connect via SSH which is almost immediately after DHCPACK.
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com

Who is online

Users browsing this forum: No registered users and 56 guests