For several years we have been using scripts to do system configuration and board burn in testing. These scripts are fairly long and generally we have only had to make minor changes to accommodate command structure changes from ROS version to version. They have been highly reliable up to now.
Recently we have started having problems with the scripts running to completion. It first started happening on RB435G boards but now seems to also be happening on RB433GL and RB411GL. The scripts simply stop running at seemingly random points.
The same script, when copied to a second identical model board from the same lot runs to completion fine. We thought it might be related to change of parts like oscillators. But we cannot find any correlation between failures and parts.
We have found that if we drop the system clock speed from 680Mhz to 600MHz, the scripts will run to completion.
We have checked power and tried many different diagnostics, but it appears to always come back to system clock speed.
Has there been a change in the script interpreter? Some setting related to process threading or something else that might have caused this to happen in more recent versions of ROS? We think it started at about 6.12 for the RB435G boards, and just started happening with 6.28 for the other boards. Could it be related to bootloader?
Any ideas welcome.