Community discussions

MikroTik App
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

v7.15beta broke backup file naming

Tue Mar 12, 2024 11:50 pm

After updating to 7.15beta6 I noticed that my automated backups were no longer working.
Upon closer inspection I noticed the issue is that /system/backup/save no longer accepts names with spaces, and will automatically convert them to _
doesn't seem like it's a file system limitation as i can still upload files with spaces to the device.

Unless there is some new limitation, I'd very much like to keep pre-7.15 behavior and allow us to use spaces for the backup file names, because if not that would break my automated backups throughout my whole network as I eventually update them...
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11646
Joined: Thu Mar 03, 2016 10:23 pm

Re: v7.15beta broke backup file naming

Wed Mar 13, 2024 7:59 am

It's a deliberate change, well published in change logs. Did you read through relevant "new version announcement post" before installing a beta version?
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Wed Mar 13, 2024 3:10 pm

It's a deliberate change, well published in change logs. Did you read through relevant "new version announcement post" before installing a beta version?
i actually read every release notes (i follow through RSS) and didn't see anything regarding backups.
reading it again i am assuming it's this, but i always assumed console changes were for the console itself.

*) console - replace reserved characters to backup and certificate export file names with underscores;

again, this would break my backup automation throughout my whole network and would require me to manually go and fix it as i update devices... why are these now reserved specifically for backup and export while previously they werent? for example, why are they not reserved for file names?
 
tangent
Forum Guru
Forum Guru
Posts: 1404
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 1:48 am

"console" is RouterOS-speak for any command you can type at the CLI. This includes your backup commands, but also scripting, etc.
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2880
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 11:30 am

Just wondering why use in filenames spaces, special characters like slashes, dots and others crucial for filesystem just for readibilty? Why not to use compact standarized easy to parse names compatibile with any type of filesystem used behind the scenes?
Sample: YYYYMMDDHHMM_DEVICENAME_OTHERSHORTINTHO.rsc with no country specific letters, just pure ANSI set.
I know that it's rather a rethorical question and does not help in that particular situation but KISS rule is the rule of thumb across the IT territories for me.
Last edited by BartoszP on Thu Mar 14, 2024 11:52 am, edited 1 time in total.
Reason: sample edit
 
holvoetn
Forum Guru
Forum Guru
Posts: 5500
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 11:50 am

Sample: YYYYMMDDHHMM_DEVICENAME_OTHERSHORTINTHO.rsc
I know that it's rather a rethorical question and does not help in that particular situation but KISS rule is the rule of thumb across the IT territories for me.
Already a "problem" since I started working with computers (mind you, that's 1980) and I always tried to follow that rule: NO spaces or special characters in filenames.
I even remember I, O or X were not allowed as filename character on a mainframe (that would cause confusion with 1, 0 or wildcard, keep in mind there was no lowercase at that time).
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2880
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 12:22 pm

I wanted to mention EBDIC but decided that it would be an overkill for many :) :)
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 12:55 pm

"replace reserved characters..." should be marked as Bold or at least !

Could someone please confirm if those are special characters not allowed?

https://engineering.purdue.edu/ECN/Supp ... aractersIn
;			Command separator
&			Background execution
(  )			Command grouping
|			Pipe
* ? [ ] ~		Filename metacharacters
{ }			String expansion characters. Usually don't require quoting.
> < & !			Redirection symbols
! ^			History substitution, quick substitution
" ' \			Using in quoting other characters
$			Variable substitution
newline space tab	Word separators
" "			Everything between " and " is taken literally, except
			for the following characters that keep their 
			special meaning:
				$	Variable substitution will occur
				`	Command substitution will occur
				"	This marks the end of the double quote
				\	Escape next character
				!	The history character
				newline	The newline character
' '			Everything between ' and ' is taken literally except 
			for ! (history) and another ', and newline.
			
\			The character following a \ is taken literally. Use
			within " " to escape ", $ and `. Often used to escape
			itself, spaces, or newlines. Always needed to escape a
			history character (usually !).
https://iubemcenter.indiana.edu/equipme ... cters.html

Probably @rextended already has a solution? Or something like this is enough?
:local Device [/system identity get name]
:local IntegerLength [:len $Device]
:local Project ""
:for Encoding from=0 to=$IntegerLength do={
    :local Characters [:pick $Device $Encoding]
    :if ($Characters = " " || $Characters = "|" || $Characters = "*") do={:set Characters "_"}
    :set Project ($Project.$Characters)
}
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 1:42 pm

This is yet another piece of evidence and major reason one should try to avoid RoS scripting in production at all costs as Mikrotik might break compatibility without notice at any time.

Since this isn't the first time (and probably not the last) that Mikrotik breaks script compatibility, I think it's more than fair to include a clear warning about this on the help page regarding scripting. Besides, spaces in file names have no significant effect that would justify this change otherwise we'd have seen a lot of complaints about this already.

Bottom line - one might wonder if R&D practices any QA sign-off at all. Excuse my French, but this is completely bonkers!
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2880
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 1:50 pm

"replace reserved characters..." should be marked as Bold or at least !

Could someone please confirm if those are special characters not allowed?
https://engineering.purdue.edu/ECN/Supp ... aractersIn
....
https://iubemcenter.indiana.edu/equipme ... cters.html
...
Wondering again: Does anyone intentionaly use characters clashing even in theory with used filesystems?

Isn't it selfcalling for problems? Not "calling", it's CALLING.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7056
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 3:27 pm

common sense would dictate not to use special chars in filenames anyway, especially when one can download files to other systems not supporting specific characters in filenames
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 3:45 pm

All major operating systems like Windows, macOS, Linux, z/OS, Android and iOS utilize UTF-8. What other OS might have the compatibility issue you are referring to?
 
tangent
Forum Guru
Forum Guru
Posts: 1404
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 3:51 pm

Windows…utilize UTF-8

It's actually UTF-16, but the real issue is that it has far more special characters than any other OS, primarily owing to its ancestry, its path scheme being a mongrel mashup of CP/M, Unix, and LAN Manager rules.

Most POSIX flavors have only two special characters: slash and null.

macOS adds colon to that, but not for the same reason as Windows; it's a Classic MacOS holdover in HFS, inherited by HFS+, then APFS.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7056
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 4:07 pm

exactly
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 4:08 pm

Well, no! ;-)

Windows defaults to UTF-16 as its internal representation but has strong support for working with UTF-8 in addition to the legacy CP-1252 and similar encodings. For example, Notepad uses either ANSI or UTF-8. The rest of the world defaults to UTF-8. However, none are limited to legacy US ASCII..

But that is still beside the point. Prohibiting spaces in file names breaks compatibility, as the OP found out. A suggestion would be to either make an exception for spaces or adjust internal file management to also replace 'special characters' with underscores to avoid breaking script compatibility.
Last edited by Larsa on Thu Mar 14, 2024 4:25 pm, edited 1 time in total.
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 4:25 pm

common sense would dictate not to use special chars in filenames anyway, especially when one can download files to other systems not supporting specific characters in filenames
what system are you talking about exactly where space is a reserved character? every system has supported space in filenames for as long as i've known.
comma would be nice too.
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 4:28 pm

common sense would dictate not to use special chars in filenames...
Sir mrz, for who that is not an expert like you, but learned using MikroTik devices from time to time (sorry not all your users are PRO), could you please spend few minutes writing a clear statement like "attention this version will break scripts if you are using special chars (like space, etc...), you can fix using..."

This can help dump users like me and professional that maybe made a mistake and didn't had the time going through all the changes.
Thanks
 
jaclaz
Long time Member
Long time Member
Posts: 667
Joined: Tue Oct 03, 2023 4:21 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 4:52 pm

what system are you talking about exactly where space is a reserved character? every system has supported space in filenames for as long as i've known.
comma would be nice too.
You must be young.
Once upon a time we were also limited to 8 characters (and we liked it :wink: ):
https://tinyapps.org/blog/200702250700_why_in_my_day
Kids today ....

Spaces - generally speaking - are a PITA, as well as a number of special characters that - while allowed on the file system may be interpreted by this (or that) program/language as an "own" special character and would need to be escaped.

For what is worth, the University of Iowa has this to say on the matter:
https://instr.iastate.libguides.com/datashare/filenames
 
holvoetn
Forum Guru
Forum Guru
Posts: 5500
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 5:04 pm

Ah, the good'ol 8 dot 3 naming convention.
360Kb on a floppy and all that ... (and now we're complaining about 16Mb not being enough :shock: )

I still remember OSes where space was not allowed... DOS, CPM, OS/2, early Windoooos, ...
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 5:43 pm

i genuinely don't understand the issue.

windows, every linux, every bsd, mac, all of these have supported spaces in filenames for a very very long time without any issues.
what is this breaking exactly? why is this an issue exactly?

regarding programming, you are correct that you need a safety check in the path, but you need it regardless for sanity, and it needs to go somewhere.
you are simply moving the safe checking from underneath, completely transparent to end user, to the surface, breaking end user formatting for no reason.
again, if someone has a reason, i would like to see it!
not asking for random characters, just asking for space and maybe comma.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 5:44 pm

@jaclaz, regarding the second link, it seems less focused on the actual problem regarding script compatibility issues caused by spaces in filenames and more like 'whataboutism' disguised as academic debate. I mean, this has a serous impact for both the OP and others who rely scripts that handle space in file names.

If you aren't personally affected, it may be difficult to grasp the implications for existing production environments. Try to imagine the disruption and effort needed to rework scripts, similar to the previous date format issue.

I do understand the temptation to avoid spaces in filenames, especially for command-line work. However, modern operating systems like Windows, macOS, and Linux have evolved to handle spaces, hyphens, commas, and even quotation marks without issues.

And as I've repeatedly emphasized, breaking script compatibility should be avoided at all costs unless there's a truly compelling reason. Forbidding spaces in filenames without a clear justification doesn't seem like a necessity IMO.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3506
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 5:45 pm

To @mrz's point...certainly avoiding any even-close-to-special file names that have to go between system, or in scripts, should be a best practice.

And, Mikrotik restricting at creation might prevent also sorts of corner-cases and avoids needing to consider encoding schemes in stuff like /tool/fetch. Since RouterOS does not support unicode, restricting diacritics or any special or multi-byte characters make sense.

It's the space in filename part seems overly restrictive. I also get that "even a space" can cause problems sometimes. But issue here is that was working. I don't use space in file name myself, but these "breaking changes" are not good since often it's not so simple as one script to change.

But I guess it just seems odd to add more "file system things" like /file/add, ROSE/SMB, and DLNA now...while also adding restriction on stuff like spaces in file names. e.g. imagine someone one day may want to rename some movie with a space on RouterOS DLNA server...
 
jaclaz
Long time Member
Long time Member
Posts: 667
Joined: Tue Oct 03, 2023 4:21 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 6:38 pm

@Larsa
I perfectly understand how breaking a tradition may be disrupting, and surely it wasn't a "smart" choice to make the change without properly making it known in advance and even when already done "hide" its significance with that generic wording:
*) console - replace reserved characters to backup and certificate export file names with underscores;
Still, given the overall (IMHO) instability of what Mikrotik calls "stable", let alone the (still IMHO) pre-alpha status of what they call "beta" I am surprised that professionals with tens, hundreds or thousands devices would update/upgrade and only later find out the issue, besides what (for those people) motivates the "rush" to update to a "beta" (again read as pre-alpha) version?

Otherwise, one can stay on the whatever version is already working fine (if it is working fine) and have a look at the scripts/whatever that use spaces and replace them with underscore, it is not as I see it (unlike the data/time format issue, that the way it has been made seems pure folly to me) the end of the world.

The Iowa reference was just for the fun of it :lol: , a more elegant way than:
https://i0.wp.com/productdoctor.co.uk/w ... ommone.jpg
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 6:50 pm

But I guess it just seems odd to add more "file system things" like /file/add, ROSE/SMB, and DLNA now...while also adding restriction on stuff like spaces in file names. e.g. imagine someone one day may want to rename some movie with a space on RouterOS DLNA server...
very good point.

@jaclaz
it's ironic that i found the issue by updating to beta because tunnels on VRF was bricked in 7.14 "stable".
beta fixed the VRF issue, but now has this breaking change for (as far as i can see in all the replies) no justifiable reason.

if at the end of the day the reason is "well we would have to think about safety checks across all our system and that's quite a bit of work, so we decided to offload all this work to the user side in their own implementations when having to deal with this" then i am not satisfied.
 
tangent
Forum Guru
Forum Guru
Posts: 1404
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 9:24 pm

Windows defaults to UTF-16 as its internal representation but has strong support for working with UTF-8 in addition to the legacy CP-1252 and similar encodings.

All true, but irrelevant in this thread's context, where we're talking about file names, because they are always encoded as UTF-16 on NTFS, regardless of Windows code page setting. Windows will indeed accept UTF-8 and old-style code pages as input in File→Save dialogs and such, and when storing file data it will happily store bytes in any encoding you like, but when storing the name of that file in the NTFS directory structure, it's translated from its input representation to UTF-16. There is no field in the NTFS file name directory entry for an encoding value.

If it were otherwise, changing your Windows code page would appear to rename all files containing untranslatable code points, as it does on FAT-based filesystems.

Even if you stick to the 7-bit ASCII subset, that doesn't necessarily help because not all Windows code pages are like ISO-8859, pure extensions of ASCII. SHIFT-JIS, for instance, doesn't have a backslash character, at all. Thus this output:

$ echo 'C:\WINDOWS' | iconv -f utf-8 -t shift-jis | iconv -f shift-jis -t utf-8
iconv: iconv(): Illegal byte sequence
C:%

…which, in turn, required this ugly hack.

But that is still beside the point. Prohibiting spaces in file names breaks compatibility

What I don't get is my observation that Windows has more illegal characters than any other OS being used to justify this change in RouterOS behavior. Space is not on the list of illegal characters, not even for FAT, as long as the LFN rules are in effect, where the file names are in UCS-2. (Not the same as UTF-16; UCS-2 restricts you to the BMP, simplifying file name processing by making all code points the same size, unlike in UTF-16.)

I'm disagreeing with you on technicalities, Larsa, but not about whether spaces should be allowed in RouterOS file names. They're risky, but not to the point that they should be considered illegal by one vendor. Who else does that?

Seriously, gimme a list. What august company has MikroTik placed themselves among with this move?
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Thu Mar 14, 2024 10:01 pm

The technical stuff you write about might very well be true, and I truly agree regarding the poor choices that MT is about to make in this case. As I wrote in an another comment:

To maintain script compatibility as much as possible, I believe it would be easier to focus on allowed characters rather than the other way around. In addition to spaces, I can think of characters like underscores, hyphens and commas. There are probably a few others I've missed.
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Fri Mar 15, 2024 12:30 am

i'm just baffled by @mrz answer that "one can download files to other systems not supporting specific characters in filenames".
what OS out there does not support spaces in filenames?
i don't think anyone is asking for / or : or ; or \
i'd like , if possible but i wouldn't lose sleep over it if it was gone.
but space is ubiquitous. the point about someone using a mikrotik device as a file sharing server is amazing, imagine putting movies or photos on it and it swaps all spaces for _. afaik that's a big con and i'd be strongly considering literally any other NAS that doesn't have this issue.
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Fri Mar 15, 2024 1:40 am

as this is likely going to go nowhere, i guess i'l just have to fix this on a per device basis as i update them.

using this regexp to get the converted filename.
:local fileName [/file/get [/file/find name~"$systemDateTime"] name]
where $systemDateTime is the variable with time gathered from system for backup creation. would not work if there was more than one backup created at that exact second, but not a realistic concern.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3506
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: v7.15beta broke backup file naming

Fri Mar 15, 2024 4:23 am

as this is likely going to go nowhere, i guess i'l just have to fix this on a per device basis as i update them.
I can see being annoyed. But likely right call. I'm with @Larsa, these breaking changes should not be handled so cavalier.

I still like to know the rational here... and/or some list of what's allowed.

I'm guessing it's not ISIS (aka DECnet) VAX VMS file transfers – even the current restrictions wouldn't help there: uppercase only with no special chars e.g.
RAID1-PART1:[DUDE.FILES.DEFAULT.MIBS]MIKROTIK.TXT;1
 
S8T8
Frequent Visitor
Frequent Visitor
Posts: 81
Joined: Thu Sep 15, 2022 7:15 pm

Re: v7.15beta broke backup file naming

Sun Mar 17, 2024 4:41 pm

Guys, we have rights to complain but I doubt they will revert the change and allow spaces in identity.
So, what are the characters not allowed?
Any suggestion to fix the identity name in scripts or get the edited backup file name created by the system?
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11646
Joined: Thu Mar 03, 2016 10:23 pm

Re: v7.15beta broke backup file naming

Sun Mar 17, 2024 5:01 pm

So, what are the characters not allowed?

I'd rather ask "which characters are safe to use?" ... and the answer would be: the same as the last 50 years: US ASCII alphabet (a-z and A-Z), roman numerals (0-9), underscore (_), dash (-) ... and that's about it. So no punctuation marks, no other language specific characters, no financial characters, no math symbols, no vogon poetry, no nothing.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3506
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: v7.15beta broke backup file naming

Sun Mar 17, 2024 6:13 pm

Guys, we have rights to complain but I doubt they will revert the change and allow spaces in identity.
I'd think there is some "space" for debate on file names. ;). In 7.15beta6, /system/identity allows spaces and most things — so not sure issue with identity.



So, what are the characters not allowed?
That be another complaint too ;). No one here can answer definitively what characters are allowed where - it is NOT documented. Not for files or identity.
But in /system/identity, does get weird in allowed characters, but it's NOT new or changed. /system/ident does NOT process "backslash escapes", and treats \ as literal – e.g. without apply any escaping rules:
[me@Mikrotik] > :put "\E2\9D\A4"
❤
[me@Mikrotik] > /system identity set name="\E2\9D\A4"
[me@\E2\9D\A4] > # it should be a heart if used via SSH with xterm (which processes UTF-8)


Any suggestion to fix the identity name in scripts or get the edited backup file name created by the system?
Not really. The rename happens automatically. It's not an :error. So a typical ":do {} on-error=" or newer ":onerror err in={} do={}" cannot capture it.
Nor is the rename logged. So you can not even brute force a check by searching logs — which be a bad way since that's a PITA and fragile.
 
jaclaz
Long time Member
Long time Member
Posts: 667
Joined: Tue Oct 03, 2023 4:21 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 10:25 am

I'd rather ask "which characters are safe to use?" ... and the answer would be: the same as the last 50 years: US ASCII alphabet (a-z and A-Z), roman numerals (0-9), underscore (_), dash (-) ... and that's about it. So no punctuation marks, no other language specific characters, no financial characters, no math symbols, no vogon poetry, no nothing.
The issue with Vogon poetry is not ASCII, and replacing spaces with underscore doesn't affect its beauty, the problem is with commas, periods and other punctuation, which are an integral part of the typical structure of the metrics:

Think of the immortal line:
Gashee morphousite, thou expungiest quoopisk!
it would become something like:
Gashee_morphousite-_thou_expungiest_quoopisk_
losing much of its original charme..
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 4:12 pm

I'd prefer if we focus on OP's issue of how to best preserve script compatibility when it comes to potential limitations in file naming.

In my opinion, at an absolute minimum, "spaces" and printable 7-bit ASCII characters that are compatible across common file systems (Windows, Linux, macOS) should be supported. This would provide a reasonable compromise between flexibility and compatibility.

If Mikrotik implements the current proposal, they SHOULD take existing files into account to maintain script compatibility during upgrades. Otherwise, it may very will disrupt critical support systems like backup, version control, and monitoring. This would ONCE AGAIN put the burden on the customer to modify their existing infrastructure to accommodate compatibility changes.

One way to maintain a reasonably fair script compatibility regardless of character choice could be to automatically rename existing files in the ROS file system during an upgrade and replace "non-approved characters" with '_', and implement a corresponding translation mechanism in ROS for file-related calls where "non-approved characters" are automatically replaced with '_'. However, the risk of filename collisions is directly related to the number of disallowed characters. Using the suggested "allowed characters" would minimize that risk.

Overall, prioritizing script compatibility and considering the implications of file naming restrictions on existing infrastructure are crucial steps for ensuring a smooth transition during upgrades and minimizing disruptions for end users.
 
t0mm13b
just joined
Posts: 18
Joined: Sat Mar 04, 2023 5:11 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 5:39 pm

I'd prefer if we focus on OP's issue of how to best preserve script compatibility when it comes to potential limitations in file naming.

In my opinion, at an absolute minimum, "spaces" and printable 7-bit ASCII characters that are compatible across common file systems (Windows, Linux, macOS) should be supported. This would provide a reasonable compromise between flexibility and compatibility.
There is a thing called best practices, and is well known through-out that any filename with spaces are a headache to deal with and pain in the ass, and not a recommended approach.

Backup tools / scripts / CLI can trip up very easily with spaces in filenames, more often than not, they would have to be escaped with a backslash to tell the shell that it is "one word" for filename with spaces due to how the shell tokenizes the input for command processing. Even NFS systems do not like spaces in the filenames either. There is a very good reason why, out of habit, to use underscore instead.

The problem stems from Microsoft and blame lies with them, ever since Windows 95 came out, MS introduced the new vFAT system with LFS, which lifted the restrictions on the DOS 8.3 format for files, as MS was popular and installed left/right/centre, the nomenclature to use spaces in the filenames became the norm and adapted everywhere, office software, even its used on AD domain controllers etc, now the users have that mentality of spaces in filenames, then when they end up using Linux, Mikrotik etc, expect the same "standard" as result, i.e. spaces in filenames.

Never write scripts that is reliant on files with spaces on them, if you need to, replace it with an underscore.

See this Superuser question/answer going back two decades ago to yield some clues.

"It is just safer to not use spaces"
 
infabo
Long time Member
Long time Member
Posts: 695
Joined: Thu Nov 12, 2020 12:07 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 6:04 pm

spaces in filename...

The real question to be asked is: why do you need them?
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 6:45 pm

@infabo The real question to be asked is: why do you need them?

@infabo: If you had read the thread from the beginning, you wouldn't have needed to ask that question.

@t0mm13b: The core issues are compatibility and why Mikrotik's proposed changes would break existing scripts and support systems. Debating "best practices" misses the point entirely.

Furthermore, modern file systems have no inherent problems storing spaces and neither does NFS. You can easily mount NFS shares that include spaces. The "spaces" issue is primarily a legacy shell scripting problem which is why the 'noclobber' option was invented.

To truly understand the fundamental issue of breaking compatibility, you need experience with mission critical support systems for backup, version control and monitoring. One problem that many might not be aware of is that since Mikrotik isn't a major player in the enterprise sector, ready-made support for ROS is often lacking and must be developed in-house. Changes that break compatibility force unnecessary spending on consultants and internal rework, redirecting resources away from more productive work.

Hobbyists or people working with smaller setups often lack this insight.
 
infabo
Long time Member
Long time Member
Posts: 695
Joined: Thu Nov 12, 2020 12:07 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 6:56 pm

Larsa, I've read the starting post. I do understand that it breaks OPs scripts.

My question remains valid: why do you need spaces? or is it just a personal decision? If so, then revise your personal preference to not using spaces because MT enforces their own "no spaces" preference.

BTW: you can already adjust your backup scripts to not use spaces in filenames before updating to 7.15. Well, if your production network consists of 1000+ of managed ROS devices and you don't have a routine for deploying scripts to the devices - OK fine. Then spaces is your tiniest problem.
 
t0mm13b
just joined
Posts: 18
Joined: Sat Mar 04, 2023 5:11 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 7:05 pm

@infabo The real question to be asked is: why do you need them?

Furthermore, modern file systems have no inherent problems storing spaces and neither does NFS. You can easily mount NFS shares that include spaces. The "spaces" issue is primarily a legacy shell scripting problem which is why the 'noclobber' option was invented.
What has noclobber got to do with "legacy shell scripting problem" or spaces in filename?
Hint: it has nothing to do with spaces in the filename.
To truly understand the fundamental issue of breaking compatibility, you need experience with mission critical support systems for backup, version control and monitoring. One problem that many might not be aware of is that since Mikrotik isn't a major player in the enterprise sector, ready-made support for ROS is often lacking and must be developed in-house. Changes that break compatibility force unnecessary spending on consultants and internal rework, redirecting resources away from more productive work.

Hobbyists or people working with smaller setups often lack this insight.
To be fair, MT did state the obvious, that's on you relying on using spaces in filenames.
What's new in 7.15beta4 (2024-Mar-04 08:04):

<snip>
*) console - replace reserved characters to backup and certificate export file names with underscores;
</snip>
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 7:07 pm

SUP-147326 - "v7.15beta brakes file naming and script compatibility"
Last edited by Larsa on Mon Mar 18, 2024 7:16 pm, edited 1 time in total.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 7:12 pm

@t0mm13b: *) console - replace reserved characters to backup and certificate export file names with underscores;

Yes @t0mm13b, you've nailed the core issue of this thread!
--

@infabo: I think it was stated pretty clear in the previous post. Is there anything I need to clarify?
 
infabo
Long time Member
Long time Member
Posts: 695
Joined: Thu Nov 12, 2020 12:07 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 7:16 pm

@Larsa I have not addressed any question towards you.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 7:22 pm

Okay, I thought your question was: 'My question remains valid: why do you need spaces? Or is it just a personal decision?' (Or did I miss something??)
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3506
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 7:49 pm

@Larsa I have not addressed any question towards you.
Why does it matter what the OP's reason?
Maybe his boss likes seeing friendly names in a folder, or he inherited the script.... If it's the backup system has rules using file names, that also have to change to any retention/etc rules as well – requiring even more time to mitigate. If used professionally, time is money. So these changes cost someone money & increase TCO of Mikrotik.

The underlying issue is there is a breaking change, without some associate bug. OP was working & then after upgrade not. Nothing in docs said don't use space & nothing still does – and scripting docs says use quotes to pass spaces.

Is there some buffer overflow issue that this fixes, if so it should be flagged as security update.
 
Railander
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 85
Joined: Thu Jun 16, 2016 11:30 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 9:26 pm

@Larsa I have not addressed any question towards you.
i'm OP.
no need to be passive-aggressive.
issue is you could use it before, you cant use it now. this thread would not exist if you could never use it.
backup pulls name from identity. identity has reserved characters (such as spaces). automated backup process stops working after an update because the backup file name no longer matches the identity name.
are you going to do the laborious work of fixing automation for hundreds of devices as they are updated because mikrotik felt like breaking compatibility and having the headache of missing backups for one or another that you missed fixing, for... no reason? again, this has worked since RouterOS's inception, there was no real reason that they've explained why they had to do it other than people in the comments (not from mikrotik) saying "it makes sense" even though something making sense or not is not a reason and either way the question is not for them to answer but mikrotik.

tl;dr
if it aint broke, don't fix it.
 
t0mm13b
just joined
Posts: 18
Joined: Sat Mar 04, 2023 5:11 pm

Re: v7.15beta broke backup file naming

Mon Mar 18, 2024 9:59 pm

@Larsa I have not addressed any question towards you.
<snip/>

Is there some buffer overflow issue that this fixes, if so it should be flagged as security update.
Only time will tell, checked CVE, nada, blog at mt has not been updated recently.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1068
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: v7.15beta broke backup file naming

Thu Mar 21, 2024 10:27 pm

Regarding 7.15beta8 (2024-Mar-21 09:12) and inconsistent rules for valid characters in filenames. Check viewtopic.php?p=1065213#p1065213

Who is online

Users browsing this forum: Amazon [Bot], GoogleOther [Bot], jurajhampel and 35 guests