[BUG] REST endpoint producing invalid JSON

RouterOS Scripting
1

I was attempting to read a backup file, and realized that the REST endpoint in 7.15.2 produces invalid JSON:

$ curl -X POST -u admin:${PASSWORD} -H "content-type: application/json" https://${YOUR_IP}/rest/file/read --data '{"file":"my-test.backup","chunk-size":"4096","offset":"0"}' | jq
parse error: Unfinished string at EOF at line 1, column 20

That seems be be because there are raw null bytes, which confuses the shell. Using NodeJS to bypass shell issues reports

SyntaxError: Bad control character in string literal in JSON at position 16

Again, that is at an un-escaped null byte, which according to the JSON spec are not allowed un-escaped: https://www.json.org/json-en.html

‘0020’ . ‘10FFFF’ - ‘"’ - ''

2

I’ve created a short script that helps parse this broken JSON, but a real fix is still needed from Mikrotik.

Typescript/Javascript:

async getBinaryBrokenJSON(url: string, upload: {[key: string]: string | string[] }) : Promise<Buffer>
{
	var response = await fetch(url, {
		method: 'POST',
		body: JSON.stringify(upload),
		headers: {'Authorization': YOUR_AUTH_STRING, 'Content-type': 'application/json'},
	});
	var data_bin = new Uint8Array(await response.arrayBuffer());
	var fixed: number[] = [];
	const append = (b:number) => {
		if (b > 9)
			fixed.push(65 + (b-10)); // 'A'+y
		else
			fixed.push(0x30 | b); // '0'+y
	}
	data_bin.forEach(b => {
		if (b < 0x20 || b >= 0x80)
		{
			fixed.push(0x5c); // '\'
			fixed.push(0x75); // 'u'
			fixed.push(0x30); // '0'
			fixed.push(0x30); // '0'
			append(b >> 4);
			append(b & 0xF);
		}
		else
			fixed.push(b);
	})
	var data = JSON.parse(new TextDecoder("ascii").decode(new Uint8Array(fixed)));
	// check data for error responses here
	return Buffer.from(data[0].data as string, "binary");
}
3

Since this is only a userforum, please report bugs directly to Mikrotik by opening a ticket at https://mikrotik.com/support or sending an email to ‘support@mikrotik.com’.

4

I too recommend file at help.mikrotik.com. The RFC for JSON requires UTF-8 formatting, so in that sense it’s a bug. And it is kinda double-whammy since you can’t use jq /etc. to fix the strings, since it’s fails on parsing… before you could even call jq expressions.

But the underlying issue, despite being Linux-based, is the default charset is Windows-1252 (Latin-1 / ISO 8859-1)… So the JSON from REST kinda follow this, odd, logic that strings are single-byte using Windows-1252 with only " double quotes escaped (AFAIK). Basically RouterOS does not deal with UTF-8 (or Unicode in general) ANYWHERE, so not just reading file from JSON/REST API that is the only issue here. So you’re JS/TS byte-reader seems like a reasonable workaround.

A better solution likely be the REST API has some method to get a file using application/octet-stream MIME-type as a binary transfer (over HTTP) and avoid the JSON entirely. Perhaps with some “Accept” / “Content-Type” header, or different/special URL for retrieving file via HTTP. Or perhaps supposing WebDAV for the RouterOS file system.