Community discussions

MikroTik App
 
byteit101
just joined
Topic Author
Posts: 6
Joined: Fri Jul 19, 2024 6:37 am

[BUG] REST endpoint producing invalid JSON

Mon Jul 22, 2024 2:46 am

I was attempting to read a backup file, and realized that the REST endpoint in 7.15.2 produces invalid JSON:
$ curl -X POST -u admin:${PASSWORD} -H "content-type: application/json" https://${YOUR_IP}/rest/file/read --data '{"file":"my-test.backup","chunk-size":"4096","offset":"0"}' | jq
parse error: Unfinished string at EOF at line 1, column 20
That seems be be because there are raw null bytes, which confuses the shell. Using NodeJS to bypass shell issues reports
SyntaxError: Bad control character in string literal in JSON at position 16

Again, that is at an un-escaped null byte, which according to the JSON spec are not allowed un-escaped: https://www.json.org/json-en.html
'0020' . '10FFFF' - '"' - '\'
 
byteit101
just joined
Topic Author
Posts: 6
Joined: Fri Jul 19, 2024 6:37 am

Re: [BUG] REST endpoint producing invalid JSON

Thu Jul 25, 2024 11:42 pm

I've created a short script that helps parse this broken JSON, but a real fix is still needed from Mikrotik.

Typescript/Javascript:
async getBinaryBrokenJSON(url: string, upload: {[key: string]: string | string[] }) : Promise<Buffer>
{
	var response = await fetch(url, {
		method: 'POST',
		body: JSON.stringify(upload),
		headers: {'Authorization': YOUR_AUTH_STRING, 'Content-type': 'application/json'},
	});
	var data_bin = new Uint8Array(await response.arrayBuffer());
	var fixed: number[] = [];
	const append = (b:number) => {
		if (b > 9)
			fixed.push(65 + (b-10)); // 'A'+y
		else
			fixed.push(0x30 | b); // '0'+y
	}
	data_bin.forEach(b => {
		if (b < 0x20 || b >= 0x80)
		{
			fixed.push(0x5c); // '\'
			fixed.push(0x75); // 'u'
			fixed.push(0x30); // '0'
			fixed.push(0x30); // '0'
			append(b >> 4);
			append(b & 0xF);
		}
		else
			fixed.push(b);
	})
	var data = JSON.parse(new TextDecoder("ascii").decode(new Uint8Array(fixed)));
	// check data for error responses here
	return Buffer.from(data[0].data as string, "binary");
}
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1452
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: [BUG] REST endpoint producing invalid JSON

Thu Jul 25, 2024 11:47 pm

Since this is only a userforum, please report bugs directly to Mikrotik by opening a ticket at https://mikrotik.com/support or sending an email to 'support@mikrotik.com'.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4011
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: [BUG] REST endpoint producing invalid JSON

Fri Jul 26, 2024 9:55 pm

I too recommend file at help.mikrotik.com. The RFC for JSON requires UTF-8 formatting, so in that sense it's a bug. And it is kinda double-whammy since you can't use `jq` /etc. to fix the strings, since it's fails on parsing... before you could even call jq expressions.

But the underlying issue, despite being Linux-based, is the default charset is Windows-1252 (Latin-1 / ISO 8859-1)... So the JSON from REST kinda follow this, odd, logic that strings are single-byte using Windows-1252 with only \" double quotes escaped (AFAIK). Basically RouterOS does not deal with UTF-8 (or Unicode in general) ANYWHERE, so not just reading file from JSON/REST API that is the only issue here. So you're JS/TS byte-reader seems like a reasonable workaround.

A better solution likely be the REST API has some method to get a file using application/octet-stream MIME-type as a binary transfer (over HTTP) and avoid the JSON entirely. Perhaps with some "Accept" / "Content-Type" header, or different/special URL for retrieving file via HTTP. Or perhaps supposing WebDAV for the RouterOS file system.

Who is online

Users browsing this forum: bbmikrotik and 7 guests