I’m writing a script that interacts with a run-of-the-mill DSL modem. When the fetch tool gets a “OK” response (HTTP/1.0 200 OK) from the modem, the tool fails and returns “status: failed”. I bellieve I pinned it down to bad practice in the modem HTTP service programming: The response header has a mix of \n and \r\n line breaks. If take the modem’s response, reformat the line breaks, and use netcat to simulate the same conversation, the fetch tool accepts this response. Look at these two examples showing the router terminal and the responses received by the fetch tool:
1. Fetch request to modem:
/system> /tool fetch url="> http://10.51.0.65/m_status.htm> " http-header-field=(“Authorization: Basic **************************”) output=user
status: failedaction failed (6)
/system>
Response data from modem:
48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f > HTTP/1 .0 200 O
4b > 0d 0a > > K..
43 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 > Conten t-type:
74 65 78 74 2f 68 74 6d 6c > 0a 0a > 3c 68 74 6d 6c > text/htm l..<html
3e 0a 3c 68 65 61 64 3e 0a3c 4d 45 54 41 20 68 > >. .<META h
2. Fetch request to netcat:
/tool fetch url="> http://10.51.0.42/m_status.htm> " http-header-field=(“Authorization: Basic *********************”) output=user
status: downloading
downloaded: 0KiB
total: 0KiB
duration: 2s
data:
Response data from netcat:
48 54 54 50 2f 31 2e 30 20 32 30 30 20 4f > HTTP/1 .0 200 O
4b > 0d 0a > > K..
43 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 > Conten t-type:
74 65 78 74 2f 68 74 6d 6c > 0d 0a > > text/htm l..
0d 0a >
3c 68 74 6d 6c 3e 0d 0a > ..
As you can see, the modem response was rejected because the HTTP header line was terminated with 0x0A (LF). The netcat response is the same except that its header is terminated with 0x0D 0x0A (CR+LF).
Although the modem’s program is breaking the rules, the fact is that the modem’s responses aren’t rejected by Firefox, Chrome or IE. There’s talk on this very subject at https://stackoverflow.com/questions/5757290/http-header-line-break-style.
The line terminator for message-header fields is the sequence CRLF. However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR.
Would Mikrotik please fix this in a future release by changing the fetch tool so it parses the header by LF instead of CR+LF?