ROS 5.1 using ssh

RouterOS General
1

Hello!
I’m having a problem with ssh connection.
I’m writing a php script, which should automatically connect to MikroTik and get information from it.
When the script executes a command on remote MikroTik router

if (!($stream = ssh2_exec($con, $commandhere))) {
					echo "unable to execute command".$b;
				}
				stream_set_blocking($stream, true);
				if ($stream) {
					echo "starting while with information get".$b;
					while (($buffer = fgets($stream, 4096)) !== false) {
						echo "each line: ".$buffer.$b;

here is the problem: router returns me tonns of encrypted (or binary) content:
each line: )™Õ³ô%0oä
each line: @h ý?RuneMag1NONEBBBBB@((((((((((((((( (((((((
each line: ‰‰‰ ‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰((((((
each line: ((((@
… etc.)

					}
					if (!feof($stream)) {
						echo "fgets fail".$b;
					}
				}

The same script worked perfect in lower versions

I noticed that in v5.x when executing, for example ping command, from terminal, the first row contains tabulated column names, there were none in previouse versions… may it be, that these symbols can’t pass through ssh connection some how?..

2

Same probem with ssh ROS 5.1
Here are the details as full as possible

After installation of RouterOS v5.1 on one of our routerboards (RB433AH) some problem have appeared with running remote commands through ssh - commands are sent from ssh-client=HP-UX SSH-2.0-OpenSSH_3.1p1 to ssh-server=MTik Router OS.
Main problem - in way of how output of such commands is sent from MTik ssh server.

In case of Router OS v 3.x and 4.x output always the same:

  1. it always contains EOF after the output of the command;
  2. output always has the same order, it arranged always in same way -
    .. confirm_rwindow … output_of_command … EOF…


    That’s the example of “normal” output from RouterOS v3.16 (with ssh -vvv debug parameter on ssh client on HP-UX)

=================v 3.16 Example ======================
… output omited …
debug1: Remote protocol version 1.99, remote software version OpenSSH_2.3.0_Mikrotik_v2.9
debug1: match: OpenSSH_2.3.0_Mikrotik_v2.9 pat OpenSSH_2.3.0*
Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.1p1
… output omited …
debug1: Sending command: /interface wireless registration-table print stats where radio-name=10.52.20.5
debug1: channel request 0: exec
debug2: callback done
debug1: channel 0: open confirm rwindow 0 rmax 16384
debug2: channel 0: rcvd adjust 32768
0 ;;; client5
interface=wlan1 radio-name=“10.52.20.5” mac-address=00:0C:42:31:C7:C0
ap=no wds=no rx-rate=“36Mbps” tx-rate=“36Mbps” packets=17461911,14967495
bytes=1887290400,2750906608 frames=10980180,9475807
frame-bytes=1909004554,2756820902 hw-frames=17118424,104344544
hw-frame-bytes=2661929939,3115194687 tx-frames-timed-out=0
uptime=1w2d17h15m22s last-activity=0s signal-strength=-63dBm@18Mbps
signal-to-noise=34dB strength-at-rates=-63dBm@18Mbps 0s,-64dBm@36Mbps 0s
tx-signal-strength=-61dBm tx-ccq=90% rx-ccq=94% p-throughput=23279
nstreme=yes framing-mode=best-fit framing-limit=3200
routeros-version=“3.10” last-ip=10.52.0.1 802.1x-port-enabled=yes
compression=no wmm-enabled=no

debug1: channel 0: rcvd eof
debug1: channel 0: output open → drain
debug1: channel 0: obuf empty
debug1: channel 0: close_write
debug1: channel 0: output drain → closed
… output omited …

=================end of example=======================


In case of Router OS v 5.x output has some differences:

  1. there’s no EOF after the output of the command;
  2. output always arranged in different way -
    ..confirm_rwindow..some_ssh_channel_commands..part_of_output..some_ssh_channel_commands..part_of_output..
    OR
    ..confirm_rwindow..output..some_ssh_channel_commands..
    and so on

That’s the 2 examples of “abnormal” output from RouterOS v5.1 (with ssh -vvv debug parameter on ssh client on HP-UX)

=================v 5.1 Example 1======================

… output omited …
debug1: Remote protocol version 2.0, remote software version ROSSSH
debug1: no match: ROSSSH
Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.1p1
… output omited …
debug1: Sending command: /interface wireless registration-table print stats where radio-name=10.52.19.3
debug1: channel request 0: exec
debug2: callback done
debug1: channel 0: open confirm rwindow 2621440 rmax 262144
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
7 ;;; client3
interface=wlan1 radio-name=“10.52.19.3” mac-address=00:0C:42:63:BF:BD
ap=no wds=no bridge=no rx-rate=“36.0Mbps” tx-rate=“36.0Mbps”
packets=728699,59938 bytes=107706513,3977622 frames=676880,56875
frame-bytes=104548417,3687570 hw-frames=748913,27625143
debug1: channel 0: rcvd close
debug1: channel 0: output open → drain
debug1: channel 0: close_read
debug1: channel 0: input open → closed
debug3: channel 0: will not send data after close
debug1: channel 0: obuf empty
debug1: channel 0: close_write
debug1: channel 0: output drain → closed
debug1: channel 0: almost dead
debug1: channel 0: gc: notify user
debug1: channel 0: gc: user detached
debug1: channel 0: send close
debug1: channel 0: is dead
debug1: channel 0: garbage collecting
debug1: channel_free: channel 0: client-session, nchannels 1
debug3: channel_free: status: The following connections are open:
#0 client-session (t4 r0 i3/0 o3/0 fd -1/-1)

debug3: channel_close_fds: channel 0: r -1 w -1 e 7
debug1: fd 1 clearing O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.1 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 0
hw-frame-bytes=134137840,1267263937 tx-frames-timed-out=0
uptime=1d12h58m14s last-activity=0ms signal-strength=-72dBm@6Mbps
signal-to-noise=25dB signal-strength-ch0=-72dBm
strength-at-rates=-72dBm@6Mbps 0ms,-71dBm@18Mbps 1d12h57m55s150ms,-
71dBm@36Mbps 60ms
tx-signal-strength=-75dBm tx-ccq=99% rx-ccq=100% p-throughput=23846
nstreme=yes framing-mode=best-fit framing-limit=2900
routeros-version=“4.5” last-ip=10.52.1.1 802.1x-port-enabled=yes
management-protection=no compression=no wmm-enabled=no

=================end of example=======================


=================v 5.1 Example 2======================

… output omited …
debug1: Remote protocol version 2.0, remote software version ROSSSH
debug1: no match: ROSSSH
Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.1p1
… output omited …
debug1: Sending command: /interface wireless registration-table print stats where radio-name=10.52.19.3
debug1: channel request 0: exec
debug2: callback done
debug1: channel 0: open confirm rwindow 2621440 rmax 262144
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: rcvd close
debug1: channel 0: output open → drain
debug1: channel 0: close_read
debug1: channel 0: input open → closed
debug3: channel 0: will not send data after close
debug3: channel 0: will not send data after close 7 ;;; Client3
interface=wlan1 radio-name=“10.52.19.3” mac-address=00:0C:42:63:BF:BD
ap=no wds=no bridge=no rx-rate=“36.0Mbps” tx-rate=“36.0Mbps”
packets=730767,60011 bytes=107940742,3980531 frames=678778,56944
frame-bytes=104773936,3690137 hw-frames=750927,27664259
hw-frame-bytes=134441191,1269046253 tx-frames-timed-out=0
uptime=1d13h1m50s last-activity=0ms signal-strength=-74dBm@6Mbps
signal-to-noise=24dB signal-strength-ch0=-74dBm
strength-at-rates=-74dBm@6Mbps 0ms,-71dBm@18Mbps 1d13h1m31s30ms,-
74dBm@36Mbps 30ms
tx-signal-strength=-75dBm tx-ccq=100% rx-ccq=95% p-throughput=23893
nstreme=yes framing-mode=best-fit framing-limit=2900
routeros-version=“4.5” last-ip=10.52.1.1 802.1x-port-enabled=yes
management-protection=no compression=no wmm-enabled=no


debug1: channel 0: obuf empty
debug1: channel 0: close_write
debug1: channel 0: output drain → closed
debug1: channel 0: almost dead
debug1: channel 0: gc: notify user
debug1: channel 0: gc: user detached
debug1: channel 0: send close
debug1: channel 0: is dead
debug1: channel 0: garbage collecting
debug1: channel_free: channel 0: client-session, nchannels 1
debug3: channel_free: status: The following connections are open:
#0 client-session (t4 r0 i3/0 o3/0 fd -1/-1)

debug3: channel_close_fds: channel 0: r -1 w -1 e 7
debug1: fd 1 clearing O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.1 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 0
=================end of example=======================


Such behaviour of RouterOS v 5.х ssh server in some cases results in wrong representation of data, received as output of remote ssh commands, in web browsers.
Here is simple cgi perl script, which runs on the same our HP-UX workstaion with the same ssh client (SSH-2.0-OpenSSH_3.1p1):

============start of script===========
#!/usr/bin/perl -W
print “Content-Type: text/html; charset=windows-1251\n\n”;
print “<meta http-equiv="Pragma" content="no-cache">\n”;
print “Snmptest host_name\n”;
print “<BODY bgcolor="yellow" link="black" alink="black" vlink="black">”;
$cmd_mtik = "/opt/ssh/bin/ssh -l test 10.52.0.19 "/interface wireless registration-table print stats where radio-name=10.52.19.3" ";
if (open ( MT, "$cmd_mtik | ")){
while ()
{
print ;
}
close(MT);
}
print “”;
============end of script==============

With receiving data from router os v3.х or 4.x it always successfully displays such data in web browser.
With receiving data from router os v5.х - with some data browser’s window is empty (for example most of the registration-table items, or /ip firewall export or so on).
I tested this script with 2 versions of apache on our HP-UX worckstaion - there’s no differents in results.

Is that some kind of bug in routeros v5.x ssh server that could be fixed?
Or may be is it a result of shh-client and ssh-server incompatibility? (“rwindow 2621440 rmax 262144” ?)

3

tried to run perl script and i get correct output

ssh used:

$ ssh -V
OpenSSH_5.5p1 Debian-4ubuntu5, OpenSSL 0.9.8o 01 Jun 2010

but output of script was incorrect according to HTML documentation. Nevertheless i got correct output every single time against RouterOS 5.2 RB433AH

Have any of you considered to use API to get data in controlled way than some interactive CLI.

4

I have not - there was no need to use any additioanl API, untill 5.x everything was OK.

Do you meen there’s no problem with ros 5.x ssh server?
EOF is not necessary after output of the command in ssh answer, аnd since 5.x we must use some additioanl API to parse mtik ssh server answers?

update: perl script in HP-UX’s console always displays data, problem begins when i try to get this data by runnig this script through apache.
I even tried in that case to sniff ssh traffic by tcpdump - there is difference in packet’s length in case when i run the script from hp-ux console and when i run the same script from apache (via web browser ..). When script runs from apache - ssh answer is much shorter.
As a result in case of apache some data are not displayed, some data - displayed sometimes, some - always.

5

exactly the same problem with PHP script insted PERL:

#!/usr/bin/php -q

<?php print "Content-Type: text/html; charset=windows-1251\n\n"; print "\n"; print "Snmptest host_name\n"; print ""; $command="/opt/ssh/bin/ssh -l test 10.52.0.19 \"/interface wireless registration-table print stats where radio-name=10.52.19.3\""; exec($command, $output, $ret_var); foreach($output as $line){ print $line . PHP_EOL; } print ""; ?>

In case with ros v5.1 most items in wireless registration-table are not displaied when this script ran from apache.
When command is just “/interface wireless registration-table print stats” i can see in web browser only 3-5 first items of about 25 items total in registration-table (numbers of this “visible” items are 0,1,2,3,4)
In case with ros v3.x/4.x everything is ok - i get full list of registration-table’s items.

6

Same problem here.

from freebsd when executed via crontab:

debug1: tty_make_modes: no fd or tio
debug1: Sending command: system clock print
debug2: channel 0: request exec confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 2621440 rmax 262144
debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: channel 0: close_read
debug2: channel 0: input open → drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug2: channel 0: input drain → closed

any command i send it will not return any results.
This ony happens when crontab runs the script. exactly same script works just fine from terminal.

output when script works from terminal:

debug1: Sending command: system clock print
debug2: channel 0: request exec confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 2621440 rmax 262144
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0



ssh -v
OpenSSH_5.8p2_hpn13v11 FreeBSD-20110503, OpenSSL 0.9.8q 2 Dec 2010

and mikrotik are rb750 or rb751 with 5.18