got same strange behaviour just after adding another user and its ssh key, thats second ssh key I added, before that all ssh connections was OK
while experimanting with ssh access to RB for scripting got this, for example:
$ sudo -u nobody ssh -i /tmp/id_dsa.pub openvpn@m66 -vvv
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to m66 [X.X.X.X] port 22.
debug1: connect to address X.X.X.X port 22: Connection timed out
debug1: Connecting to m66 [X.X.X.X] port 22.
debug1: connect to address X.X.X.X port 22: Connection timed out
ssh: connect to host m66 port 22: Connection timed out
$
another one:
$ sudo -u nobody ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentityFile=/etc/openvpn/keys/id_dsa.pub -o UserKnownHostsFile=/etc/openvpn/keys/known_hosts -vvvvvv openvpn@gw.m66
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 gw.m66
debug1: permanently_drop_suid: 99
Could not create directory '/.ssh'.
debug3: Not a RSA1 key file /etc/openvpn/keys/id_dsa.pub.
debug1: identity file /etc/openvpn/keys/id_dsa.pub type 2
debug1: identity file /etc/openvpn/keys/id_dsa.pub-cert type -1
ssh_exchange_identification: Connection closed by remote host
and another:
$ ssh -i id_dsa.pub openvpn@gw.m66 -vvv
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 gw.m66
debug1: permanently_drop_suid: 2525
debug3: Not a RSA1 key file id_dsa.pub.
debug1: identity file id_dsa.pub type 2
debug1: identity file id_dsa.pub-cert type -1
ssh_exchange_identification: Connection closed by remote host
and soon later I was surprised by finding what no ssh connections are possible for some IPs on RB interfaces, thanks god I found one IP on vlan interface for which ssh connection was possible to reboot router