Reducing OpenSSH Login Time
We are seeing slow ssh connections from our servers. After some analysis , we found that the issue is with default SSH Configuration and not because of slow network.
We did a verbose while using ssh and can see ssh getting stucked at a particular point for all outgoing ssh connections and for incoming connections getting stucked at another point.
Following below shows ssh getting stucked for around 1 minutes at "Next authentication method: gssapi-with-mic" for all outgoing connections from the server.
==================================================
[zaman@server ~]$ time ssh -v 192.168.2.171
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.2.171 [192.168.2.171] port 22.
debug1: Connection established.
debug1: identity file /home/zaman/.ssh/identity type -1
debug1: identity file /home/zaman/.ssh/id_rsa type -1
debug1: identity file /home/zaman/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_4*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 br="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.2.171' is known and matches the RSA host key.
debug1: Found key in /home/zaman/.ssh/known_hosts:14
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: gssapi-with-mic ------> Waiting
here for around 1 minutes.
debug1: An invalid name was supplied
Cannot determine realm for numeric host address
debug1: An invalid name was supplied
Cannot determine realm for numeric host address
debug1: An invalid name was supplied
debug1: An invalid name was supplied
debug1: Next authentication method: publickey
debug1: Trying private key: /home/ssdg/.ssh/identity
debug1: Trying private key: /home/ssdg/.ssh/id_rsa
debug1: Trying private key: /home/ssdg/.ssh/id_dsa
debug1: Next authentication method: password
ssdg@192.168.2.171's password:
.....
.....
real 1m43.363s
user 0m0.005s
sys 0m0.004s
===================================================
It is getting stucked while trying to do a GSI authentication and since we are not using any kind of GSI authentication , we can disable GSI by editing the SSH Config file as shown below.
$ cat .ssh/config
GSSAPIAuthentication no
$ chmod 400 .ssh/config
After doing the above configuration , it can be noticed that the time to do an SSH has come down drastically.
Next we will see another issue of SSH taking long time.This issue can be resolved by adding a config parameter in SSH Server Config File (sshd_config) .
For the above issue , SSH is getting stucked at "debug1: SSH2_MSG_SERVICE_ACCEPT received" as shown below.
$ time ssh -v 192.168.2.149
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /home/zaman/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.2.149 [192.168.2.149] port 22.
debug1: Connection established.
debug1: identity file /home/zaman/.ssh/identity type -1
debug1: identity file /home/zaman/.ssh/id_rsa type -1
debug1: identity file /home/zaman/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 br="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.2.149' is known and matches the RSA host key.
debug1: Found key in /home/zaman/.ssh/known_hosts:5
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received ----------waiting here for around 30 seconds
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/zaman/.ssh/identity
debug1: Trying private key: /home/zaman/.ssh/id_rsa
debug1: Trying private key: /home/zaman/.ssh/id_dsa
debug1: Next authentication method: password
zaman@192.168.2.149's password:
real 0m21.343s
user 0m0.008s
sys 0m0.003s
====================
To resolve the above issue , the following parameter needs to be configured in /etc/ssh/sshd_config file as shown below.
UseDNS no
Restart SSH service after doing the above configuration change
$service sshd restart
$ time ssh -v 192.168.2.149
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /home/zaman/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.2.149 [192.168.2.149] port 22.
debug1: Connection established.
debug1: identity file /home/zaman/.ssh/identity type -1
debug1: identity file /home/zaman/.ssh/id_rsa type -1
debug1: identity file /home/zaman/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 br="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.2.149' is known and matches the RSA host key.
debug1: Found key in /home/zaman/.ssh/known_hosts:5
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/ssdg/.ssh/identity
debug1: Trying private key: /home/ssdg/.ssh/id_rsa
debug1: Trying private key: /home/ssdg/.ssh/id_dsa
debug1: Next authentication method: password
ssdg@192.168.2.149's password:
real 0m1.096s
user 0m0.006s
sys 0m0.004s
Thus we had seen how to reduce SSH Login time for two different issues.
We did a verbose while using ssh and can see ssh getting stucked at a particular point for all outgoing ssh connections and for incoming connections getting stucked at another point.
Following below shows ssh getting stucked for around 1 minutes at "Next authentication method: gssapi-with-mic" for all outgoing connections from the server.
==================================================
[zaman@server ~]$ time ssh -v 192.168.2.171
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.2.171 [192.168.2.171] port 22.
debug1: Connection established.
debug1: identity file /home/zaman/.ssh/identity type -1
debug1: identity file /home/zaman/.ssh/id_rsa type -1
debug1: identity file /home/zaman/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_4*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 br="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.2.171' is known and matches the RSA host key.
debug1: Found key in /home/zaman/.ssh/known_hosts:14
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: gssapi-with-mic ------> Waiting
here for around 1 minutes.
debug1: An invalid name was supplied
Cannot determine realm for numeric host address
debug1: An invalid name was supplied
Cannot determine realm for numeric host address
debug1: An invalid name was supplied
debug1: An invalid name was supplied
debug1: Next authentication method: publickey
debug1: Trying private key: /home/ssdg/.ssh/identity
debug1: Trying private key: /home/ssdg/.ssh/id_rsa
debug1: Trying private key: /home/ssdg/.ssh/id_dsa
debug1: Next authentication method: password
ssdg@192.168.2.171's password:
.....
.....
real 1m43.363s
user 0m0.005s
sys 0m0.004s
===================================================
It is getting stucked while trying to do a GSI authentication and since we are not using any kind of GSI authentication , we can disable GSI by editing the SSH Config file as shown below.
$ cat .ssh/config
GSSAPIAuthentication no
$ chmod 400 .ssh/config
After doing the above configuration , it can be noticed that the time to do an SSH has come down drastically.
Next we will see another issue of SSH taking long time.This issue can be resolved by adding a config parameter in SSH Server Config File (sshd_config) .
For the above issue , SSH is getting stucked at "debug1: SSH2_MSG_SERVICE_ACCEPT received" as shown below.
$ time ssh -v 192.168.2.149
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /home/zaman/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.2.149 [192.168.2.149] port 22.
debug1: Connection established.
debug1: identity file /home/zaman/.ssh/identity type -1
debug1: identity file /home/zaman/.ssh/id_rsa type -1
debug1: identity file /home/zaman/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 br="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.2.149' is known and matches the RSA host key.
debug1: Found key in /home/zaman/.ssh/known_hosts:5
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received ----------waiting here for around 30 seconds
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/zaman/.ssh/identity
debug1: Trying private key: /home/zaman/.ssh/id_rsa
debug1: Trying private key: /home/zaman/.ssh/id_dsa
debug1: Next authentication method: password
zaman@192.168.2.149's password:
real 0m21.343s
user 0m0.008s
sys 0m0.003s
====================
To resolve the above issue , the following parameter needs to be configured in /etc/ssh/sshd_config file as shown below.
UseDNS no
Restart SSH service after doing the above configuration change
$service sshd restart
$ time ssh -v 192.168.2.149
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /home/zaman/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 192.168.2.149 [192.168.2.149] port 22.
debug1: Connection established.
debug1: identity file /home/zaman/.ssh/identity type -1
debug1: identity file /home/zaman/.ssh/id_rsa type -1
debug1: identity file /home/zaman/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 br="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '192.168.2.149' is known and matches the RSA host key.
debug1: Found key in /home/zaman/.ssh/known_hosts:5
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/ssdg/.ssh/identity
debug1: Trying private key: /home/ssdg/.ssh/id_rsa
debug1: Trying private key: /home/ssdg/.ssh/id_dsa
debug1: Next authentication method: password
ssdg@192.168.2.149's password:
real 0m1.096s
user 0m0.006s
sys 0m0.004s
Thus we had seen how to reduce SSH Login time for two different issues.
Comments