I have 2 EC2 instances - one is in a private subnet connected to a nat gateway and the other is in a public subnet connected to an internet gateway. I have a jenkins server on the private subnet and a bastion server in the public subnet.
I used scp to copy the pem key used when creating my jenkins instance over to my bastion server and am now trying to connect to the jenkins instance via the bastion.
I run the command ssh -i pem.key ubuntu#privateipaddress but keep getting a permission denied (publickey) error.
I've checked my security group settings for both instances and they check out. The inbound rules for the jenkins security group allow SSH and HTTP traffic from the bastion security group and the outbound rules for the jenkins security group currently allow all traffic through.
I have the reverse for the bastion security group.
EDIT: Verbose:
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 10.0.1.49 [10.0.1.49] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file key.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file key.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
debug1: match: OpenSSH_7.6p1 Ubuntu-4ubuntu0.3 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.0.1.49:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:EgAjWBAxVLz8L+MQNQXZeIwh51QZOPxPhvugsxv1XGs
debug1: Host '10.0.1.49' is known and matches the ECDSA host key.
debug1: Found key in /home/ubuntu/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: key.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
ubuntu#10.0.1.49: Permission denied (publickey).
I managed to fix this problem by following this guide: https://aws.amazon.com/blogs/security/securely-connect-to-linux-instances-running-in-a-private-amazon-vpc/
I had come across it before, but never considered trying because a trainer on my course said I wouldn't need to.
Related
In AWS, I have created a Bastion host (10.0.10.182) using Amazon Linux 2 and from there I am able to connect to a EC2 private subnet instance (10.0.20.121) (amazon linux 2). (However, this works fine only for the first time.)
After connecting to the Private instance, in order to pull a git repo from github on the private instance, I run the ssh-keygen on the private instance and copy that to github Keys. I can see the .ssh dir in the home dir with the usual files - known_hosts, authorized_keys, id_rsa, id_rsa.pub.
When the original connection from bastion host to private ec2instance times out, I am unable to re-login to the private instance via the bastion host. I get the following message:
ssh -i TestVPC_NCal.pem ec2-user#10.0.20.121
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Here is the ssh debug log generated on the Bastion host:
######Begin ssh debug log ######### [ec2-user#ip-10-0-10-182 ~]$ ssh -v -i TestVPC_NCal.pem ec2-user#10.0.20.121 OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 58: Applying
options for * debug1: Connecting to 10.0.20.121 [10.0.20.121] port 22.
debug1: Connection established. debug1: key_load_public: No such file
or directory debug1: identity file TestVPC_NCal.pem type -1 debug1:
key_load_public: No such file or directory debug1: identity file
TestVPC_NCal.pem-cert type -1 debug1: Enabling compatibility mode for
protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_7.4 debug1:
Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000 debug1:
Authenticating to 10.0.20.121:22 as 'ec2-user' debug1:
SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex:
algorithm: curve25519-sha256 debug1: kex: host key algorithm:
ecdsa-sha2-nistp256 debug1: kex: server->client cipher:
chacha20-poly1305#openssh.com MAC: compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC:
compression: none debug1: kex: curve25519-sha256 need=64
dh_need=64 debug1: kex: curve25519-sha256 need=64 dh_need=64 debug1:
expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key:
ecdsa-sha2-nistp256 SHA256:5W++Ewk+lx2YXUUY1xhhttjKG3KVWvIOTvtp7THBFJc
debug1: Host '10.0.20.121' is known and matches the ECDSA host key.
debug1: Found key in /home/ec2-user/.ssh/known_hosts:2 debug1: rekey
after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting
SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: rekey after
134217728 blocks debug1: SSH2_MSG_EXT_INFO received debug1:
kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that
can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next
authentication method: gssapi-keyex debug1: No valid Key exchange
context debug1: Next authentication method: gssapi-with-mic debug1:
Unspecified GSS failure. Minor code may provide more information No
Kerberos credentials available (default cache:
KEYRING:persistent:1000)
debug1: Unspecified GSS failure. Minor code may provide more
information No Kerberos credentials available (default cache:
KEYRING:persistent:1000)
debug1: Next authentication method: publickey debug1: Trying private
key: TestVPC_NCal.pem debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic debug1: No more authentication
methods to try. Permission denied
(publickey,gssapi-keyex,gssapi-with-mic). [ec2-user#ip-10-0-10-182 ~]$
########### End debug log ########
I wonder if the running the ssh-keygen on EC2 private instance is somehow causing the error. Any pointers to resolve this are very welcome!
I was able to resolve this issue by creating the TestVPC_NCal.pem file (associated with the EC2 private instance) on the EC2 private instance while the connection was working. The clue was in the log I posted in my question:
##############
Connecting to 10.0.20.121 [10.0.20.121] port 22. debug1: Connection established. debug1: key_load_public: No such file or directory debug1: identity file TestVPC_NCal.pem type -1 debug1: key_load_public: No such file or directory debug1: identity file TestVPC_NCal.pem-cert type -1 debug1:
##############
When the connection timed-out overnight, I was able to log back in to EC2 private instance with no issues.
I have downloaded the default private key and am able to connect via SSH with no problem using that private key. In my Lightsail instance, I went to the SSH Keys tab, created a new key pair and downloaded the new private key (savng it in the correct location on my local machine with proper permissions). However, i am unable to connect using that new private key. Here is the output I get from the command: ssh -v -i ~/.ssh/test.pem me#x.x.x.x
OpenSSH_7.8p1, LibreSSL 2.6.2
debug1: Reading configuration data /Volumes/Norman Data/daveh0/.ssh/config
debug1: /Volumes/Norman Data/daveh0/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to x.x.x.x port 22.
debug1: Connection established.
debug1: identity file .ssh/test.pem type -1
debug1: identity file .ssh/test.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.8 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to x.x.x.x:22 as 'me'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Bajjqc9SJlMHTB/OrEWKl4ATi6/wI+fB1C351fi5Iwk
debug1: Host 'x.x.x.x' is known and matches the ECDSA host key.
debug1: Found key in /Volumes/Norman Data/daveh0/.ssh/known_hosts:10
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: .ssh/test.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
me#x.x.x.x : Permission denied (publickey).
I've got to be missing a step on the SSH Keys screen, but I can't seem to figure out what it would be. Can anyone help?
Keypairs are a feature of Linux. The way it works is:
Somebody tries to connect to the Linux computer using SSH, eg ssh -i key.pem username#IP-ADDRESS
The Linux computer looks in /home/USERNAME/.ssh/authorized_keys
If it finds a public key that matches the private key supplied in key.pem, then the connection is permitted
Therefore, since you created a new keypair, you will need to add the new keypair to the appropriate user's ~/.ssh/authorized_keys file.
Your example shows you as logging in as a user called me, so the public keypair should be added to /home/me/.ssh/authorized_keys.
When first launching a Lightsail or EC2 instance, you can specify a keypair and software on the instance will automatically add the associated public key to the authorized_keys file. However, you will need to do this step manually for an already-running instance.
For AWS Lightsail, I was able to login via SSH by appending my public key id_rsa.pub to remote authorized_keys, I used SFTP (Filezilla) to update authorized_keys file. For SFTP connection I downloaded ssh key from Accounts page.
I'm trying to connect to the problematic Amazon EC2 instance with this command:
ssh -vvv -i MY_KEY.pem root#ec2-XX-XX-XXX-XX.compute-1.amazonaws.com
and getting this result:
OpenSSH_7.5p1, OpenSSL 1.1.0g-fips 2 Nov 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 8: Applying options for *
debug1: Connecting to ec2-XX-XX-XXX-XX.compute-1.amazonaws.com [XX.XX.XXX.XX] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file MY_KEY.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file MY_KEY.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.5
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:22 as 'root'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm#openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256#libssh.org need=32 dh_need=32
debug1: kex: curve25519-sha256#libssh.org need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:rRMQg/hs
debug1: Host 'ec2-XX-XX-XXX-XX.compute-1.amazonaws.com' is known and matches the ECDSA host key.
debug1: Found key in /home/dina/.ssh/known_hosts:8
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: MY_KEY.pem
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Offering RSA public key: dina#localhost.localdomain
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Trying private key: MY_KEY.pem
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: keyboard-interactive
Password:
The key MY_KEY.pem should be correct, I've checked the fingerprint under NETWORK & SECURITY -> Key Pairs on AWS console, and it's the same as for the key I'm using.
Permissions for the .pem file is 400, owner is current owner. SSH inbound traffic to the instance is allowed from all addresses.
Also tried this with different users (root, ubuntu, admin, ec2-user, centos, fedora) as I have no contact with the person who launched the instance and have no idea what system is there.
Any idea about what is going on and what could be done about it? Your help is very much appreciated
What AMI are you using?
If you are using an Ubuntu AMI, you should connect as ubuntu.
$ ssh -vvv -i ~/.ssh/MY_KEY.pem ubuntu#ec2-XX-XX-XXX-XX.compute-1.amazonaws.com
On Amazon Linux, the default user is ec2-user.
$ ssh -vvv -i MY_KEY.pem ec2-user#ec2-XX-XX-XXX-XX.compute-1.amazonaws.com
Once you are in both of these users will have sudo privileges.
I am trying to connect to my ec-2 free(t2.micro) instance through ssh from my PC.
I have created instance with default VPC. I am not able to connect it from my PC.
It is throwing me permission denied error.
I have checked the rules in the security group.
I have gone through the below url's to check the answer but no success.
AWS SSH connection error: Permission denied (publickey)
Troubleshooting Connecting to Your Instance
SSH: Permission denied (publickey)
Also when I run below command
sudo ssh -v -i tep-keyPair.pem ubuntu#ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com
The below error came:
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com [52.XX.XXX.XX] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file tep-keyPair.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file tep-keyPair.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Authenticating to ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ZeJ4XQUfgLkaMUEvjGohL/6FWKN9Gq4AXrPwL/i9t3M
debug1: Host 'ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com' is known and matches the ECDSA host key.
debug1: Found key in /var/root/.ssh/known_hosts:3
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: tep-keyPair.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
Please help me.. I am really stuck here
The fact that you are receiving a Permission denied (publickey) error indicates that you are successfully communicating with the instance, so the problem is not related to networking nor security groups.
Rather, the instance is not accepting a connection via the keypair you are providing. Therefore, you either need to provide it with the keypair it expects, or you can copy a new keypair to the instance.
To copy a different keypair to the instance, follow instructions on this StackOverflow answer, which is written for Ubuntu: Change key pair for ec2 instance
I've configured my EC2 instance, and connected with SSH. But when I created a new Security Group with port rules I couldn't access via SSH anymore. Currently, my custom Security Group rules are:
SSH 0.0.0.0/0
HTTP 0.0.0.0/0
HTTPS 0.0.0.0/0
When I try ssh -v -i bodruk.pem ubuntu#ec2-54-149-134-92.us-west-2.compute.amazonaws.com I have the following error:
OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug1: Connecting to ec2-54-149-134-92.us-west-2.compute.amazonaws.com [54.149.
134.92] port 22.
debug1: Connection established.
debug1: identity file bodruk.pem type -1
debug1: identity file bodruk.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubu
ntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH_6.6.1* compat 0x04000
000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm#openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm#openssh.com none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA e2:13:af:e1:1b:70:f9:70:3b:cd:1d:7f:14:de:ce:90
debug1: Host 'ec2-54-149-134-92.us-west-2.compute.amazonaws.com' is known and ma
tches the ECDSA host key.
debug1: Found key in /c/Users/Thiago/.ssh/known_hosts:2
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: bodruk.pem
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
Already tried this solution, but doesn't work. I changed the Key Pair twice and deleted the known_hosts file with no success.
Any idea?
Can you telnet to the instance with the ssh port? (telnet 'ip' 'port')
If you can telnet, so the problem probably in the Key Pair or something in your computer. And if not, its probably something with the Security Group and network.
I ran into this issue recently and the funny part is my pem file was owned by root instead of my user. When I did sudo chown user:group {pem file name}, I was able to ssh in without a problem.