The difference between this question and all the others comes down to this: yesterday, I could connect via SSH just fine using this command:
ssh -i "~/.ssh/[.cer file]" ubuntu#[Public IPv4 DNS]
(A .cer file is created instead of .pem since I use Chrome.) Then I shut down the instance and deleted the volume (after creating a snapshot) then I went home.
Today, I did the following:
Created a new volume from the snapshot, and attached it to the instance
Created an Elastic IP address and assigned it to the instance
Started the instance
Tried connecting via SSH using the new Public IPv4 DNS. I got a Permission denied (publickey) error.
Thinking it may have something to do with the Elastic IP, I dissociated the Elastic IP and rebooted the instance to get a new temporary IP.
I tried using ssh using the same command (with the new Public DNS) and am still getting the Permission denied (publickey) error.
Here is the log I get when adding -v to the command:
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/WonderWolff/.ssh/config
debug1: /Users/WonderWolff/.ssh/config line 14: Applying options for *.compute.amazonaws.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to ec2-13-57-238-91.us-west-1.compute.amazonaws.com port 22.
debug1: Connection established.
debug1: identity file /Users/WonderWolff/.ssh/rei_development.cer type -1
debug1: identity file /Users/WonderWolff/.ssh/rei_development.cer-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.10
debug1: compat_banner: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.10 pat OpenSSH_6.6.1* compat 0x04000002
debug1: Authenticating to ec2-13-57-238-91.us-west-1.compute.amazonaws.com:22 as 'ubuntu'
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: aes128-ctr MAC: umac-128-etm#openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: umac-128-etm#openssh.com compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:LnRbxnhhpoLZeIUFXFzOybmc+cPvutkYqZCmUmq+zVw
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
Warning: Permanently added 'ec2-13-57-238-91.us-west-1.compute.amazonaws.com' (ED25519) to the list of known hosts.
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /Users/WonderWolff/.ssh/rei_development.cer explicit
debug1: SSH2_MSG_SERVICE_ACCEPT received
Unauthorized use is strictly prohibited. All access and activity
is subject to logging and monitoring.
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/WonderWolff/.ssh/rei_development.cer
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
ubuntu#ec2-13-57-238-91.us-west-1.compute.amazonaws.com: Permission denied (publickey).
If it worked yesterday after multiple stops and reboots (I stopped and rebooted multiple times yesterday), what could possibly be the issue? Was attaching a new volume created from a snapshot the issue?
UPDATE: I created a new instance using the same keys. Connecting to that instance works, no problem. But I would still like to understand why either the removal and re-association of a volume, OR the assigning an Elastic IP address, OR a different reason I don't yet know causes me to get a Permission Denied error with the first instance.
Related
In AWS, I have created a Bastion host (10.0.10.182) using Amazon Linux 2 and from there I am able to connect to a EC2 private subnet instance (10.0.20.121) (amazon linux 2). (However, this works fine only for the first time.)
After connecting to the Private instance, in order to pull a git repo from github on the private instance, I run the ssh-keygen on the private instance and copy that to github Keys. I can see the .ssh dir in the home dir with the usual files - known_hosts, authorized_keys, id_rsa, id_rsa.pub.
When the original connection from bastion host to private ec2instance times out, I am unable to re-login to the private instance via the bastion host. I get the following message:
ssh -i TestVPC_NCal.pem ec2-user#10.0.20.121
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Here is the ssh debug log generated on the Bastion host:
######Begin ssh debug log ######### [ec2-user#ip-10-0-10-182 ~]$ ssh -v -i TestVPC_NCal.pem ec2-user#10.0.20.121 OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 58: Applying
options for * debug1: Connecting to 10.0.20.121 [10.0.20.121] port 22.
debug1: Connection established. debug1: key_load_public: No such file
or directory debug1: identity file TestVPC_NCal.pem type -1 debug1:
key_load_public: No such file or directory debug1: identity file
TestVPC_NCal.pem-cert type -1 debug1: Enabling compatibility mode for
protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_7.4 debug1:
Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000 debug1:
Authenticating to 10.0.20.121:22 as 'ec2-user' debug1:
SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex:
algorithm: curve25519-sha256 debug1: kex: host key algorithm:
ecdsa-sha2-nistp256 debug1: kex: server->client cipher:
chacha20-poly1305#openssh.com MAC: compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC:
compression: none debug1: kex: curve25519-sha256 need=64
dh_need=64 debug1: kex: curve25519-sha256 need=64 dh_need=64 debug1:
expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key:
ecdsa-sha2-nistp256 SHA256:5W++Ewk+lx2YXUUY1xhhttjKG3KVWvIOTvtp7THBFJc
debug1: Host '10.0.20.121' is known and matches the ECDSA host key.
debug1: Found key in /home/ec2-user/.ssh/known_hosts:2 debug1: rekey
after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting
SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: rekey after
134217728 blocks debug1: SSH2_MSG_EXT_INFO received debug1:
kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that
can continue: publickey,gssapi-keyex,gssapi-with-mic debug1: Next
authentication method: gssapi-keyex debug1: No valid Key exchange
context debug1: Next authentication method: gssapi-with-mic debug1:
Unspecified GSS failure. Minor code may provide more information No
Kerberos credentials available (default cache:
KEYRING:persistent:1000)
debug1: Unspecified GSS failure. Minor code may provide more
information No Kerberos credentials available (default cache:
KEYRING:persistent:1000)
debug1: Next authentication method: publickey debug1: Trying private
key: TestVPC_NCal.pem debug1: Authentications that can continue:
publickey,gssapi-keyex,gssapi-with-mic debug1: No more authentication
methods to try. Permission denied
(publickey,gssapi-keyex,gssapi-with-mic). [ec2-user#ip-10-0-10-182 ~]$
########### End debug log ########
I wonder if the running the ssh-keygen on EC2 private instance is somehow causing the error. Any pointers to resolve this are very welcome!
I was able to resolve this issue by creating the TestVPC_NCal.pem file (associated with the EC2 private instance) on the EC2 private instance while the connection was working. The clue was in the log I posted in my question:
##############
Connecting to 10.0.20.121 [10.0.20.121] port 22. debug1: Connection established. debug1: key_load_public: No such file or directory debug1: identity file TestVPC_NCal.pem type -1 debug1: key_load_public: No such file or directory debug1: identity file TestVPC_NCal.pem-cert type -1 debug1:
##############
When the connection timed-out overnight, I was able to log back in to EC2 private instance with no issues.
I have downloaded the default private key and am able to connect via SSH with no problem using that private key. In my Lightsail instance, I went to the SSH Keys tab, created a new key pair and downloaded the new private key (savng it in the correct location on my local machine with proper permissions). However, i am unable to connect using that new private key. Here is the output I get from the command: ssh -v -i ~/.ssh/test.pem me#x.x.x.x
OpenSSH_7.8p1, LibreSSL 2.6.2
debug1: Reading configuration data /Volumes/Norman Data/daveh0/.ssh/config
debug1: /Volumes/Norman Data/daveh0/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to x.x.x.x port 22.
debug1: Connection established.
debug1: identity file .ssh/test.pem type -1
debug1: identity file .ssh/test.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.8 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to x.x.x.x:22 as 'me'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Bajjqc9SJlMHTB/OrEWKl4ATi6/wI+fB1C351fi5Iwk
debug1: Host 'x.x.x.x' is known and matches the ECDSA host key.
debug1: Found key in /Volumes/Norman Data/daveh0/.ssh/known_hosts:10
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: .ssh/test.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
me#x.x.x.x : Permission denied (publickey).
I've got to be missing a step on the SSH Keys screen, but I can't seem to figure out what it would be. Can anyone help?
Keypairs are a feature of Linux. The way it works is:
Somebody tries to connect to the Linux computer using SSH, eg ssh -i key.pem username#IP-ADDRESS
The Linux computer looks in /home/USERNAME/.ssh/authorized_keys
If it finds a public key that matches the private key supplied in key.pem, then the connection is permitted
Therefore, since you created a new keypair, you will need to add the new keypair to the appropriate user's ~/.ssh/authorized_keys file.
Your example shows you as logging in as a user called me, so the public keypair should be added to /home/me/.ssh/authorized_keys.
When first launching a Lightsail or EC2 instance, you can specify a keypair and software on the instance will automatically add the associated public key to the authorized_keys file. However, you will need to do this step manually for an already-running instance.
For AWS Lightsail, I was able to login via SSH by appending my public key id_rsa.pub to remote authorized_keys, I used SFTP (Filezilla) to update authorized_keys file. For SFTP connection I downloaded ssh key from Accounts page.
I am trying to connect to my ec-2 free(t2.micro) instance through ssh from my PC.
I have created instance with default VPC. I am not able to connect it from my PC.
It is throwing me permission denied error.
I have checked the rules in the security group.
I have gone through the below url's to check the answer but no success.
AWS SSH connection error: Permission denied (publickey)
Troubleshooting Connecting to Your Instance
SSH: Permission denied (publickey)
Also when I run below command
sudo ssh -v -i tep-keyPair.pem ubuntu#ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com
The below error came:
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com [52.XX.XXX.XX] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file tep-keyPair.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file tep-keyPair.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Authenticating to ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ZeJ4XQUfgLkaMUEvjGohL/6FWKN9Gq4AXrPwL/i9t3M
debug1: Host 'ec2-52-XX-XXX-XX.us-west-2.compute.amazonaws.com' is known and matches the ECDSA host key.
debug1: Found key in /var/root/.ssh/known_hosts:3
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: tep-keyPair.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
Please help me.. I am really stuck here
The fact that you are receiving a Permission denied (publickey) error indicates that you are successfully communicating with the instance, so the problem is not related to networking nor security groups.
Rather, the instance is not accepting a connection via the keypair you are providing. Therefore, you either need to provide it with the keypair it expects, or you can copy a new keypair to the instance.
To copy a different keypair to the instance, follow instructions on this StackOverflow answer, which is written for Ubuntu: Change key pair for ec2 instance
After changing the owner of .ssh folder from user to root, i cannot login the remote server with ssh. Here is the error message:
OpenSSH_6.9p1, LibreSSL 2.1.7
debug1: Reading configuration data /Users/qj/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to ec2-52-193-83-231.ap-northeast-1.compute.amazonaws.com [52.193.83.231] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file gmail.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file gmail.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Authenticating to ec2-52-193-83-231.ap-northeast-1.compute.amazonaws.com:22 as 'ec2-user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client chacha20-poly1305#openssh.com <implicit> none
debug1: kex: client->server chacha20-poly1305#openssh.com <implicit> none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:EahONyMKzM6Q4tdEBSa9LwyOFI65KB02GesJGuGE9Ss
debug1: Host 'ec2-52-193-83-231.ap-northeast-1.compute.amazonaws.com' is known and matches the ECDSA host key.
debug1: Found key in /Users/qj/.ssh/known_hosts:25
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/qj/.ssh/dqj
debug1: Authentications that can continue: publickey
debug1: Trying private key: gmail.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
It's really my fault to change owner of .ssh folder.
Anyone help me~
If you really need to preserve the server, you could try creating an AMI from the machine. Then relaunching from that. AWS will then attempt to put your public key in authorized_keys again, and may well fix the permissions issue in doing so.
If not, you can always launch a new server and attach the broken servers EBS volume to the new server to fix the permissions on the folder. Not garunteed to work though if you've got ephemeral storage or a weird file system.
I found the answer from https://forums.aws.amazon.com/thread.jspa?threadID=133054&tstart=0
Here is the answer:
Stop the instance
Detach the Root Volume
Launch another instance(or if you have one already you can skip this step)
Attached the Volume in 2 to the new(or already existing other) instance
Log in into the instance
Mount the Volume
Change the folder permissions as appropriate
Umount the Volume and detach it
Attach it back to the original instace
Start the instance and connect
It occurs some problems at step 6 when mounting the Volume to the new instance using the shell mount xvdf /ebs/ -t ext4(mkdir /ebs //this folder is a mount point, more details from Making an Amazon EBS Volume Available for Use). The error message is :
mount: wrong fs type, bad option, bad superblock on /dev/xvdf,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so.
Since the file system for the Volume is GPT. Fortunately, I got the reason from this post Problem mounting GPT disk partition. And the solution is that i need to mount /dev/xvdf1, not just /dev/xvdf, such as mount xvdf1 /ebs/ -t ext4.
Finally, mounting the volume is successful.
I've configured my EC2 instance, and connected with SSH. But when I created a new Security Group with port rules I couldn't access via SSH anymore. Currently, my custom Security Group rules are:
SSH 0.0.0.0/0
HTTP 0.0.0.0/0
HTTPS 0.0.0.0/0
When I try ssh -v -i bodruk.pem ubuntu#ec2-54-149-134-92.us-west-2.compute.amazonaws.com I have the following error:
OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug1: Connecting to ec2-54-149-134-92.us-west-2.compute.amazonaws.com [54.149.
134.92] port 22.
debug1: Connection established.
debug1: identity file bodruk.pem type -1
debug1: identity file bodruk.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubu
ntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH_6.6.1* compat 0x04000
000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm#openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm#openssh.com none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA e2:13:af:e1:1b:70:f9:70:3b:cd:1d:7f:14:de:ce:90
debug1: Host 'ec2-54-149-134-92.us-west-2.compute.amazonaws.com' is known and ma
tches the ECDSA host key.
debug1: Found key in /c/Users/Thiago/.ssh/known_hosts:2
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: bodruk.pem
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
Already tried this solution, but doesn't work. I changed the Key Pair twice and deleted the known_hosts file with no success.
Any idea?
Can you telnet to the instance with the ssh port? (telnet 'ip' 'port')
If you can telnet, so the problem probably in the Key Pair or something in your computer. And if not, its probably something with the Security Group and network.
I ran into this issue recently and the funny part is my pem file was owned by root instead of my user. When I did sudo chown user:group {pem file name}, I was able to ssh in without a problem.