I have my NAT and Bastion set up to login with SSH forwarding:
ssh-add -K keyfile.pem
ssh -A ec2-user#bastionhost
ssh ec2-user#privateSubnetServer
What's the best method for handling ssh and users at this point between hosts in the private subnet?
I get:
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
When trying to do it the traditional linux way. I can get to them if I use the AWS created key pairs.
I can't set up or connect to a directory service at this point.
There can be two reasons, either the key is invalid or the proxy command not working as you expecting.
Before that, you set you proxy command like this
host bastion
HostName bastion_Adress
User centos|whatever
identityFile /mykeys/ec2.pem
Now try to ssh to bastion
ssh bastion
If the above then bastion working fine, we can set proxy command now.
host private_server
Hostname 10.0.5.45
user centos
IgnoreUnknown UseKeychain
AddKeysToAgent yes
ProxyCommand ssh bastion -W %h:%p
Now you can ssh to private server
ssh private_server
Normally these are necessary but just in case of mac happen sometime
IgnoreUnknown UseKeychain
AddKeysToAgent yes
.ssh/config: "Bad configuration option: UseKeychain" on Mac OS Sierra 10.12.6
You can debug the issue using this flow
ssh to bastion
copy your private server ssh key to the bastion
ssh to private instance
If the above worked it mean bastion ssh config is not valid
If the above does not work then the key is not valid.
Host bastion.ip.address
User ec2-user
IdentityFile ~/.ssh/bastionkey.pem
CheckHostIP no
Host private.subnet.ip
User ec2-user
IdentityFile ~/.ssh/bastionkey.pem
ProxyCommand ssh ec2-user#bastion.host.ip -W %h:%p
Related
Overview
I'm trying to configure ~/.ssh/config to connect my local VSCode to remote (EC2). I've done a lot of testing and can't understand why situation one is working and others are failing. I'm able to get RemoteCommand to work successfully for ssh'ing into EC2 instance after ssh'ing into BastionHost, however I'm unable to achieve the same with ProxyJump or ProxyCommand. VSCode doesn't list the EC2 filesystem when using the RemoteCommand example (just gets to BastionHost), so thinking I'll need to resolve to either ProxyJump/ProxyCommand based on most of the documentation.
I've tried to follow the instructions here exactly along with trying out different approaches from other articles to no avail.
##WORKS
Host dev-ec2
HostName 10.248.000.206
User meme1
RemoteCommand ssh 10.248.000.201
RequestTTY yes
IdentityFile ~/.ssh/mykey
##WORKS
Host bastion-dev
HostName 10.248.000.206
User meme1
IdentityFile ~/.ssh/mykey
RequestTTY yes
##FAILS (times out)
Host dev-ec2-proxycommand
HostName 10.248.000.201
User meme1
ProxyCommand ssh.exe bastion-dev -W %h:%p
##FAILS (Permission denied on public key, even though no issue in the RemoteCommand example)
Host ec2-dev-proxyjump
HostName 10.248.000.201
User meme1
ProxyJump bastion-dev
IdentityFile ~/.ssh/mykey
System Info
OS: Windows 10
Bastion OS: Linux (Amazon Linux AMI)
Disclaimer
I've been trolling StackOverflow and other forums for the past couple days to no avail, and although I've found similar questions none have provided viable answers for resolving.
I suppose this one below is failing because you are using a command from Windows at your bastion, that is Linux.
Command ssh.exe will not work on Linux. Everything that you put on ProxyCommand will run inside your bastion host, in your case it is a Linux OS.
Also make sure your instance Security Group allow connection from bastion IP, not from your computer.
##FAILS (times out)
Host dev-ec2-proxycommand
HostName 10.248.000.201
User meme1
ProxyCommand ssh.exe bastion-dev -W %h:%p
I have the config below on my ~/.ssh/config file and it works fine for me to connect on my instance behind bastion host.
Bastion IP: 172.31.4.238
Host IP (behind bastion): 172.31.11.98
Host 172.31.11.98
HostName 172.31.11.98
User ec2-user
ProxyCommand ssh -W %h:%p ec2-user#172.31.4.238
See it below
$ ssh 172.31.11.98
The authenticity of host '172.31.11.98 (<no hostip for proxy command>)' can't be established.
ECDSA key fingerprint is SHA256:vy....
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '172.31.11.98' (ECDSA) to the list of known hosts.
__| __|_ )
_| ( / Amazon Linux 2 AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-2/
[ec2-user#ip-172-31-11-98 ~]$
current my .ssg/config has
Host git-codecommit.*.amazonaws.com
User APKAS2GIPODK72AAAAAA
IdentityFile ~/.ssh/codecommit_rsa
Host github.com
User durrantm
IdentityFile ~/.ssh/id_rsa
I believe I can add an entry for a specific EC2 machine that I want to ssh into.
Is there a way to make the pem file by a default for all my EC2 ssh connections (while I am using the awscli, configured for my account-user) so that I don't have to -i "abc.pem" for each ec2 connection?
Yes, you can if you connect to the instances via their public DNS.
Add this to your SSH config:
Host ec2-*.compute.amazonaws.com
IdentityFile abc.pem
This is identical to what you have with Host git-codecommit.*.amazonaws.com
Yes, you can start an ssh-agent on your EC2 instance and have it load your private key. Then whenever you try and SSH to any instance that offers public key authentication, the ssh-agent will attempt to authenticate you using the loaded private key. This article describes it nicely.
eval $(ssh-agent) && ssh-add ~/.ssh/*.pem
I understand that it is not recommended to ask multiple question in a single post but they are all tightly coupled hence asking them under one post.
I was trying to SSH to a private EC2 instance in a VPC from the internet using Bastion host. I came across 2 approaches.
NOTE When I launched my private EC2 instance and Bastion host I chose different Keys.(In both the approaches I saw the authors used same Keys for EC2 and the Bastion)
Approach 1: Configuring SSH ProxyCommand as illustrated in https://www.youtube.com/watch?v=EpFAHis4O4g
Approach 2: ssh-agent command with ssh -A option https://aws.amazon.com/blogs/security/securely-connect-to-linux-instances-running-in-a-private-amazon-vpc/
Question 1: In approach 2, I did
ssh-add ~/Downloads/Ec2.pem
ssh-add ~/Downloads/BastionKey.pem
Is my approach of adding both the keys to the ssh-agent correct? This way I was able ssh into Bastion as a first step and then as a second step I had to explicitly ssh into private EC2.
Question 2: Approach 1 is against the idea of (Approach 2) 2 step ssh process but proposes ProxyCommand so that end user should be able to directly ssh into the private EC2 in a single step. This way a system admin of Bastion host will not have control of ssh-ing to individual EC2 instances.In fact the author demonstrates that if you ssh into Bastion host (it will succeed) and then ssh to EC2 it fails. Is my understanding correct here?
Question 3: Approach 1 is not working for me. I used the same structure for ~/.ssh/config but with different Key file paths. I am unable to ssh to my private EC2 directly. I even tried the 2 step process but I can only login into Bastion host, I cannot ssh to EC2. All my Security groups, ACL, Internet gateway, NAT gateway and VPC settings are fine, otherwise Approach 2 wouldn't have worked. What could I be doing wrong?
cat ~/.ssh/config
HOST bastion
Hostname ec2-5x-xx-xx-xx.compute-1.amazonaws.com
User ec2-user
IdentityFile /Users/myname/Downloads/BastionKey.pem
HOST *.ec2.internal
User ec2-user
IdentityFile /Users/myname/Downloads/Ec2.pem
ProxyCommand ssh -q -W %h:%p bastion
Question 4: Which is the recommend approach among 1 & 2 and what additional steps must be followed when different keys are chosen for Bastion and EC2?
Error output:
ssh -v ip-10-0-1-12.ec2.internal
OpenSSH_7.8p1, LibreSSL 2.6.2
debug1: Reading configuration data /Users/myname/.ssh/config
debug1: /Users/myname/.ssh/config line 6: Applying options for *.ec2.internal
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Executing proxy command: exec ssh -q -W ip-10-0-1-12.ec2.internal:22 bastion
debug1: identity file /Users/myname/Downloads/Ec2.pem type -1
debug1: identity file /Users/myname/Downloads/Ec2.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.8
/bin/false: No such file or directory
ssh_exchange_identification: Connection closed by remote host
HOST bastion
Hostname ec2-example-ip.compute-1.amazonaws.com
User ec2-user
IdentityFile /Users/myname/Downloads/BastionKey.pem
HOST *.ec2.internal
User ec2-user
IdentityFile /Users/myname/Downloads/Ec2/Ec2.pem
ProxyCommand ssh -q -W %h:%p bastion
Try using the bastions public DNS name in your ssh config, also use the internal ec2 DNS name for the one behind the VPC. (AWS DNS is sketchy at best sometimes)
Note: *.ec2.internal assums you will use the same ssh key for each ec2 you need to access proxying through your bastion. If this is not the case replace *.ec2.internal with whatever-internal-ip.ec2.internal and add an entry for each individual ec2.
Hope this can fix your issue.
Situation : So Basically I have Ec2 instance and to login to Ec2 instance, first I will login to jump server and then I login to my instance with private ip, and all my pem files are stored inside the jump server and in private instance(EC2 instance).
What I have Done : I have edited sshd-config file with "PasswordAuthentication No and change it to PasswordAuthentication Yes" and created password like "xyz123" now I able to login to my machine with,
ssh ubuntu#ipadrrs and psswd:xyz123
but this is not serving my open item.
what I am looking for : actually I used to share my .pem file to my team member to access instance, and if I enable password with ip they still can login with username and password and if they leave my org if they have my pem file handy still they can access my instance with pem file or password.
what is the best way to avoid this problem.
will Active directory will help here or LDAP will help here?if so how?
Please help me with quires.
The best solution now is to use AWS Systems Manager Session Manager. This requires no pem sharing. SSH access direct from the browser, I am using this and I did not find any issues. Also, entire sessions can be audited.
Otherwise, rotate your key pairs. But I am not sure if there is an AWS way to do it. Also, use the best practices by enabling security groups to open only to known ips instead of the entire whole.
Here you can convert your jump server to a SSH Bastion server. The same can be done using setting up iptables rules on this server. Below mentioned is the example of a rule which you can setup:
iptables -t nat -A PREROUTING -d xx.xx.xx.xx -p tcp --dport yyyy -j DNAT --to zz.zz.zz.zz:22
Here xx.xx.xx.xx is the private IP of the SSH bastion server. yyyy is the port which will be used for inbound access. zz.zz.zz.zz will be the private IP of the destination server. This rule simply means that you ssh into a machine using port yyyy, which will port forward the traffic to port 22 on zz.zz.zz.zz machine.
In this case you will only have to configure SSH public on the destination machine(zz.zz.zz.zz) only and the client machine will have the private key. Command to connect from the client machine will be ssh -i <path-to-private-key> username#BastionPublicIP -p yyyy
Below mentioned are the ports to be opened at security groups:
Bastion - Inbound - yyyy(from your IP)
Bastion - Inbound - 22(from your IP)
Destionation - Inbound - 22(from bastion machine)
I suggest, you use an Amazon Linux AMI for SSH Bastion server.
Assuming Machine A is target machine which I want to SSH into finally while Machine B is a bridge machine (bastion host). These two machines are accessible using the same PEM file.
The security group of Machine A allows SSH connections only from Machine B. So If I want to connect to Machine A, I need to connect through Machine B.
How can this be accomplished without placing the PEM file on the bastion host?
You can use ProxyCommand. I prefer defining the following in your ~/.ssh/config file.
host MachineB
HostName <MachineB-IP>
IdentityFile <Full Path of .pem file>
User username
host MachineA
HostName <MachineA-IP>
ProxyCommand ssh MachineB nc -w 120 %h %p
IdentityFile <Full Path of .pem file>
User username
Then access MachineA like:
$ ssh MachineA
To reach an EC2 instance in a private subnet via a bastion host in a public subnet, without placing your SSH private key on the bastion, you need to use SSH agent forwarding.
Specific instructions are provided here.