I am bringing up an ec2 instance based on an in house AMI that used a different ssh key for authentication than the one I'd like to use on the instance I create using knife (in the example I call it original-pem-for-ami.pem):
knife ec2 server create -I ami-0123456 -f m2.xlarge \
--ssh-user username --groups sg-1234 \
--identity-file ~/.ssh/original-pem-for-ami.pem \
--node-name solr1 --hint ec2 -a public_ip_address \
--ssh-key name-of-key-i-want-to-use-to-login-to-new-instance
When I run this command the server comes up correctly, the correct security group is assigned etc, but I can only connect it to using:
ssh -i ~/.ssh/original-pem-for-ami.pem username#assigned-ec2-public-dns-name
Is there a way to make the new instance the key associated with the named keypair name-of-key-i-want-to-use-to-login-to-new-instance. I thought using --ssh-key name-of-key-i-want-to-use-to-login-to-new-instance would do this.
Check which version of knife-ec2 you have. --ssh-key is correct in 0.12 but before that (0.11 and earlier) I think it was something different. Also make sure this works through the normal AWS tools, it is possible the AMI wasn't prepared correctly and uses a hardwired key.
Related
Does anyone have an set of instructions for creating an AWS AMI from a chef 12, Linux based OpsWorks instance?
AWS publishes instructions on Creating a Custom Linux AMI from an AWS OpsWorks Instance. But it looks like they are out of date for Chef 12, Linux based OpsWorks stacks.
For example, they do not say you should remove the /opt/chef or /var/chef folders.
Here are the current instructions from AWS:
Create a Custom Linux AMI from an AWS OpsWorks Instance
If you want to use a customized AWS OpsWorks Linux instance to create an AMI, you should be aware that every Amazon EC2 instance created by OpsWorks includes a unique identity. If you create a custom AMI from such an instance, it will include that identity and all instances based on the AMI will have the same identity. To ensure that the instances based on your custom AMI have a unique identity, you must remove the identity from the customized instance before creating the AMI.
To create a custom AMI from an AWS OpsWorks instance
Create a Linux stack and add one or more layers to define the configuration of the customized instance. You can use built-in layers, customized as appropriate, as well as fully custom layers. For more information, see Customizing AWS OpsWorks.
Edit the layers and disable AutoHealing.
Add an instance with your preferred Linux distribution to the layer or layers and start it. We recommend using an Amazon EBS-backed instance. Open the instance's details page and record its Amazon EC2 ID for later.
When the instance is online, log in with SSH and run the following commands, in order:
sudo /etc/init.d/monit stop
sudo /etc/init.d/opsworks-agent stop
sudo rm -rf /etc/aws/opsworks/ /opt/aws/opsworks/ /var/log/aws/opsworks/ /var/lib/aws/opsworks/ /etc/monit.d/opsworks-agent.monitrc /etc/monit/conf.d/opsworks-agent.monitrc /var/lib/cloud/
If you are creating an AMI based on Amazon Linux 2014.09, run rpm -e opsworks-agent-ruby to ensure that the agent is running.
If you are creating an AMI based on Ubuntu, run dpkg -r opsworks-agent-ruby to ensure that the agent is running.
This step depends on the instance type:
For an Amazon EBS-backed instance, use the AWS OpsWorks console to stop the instance and create the AMI as described in Creating an Amazon EBS-Backed Linux AMI..
For an instance store-backed instance, create the AMI as described in Creating an Instance Store-Backed Linux AMI and then use the AWS OpsWorks console to stop the instance.
When you create the AMI, be sure to include the certificate files. For example, you can call the ec2-bundle-vol command with the -i argument set to -i $(find /etc /usr /opt -name '.pem' -o -name '.crt' -o -name '*.gpg' | tr '\n' ','). Do not remove the apt public keys when bundling. The default ec2-bundle-vol command handles this task.
Clean up your stack by returning to the AWS OpsWorks console and deleting the instance from the stack.
I've setup a new EC2 Instance in AWS with a Private Key (downloaded and added to my ~/.ssh folder).
However, once the EC2 Instance has started, I try to ssh to that instance "a.a.a.a" public IP using the Private Key, however it logs me in to a different IP/instance.
Is there an ssh or private key cache of some sort I don't know about, or howcome I get ssh'd into a different EC2 Instance (in a different subnet)?
Instead of guessing, do this. Once you ssh into the instance, invoke ec2metadata which will list among other data, the private ip and public ip (if it is assigned one) of the instance.
/usr/bin/ec2metadata
~$ ec2metadata
ami-id: ami-xxxxx
...
availability-zone: us-east-1a
...
instance-id: i-8080abcd
instance-type: m3.medium
...
local-ipv4: 10.2.1.40
...
public-hostname: ec2-23-64-195-76.compute-1.amazonaws.com
public-ipv4: 23.64.195.76
...
In case you do not find ec2metadata, download it:
$ wget http://s3.amazonaws.com/ec2metadata/ec2-metadata
EC2 Instance Metadata Query Tool
I believe you have already found solution, if not, you may consider trying below.
In your Mac/Windows there should be a file called '/xxx/xxx/.ssh/known_hosts' and please try to find entries of both IP's and remove those lines (These lines are added when you are trying to SSH to new instance. So there could be some conflict due to the old entries). I had faced similar issues and I did and it was working. Thanks
You can still get all meta-data or user-data of the instance by doing simple http request using curl/wget inside your instance:
$ curl http://169.254.169.254/latest/meta-data/
It should return all keys that you might be interested in getting its value like the following:
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
instance-action
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups
Now get only the key/data you want by specifying its name, the following will help distinguish your instance(s):
$ curl http://169.254.169.254/latest/meta-data/public-ipv4
$ curl http://169.254.169.254/latest/meta-data/local-ipv4
$ curl http://169.254.169.254/latest/meta-data/ami-id
$ curl http://169.254.169.254/latest/meta-data/public-hostname
$ curl http://169.254.169.254/latest/meta-data/local-hostname
$ curl http://169.254.169.254/latest/meta-data/mac
To know more about EC2 Metadata, you can check http://www.dowdandassociates.com/blog/content/howto-get-amazon-ec2-instance-metadata/
I have a docker swarm all hosted on AWS, created basically along the lines of this tutorial.
To deploy our code, I need to be able to access this swarm separate of the computer where I created these instances. I don't see anywhere in the docs for the docker-machine amazonec2 driver where I can use my AWS credentials to connect to these existing instances.
Some tutorials I came across use a --url argument to specify via the docker-machine url to connect to an existing instance, but I don't see that argument in my most recent docker-machine version.
Other tutorials mention TLS configuration and using that in conjunction with docker-machine to connect to existing instances, but given unique/secret AWS credentials, this seems redundant and adds a layer of complexity I hope I can avoid.
What is the recommended approach to this?
Unable to connect:
puttygen my-key.pem -L > id_rsa
docker-machine create --driver generic --generic-ip-address=ec2-....compute.amazonaws.com --generic-ssh-key id_rsa Swarm-Dev01
Running pre-create checks...
Creating machine...
(Swarm-Dev01) Importing SSH key...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
To get access to an existing instance, you can use the docker-machine create --driver generic command. The command will ssh onto the machine, make sure docker is installed, and then download certificates that it stores for future access e.g. using docker-compose.
Command:
docker-machine create \
--driver generic \
--generic-ip-address=<your_ip> \
--generic-ssh-key ~/.ssh/id_rsa \
vm
Documentation:
https://docs.docker.com/machine/drivers/generic/
if I gave --node-name option in my knife ec2 server create node name is taking
for e.g:
knife ec2 server create -r "role[test]" -I ami-59aas30 --flavor
t1.micro -x ubuntu --ssh-key VS_Key --availability-zone us-east-1a -p
22 --tags Name=filetext --subnet subnet-f7sdfdf -g sg-8adsf3e4
--node-name Testnode
my question is if I don't mention any node name how it is taking the instance-id as node name, I want to use instance ids in my code. I would like to know, how it is taking the instance's ids
On ubuntu you can use ec2metadata, which is part of the cloud-utils apt-get package:
apt-get install cloud-utils
ec2metadata --instance-id
You cannot specify Amazon EC2 instance IDs - EC2 generates those values for you when you create a new virtual machine. The --node-name flag corresponds to the node's name in the Chef Server, not the instance id.
To get the id for a specific instance, you can use the AWS Command Line Tools:
ec2-describe-instances
Or you can use knife:
knife ec2 server list
Or you can search the node attributes for this value:
knife search node "node_name:YOURNAME" -a ec2.instance_id
You can retrieve the instance-id from instance identity document
instance_info =
Chef::HTTP.new('http://169.254.169.254/latest/dynamic/instance-identity/document').get('/')
instance_id = JSON.parse(instance_info)['instanceId']
169.254.169.254 is the IP of Amazon's service that serves these documents so this should work as long as you run it from your machine running on AWS.
I have name of the a ec2 instance and want to do ssh to it. How can I figure out the 'Public DNS' of the ec2 instance using the ec2 instance name.
I want to do it using bash.
aws ec2 describe-instances --instance-ids i-12abc34 --query 'Reservations[].Instances[].PublicDnsName'
Where i-12abc34 is your instance id
You can query the instance metadata service.
Using curl:
curl -s http://169.254.169.254/latest/meta-data/public-hostname
Using wget:
wget -qO - http://169.254.169.254/latest/meta-data/public-hostname
If brave, actual bash:
exec 3<> /dev/tcp/169.254.169.254/80
echo -e "GET /latest/meta-data/public-hostname HTTP/1.0\r\n\r\n" >&3
cat <&3
(The last one leaves the connection open for me, so the cat gets stuck. The headers are also present in the output)
(This is from the instance itself and need access to the instnace - it is not the instance name-related version. There are enough of those answers here)
Using the EC2 API tools:
# Region is only needed if not in us-east-1
$ ec2-describe-instances --region <region> <instance id>
Using the unified AWS CLI tool:
$ aws --region <region> ec2 describe-instances --instance-ids <instance id≥
I prefer the unified tool as it offers comprehensive and consistent data.
If you install the cloud-utils tool as described in this answer it's much more straight
forward.
https://stackoverflow.com/a/10600619/28672
ec2-metadata --public-ipv4
> public-ipv4: 54.200.4.52
It depends on what you mean by "figure out". If you mean figuring out yourself, you cannot. The public DNS name has nothing to do with the ec2 instance name. The public DNS name is composed of public IP address, region/availability zone, type of service, aws domain name, etc. For example, ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com. Because the public IP address is changed every time you stop and start your instance, unless you use an elastic IP address, your public DNS name will be changed.
If you mean figure out by using AWS API or CLI tool, you can. Using EC2 CLI, you should use command
ec2-describe-instances instance_id. Again, the instance has to be running and the public DNS does change after stop/start.