I need to create AWS CentOS 7 instance images for a customer, and need it to automatically send the ip and instance id to our AWS server every time the instance boots. For example, this is the very basic test version of the script I need to run:
#!/bin/bash
$serverIP=""
curl "https://$serverIP"/myphp.php?id='sentid'&ip='sentip'"
If the script is run directly, it works fine and is received by the server and processed there. But I can't get it to run at boot. I cannot put the script in the "User Data" directly due to security concerns as the customer can then see it easily, it needs to be in a script in the filesystem of the image.
I've tried several things that work fine on a physical Linux server, but not on AWS. I know profile.d runs every time someone logs in but over-sending like that is fine.
/etc/profile.d/myscript.sh
This stops the AWS instance from booting. Even just
#!/bin/bash/
echo "hello world"
prevents it from booting. The instance starts, but when you go to ssh into it you get 'Network Error: connection timed out', which is the standard error if you put a wrong ip in, or upset it by leaving a service like httpd enabled.
However, a blank bash script with just #!/bin/bash will allow the instance to start. Removing the script via user data usually makes it boot, sometimes it just dies.
The first thing I tried was crontab. I did:
crontab -e
#reboot /var/ook/myscript.sh
systemctl enable crond.service
But the instance wouldn't start. So I put "systemctl disable crond.service" in the User Data and one booted, but another still stayed dead. Myscript.sh was just another echo "doob" >> file which worked fine when run directly.
I tried putting in /etc/systemd/system/my-startup.service:
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/var/ook/writedood.sh
[Install]
WantedBy=multi-user.target
then:
systemctl enable my-startup.service
But this did nothing. My script "writedood.sh" was just echo "doob" >> ./file.txt ensuring file.txt was chmod 777. At least it didn't prevent the instance from starting.
To give context, an instance won't start if httpd is left enabled on shutdown, but will if you disable it in User Data.
I wanted to have a go at putting something in init.d but I'm not sure how to simply tell it to run a script once in the background, and given the plethora of success I've had so far with the instance not restarting, I'm not holding out much hope that that would work.
Thanks in advance!
EDIT::: I realised that sometimes AWS EC2 Instances Console is causing the problem where I can't ssh in after stopping and starting. It blanks the public ipv4 address when I click stop, but when I start, it puts the old address up and hangs. If I refresh the page, or uncheck/check the instance; the ip changes to the new address. This has caused much consternation.
Crontab worked if I placed the scripts and output file in different folders. It's very finicky; any errors, such as it not being able to write to the output file, and the instance won't start. I put startscript.sh in /usr/local/src, and output.out to /tmp/ to ensure there were no permissions problems, and now the instance starts and runs the script on boot.
I then realised that sometimes AWS EC2 Instances Console is causing the problem where I can't ssh in after stopping and starting. It blanks the public ipv4 address when I click stop, but when I start, it puts the old address up and hangs. If I refresh the page, or uncheck/check the instance; the ip changes to the new address. This has caused much consternation.
I have been using AWS educate for an ec2 instance that I connect using terminal on my macbook pro Catalina. I use ubuntu to connect (I have putty running on terminal) and I have been able to connect perfectly fine and changed nothing, but after about a week or 2 when I use the command
sophiadhanani$ ssh -i "name of pem file.pem" ubuntu#ec2-IPnumbers.compute-1.amazonaws.com it returns permission denied (publickey).
No idea why this is happening and the trouble shoot says to create a temp instance and then mount point- is there an easier way to solve this issue?
Are you using an encrypted home directory on your Macbook?
Apparently, using an encrypted home, you can experience login problems.
You may want to remove the file from ~/.ssh/authorized_keys to somewhere else e.g. /etc/ssh/.authorized_keys
And then edit your the /etc/ssh/sshd_config file and add /edit to point to the new location.
AuthorizedKeysFile /etc/ssh/.authorized_keys
I'm trying to give access to a GCE VM by adding SSH Keys to the project metadata. My current SSH key is in the project metadata and I can connect just fine using:
ssh -i ~/.ssh/<private_key> <username>#<instance_ip>
Now, I generated another key:
ssh-keygen -t rsa -f ~/.ssh/<new_key> -C <new_username>
After adding the generated public key to the project metadata, I then run:
ssh -i ~/.ssh/<new_private_key> <new_username>#<instance_ip>
But I get Permission denied (publickey,gssapi-keyex,gssapi-with-mic). Running with -vvv flag doesn't show me much besides the key being rejected.
Things I know/checked:
firewall isn't an issue because I can connect using my original key from same location
the instance is running SSH (running nc <instance_nat_ip> 22 shows "OpenSSH" etc.)
no passphrases were used with the generation of any SSH key
there are no instance-level restrictions on project-wide metadata
there are no instance-level ssh keys already added
there are no newlines/breaks causing the key to be malformed
permissions on ~./ssh aren't an issue since another key pair works fine from the same directory, additionally, both key pairs have the same permissions anyways
OSLogin isn't enabled either on the project or instance
Things I've tried:
removing and readding the SSH keys in project metadata
trying with new key pairs generate on another person's machine
restarting sshd service
Questions:
Does the username specified during the ssh-keygen step have to already exist on the remote instance prior to adding the key to the metadata? i.e. do I have to run sudo useradd <new_username> while SSH'd into the instance Creating a new test instance revealed this to not be the case, all users in the project metadata were created automatically
why does my existing SSH key work and not new ones even though they are added the same way?
there's a chance the enable-oslogin:TRUE was applied to the instance briefly a long time ago (I'm not sure since I'm not the one who created the instance) but it's no longer there in the instance or project metadata. Would having that been enabled, even briefly, cause some issues?
EDIT: I started up a new instance in the same project with the same network details and I was able to SSH to that instance using the new key. Original instance is still denying the key
Did some digging around and found out that the systemd service that propagates accounts information from the metadata server is a daemon called google-accounts-daemon.
When I ran sudo ps aux | grep daemon I didn't see it running as I did on the test instance I created.
So when I ran sudo systemctl restart google-accounts-daemon the SSH keys magically propagated and everything worked.
I have no idea what caused the daemon to stop running in the first place, so if anyone has ideas, that'd be appreciated in case this comes up in the future.
This may be an extremely dumb question but I am new to AWS and terminal controls on Mac. I am trying to SSH into my EC2 instance and following the documentation here.
I am perplexed because it is asking for the PATH in order to chmod 400 my private .pem file. I am unsure which path they are asking for here and would love some clarification. I have already downloaded AWS CLI so I am unsure what PATH it is asking for. Any help is very appreciated.
As i checked document link shared by you, here path means the location where you have downloaded .pem key file during launching a instance on AWS.
If you used Safari browser, you can find Download location:-
http://support.topspinmedia.com/hc/en-us/articles/204262743-I-m-on-a-Mac-using-Safari-where-s-my-download-
ssh -i /Users/Onicha/Downloads/my-key-pair.pem ec2-user#ec2-198-51-100-1.compute-1.amazonaws.com
When you created that EC2 instance, you must have downloaded a private key. If not or you do not have the key anymore, I am afraid you must remove that instance and set up a new one.
Once you have the key, in your terminal, change the directory to where the private key is. You must first change the permissions on the key to make it only readable by your user and then you can SSH to it.
chmod 400 key.pem
ssh -i key.pem [user]#[host]
You will need to specify the path to the key, instance key pairs are described here http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
It may help to verify a few things-
Are you creating the key pair on Amazon Ec2 (or whichever instance you are using) and then downloading and saving (as a .pem file- example yourpemfile.pem) on your computer? If you are, you would know which directory (the PATH that is being discussed) you saved it in. If you already did it and cannot find it, you can do it again creating a new key pair with a new filename yourpemfile2.pem, but you have to stop the Ec2 instance and remove the old keypair file associated with it. (I have seen ways online to change keypairs while instance is running as well).
Go to that particular folder on your Mac using Terminal. You don't need Amazon CLI if you use the terminal with MacOS. (You can do so by pressing Command+Space and then type terminal in the blank typing space that appears). Once the terminal window opens, at the prompt- type linux command 'pwd' to see which directory/folder your are in. Use 'cd' command for going to your folder where the .pem file is downloaded or saved (by you). List the particular folder by using linux command 'ls -l' at the prompt to see if your .pem file is actually there and what are its permissions.
Then change permissions using 'chmod' command-> chmod 400 yourpemfile.pem. ls -l yourpemfile.pem' again the file permissions should become r--------.
Now you are in the directory where you pem file is and you can directly ssh to your Ec2 instance from here. Now (using web browser) go to Amazon Ec2 Management Console and click on Instances (within folder Instances) on the left of dashboard you will see details of your instance which is running. Make sure the key pair associated with that instance is this particular .pem file. Then at the top of the page, where it says "Launch instance" and also has two other tabs, "Connect" and "Actions", click on "Connect".
A new window pops up which provides details of ssh commands to use and instance details.
Make sure to select standalone SSH client.
You see a suggested command like this- (helps in making sure you are using the right amazon instance and keypair)
ssh -i "AmcEc2mykeypair.pem" ec2-user#ec2-134-17-351-22.us-east-2.compute.amazonaws.com
Copy paste this on your linux terminal (Use Command+c and Command+v on Mac). Press Enter.
You should be 'logged in' now and a new prompt for the Amazon Ec2 instance machine- specifying clearly that it is an Amazon linux or ubuntu image should appear.
I've just setup my EC2 server following this video here exactly - http://www.youtube.com/watch?v=bBajLxeKqoY
I even chose the same server type, everything went well until it asked for the root password to my EC2 server...
Any suggestions?
Update
Updating this answer because of the activity:
Depending on if the system is ubuntu or Rhel the user varies.
For ubuntu it is
ssh -i my-pem-file.pem ubuntu#my-ec2-instance-address
For RHEL it is
ssh -i my-pem-file.pem root#my-ec2-instance-address
Connecting to an ec2 instance does not require a password, it would require only a pem file
and this is how you connect to it
ssh -i my-pem-file.pem ec2-user#my-instance-address
and remember to chmod 400 your pem file before ssh'ing
If you need to do things as root once you are in as ec2-user, use sudo su - that gets you to root and doesn't need a password. Somethings you do need that for, like looking at the tomcat log files
I had the same problem and after a lot of struggle, I read this page again:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html
Notice in the 4th topic it talks about the user name. It is usually ec2-user but if you are using RHEL5 distro it can be root and if you are using the Ubuntu the user name will be ubuntu - which was my case.
So alternatively try those:
ssh -i my-pem-file.pem ubuntu#my-ec2-instance-address for Ubuntu
or
ssh -i my-pem-file.pem root#my-ec2-instance-address for RHEL5
Hope it helps!
You will be asked for password when you enable PasswordAuthentication yes in your sshd_config. Try changing that to no. that should fix it.
Just for anyone else that might have the same problem
Just do
sudo su -
Worked for me
If you've moved/copied the pem file from another machine, the owner of the file may be different to the user that is running the ssh client. To change the owner of the file:
sudo chown <currentusername> <filename.pem>
Please use --query 'KeyMaterial' while generating key.
Info:
keyMaterial - an unencrypted PEM encoded RSA private key.
In my case, we had been copying a text file to keep a ubiquitous key. Someone accidentally added characters to this file, and we started getting prompted for a password from the now-corrupt .pem file, when we had never set a password.
In my case, the problem was the file's break type. Try this:
1.- Open the .pem file with TextWrangler
2.- At Bottom of app, verify if the Break Type is "Windows(CRLF)".
Regards
This is the way to connect:
ssh -i /path/my-key-pair.pem ec2-user#public-ip
Now, instead of ec2-user, it could be root, or centos, or ubuntu, or something else. You can check under the "Usage Instructions". If it's not there, and you've tried all the above users, find out from the documentation that came with the AMI.
There is one bug on AWS that cost me a lot of time. If you're launching an instance from a saved AMI under "Images > AMIs", note that it uses the original keys of the running instance it was created from. When launching the AMI, it will prompt you to choose a new key, and even show such key under the description, but the truth is those keys will never work!
So if you're using a custom/saved AMI you'll have to either get the original keys and use them, or just create a brand new one from AWS or the market place. If you haven't created the AMI or can't remember where it came from, look under the details tab. You can then launch a new instance or create a new AMI from the same source. This will then use the keys that you specify.
chmod 400 pem_file
ssh -i /path_to_the_pem_file ec2-user#ServerIP
or
ssh -i /path_to_the_pem_file ubuntu#ServerIP
root access is prohibited by default by AWS AMIs.
hope it helps.
Check your .pem file is not corrupt. Using this line
openssl rsa -check -in test.pem -noout
It should return RSA Key OK. If it doesn't then there is most likely a problem with some sort of formatting or something in your pem file.
You should use the *.pem instead of a root password. Once you have logged in use passwd to set a password.
It could be a Linux problem.
But there is also a chance that you use the wrong address/key (you started a new instance but still using the old address; or 2 pem have similar name, used the wrong one), "ssh with non-existing user".
I am using TurboLinux instance, on Mac OS system, please try this command:
ssh -i xxxxx.pem root#xxx.xxx.xxx.xxx (public address of allocated VM)
I can get through without asking password anymore.
In case someone else bumps into this, the solution for my problem was that I had to run it with sudo:
sudo ssh -i my-pem-file.pem root#my-ec2-instance-address
For me, the issue was that I had created an AMI from an existing instance, so when I launched the AMI, even though I selected a new key, the instance's original key was what worked.
Login to Amazone console in browser:
https://us-east-2.console.aws.amazon.com/ec2/xxxxxxxx
Click on instances from left panel -> then select your instance -> click on connect button at top right
You will see a window open and there will be a button ssh Client
Here you can see exact command to connect with your instance.