So I've been using AWS AMI in my cloud formation template.
It seems they create new images every month and deprecate the old ones 2 weeks or so after the new one's released. This creates many problems:
Old template stacks becomes broken.
Templates need to be updated.
Am I missing something?
E.G.
I'm staring at
API: ec2:RunInstances Not authorized for images: [ami-1523bd2f]
error in my
cloud formation events.
Looking it up that's the 02.12 image id:
http://thecloudmarket.com/image/ami-1523bd2f--windows-server-2012-rtm-english-64bit-sql-2012-sp1-web-2014-02-12
Where as now there's a new image id:
http://thecloudmarket.com/image/ami-e976efd3--windows-server-2012-rtm-english-64bit-sql-2012-sp1-web-2014-03-12
You are correct indeed. Windows AMI are deprecated when a new version is released (see http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/Basics_WinAMI.html)
There is no "point and click" solution as of today, documentation says : "AWS updates the AWS Windows AMIs several times a year. Updating involves deprecating the previous AMI and replacing it with a new AMI and AMI ID. To find an AMI after it's been updated, use the name instead of the ID. The basic structure of the AMI name is usually the same, with a new date added to the end. You can use a query or script to search for an AMI by name, confirm that you've found the correct AMI, and then launch your instance."
One possible solution might be to develop a CloudFormation Custom Resource that would check for AMI availability before launching an EC2 instance.
See this documentation about CFN Custom Resources : http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/crpg-walkthrough.html
And this talk from re:Invent : https://www.youtube.com/watch?v=ZhGMaw67Yu0#t=945 (and this sample code for AMI lookup)
You also have the option to create your own custom AMI based on an Amazon provided one.Even if you do not modify anything. Your custom AMI will be an exact copy of the one provided by Amazon but will stay available after Amazon AMI's deprecation.
Netflix has open sourced tools to help to manage AMIs, have a look at Aminator
Linux AMI are deprecated years after release (2003.11 is still available today !) but Windows AMI are deprecated as soon as a patched version is available. This is for security reason.
This ps script works for my purposes, we use windows 2012 base image:
$imageId = "xxxxxxx"
if ( (Get-EC2Image -ImageIds $imageId) -eq $null ) {
$f1 = New-Object Amazon.EC2.Model.Filter ; $f1.Name="owner-alias";$f1.Value="amazon"
$f2 = New-Object Amazon.EC2.Model.Filter ; $f2.Name="platform";$f2.Value="windows"
$img = Get-EC2Image -Filters $f1,$f2 | ? {$_.Name.StartsWith("Windows_Server-2012-RTM-English-64Bit-Base")} | Select-Object -First 1
$imageId =$img.ImageId
}
I recently ran into the same error. I had built a custom ami in one account, and was trying to run an EC2 instance from another account.
The issue for me was that the AMI did not have the correct permissions to enable my user from the other account to run it.
To fix it, I logged in the other account and added the required permissions to the ami:
aws ec2 modify-image-attribute --image-id youramiid --launch-permission "Add=[{UserId=youruserid}]"
More information at this documentation page.
If you are using a training material and copied the code, make sure to replace the AMI name with the correct AMI Image values available under list of AMI's visible under your account. Similar with other values. If you are just cut and paste the values from training code may not be available now.
Related
I am having a template where I am creating a Ubuntu EC2 instance based on the region and the associated image id mapped in the template.Is there anyway through which the the latest Ubuntu image id will get selected based on the region.This will happen during template execution.It would be helpful to get any sample template for the same.
There are few ways you can achieve this:
A) You can use the Mappings section of the template to specify an AMI for each region. You would then use Fn::FindInMap to retrieve the value of the AMI according to the evaluation of the pseudo parameter AWS::Region.
See:
Mappings - AWS CloudFormation
Fn::FindInMap - AWS CloudFormation
Pseudo Parameters Reference - AWS CloudFormation
B) You can use a lambda backed custom resource to retrieve the latest ubuntu AMI during stack creation. There is a getting started guide for the same, you can use it as a starting point.
See: Walkthrough: Looking Up Amazon Machine Image IDs - AWS CloudFormation
C) If you can migrate to an Amazon Linux AMI, based on RHEL, you can reference public systems manager parameters for the latest AMI id for that region. I have an example template in github you can use as a reference.
See: CloudFormationExamples/highlyavailable-asg-lamp-server-alb at master ยท smith-b/CloudFormationExamples
I am new to Terraform. How to stop GCP vm instances using terraform?
I have tried changing status of the vm instance, it's available for AWS but couldn't find the way to do it for GCP.
Edit
Since version v3.11.0 of Google provider (released 2020/03/02), it is possible to shutdown and start a Compute instance with desired_status field :
compute: added the ability to manage the status of google_compute_instance resources with the desired_status field
Just declare in your Terraform resource :
resource "google_compute_instance" "default" {
name = "test"
machine_type = "n1-standard-1"
zone = "us-central1-a"
[...]
desired_status = "TERMINATED"
}
And apply your changes. If your instance was running before, it should be shut down. This PR shows the modifications that have been added, if you are interested to take a look. The desired_status can either take RUNNING or TERMINATED values.
Previous answer (as of 2019/10/26)
As of the time of the question (2019/09/18), with the latest Google provider available then (version v2.15.0), this is not possible to update the status of a Google Compute instance.
The following issue is opened on the Google Terraform provider on Github :
google_compute_instance should allow to specify instance state #1719
There is also a Pull Request to add this feature :
ability to change instance_state #2956
But unfortunately, this PR seems to be stale (not updated since 2019/03/13).
I launched a windows EC2 instance on AWS but I can't get the password for login. I keep getting this warning message even one day after launching the server.
Password not available yet.
Please wait at least 4 minutes after launching an instance before trying to retrieve the auto-generated password.
Note: Passwords are generated during the launch of Amazon Windows AMIs or custom AMIs that have been configured to enable this feature. Instances launched from a custom AMI without this feature enabled use the username and password of the AMI's parent instance.
And I also tried below command line:
$ aws --profile ie ec2 get-password-data --instance-id i-xxxxx --priv-launch-key my.pem --region ap-southeast-2
but it returns an empty password:
{
"InstanceId": "i-xxxx",
"PasswordData": "",
"Timestamp": "2019-08-05T23:12:04.000Z"
}
So how can I get the password for this EC2 instance?
I have tried to stop/start the instance but it doesn't help.
One possible reason is that the instance is launched from a customised AMI but I also don't know that AMI's password. Is there a way to reset the password?
It's possible that EC2Config is disabled. The empty string you are getting for console output caused by this. It could be an issue with the EC2Config service; either the misconfigured configuration file or that Windows failed to boot properly.
For recovery, I'd say try the password you used at the machine used to create the AMI and if it's not a custom made AMI, try a different one altogether. I'd be more helpful if you can share the AMI ID.
Additionally, if you are looking to recover data on an EBS volume of the server, you can follow this
I have the AWS CLI installed on my Windows computer, and running this command "works" exactly like I want it to.
aws ec2 describe-images
I get the following output, which is exactly what I want to see, because although I have access to AWS through my corporation (e.g. to check code into CodeCommit), I can see in the AWS web console for EC2 that I don't have permission to list running instances:
An error occurred (UnauthorizedOperation) when calling the DescribeImages operation: You are not authorized to perform this operation.
I've put terraform.exe onto my computer as well, and I've created a file "example.tf" that contains the following:
provider "aws" {
region = "us-east-1"
}
I'd like to issue some sort of Terraform command that would yell at me, explaining that my AWS account is not allowed to list Amazon instances.
Most Hello World examples involve using terraform plan against a resource to do an "almost-write" against AWS.
Personally, however, I always feel more comfortable knowing that things are behaving as expected with something a bit more "truly read-only." That way, I really know the round-trip to AWS worked but I didn't modify any of my corporation's state.
There's a bunch of stuff on the internet about "data sources" and their "aws_ami" or "aws_instances" flavors, but I can't find anything that tells me how to actually use it with a Terraform command for a simple print()-type interaction (the way it's obvious that, say, "resources" go with the "terraform plan" and "terraform apply" commands).
Is there something I can do with Terraform commands to "hello world" an attempt at listing all my organization's EC2 servers and, accordingly, watching AWS tell me to buzz off because I'm not authorized?
You can use the data source for AWS instances. You create a data source similar to the below:
data "aws_instances" "test" {
instance_tags = {
Role = "HardWorker"
}
filter {
name = "instance.group-id"
values = ["sg-12345678"]
}
instance_state_names = ["running", "stopped"]
}
This will attempt to perform a read action listing your EC2 instances designated by the filter you put in the config. This will also utilize the IAM associated with the Terraform user you are performing the terraform plan with. This will result in the error you described regarding lack of authorization, which is your stated goal. You should modify the filter to target your organization's EC2 instances.
Problem:
I have an EMR cluster (along with a number of other resources) defined in a cloudformation template. I use the AWS rest api to provision my stack. It works, I can provision the stack successfully.
Then, I made one change: I specified a custom AMI for my EMR cluster. And now the EMR provisioning fails when I provision my stack.
And now my stack creation fails, due to EMR provisioning failing. The only information I can find is an error on the console: null: Error provisioning instances.. Digging into each instance, I see that the master node failed with error Status: Terminated. Last state change reason:Time out occurred during bootstrap
I have s3 logging configured for my EMR cluster, but there are no logs in the s3 bucket.
Details:
I updated my cloudformation script like so:
my_stack.cfn.yaml:
rMyEmrCluster:
Type: AWS::EMR::Cluster
...
Properties:
...
CustomAmiId: "ami-xxxxxx" # <-- I added this
Custom AMI details:
I am adding a custom AMI because I need to encrypt the root EBS volume on all of my nodes. (This is required per documentation)
The steps I took to create my custom AMI:
I launched the base AMI that is used by AWS for EMR nodes: emr 5.7.0-ami-roller-27 hvm ebs (ID: ami-8a5cb8f3)
I created an image from my running instance
I created a copy of this image, with EBS root volume encryption enabled. I use the default encryption key. (I must create my own base image from a running instance, because you are not allowed to create an encrypted copy from an AMI you don't own)
I wonder if this might be a permissions issue, or perhaps my AMI is misconfigured in some way. But it would be prudent for me to find some logs first, to figure out exactly what is going wrong with node provisioning.
I feel stupid. I accidentally used a completely un-related AMI (a redhat 7 image) as the base image, instead of the AMI that EMR uses for it's nodes by default: emr 5.7.0-ami-roller-27 hvm ebs (ami-8a5cb8f3)
I'll leave this question and answer up in case someone else makes the same mistake.
Make sure you create your custom AMI from the correct base AMI: emr 5.7.0-ami-roller-27 hvm ebs (ami-8a5cb8f3)
You mention that you created your custom AMI based on an EMR AMI. However, according to the documentation you linked, you should actually base your AMI on "the most recent EBS-backed Amazon Linux AMI". Your custom AMI does not need to be based on an EMR AMI, and indeed I suppose that doing so could cause some problems (though I have not tried it myself).