I can't find a way to specify a user-data after creating ECS instance definition.
Document says You can pass this user data into the Amazon EC2 launch wizard in Step 6.g of Launching an Amazon ECS Container Instance.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bootstrap_container_instance.html#multi-part_user_data
ECS is launched automatically, how do you specify the user data?
I want to send /var/log/syslog to cloudwatch and I need to add user data (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_cloudwatch_logs.html)
I had to add the user data as a autoscaling group property
steps are
copy existing launch configuration
edit user data of the launch configuration
edit autoscaling group to use the created launch configuration
terminate ecs instances so that the modified autoscaling group launches new ec2 with new launch configuration
via terraform we can pass it as template file within launch config
data "template_file" "user_data" {
template = "${file("${path.module}/templates/user_data.sh")}"
vars = {
ecs_config = "${var.ecs_config}"
ecs_logging = "${var.ecs_logging}"
cluster_name = "${var.cluster}"
env_name = "${var.environment}"
custom_userdata = "${var.custom_userdata}"
cloudwatch_prefix = "${var.cloudwatch_prefix}"
}
By default, user data scripts and cloud-init directives run only during the first boot cycle when an EC2 instance is launched.
https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/
In the article it also explain further possible workaround.
Related
Our EKS cluster is terraform managed and was specified with EC2 Launch Template as terraform resource. Our aws_eks_node_group includes Launch template section as shown below.
resource "aws_eks_node_group" eks_node_group" {
.........................
..........................
launch_template {
id = aws_launch_template.eksnode_template.id
version = aws_launch_template.eksnode_template.default_version
name = aws_launch_template.eksnode_template.name
}
}
However, after a while, EKS self-deployed the new Launch Template and linked it to the relevant auto-scaling group.
Why this has happened at the first place and how to avoid it in the future?
How can we link the customer managed Launch Template to EKS Autoscaling Group via terraform again? I tried changing the name or version of the launch template, but it is still using one created by EKS (self-managed)
The Amazon EKS API creates this launch template either by copying one you provide or by creating one automatically with default values in your account.
We don't recommend that you modify auto-generated launch templates.
(source)
I want to create AWS CloudWatch dashboard with some metrics via Terraform. All my infrastructure are described in Terraform and I understand how to create AWS CloudWatch dashboard (Terraform + json template). The stuck is in AWS AutoScaling Group. When I want to display some metrics on dashboard, I just use constructions like
some_monitored_instance_id = "${aws_instance.some_instance.id}"
which then puts to json template like
"metrics": [
["AWS/EC2", "CPUUtilization", "InstanceId", "${some_monitored_instance_id}"]
],
All fine when instances are started via
resource "aws_instance" "some_instance" {}
But I can not use such method when instances are started via AutoScaling Group. How can I extract instance ids when instances launched via AutoScaling Group (and Launch Configuration) for future use in Terraform?
First, you really shouldn't. ASGs change out instances and those IDs will change. Cloudwatch offers metrics for ASGs. So you can see metrics for instances made by the ASG. You can also create a resource group and have metrics by resource group.
But, if you really wanted to do this:
data "aws_instances" "test" {
instance_tags = {
SomeTag = "SomeValue"
}
instance_state_names = ["running", "stopped"]
}
output ids {
value = data.aws_instances.test.ids
}
This will work if you put a tag in your launch config that is set on EC2s at launch.
This works because:
instance_tags - (Optional) A map of tags, each pair of which must exactly match a pair on desired instances
see docs
I have a instance in aws and private key file is authorized keys which is stored in .ssh path.When auto scaling AMI is launched then that file should copy to new server in the same location i.e .ssh how can we do it by using Cloud formation template.
what code or what commands should I keep in CFN template.
Thanks
you can create a new ami with your private key stored and use the ami in your CloudFormation template.
steps:
1. launch new instance from the current ami you are using
2. ssh to your instance and copy the private key file
3. stop the instance in aws console and right click the instance -> image -> create image
4. use your new ami as EcsAmiId
You can specify user data script in your CF template where you are describing your EC2 instance. user data script is just a normal bash script that will be executed when the instance boots up, which means that you can automate any such task as copying files using it, which you would otherwise needed to do manually.
You can also pre-bake a custom AMI, or in other words, create a new AMI that has all the settings already in place and use that AMI in your CF template instead of whatever default AMI you are using right now.
A Keypair can be created within the EC2 console, or the public half of an existing Keypair can be uploaded to EC2.
Once this is done, an Amazon EC2 instance can be launched with a reference to this Keypair. Software on the instance (if using an Amazon Linux AMI) will automatically copy the public half of the nominated keypair to the /home/ec2-user/.ssh/authorized_keys file.
This applies for launching an EC2 instance via any method, eg console, API, CloudFormation.
This is much easier than trying to manipulate the authorized_keys file yourself via User Data.
I have a batch job which is trigged with boto3, I run about 10 workers on an ec2 compute enviroment and they used to inherit the jobname from batch but now all my instances have no name tag so it's difficult to figure out which is which.
How can I pass on a name tag to the ec2 instances for my batch job via boto3?
batch = boto3.client('batch', my_arguments)
batch_detils = batch.submit_job(
jobName = 'worker_{}'.format(i),
jobQueue = 'my_queue',
jobDefinition = 'my_job_definition',
containerOverrides = {'command': ['sleep 100']},
)
I should specify the ultimate goal is to get each worker with a unique name, not to get the all the same name as the compute environment.
The Compute Environment Parameters - AWS Batch documentation page talks of a tags parameter:
Key-value pair tags to be applied to instances that are launched in the compute environment. For example, you can specify "Name": "AWS Batch Instance - C4OnDemand" as a tag so that each instance in your compute environment has that name. This is helpful for recognizing your AWS Batch instances in the Amazon EC2 console.
Have you tried this
Boto3 create_tags
In order to run AWS monitoring scripts (http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts-perl.html) on one of my ec2 existing instance, I need a IAM role associated to the instance.
I found a lot of documentation to launch an instance with a role... But nothing to link a role to an existing instance.
Is it possible?
If not, what can I do? Launch a new instance with a role and transfert the volumes to it?
If not, what can I do? Launch a new instance with a role and transfert
the volumes to it?
This is one option but it might take you a long time if you have a lot instances.
The other option is to simply use IAM by creating a new user then add the Amazon CloudWatch PutMetricData operation permission to that user. Then, create AWS credentials for that user and finally use them as per the docs that you specified:
Optional: If you aren't using an IAM role, update the
awscreds.template file that you downloaded earlier. The content of
this file should use the following format:
AWSAccessKeyId=YourAccessKeyID
AWSSecretKey=YourSecretAccessKey
Also set the environment variable AWS_CREDENTIAL_FILE to point that awscreds.template in the environment of the user that is running the mon-scripts-perl
There is a new method available associate-iam-instance-profile to Associates an IAM instance profile with a running or stopped instance.
Example:
aws ec2 associate-iam-instance-profile --instance-id YourInstanceId --iam-instance-profile Name=YourNewRole-Instance-Profile
Doco
AWS doesn't allow you to modify the instance role after launching the instance.
You can either:
1) Launch a new instance with the role needed by taking the AMI of the already running instance and reassigning the EIP.
or
2) Create a headless user, generate access key and secret key for the user with specific permssion, and use those keys.
Earlier there was no solution. You had to create and AMI and launch the server again with the appropriate role.
On Feb 9, AWS launched these new CLI options which can help you solve your problem.
See this link: https://aws.amazon.com/blogs/security/new-attach-an-aws-iam-role-to-an-existing-amazon-ec2-instance-by-using-the-aws-cli/