I am trying to create a docker image based on httpd with a custom information about the docker image. So for that am trying to set the ECS_ENABLE_CONTAINER_METADATA=true in /etc/ecs/ecs.config.
I am trying to do it in the user data of the ecs instance. First thing i noticed is there is no provision to specify the user data while creating the cluster.
Then tried copying the launch configuration and edited the user data per below stackoverflow,
ECS, how to add user-data after creating ecs instance
But when i try to run tasks, I found that no ecs instance is linked with the cluster.
Any suggestions if you had run to similar issue ?
It seems that the ECS instance is not registered with the cluster. You need to ensure that the AMIs you use to create the ECS instance has the ECS agent enabled and running. The full list of AMIs is available in the ECS developer docs under container instances.
Related
I am new to ECS
I need Help on few of the things:
FIRST
I am gonna use docker images tagged according to git tags then pushed to ECR and update the task Definition. Is there any way by which i can know that which revision consists of which tag. As the revisions are numbered in a sequential manner rather than the docker image tag. And i am gonna use bitbucket pipelines to push the image then update the service.
I want this feature so that i can revert to a desired tag anytime.
Will a python script with boto3 be helpful? Can anybody help with that
SECOND
An EC2 instance is not launched on creating a new cluster other then t2.micro...
and then it leads to "No Container Instances were found in your cluster"
error on creating any service on that cluster.
I checked that i have 'AmazonEC2ContainerServiceforEC2Role' attached to ecsinstance Role and i explicitly added this as a policy to my IAM user. BUt still the same issue. ANy Help!!!
For SECOND:
As per AWS Docs, the instances containing the Docker Images should reside in public subnet to query repositories.
I had this issue earlier and it was fixed when I changed the subnets in AutoScaling groups to public instead of private.
I've faced with the problem while using AWS SDK. Currently I am using SDK for golang, but solutions from other languages are welcome too!
I have ECS cluster created via SDK
Now I need to add EC2 containers for this cluster. My problem is that I can't use Amazon ECS Agent to specify cluster name via config:
#!/bin/bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
or something like that. I can use only SDK.
I found method called RegisterContainerInstance.
But it has note:
This action is only used by the Amazon ECS agent, and it is not
intended for use outside of the agent.
It doesn't look like working solution.
I need to understand how (if it's possible) to create working ECS clusterusing SDK only.
UPDATE:
My main target is that I need to start specified count of servers from my Docker image.
While I am investigating this task i've found that I need:
create ECS cluster
assign to it needed count of ec2 instances.
create Task with my Docker image.
run it on cluster manually or as service.
So I:
Created new cluster via CreateCluster method with name "test-cluster".
Created new task via RegisterTaskDefinition
Created new EC2 instance with ecsInstanceRole role with ecs-optimized AMI type, that is correct for my region.
And there place where problems had started.
Actual result: All new ec2 instances had attached to "default" cluster (AWS created it and attach instance to it).
If I am using ECS agent I can specify cluster name by using ECS_CLUSTER config env. But I am developing tool that use only SDK (without any ability of using ECS agent).
With RegisterTaskDefinition I haven't any possibility to specify cluster, so my question, how I can assign new EC2 instance exactly to specified cluster?
When I had tried to just start my task via RunTask method (with hoping that AWS somehow create instances for me or something like that) I receive an error:
InvalidParameterException: No Container Instances were found in your cluster.
I actually can't sort out which question you are asking. Do you need to add containers to the cluster, or add instances to the cluster? Those are very different.
Add instances to the cluster
This is not done with the ECS API, it is done with the EC2 API by creating EC2 instances with the correct ecsInstanceRole. See the Launching an Amazon ECS Container Instance documentation for more information.
Add containers to the cluster
This is done be defining a task definition, then running those tasks manually or as services. See the Amazon ECS Task Definitions for more information.
I am trying to programmaticly create a ECS cluster with EC2 instance in it. As far as I understand I should first create an ECS cluster , than EC2 instance and then register instance using this method :
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/ECS.html#registerContainerInstance-property
Is it how I should do it? Which arguments are mandatory? How to get instanceIdentityDocument and instanceIdentityDocumentSignature?
thanks
I would use the User Data of the EC2 instance to launch the instance directly into the ECS cluster. This is the User Data you'll want to use:
#!/bin/bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
The details of this are described in the AWS docs. You can also use this user data in an Auto Scale Group Launch Configuration.
Apart from that, it might be worth it to look into languages that where made to provision infrastructure, like Terraform (also for AWS) or CloudFormation (specifically for AWS).
We have an AutoScaling Group that runs containers (using ECS). When we add OR replace EC2 instances in the ASG, they don't have the docker images we want on them. So, we run a few docker pull commands using cloud-init to fetch the images when they boot up.
However, the ASG thinks that the new instance is ready, and terminates an old instance. But in reality, this new instance isn't ready until all docker images have been pulled.
E.g.
Let's say my ASG's desired count is 5, and I have to pull 5 containers using cloud-init. Now, I want to replace all EC2 instances in my ASG.
As new instances start to boot up, ASG will terminate old instances. But due to the docker pull lag, there will be a time during the deploy, when the actual valid instances will be lesser than 3 or 2.
How can I "mark an instance ready" only when cloud-init is finished?
Note:I think Cloudformation can bridge this communication gap using CFN-Bootstrap. But I'm not using Cloudformation.
What you're look got is AutoScaling Lifecycle Hooks. You can keep an instance in the Pending:Wait state until you're docker pull has completed. You can then move the instance to InService. all of this can be done with the AWS CLI so it should be achievable with an AWS AutoScaling command before and after your docker commands.
The link to the documentation I have provided explains this feature in detail and provides great examples on how to use it.
I have an instance in AWS that I set up my entire environment (I'll call it my ready instance) on and is running perfectly. I then created a load balancer (ELB) with an autoscaling policy.
When I created a load balancer with an autoscaling policy (min of 2 instances), 2 instances sprung up. The instances were empty, however. For the launch configuration, I specified my ready instance AMI. Isn't this supposed to tell the autoscaling policy which instance to clone? In this case, shouldn't my ready instance be cloned into them and they should have the same content?
Instances are not created based on a clone of a running instance, but rather just the disk image stored in the AMI. It might be a case where you need to create a new AMI from your running instance and use that AMI as the basis for your autoscaling group.
once upon a time even I was this phase of my life.Basically it will just the boot the AMI which you have specified in your as configuration. If your AMI has got old code, then it will boot and serve your client from the out-dated code. Do solve this you can automate the code management process, all you have to do is boot the new ami with a user-data script which will perfom certain actions during the boot. Using user-data script you can automate this process. Autoscaling configuration also have a provision to accept the user-script data.