I would like to start a task definition on an instance within my cluster (not in the default one). So something like:
create a cluster
create a task definition with a docker image (I have a docker image
already pushed to ecs)
run the task definition in the cluster
I would like to add a keypair to the ec2 instance for ssh access
I have tried to use these functions form boto3 (ec2, ecs)
create_cluster
run_task
register_container_instance
register_task_definition
run_instances
I managed to run an instance with run_instances, it works perfectly well but I want to run an instance in my cluster. Here is my code:
def run_instances():
response = ec2.run_instances(
BlockDeviceMappings=[
{
'DeviceName': '/dev/xvda',
'Ebs': {
'DeleteOnTermination': True,
'VolumeSize': 8,
'VolumeType': 'gp2'
},
},
],
ImageId='ami-06df494fbd695b854',
InstanceType='m3.medium',
MaxCount=1,
MinCount=1,
Monitoring={
'Enabled': False
})
return response
There is a running instance on ec2 console but it doesn't appear in any of the clusters in the ecs console (I tried it with an ecs-optimized ami and with a regular one).
I also tried to follow these steps for getting my system up and running in a cluster without success:
https://github.com/spulec/moto/blob/master/tests/test_ecs/test_ecs_boto3.py
Could you please help me find out what do I miss? Is there ant other setup have to make beside calling these SDK functions?
Thank you!
You will need to run an instance that uses ECS Optimized AMI since those AMIs have ECS agent preinstalled on them otherwise you would need to install ECS agent yourself and bake a custom AMI.
By default, your ECS optimized instance launches into your default cluster, but you can specify alternative cluster name in UserData property of run_instances function
#!/bin/bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
The list of available ECS AMIs is available here
Related
As you can see in the next image, got atm three instances registered
and would like to have none. If I click in one of them, for instances, this is the only thing I get
so there's not anywhere a button to remove it.
The right term to be applied here is deregister. So, one wants to deregister the on-premises instances.
Using AWS-CLI you can check the current instances that are registered using the following command
aws deploy list-on-premises-instances --region eu-west-2
This gives something like this with the names of the current registered instances
{
"instanceNames": [
"Amazon_Linux_1",
"Amazon_Linux_2",
"codedeploy"
]
}
To do unregister the instance named Amazon_Linux_1 run
aws deploy deregister-on-premises-instance --instance-name Amazon_Linux_1 --region eu-west-2
Note: if you try to run it without the region, you'll get
You must specify a region. You can also configure your region by
running "aws configure".
I am creating a ECS cluster (EC2 launch type) using the ecs-cli. I want to run a script to modify vm.max_map_count setting in /etc/sysctl.conf once the EC2 instance is created. At the moment, I am doing it manually by ssh'ing into the instance and running the script as sudo.
Is it possible to run automation script on the EC2 instance created as part of cluster creation? Any reference/documentation will be really helpful.
Thanks
Since you've tagged your question with amazon-cloudformation I assume that you are defining your ECS container instances using CFN.
If so, you can use UserData in your AWS::EC2::Instance to execute commands when the instances are launched:
Running commands on your Linux instance at launch
You are probably already using it to specify cluster name for the ECS agents running on your instances. So probably you already have something similar in your UserData;
echo ECS_CLUSTER=${ClusterName} >> /etc/ecs/ecs.config
echo ECS_BACKEND_HOST= >> /etc/ecs/ecs.config
You can extend the UserData with extra commands that would modify /etc/sysctl.conf.
There are some other possibilities, such as using SSM State Manager to perform actions when your instances launch.
I'm using AWS Batch and my Docker image is hosted on private Nexus repo. I'm trying to create the Job Definition but i can't find anywere how to specify the Repo Credentials like we did with a Task Definition in ECS.
I tried to manually specify it in the Json like that :
{
"command": ["aws", "s3", "ls"],
"image": "nexus-docker-repo.xxxxx.xxx/my-image",
"memory": 1024,
"vcpus": 1,
"repositoryCredentials": {
"credentialsParameter": "ARN_OF_CREDENTIALS"
},
"jobRoleArn" : "ARN_OF_THE_JOB"
}
But when i apply the changes the parameter credentialsParameter was removed . I think that it's not supported.
So how to pull an image from a private repo with AWS Batch ? Is it possible ?
Thank you.
I do not see the option repositoryCredentials either in the batch job definition.
A secure option could be
Generate the config.json for docker login
Place that file in s3
Generate an IAM role that has access to that file.
Create a compute environment with a
Launch Template and user data to download the config.json
Run the jobs with that compute environment.
Ok i was able to do it by modifying the file /etc/ecs/ecs.config
If the file is not there you have to create it.
Then I had to add these 2 lines in that file :
ECS_ENGINE_AUTH_TYPE=docker
ECS_ENGINE_AUTH_DATA={"https://index.docker.io/v1/":{"username":"admin","password":"admin","email":"admin#example.com "}}
Then i had to restart the ECS agent :
sudo systemctl restart ecs ## for the Amazon ECS-optimized Amazon Linux 2 AMI
Or
sudo stop ecs && sudo start ecs ## for For the Amazon ECS-optimized Amazon Linux AMI
I launched an ECS Optimised instance in ap-south-1 region of AWS from ami id: ami-0a8bf4e187339e2c1 using the link https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html but there is no ecs agent present. Even /var/log/ecs directory is not present so I cannot check logs. I have correct cluster name configured in /etc/ecs/ecs.config
If you look at the instances in the EC2 console in AWS, can you see the AMI ID? Is it the AMI ID you expect?
Just to have a point of comparison, I just SSH'd to an ECS-optimized EC2 instances and I can see ecs-agent in a docker ps listing, I can see /var/log/ecs, so my first instinct is that this EC2 instance didn't end up using the AMI you expected it to.
If you want to check logs go to tasks and click on the task in which you wan to see logs and then click on logs yo will see the logs of your container.
I have a cluster "my-cluster"
If I try and add an ECS instance, there are non available. However, if I create a cluster "default", then I have an instance available.
I have deleted the file /var/lib/ecs/data/ecs_agent_data.json as suggested here:
Why can't my ECS service register available EC2 instances with my ELB?
Where can I change my instance/load balancer to allow me to use an EC2 instance in "my-cluster" rather than having to use the "default" cluster?
Per the ECS Agent Configuration docs:
If you are manually starting the Amazon ECS container agent (for non-Amazon ECS-optimized AMIs), you can use these environment variables in the docker run command that you use to start the agent with the syntax --env=VARIABLE_NAME=VARIABLE_VALUE. For sensitive information, such as authentication credentials for private repositories, you should store your agent environment variables in a file and pass them all at once with the --env-file path_to_env_file option.
One of the environment variables in the list is ECS_CLUSTER. So start the agent like this:
docker run -e ECS_CLUSTER=my-cluster ...
If you're using the ECS-optimized AMI you can use an alternative approach as well.