I have have created two clusters ECS in the same subnetwork, one for jenkins master and other for jenkins slave(empty cluster). I have installed Amazon ECS plugin on jenkins master but I am not able to configure jenkins slave node. I created both clusters using ecs-cli up command and following are my settings for Amazon ECS plugin similar to my cluster. After running this job a task definition is created in ECS but the service and task definition are not created in the cluster.
Name: ecs-jenkins-slave
Amazon ECS Credential: aws_credentials
ECS Region Name: cluster_region
ECS Clutser: cluster_cluster
ECS Agent Template
Label: ecs-jenkins-slave
Docker Image: jenkinsci/jnlp-slave
Subnet: cluster_subnet
security_group: cluster_sg
(rest of the fields are default)
I created a test job to verify configuration and under Restrict where this project can be run of my test job, I am getting Label ecs-jenkins-slave is serviced by no nodes and 1 cloud. Permissions or other restrictions provided by plugins may prevent this job from running on those nodes message. When I am running the job, it is going in pending state with message '(pending—‘Jenkins’ doesn’t have label ‘ecs-jenkins-slave’) '
Related
I was trying to deploy webserver in EC2 instance, this is the error i'm getting in deployment phase
I'm uploading code to s3 and implementing via CodeDeploy to EC2.
My Ec2 CodeDeploy agent service running.
even though I provided role for EC2 - Awscodedeployfullacces,ec2fullaccess,S3fullacess. And codedeploy-access for my deployment grp, I was implementing in a (amazon) linux machine.
Deployment events log in aws:
this is my yaml code:
and the corresponding codes in scripts/ folder.
Event Log failed:
Is there easy way to run an ECS Task attached or to follow the logs only while the container is Running (ie. Detach after displaying all of the logs associated)?
Using the AWS CLI (1.17.0) and ecs-cli (1.21.0), I have gotten decently close with the following two commands:
aws ecs run-task --cluster "mycluster" --task-definition testhelloworldjob --launch-type FARGATE --network-configuration etc.etc.etc.
ecs-cli logs --task-id {TASK_ID_HERE_FROM_OUTPUT_OF_PREVIOUS_COMMAND} --follow
I am currently have two issues with the above approach:
There is a race condition being that the logs are not available when the task is in a pre "running" state. Instead of ecs-cli logs waiting for the logs to exist, there is an error immediately thrown.
Even after waiting for the task to be in a running state, and issuing the ecs-cli logs the command refuses to detach even AFTER the task is finished and in a Post Running status.
For the first issue I could poll until there is a post activating/pending status, prior to calling logs. For the second issue I could draft some type of threaded call that would poll to stop the following of a log after the container in question is no longer running.... But there has to be an easier way?
To clarify I am coming from numerous other container orchestration tools/technologies that seemingly supported this very seamlessly. Here are some examples of tools and their associated commands that would yield me my intended results:
Docker CLI:
docker run hello-world
Docker-Compose Yaml:
docker-compose up
K8 Kubectl Yaml:
kubectl apply -f ./hello-k8.yaml && kubectl logs --follow hello-world
I think ecs-cli is the best option available at the moment.
Apart from that, you can change the logs driver of the AWS ECS task to syslog and then watch the logs file from the terminal after doing SSH into the EC2 container instance in which it is running.
Another thing you can do is SSH into the EC2 container instance in which it was running before and then run the container of that AWS ECS task by yourself in it using docker run and once the testing is done, you can stop and remove that container and then get that task started via AWS ECS.
Note: You can use AWS SSM Session Manager in order to avoid using EC2 key pair and adding an inbound rule for SSH.
I'm trying to deploy Kubernetes application in AWS EKS through Jenkins.
I visited few of blogs, they mentioned Jenkins X. But JenkinsX need to be configured separately. But as per instruction, we need to use our existing Jenkins for K8S app deployment.
Note : AWS EKS and Jenkins are Separate machine(We using our existing Jenkins). I may need to create New EKS environment based on requirement.
Please suggest if any AWS EKS plugin for Jenkins which can be used for deployment.
Else
Is there any way to create custom Bash script(automation script) for deploying K8S application in AWS EKS?
My Research here is : Actually AWS is providing api/sdk support for only Creating/Managing Clusters but not deploying the application in k8s environment(using kubectl).
Probably creating cluster we can do it through SDK. but How to deploy k8s application on remotely(because Jenkins is running in another machine).
Why not configuring kubectl for jenkins and deploy apps using kubectl apply deployment.yaml command?
Once you have kubectl config you can save it as secret text. I had an assignment for the interview and here is an example of such deployment:
https://github.com/mtuktarov/hello
It uses shared lib:
https://github.com/mtuktarov/hello-jenkins-lib
Finally I'm done this exercise by creating Bash automation script, following these steps:
Created Docker image with application binary.
Created EKS Cluster using eksctl create cluster <PARAM>, which creates EKS Control Plane and Worker nodes.
Created Kubernetes Deployment File using Docker image and Deployed using kubectl apply <PARAM> commandline.
Exposed the application using kubectl expose <PARAM> cli.
Latest Update From AWS EKS Service:
AWS recently announced AWS EKS Worker node creation support using AWS SDK. So now Creating EKS environment can be done using SDK itself.
===================
Update:
Now AWS has Supported Creating worker node thorugh UI and AWS SDK.
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/EKS.html#createNodegroup-property
I have recently changed AMI on which my ECS EC2 instances are running from Amazon Linux to Amazon Linux 2 (in both cases I am using ECS optimized images). I am deploying my instances using cloudformation and having a real headache as those new instances sometimes are being run successfully and sometimes not (same stack, no updates, same code).
On the failed instances I see that there is an issue with ECS service itself after executing ecs-logs-collector.sh I see in ecs file log "warning: The Amazon ECS Container Agent is not running". Also directory "/var/log/ecs" doesn't even exist!.
I have correct IAM role attached to an instance.
Also as mentioned, it is the same code being run, and on 75% of attempts it fails with ECS service, I have no more ideas, where else to look for some issues/logs/errors.
AMI: ami-0650e7d86452db33b (eu-central-1)
Solved. If someone will fall into this issue adding this to my userdata helped:
cp /usr/lib/systemd/system/ecs.service /etc/systemd/system/ecs.service
sed -i '/After=cloud-final.service/d' /etc/systemd/system/ecs.service
systemctl daemon-reload
I'm using Jenkins Pipeline and Packer to create AMI inside an AWS Account.
The Jenkins uses Kubernetes cluster as slave (using a cloud plugin that allows me to parameter docker pods template),
I have a pipeline that pull git project with the packer template in it and run packer validate command which is a success. Than, it runs packer build and i get the following error:
[1;31mBuild 'Amazon Linux 2 Classic' errored: No valid credential sources found for AWS Builder. Please see https://www.packer.io/docs/builders/amazon.html#specifying-amazon-credentials for more information on providing credentials for the AWS Builder.[0m
I also use Kube2iam to provide roles on my slave containers.
In my packer template, i don't define any aws credentials since I don't want to use it but role. Do you know if I have something to do inside the packer template to indicate the role to use ?
Best Regards,
Tony.
From what I understand, you are running Jenkins inside a Kubernetes cluster running on AWS EC2 instances? If so, the Jenkins agents running the build should be able to read available roles from the metadata of the instance they're running on.
In this case, the process would be to assign the desire IAM role to instances and Kubernetes should be able to handle that.