Trying to create a service in ECS and unbelievably looks like it is not possible to specify any environmental variables...
Is it possible to do that without updating my task definition and recreate the service? Or a task override option?
This looks a bit cumbersome
The environmental variables are provided through task definition. Thus you have to update the definition to add/change the variables.
You don't have to re-create the service from scratch. You can update your service to use the new version of your task definition. For updating existing service you can use update-service AWS CLI call. The cli also provides --force-new-deployment if you want to force the deployment (but changing task should be enough and forcing would not be required).
You can't define environment variables at service creation time as explained by the other answer but you can define tags: thus one workaround is to
Create a service with a set of tags and propagateTags: SERVICE
On container startup read the cluster and task ARNs from the task metadata endpoint
Read service tags with the DescribeTasks API (note the include: ['TAGS'] parameter)
Configure the environment using the tag values
Related
CONTEXT:
We have a platform where users can create their own projects - multiple projects per user. We need to provide them with a browser-based IDE to edit those projects.
We decided to go with coder-server. For this we need to configure an auto-scalable cluster on AWS. When the user clicks "Edit Project" we will bring up a new container each time.
https://hub.docker.com/r/codercom/code-server
QUESTION:
How to pass parameters from the url query (my-site.com/edit?project=1234) into a startup script to pre-configure the workspace in a docker container when it starts?
Let's say the stack is AWS + ECS + Fargate. We could use kubernetes instead of ECS if it helps.
I don't have any experience in cluster configuration. Will appreciate any help or at least a direction where to dig further.
The above can be achieved using multiple ways in AWS ECS. The basic requirements for such systems are to launch and terminate containers on the fly while persisting the changes in the files. (I will focus on launching the containers)
Using AWS SDK's:
The task can be easily achieved using AWS SDKs, Using a base task definition. AWS SDK allows starting tasks with overrides on the base task definition.
E.G. If task definition has a memory of 2GB then the SDK can override the memory to parameterised value while launching a task from task def.
Refer to the boto3 (AWS SDK for Python) docs.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.run_task
Overall Solution
Now that we know how to run custom tasks with python SDK (on demand). The overall flow for your application is your API calling AWS lambda function whit parameters to spin up and wait to keep checking task status and update and rout traffic to it once the status is healthy.
API calls AWS lambda functions with parameters
Lambda function using AWS SDK create a new task with overrides from base task definition. (assuming the base task definition already exists)
Keep checking the status of the new task in the same function call and set a flag in your database for your front end to be able to react to it.
Once the status is healthy you can add a rule in the application load balancer using AWS SDK to route traffic to the IP without exposing the IP address to the end client (AWS application load balancer can get expensive, I'll advise using Nginx or HAProxy on ec2 to manage dynamic routing)
Note:
Ensure your Image is lightweight, and the startup times are less than 15 mins as lambda cannot execute beyond that. If that's the case create a microservice for launching ad-hoc containers and hosting them on EC2
Using Terraform:
If you looking for infrastructure provisioning terraform is the way to go. It has a learning curve so recommend it as a secondary option.
Terraform is popular for parametrising using variables and it can be plugged in easily as a backend for an API. The flow of your application still remains the same from step 1, but instead of AWS Lambda API will be calling your ad-hoc container microservice, which in turn calls terraform script and passing variables to it.
Refer to the Terrafrom docs for AWS
https://registry.terraform.io/providers/hashicorp/aws/latest
Currently I have several redis clusters for different env. In my code, I will write data to redis inside my lambda function, if I deploy this lambda to my aws account, how can it update the corresponding redis cluster? Since every environment has its own redis cluster.
You can have a config file with Redis cluster name and their hostnames and in code, you can pick different clusters based on the env provided.
If you are using roles in the AWS account, for each environment then you should also do STS on requirred role
Your resource files are just files. They will be loaded by your application following different strategies, depend on your application's framework and how it has been configured.
Some application, at build-time, apply the correct configuration, by passing a flag to the build for example. --uat or --prod. If your application is one of those kind, you can just build the correct version and push it to AWS. They will connect to the correct Redis, given that you put the redis configuration into the ENV files
The other option is to use Environment Variable
I have an ECS service that will be running many instances of one task. All these instances use the exact same task definition but all need to be provided a specific environment variable INDEX that is unique to that instance. I would like the service to monitor each instance and restart it with the same environment variable value if it fails (I.E. If a task with INDEX=555 fails, I would like the service to spin up a new task with INDEX=555 to replace it). Currently I only see the option to set environment variables in the task definition itself, which would require me to create many versions of the exact same task (and the corresponding service as well) with the only difference being the environment variable (this seems wasteful and would clutter the task definition list and service list in that cluster).
In short I want to configure my ECS service such that I can provide a list of values for a specific environment variable along with one task definition and have it create a 1:1 map of a task instance to environment variable. Is it possible to do so? If so, how can I accomplish this?
Important Note: This service is running on Fargate and not EC2 Instances
Sadly, you can't do this for ECS Service. The ECS services are meant to run exact copies of same tasks which are disposable, exchangeable and scalable.
However, you can provide overwrites for the varaibles for tasks outside of a service, which you can do using run-task or start-task with --overrides of AWS CLI or equivalent in any AWS SDK.
I have an ECS service and within that ECS service, I want to boot up 3 tasks all from the same task definition. I need each of these tasks to be on a separate EC2 instance, this seems simple enough however I want to pass a different command to each one of the running tasks to specify where their config can be found and some other options via the CLI within my running application.
For example for task 1 I want to pass run-node CONFIG_PATH="/tmp/nodes/node_0 and task 2 run-node CONFIG_PATH="/tmp/nodes/node_1" --bootnode true and task 3 run-node CONFIG_PATH="/tmp/nodes/node_0 --http true"
I'm struggling to see how I can manage individual task instances like this within a single service using Terraform, it seems really easy to manage multiple instances that are all completely equal but I can't find a way to pass custom overrides to each task that are all running off the same task definition.
I am thinking this may be a job for a different dev-ops automation tool but would love to carry on doing it in Terraform if possible.
This is not a limitation of a terraform. This is how ECS service works - runs exact copies of same task definition. Thus, you can't customize individual tasks in an ECS service as all these tasks are meant to be identical, interchangeable and disposable.
To provide overwrites you have to run the tasks outside of a service, which you can do using run-task or start-task with --overrides of AWS CLI or equivalent in any AWS SDK. Sadly there is no equivalent for that in terraform, except running local-exec with AWS CLI.
I’m new to AWS ECS deployment. This is my first time.
I have updated the .env in my container definition on my AWS account.
But when I run docker exec e718a29eb0e3 env in my container I still seeing the latest value updated.
I even tried
node#db39b382163a:/api$ pm2 restart all
I still not seeing it updated.
Do I need to restart something else ?
The native CodePipeline -> ECS integration will only update the container definitions' image attribute so you cannot use it to manage environment variables. you have a couple of other options:
You can use a Lambda function instead to drive your deployment and do something similar to the above to edit both the image and environment attributes.
If you're using CloudFormation to manage your task definition and service, you can use these templates to manage those fields instead of the native integration.