I have currently deployed a web server via aws ecs cli compose service up cli command, and I further registered a domain in route 53 service, register a certificate through Amazon Certificate Manager. Making use of the ALB (application load balancer), I am able to perform dynamic port mapping and https for my web application, but here is the problem.
Using docker compose as the blueprint for my web application, which consists of 3 containers, frontend, loopback and database (mongo), my frontend container's dynamic port mapping and https are up and running fine
However the problem comes to the loopback container, there are chances frontend needs to fetch something via loopback API server (which makes use of 3002 port), but the loopback container does not is not have configured in https which causes the error below when calling the API.
Through ecs cli compose service up command, I can configure the target group to allow elb to forward the request to frontend container (using --target-group-arn, --container-name and --container-port attributes to specify the frontend container with the specific target group), but this command seems unable to map the 2nd target group to my loopback container. Reading https://docs.aws.amazon.com/AmazonECS/latest/developerguide/register-multiple-targetgroups.html which seems to allow the possibility of multiple target groups for a service, but I cannot figure out how to use create service command to link up my docker containers without using user ecs cli compose service up command.
Is there a way to
Use ecs cli compose service up command to register multiple target groups on my containers?
Apply https also on my loopback URL (which domain name is myDomain.com:3002)?
======================================================
Follow-up tasks
Created 2 target groups
Configured rules and listeners
Knowing ecs cli service up cannot register multiple groups, I tried to do via console, still only 1 container can be registered
Thanks and appreciate for all helps
As far your question is a concern, it possible to perform that using AWS console, but ecs cli currently does not support multiple target group at the moment.
you can check this ecs-cli compose service up with a load balancer also consider this amazon-ecs-cli-register-service.
The second error occurred when the frontend application tries to use load mix HTTP and https resources. you can look into the error, there can be static or API calls that are based on HTTP, convert all these calls to HTTPS then it should work fine. you can check error seems like static file loading from http site.
Once you applied HTTPS it should point to https://example.com or https://api.example.com, the port is not required with HTTPs call if its bind with standard HTTPS port.
Update:
ALB target group route traffic base on the target group, so the target group contain the desired container. adding screenshot to make it more clear.
ecs cli compose service up contains a parameter --target-groups allowing you to add multiple target groups at once.
ecs-cli compose --file "../../src/docker-compose.yml" `
--ecs-params "../../src/ecs-params.yml" `
--project-name xxxxx service up `
--target-groups "targetGroupArn=arn:aws:elasticloadbalancing:eu-west-3:xxxxx:targetgroup/xxxx_tg1,containerPort=80,containerName=webapi" `
--target-groups "targetGroupArn=arn:aws:elasticloadbalancing:eu-west-3:xxxxx:targetgroup/xxxx_tg2,containerPort=81,containerName=webapi2" `
--cluster-config myconfig`
--ecs-profile myprofile
ecs-cli compose service up documentation
Related
I have a setup with an ALB and a target group created by ECS, I'm using Fargate and created a build pipeline by following this article. My app is built with NET core, I have an Angular frontend. Got all this working, I'm able to deploy my code changes, but I'm a bit stuck with the following issue.
I'm using Cognito for authentication and a custom domain that I set for the hosted UI. It seems that, from the browser, when I try to hit an endpoint that is secured, I get a 504 Gateway error, which somehow is not doing the redirect to Cognito in the browser. All this works fine when I run the application on localhost.
When I looked at the logs, I noticed the following exception:
System.InvalidOperationException: IDX20803: Unable to obtain configuration from: 'https://cognito-idp.<region>.amazonaws.com/<region_and_a_code>/.well-known/openid-configuration'
Apparently, it can't establish a connection to Cognito. My containers are using only port 80, my target group instances are also using port 80, ALB uses HTTPS on 443 which directs the traffic to the target group, and for ALB port 80 I just redirect to 443.
I tried a few different things, like setting the authority value instead of the metadata address, tried using a BackChannelHttpHandler to execute the HTTPS call, tried updating the port mappings to allow communication on 443, but somehow it seems that it gets overridden by the task definition that I have created when I set up the build pipeline. The network mode in my task definition is now awsvpc, and if I try to set it to host, it will complain that I can't use it with Fargate.
What do I need to do to allow the HTTPS request from my Docker container instances to reach Cognito?
You are trying to set this up in a public ALB. This setup using a private NLB will work, might work with a private ALB as well. You can then set up vmc private links to get at the service's you need access too.
I have dockersied our webapp into two different docker files, such as:
1.Frontend
2.Backend
Both docker apps have their own .env files , i have to defined deployed server Ip address to connect them.
In frontend-env : configure backend ip
but deploying in ECS service, each container will be different ip ,how to solve this issue scale out and still connect the service each other.
so far :
Create spreate ecs cluster for both frontend and backend, with ALB.
Give the ALB in env files to connect them or hit the api.
any other solutions for this deployment?
You should be using Service Discovery to achieve this. You can read the announcement blog here. In a nutshell the way it works is that if you have two ECS services frontend and backend you want to expose frontend with an ALB for public access but enabling service discovery you will be able to have all tasks that belong to frontend to be able to connect to the tasks that belong to backend by calling backend.<domain> (where is the SD namespace you defined). When you do so ECS Service Discovery will resolve backend.<domain> to the private IP addresses of the tasks in backend thus eliminating the need for a load balancer in front of it.
If you want a practical example of how this works you can investigate this basic demo app:
When developing locally, I run docker-compose, where I have two services Service1 and Service2. Service2 depends on Service1. When I deploy them to ECS, I create them within one task definition and provide JSON array of container definitions to spin them up.
When I run them locally, within docker-compose, from Service2 I can call http://Service1:8080/v1/graphql (since they're in docker-compose together I can call it by the service name) ... however, when I deploy to ECS and I make that same API call, I get a 404.
Based on this: Docker links with awsvpc network mode I've also tried http://localhost:8080/v1/graphql ... I'd appreciate any help!
I'd try service discovery as mentioned here:
Amazon ECS now includes integrated service discovery. This makes it possible for an ECS service to automatically register itself with a predictable and friendly DNS name in Amazon Route 53. As your services scale up or down in response to load or container health, the Route 53 hosted zone is kept up to date, allowing other services to lookup where they need to make connections based on the state of each service.
See an example here.
I've gotten a bit lost in the number of services in AWS and I'm having a difficult time finding the answer to what I think is probably a very simple question.
I have a Docker image that's serving a RestAPI over HTTP on port 80. I am currently hosting this on AWS with ECS. It's using Faregate but I could make an EC2 cluster if need be.
The problems are:
1) I currently get a new IP address whenever I run my task, I want a consistent address to access it from. Doesn't need to be a static IP, it could be routed from DNS.
2) It's not using my hostname which I would like to have api.myhostname.com go to the Docker image while www.myhostname.com currently already goes to my Cloudfront CDN serving the web application.
3) There's no SSL and I would need this to be encrypted.
Which services should I be using to make this happen? I looked into API Gateways and didn't find a way to use an ECS task as a backend. I looked into ELB for ECS but load balancers didn't seem to provide a way to make static IPs out of the Docker images.
Thanks.
I'll suggest a service for each of you requirements:
you want to run a Docker container: ECS using FARGATE is the right solution
you want a consistent address: use the Service Load Balancing which is integrated into ECS. [1] You can also achieve consistent addressing using Service Discovery if the price for running a load balancer is too high in your scenario. [2]
you want SSL: AWS Elastic Load Balancing integrates with AWS Certificate Manager (ACM) which allows you to create HTTPS listeners. [3]
you want to use your hostname: use AWS Route53 and an Application Load Balancer. The load balancer receives a hostname by aws automatically and you can then point your custom dns at that entry. [4]
So my advice is:
Create an ECS service which starts your docker container as FARGATE task.
Create a certificate for your HTTPS listener in AWS Certificate Manager. ACM manages your certificates and sends you an email if they are expiring soon. [5]
Use Service Load Balancing with an Application Load Balancer to automatically register any newly created ECS tasks to a target group. Configure the load balancer to listen for incoming traffic on an HTTPS listener and routes it to the target group which has your ECS tasks registered as targets.
References
[1] https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-load-balancing.html
[2] https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-service-discovery.html
[3] https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-create-https-ssl-load-balancer.html
[4] https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/using-domain-names-with-elb.html
[5] https://docs.aws.amazon.com/acm/latest/userguide/acm-overview.html
I used the cloud formation template provided by Docker for AWS setup & prerequisites to set up a docker swarm.
I created a REST service using Tibco BusinessWorks Container Edition and deployed it into the swarm by creating a docker service.
docker service create --name aka-swarm-demo --publish 8087:8085 akamatibco/docker_swarm_demo:part1
The service starts successfully but the CloudWatch logs show the below exception:
I have tried passing the JVM environment variable in the Dockerfile as :
ENV JAVA_OPTS= "-Dbw.rest.docApi.port=7778"
but it doesn't help.
The interesting fact is at the end the log says:
com.tibco.thor.frwk.Application - TIBCO-THOR-FRWK-300006: Started BW Application [SFDemo:1.0]
So I tried to access the application using CURL -
curl -X GET --header 'Accept: application/json' 'URL of AWS load balancer : port which I exposed while creating the service/resource URI'
But I am getting the below message:
The REST service works fine when I do docker run.
I have checked the Security Groups of the manager and load-balancer. The load-balancer has inbound open to all traffic and for the manager I opened HTTP connections.
I am not able to figure out if anything I have missed. Can anyone please help ?
As mentioned in Deploy services to swarm, if you read along, you will find the following:
PUBLISH A SERVICE’S PORTS DIRECTLY ON THE SWARM NODE
Using the routing mesh may not be the right choice for your application if you need to make routing decisions based on application state or you need total control of the process for routing requests to your service’s tasks. To publish a service’s port directly on the node where it is running, use the mode=host option to the --publish flag.
Note: If you publish a service’s ports directly on the swarm node using mode=host and also set published= this creates an implicit limitation that you can only run one task for that service on a given swarm node. In addition, if you use mode=host and you do not use the --mode=global flag on docker service create, it will be difficult to know which nodes are running the service in order to route work to them.
Publishing ports for services works different than for regular containers. The problem was; the image does not expose the port after running service create --publish and hence the swarm routing layer cannot reach the REST service. To resolve this use mode = host.
So I used the below command to create a service:
docker service create --name tuesday --publish mode=host,target=8085,published=8087 akamatibco/docker_swarm_demo:part1
Which eventually removed the exception.
Also make sure to configure the firewall settings of your load balancer so as to allow communications through desired protocols in order to access your applications deployed inside the container.
For my case it was HTTP protocol, enabling port 8087 on load balancer which served the purpose.