Meteor on AWS using Mup - SSL with ELB - amazon-web-services

I'm migrating my Meteor app to AWS, want to use ACM issued SSL cert attached to ELB.
My current setup is:
ELB with ACM SSL cert(verified that load balancing and HTTPS is working on simple HTTP server inside EC ubuntu machine)
Meteor up is deployed on EC2 machine using Mup (Please see my mup.js which works well with SSL cert physically available from file system)
I want to stop using reverse proxy from mup.js config completely and let ELB run all SSL stuff. Problem is that ELB is not able to communicate with Meteor up,
have tried different ROOT_URLs but none are working:
EC2 Elastic IP with HTTP and HTTPS
(i.e. ROOT_URL: 'https://my-ec2-elastic-ip.com', ROOT_URL: 'http://my-ec2-elastic-ip.com')
ELB domain name with HTTP and HTTPS
What should I put for ROOT_URL and is it game changer in accepting requests? i.e. if I have wrong ROOT_URL, will Meteor still be able to accept incoming requests?
Mup version: 1.4.3
Meteor version: 1.6.1
Mup config
module.exports = {
servers: {
one: {
host: 'ec2-111111.compute-1.amazonaws.com',
username: 'ubuntu',
pem: 'path to pem'
}
},
meteor: {
name: 'my-app',
path: 'path',
servers: {
one: {}
},
buildOptions: {
serverOnly: true,
},
env: {
ROOT_URL: 'https://ec2-111111.compute-1.amazonaws.com',
MONGO_URL: 'mongo url',
},
dockerImage: 'abernix/meteord:node-8.9.1-base',
deployCheckWaitTime: 30,
},
proxy: {
domains: 'ec2-111111.compute-1.amazonaws.com,www.ec2-111111.compute-1.amazonaws.com',
ssl: {
crt: './cert.pem',
key: './key.pem'
}
}
};

Resolved, first and general issue was that I was using classic ELB, which doesn't support WebSockets and was preventing DDP connection. Newer Application Load Balancer which comes with WebSocket and Sticky Sessions helped. More on the diff here: https://aws.amazon.com/elasticloadbalancing/details/#details
Another issue more specific to my use case was having no endpoint for ELB health check, I was hiding/securing everything behind basic_auth, health check was getting 403 unauthorized failing and not registering EC2 instance in ELB. Make sure you have endpoint for health check that returns 200 OK, and also revisit your security groups - check out inbound rules and make sure ELB has access to corresponding ports to EC2 instance(80, 443 etc.).

Related

Strapi AWS Elastic Beanstalk - Deploy success but cannot access URL

I'm trying to deploy Strapi 4 to AWS elastic beanstalk.
After deploying, the app is not accessible by the URL (e.g http://lrd-api.ap-southeast-1.elasticbeanstalk.com).
The instance is running, and docker logs show a successful deployment. The Strapi app is running and connected to the RDS database, but I am unable to access it through the URL.
Are there any additional steps I'm unaware of?
My server.ts file (config/server.ts for Strapi)
import cronTasks from './functions/cron_tasks';
export default ({ env }) => ({
host: env('HOST', '0.0.0.0'),
port: env.int('PORT', 1337),
app: {
keys: env.array('APP_KEYS'),
},
webhooks: {
populateRelations: env.bool('WEBHOOKS_POPULATE_RELATIONS', false),
},
cron: {
enabled: true,
tasks: cronTasks,
},
});
I have tried changing the EB to a load balanced instance to listen to both ports 80 and 433.
For anyone facing this same issue, this was the problem:
Elastic beanstalk load balance was listening to port 80 for the instance port 80
The app itself was on port 1337
To make it work, what I did was:
Change the port mapping in my docker-compose.yml
Instead of port 1337:1337, I used 80:1337

aws nlb socket.io ssl via copilot

Context
I'm trying to create a load balancer with AWS Copilot that runs socket.io instances.
Locally and in an HTTP environment, I can successfully connect to the socket instance via the load balancer (ws://{id}.elb.{region}.amazonaws.com:8080/socket.io/?EIO=4&transport=websocket).
Problem
When I'm in the HTTPS context and try to connect, it fails and I get the error message Error: Websocket error with no real error message or status code (wss://{id}.elb.{region}.amazonaws.com:8080/socket.io/?EIO=4&transport=websocket).
With ws:// everything works and in wss:// it does not work.
Code Snippets
AWS Copilot manifest.yml
name: websocket-service
type: Load Balanced Web Service
http:
path: '/'
healthcheck: '/health'
nlb:
port: 8080/tcp
Server
import {Server} from 'https'; // I use HTTPS here
// ...
this._io = socketIo(server);
// ...
Client
// ...
const client = socketClient(url, {
transports: ['websocket']
});
// ...
AWS Console
EC2 -> Load Balancing -> Load Balancers -> {target} -> Listeners

AWS Global Accelerator in front of ALB managed with EKS alb ingress health checks fail

got an EKS cluster with alb ingress controller and external DNS connected to route53, now some clients want static IPs or IP range for connecting to our servers and whitelisting these IPs in their firewall.
Tried the new AWS Global Accelerator, followed this tutorial https://docs.aws.amazon.com/global-accelerator/latest/dg/getting-started.html but it fails with :
Listeners in this accelerator have an unhealthy status. To make sure that Global Accelerator can run health checks successfully, ensure that a service is responding on the protocol and port that you specified in the health check configuration. Learn more
With further reading understood that the healthchecks will be the same configured at the ALB, also that it might fail because of the route53 Healthchecks ips are not whitelisted but all the inbound traffic is open in ports 80 and 443, so not quite sure how to further debug this or if there is any other solution for getting an ip range or static ip for the ALB.
You need to add a healthcheck rule like this one to the ingress controller:
- http:
paths:
- path: /global-accelerator-healthcheck
backend:
serviceName: global-accelerator-healthcheck
servicePort: use-annotation
Then an annotation:
alb.ingress.kubernetes.io/actions.global-accelerator-healthcheck: '{"Type": "fixed-response", "FixedResponseConfig": {"ContentType": "text/plain", "StatusCode": "200", "MessageBody": "healthy" }}'
Then configure the global accelerator to the health checks to that endpoint
When it comes to AWS ALB Ingress controller, always try to think of
it as you are working with AWS ALB, and its Target Groups.
You can even identify the ALB and its target groups by logging in to AWS console UI.
To answer your question try adding following details to your ingress,
code:
annotations:
alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
alb.ingress.kubernetes.io/healthcheck-port: "8161"
alb.ingress.kubernetes.io/healthcheck-path: /admin
alb.ingress.kubernetes.io/success-codes: '401'
alb.ingress.kubernetes.io/backend-protocol: HTTP`
Note: If you have different health check settings for different services, remove this block from K8s "Ingress" and add blocks per K8s "Service".
If more information required, please refer to: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/ingress/annotations/

How to fix 'Health checks failed with these codes: ' on elasticbeanstalk instace?

My application has Health Status of Severe because of Target.ResponseCodeMismatch error.
I've tried following Redirection is not configured on the backend in this aws instruction . And I've changed my port to '443' and my protocol to 'HTTPS' on eb config and redeployed. It changes the Health Status to Ok but when I access my url I get the page 'Index of' only
Here is what eb status --verbose returns:
Description: Health checks failed with these codes: [301]
Reason: Target.ResponseCodeMismatch
And this is from eb config:
AWSEBV2LoadBalancerListener.aws:elbv2:listener:default:
DefaultProcess: default
ListenerEnabled: 'true'
Protocol: HTTP
Rules: null
SSLCertificateArns: null
SSLPolicy: null
AWSEBV2LoadBalancerListener443.aws:elbv2:listener:443:
DefaultProcess: default
ListenerEnabled: 'true'
Protocol: HTTPS
Rules: null
SSLCertificateArns: arn:aws:acm:us-east-2:XXXX:certificate/XXXXXX
SSLPolicy: ELBSecurityPolicy-XX-XX-XXXX
aws:elasticbeanstalk:environment:process:default:
DeregistrationDelay: '20'
HealthCheckInterval: '15'
HealthCheckPath: /
HealthCheckTimeout: '5'
HealthyThresholdCount: '3'
MatcherHTTPCode: null
Port: '443'
Protocol: HTTPS
For someone that may come across this as I did, I found the solution to be setting up the Health Check endpoint of the ELB target group to an actual URL on my website that returned an HTTP 200 code.
On the EC2 dashboard, under Load Balancing -> Target Groups, go to the tab Health Checks and edit the path to a path in your site that returns an 200 code.
I had a similar problem. In my case the fix was to change the health check's "Success codes" setting from 200 to 200,301.
Typically your app is going to get deploy exposing it's native port. In the case of java this is usually 8080, with node it's 3000. Then AWS will, as part of EB, will still a proxy of either apache or nginx in front of your app exposing port 80. It's ELB that exposes port 443 to the outside.
So you probably want to change the port and protocol to 80 / HTTP
This seem to have helped me, in
"health check settings" > "Advanced health check settings"
Under: "Port" from "Traffic port" to "Override" and used "80"
Under: "Success codes"
One Instance of EB
from "200" to "200,300,301,302,303,304,305,306,307,308" (In Elastic Beanstalk Health was down because of "3xx Responses")
Second Instance of EB
from "200" to "200,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,422,425,426,428,429,431,451" (In Elastic Beanstalk Health was down because of "4xx Responses")
I had the following problem:
Description: Health checks failed with these codes: [400]
Reason: Target.ResponseCodeMismatch
What fixed for me was entering the Success codes value to 400.
EC2 (service) > Load Balancing (from left side menu) > Target Groups (click on Name) > Health Check (click on tab) > Edit > Advanced health check settings > Success codes = 400
For me, this was as simple as making sure the route that the health check is pointed to, like '/' for my example, returns a 200 result.
const router = require("express").Router();
// Health Check
router.get('/', (req, res)=>{
res.sendStatus(200);
})
module.exports = router;
In DotNet I added this controller that will allow anonymous access and then in EC2 > Load Balancing > Target Group set the health check path to /health
public class HealthController : Controller
{
[AllowAnonymous]
public IActionResult Index()
{
return Ok($"HEALTH: OK - {DateTime.Now}");
}
}
Under Advanced Settings, include the HTTP code 302 to the expected results

Kubernetes nginx ingress proxy pass to websocket

We are running rails application with unicorn and websocket.
We are using AWS ELB as ingress
SSL terminates on ELB and forwards traffic to application.
Nginx ingress routes traffic to web app running unicorn/puma on port 8080.
App works but our websocket responds with 200 instead of 101. We have enabled CORS and used required annotations in ingress.
This are annotations used for the ingress controller service
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
service.beta.kubernetes.io/aws-load-balancer-ssl-cert::arn:aws:iam::xxx:server-certificate/staging
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
When we use aws loadbalancer protocol as tcp and load balancer ports as 443 it fails on infinite redirect loop.
Following are the annotations used in the ingress:
nginx.ingress.kubernetes.io/service-upstream: true
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type"
nginx.ingress.kubernetes.io/cors-allow-origin: "*"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
ingress.kubernetes.io/force-ssl-redirect: "true"
Our sample nginx configuration we used earlier without ingress is here
How to get websockets working with nginx ingress controller with AWS ELB ?
Is it possible to try without CORS?
Part of the handshake is the client must send at least these headers:
Sec-WebSocket-Key
Sec-WebSocket-Version
And maybe something else. Look at https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#The_WebSocket_Handshake