https timeouts on AWS elastic beanstalk app - amazon-web-services

We're having a bizarre issue on one of our Elastic Beanstalk apps, where every https connection results in a timeout. Verified that the cert itself is fine via openssl s_client, so it looks like an issue with the load balancer - but everything reads fine there too, so I'm a little stumped.
We're not using Route 53 (managing DNS over on namecheap) and our cert was purchased externally (not using AWS cert manager).
My main questions are:
I don't have access to our Namecheap account but I've been assured everything is in line over there - am I on the right track by focusing on the load balancer/aws config as the culprit, or does this sound like it could be some weird DNS misconfiguration?
Are there good reasons to avoid using the AWS cert manager when not managing DNS via route 53? If using an AWS-supplied cert could resolve this, I'm happy to go that route instead. This question is very similar but deals with AWS-supplied certs.

Maybe you are pointing Load Balancer to port 443 + HTTPS instead of 80 + HTTP? Try setting up Classic Load Balancer as shown on the pictures.
Example 1
Example 2

There are two places to enable https on elastic beanstalk:
Place 1: From elastic beanstalk environment
select Services
select Elastic Beanstalk
select the environment within the application
select Configuration
in Load balancer section: select modify
select "Add listener" with the following info:
listener port: 443
listener protocol: HTTPS
Instance port: 80
Instance protocol: HTTP
SSL certificate: Add an ACM certificate created from AWS or another
Place 2: The load balancer
select Services
select EC2
scroll down left panel: select Load balancers
select Action
select Edit Listeners
select "Add listener" with the following info:
listener port: 443
listener protocol: HTTPS
Instance port: 80
Instance protocol: HTTP
SSL certificate: Add an ACM certificate created from AWS or another
select Save

Related

Amazon EC2 - Listen for HTTPS request and Redirect to SpringBoot

I have a frontend React application hosted on Amazon Amplify and a backend SpringBoot application hosted on Amazon EC2.
My domain can only send https request but SpringBoot by default is http. My question is how can my EC2 instance listen to HTTPS request and then redirect to http port in SpringBoot.
I checked other posts and seems like you should not add SSL to your SpringBoot application, but rather to the Load Balancer in front of it. At the end of the today, this is what I want:
POST https: xxx.xxx.xxxx:443/user/signin
---> http: xxx.xxx.xxxx:8080/user/signin
---------------------------------------- update ----------------------------------------
Marcin has provided a top level idea on how to solve this, thank you!
I also attached the step-by-step solution for people like me, please see answers below
Below is the complete steps to take to convert your
http api to https using aws ec2
disclaimer: I only researched for couple hours, some concept might be inaccurate or wrong, but following this guide does gets the job done, correct me for misleading information
(1) springboot:
keep your server port on 8080 and don't change it to https (443)
(2) make sure your EC2 instance has the correct *VPC* and *IPv4 CIDR* set up
go to Instances -> Description -> VPC ID and then click on it
you should now see the list of VPCs, find the one that associated with your instance
In detail -> IPv4 CIDR -> check if it has two or more values in below format:
xxx.xx.0.0/16
xxx.xx.0.0/16
(3) skip this step if you have two IPv4 CIDR set up
select your VPC instance -> click Actions -> EDIT CIDRS -> Add new IPv4 CIDR
make sure two IPv4 CIDR are in different zone
more information on IPv4 CIDR:
https://docs.aws.amazon.com/vpc/latest/userguide/working-with-vpcs.html
(4) now we want to create an application load balancer that listens to https:443 request
select HTTP HTTPS Application Balancer and for each step (as shown in aws)
step 1.
Load Balancer Protocol and Port: https: 443
step 1.
Availability Zones: now is the time to select your VPC and two zones
step 2.
Security Setting choose a certificate from ACM (assume you have one on Route 53)
step 3.
Security Group: make sure to select the same group as your EC2 instance
step 4.
Routing http: 8080, target type: instance
step 5.
Register Targets select your EC2 instance, on port 80, please don't forget!
(5) now load balancer set up, double check security group of your EC2 instance
go to instances -> Description -> Security Group and click on it
for inbound rule, keep port 443, 22, 80, 8080 don't remove 8080
443 is for https, 22 for ssh client, 80 for tomcat
(6) now find the ips to use for the https request
this is not the public ip address of your EC2 instance
your application is behind a load balancer, the ip address should be the network interface IP. each network interface IP associates with a subnet ID that your VPC uses.
so go to Network Interfaces in your EC2 console. select any of your network interface IDs with a subnetId under your EC2's VPC.
click Details -> and scroll down to find the public IP you need
(7) before start next step, make sure you have the following:
a domain hosted in Route 53 (I have one for my frontend UI)
SSL certificated got from aws Certificate Manager
you should have put this SSL to your load balancer in step 3
if don't know what to do, check this stackOverFlow post for answers:
Adding SSL to domain hosted on route 53 AWS
(8) before next step, make sure you understand the following:
If you directly test your https request in postman, you will likely succeed.
However you will fail if using in production, like this:
axios.get("https:xxx.xxx.xxx:443/user/signin");
(failed)net:ERR_CERT_COMMON_NAME_INVALID
This is because whatever static IP you are using, does not match the AName for your SSL certificate. For example, if your domain name is helloworld.com, your backend API request should be https://helloworld.com/user/signin
(9) create a subdomain and config it in Route 53
I'm getting lazy, please see the link below:
https://aws.amazon.com/premiumsupport/knowledge-center/create-subdomain-route-53/
(10) final step!!!!!!
Now you have a working subdomain, let's use it for your network interface IP
For test, you can pick any IP from the list of network interface IPs. Go to Route 53 -> Hosted Zones -> select your subdomain, example: api.helloworld.com -> create record
Record Format
name: api.helloworld.com
type: A
Routing: Simple
Value: <your_network_interface_ip_address_multiple>
(11) sorry not yet ready
please wait for couple of days for DNS record to be updated, if DNS can interpret your subdomain, let's say api.helloworld.com into your network interface IP and since you can already test the correctness of your IP in postman, you should be ready to go!
--------------------------- end of useful information ---------------------------
If you want to keep using only the instance (no load balancer or cloudfront), then you need to get your own domain for it. Then you have to register a valid, public SSL certificate for that using, e.g. letsencyrpt. Once you have that, you can setup nginx on the instance to accept the https connections, and forward to your spring boot as http.
The easier route is to use application load balancer. You still need your own domain, but once you have it, you can easly get free SSL cert from AWS ACM and then deploy it on the balancer. No need to change your instance. So it would be:
Client ---(HTTPS)---> ALB ---(HTTP)---> EC2

Add SSL to a domain serving kubernetes app

I am using kops to deploy my kubernetes cluster. in my cluster, I have a simple Nginx deployment, a service, and an ingress. Its configured with route 53 (not using external DNS, manually creating A record in route 53 pointing to a classic load balancer (generated by kops)).
I can hit the domain www.XXXX.com as well as a subdomain on it but, there is not SSL certificate on it.
I know we can apply SSL on the loadbalancer. So, went in ACM, created a certificate and when I try to apply it, I see 80 and 443 are serving TCP traffic, that's why I cannot add these two ports serving HTTP and HTTPS with certificate (as shown in image)
If I delete the two TCP listeners and add HTTP and HTTPS listners I can, but then my app is not reachable on the domain.
How can I configure ACM on this loadbalancer. Is this even a correct way of adding https for an app deployed on Kubernetes?
I know about https://kubernetes.github.io/ingress-nginx/user-guide/tls/, but if there is a way to do it using above approach I would prefer that.
UPDATE 1:
when I describe my nginx pod I see Ports as follows:
Ports: 80/TCP, 443/TCP
shouldn't that be
Ports: 80/HTTP, 443/HTTPS
?
Turns out I deployed the ingress controller incorrectly.
the documentation clearly says download and update values in the file before applying I missed the updating part.
You have to update CIDR proxy-real-ip-cidr and service.beta.kubernetes.io/aws-load-balancer-ssl-cert
literally that all you need.
make sure you are creating a certificate before applying this YAML file.
and while creating cert add
DOMAIN.com
*.DOMAIN.com
both to make it work.

Gatsby site serving on EC2 with pm2 node with aws classic load balancer needs https

I am running a Gatsby site in development mode as a dev server on EC2 with a loadbalancer pointing from port 80 to 8000. I have setup a cname on my domain dns to point to the load balancer this works fine. However I need to display this page as an iframe in sanity.io as a web preview and it requires https.
I've read through this https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-create-https-ssl-load-balancer.html and most of it is pretty straight forward for the most part.
What I have done so far is created a listener for 443 https on the loadbalancer and added https 443 to the security group. i have succsufully issued a certificate to the subdomain I am using with aws and attached it to the loadbalancer listener.
Gatsby has a article about custom certs for development mode here https://www.gatsbyjs.org/docs/local-https/#custom-key-and-certificate-files What I am looking for is the cert file, the authority file and the key file in order to pass this command below
Where in the aws certificate manager do I find these files. I think that is the last piece I need to get https working, correct me if I am wrong.
thanks ahead of time.
gatsby develop --https --key-file ../relative/path/to/key.key --cert-file ../relative/path/to/cert.crt --ca-file ../relative/path/to/ca.crt
This is the process I used to request my certficate and it says it's issued
https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html
But how do I use it with the custom https command with gatsby?
There is a export option but it says only for private keys. Do I need to create a private key and then I can export these files I need?
Do I even need to run https on gatsby's side. I watched a video using apache and no change was made to the apache server to get https working with the loadbalancer.
Here is a screenshot of my loadbalancer listenr
Here is a image of my security groups
If I run the --https for gatsby develop it breaks my site I can no longer visit it via the loadbalancer or port 8000. So not sure what to do here.
I would suggest not to encrypt the connection between your ELB and the EC2 instances. If your EC2 instances are not publicly reachable, but only through the load balancer instead, it is best practice to terminate the SSL connection on the load balancer. No need to encrypt HTTP requests inside an AWS VPC (i.e. between ELB and target instances).
You can create a load balancer that listens on both the HTTP (80) and HTTPS (443) ports. If you specify that the HTTPS listener sends requests to the instances on port 80, the load balancer terminates the requests and communication from the load balancer to the instances is not encrypted. [1]
There is some discussion (e.g. on the blog of Kevin Burke) whether it is necessary to encrypt traffic inside a VPC. [2] However, most people are probably not doing it.
What it means for you: Use the same instance protocol for your targets as before: HTTP via port 8000 for both listeners. Do not set up SSL for your Gatsby service. Use a plain HTTP server config instead. No changes are necessary to ELB targets when using SSL termination on the load balancer.
References
[1] https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-create-https-ssl-load-balancer.html
[2] https://acloud.guru/forums/aws-certified-security-specialty/discussion/-Ld2pfsORD6ns5dDK5Y7/tlsssl-termination?answer=-LecNy4QX6fviP_ryd7x

Elastic Beanstalk with Classic Load Balancer working with HTTP but not HTTPS

I have set up a load Balancer to my elastic beanstalk app. It has provided a DNS / URL which works on http.
I have set up my domain on Route 53. I'm trying to make it work with HTTPS for a subdomain app.example.com
I have set up a wildcard certificate using ACM *.example.com.
I have enable HTTPS and port 443 on the loadBalancer. But I can't access it using the domain name https://app.example.com but works with http://app.example.com
In Route 53, I have created an A record with Alias set to the load Balancer DNS Name.
I have gone through tonnes of answers on stackoverflow but nothing worked for me. No idea what I'm missing.
You do not need to map 443 of the target as the TLS will terminate before sending the request to target.
Map 8080 port of the target to 443 of the load balancer.
Below diagram show SSL/TLS termination work with ALB.
SSL Certificates
To use an HTTPS listener, you must deploy at least one SSL/TLS server
certificate on your load balancer. The load balancer uses a server
certificate to terminate the front-end connection and then decrypt
requests from clients before sending them to the targets.
/application/create-https-listener
This is what worked for me. Changing Instance Protocol to HTTP in first row and changing Instance Port to 80.

AWS SSL certificate with Elastic Beanstalk: HTTPS site not reachable

When I associate an AWS certificate with my Elastic Beanstalk app and visit the domain using https, I get 'This site can’t be reached, mydomain.com refused to connect.' I can visit the site using http.
I created a security certificate with AWS's ACM. I added my domain name, example.com, along with additional names that were sub domains to the certificate. In my app's environment 'Loading balancing' section I set this up:
I setup the SecurityGroup named awseb-e-abc123-stack-* for my environment as follows:
There is another security group named awseb-e-abc123-stack-AWSEBLoadBalancerSecurityGroup-*, which is as follows. It has the same name tag as the above group, which is the same as my environment name:
It seems, though, that the entries of the "AWSEBLoadBalancerSecurityGroup" security group does nothing, as removing all the entries still allows HTTP traffic to work.
In .elasticbeanstalk\securelistener.config, I have the following
option_settings:
aws:elb:listener:443:
SSLCertificateId: <my certificate's ARN>
ListenerProtocol: HTTPS
InstancePort: 80
It seems, though, if I add a syntax error in this file, the deployment still succeeds.
Here is the output of curl -vL https://<my domain>:
* Rebuilt URL to: https://<my domain>/
* Trying <my elastic IP>...
* connect to <my elastic IP> port 443 failed: Connection refused
* Failed to connect to <my domain> port 443: Connection refused
* Closing connection 0
curl: (7) Failed to connect to <my domain> port 443: Connection refused
I used a separate domain name register to setup my domain name, and set up my domain's DNS A records IP address equal to my Elastic IP.
[edit]
I had mentioned above that changing the rules of the load balancer security group does nothing. This was because my EC2 instances's security group was pointing to the instance's security group, not the security group of the load balancer. When I pointed the EC2's security group to the load balancer's security group, the security group's rules are execercized. I verified this by removing all the rules from the load balancer security group, and seeing that no requests are accepted. However, if I add back the HTTP and HTTPS rules to the load balancer security group but remove all the rules from the instance security group, all HTTP requests are going through. This is NOT expected behavior because the load balancer is supposed to forward traffic to the instance. What seems to be happening is (1) the instance and load balancer security groups are not at all related to the instance and load balancer and (2) no traffic is going to the load balancer.
Is there anything else I'm missing?
[edit 2]
I misread gkrizek's comment. If I use the public DNS of the load balancer, I am able connect using either HTTP or HTTPS. I am able to connect to both versions using telnet. So instead of creating an A record that set testdomain.com to my Elastic IP, I created a sub.testdomain.com CNAME record set to the load balancer. Now I am able to browse to both http://sub.testdomain.com and https://sub.testdoamin.com. Two questions:
Is it OK to use the load balancer's public DNS in the CNAME record? I can't use the *.elasticbeanstalk.com URL because it could change, so I'm wondering if the same situation applies here
How can I secure https://testdomain.com (no sub domain). It seems like with this method, it is impossible to have https://testdomain.com since I cannot create an A record with an domain name.
The issue is that you have to create a CNAME record with your domain and use the load balancer DNS name as the address. If you use the Elastic IP, the requests will not go to the load balancer.
To answer my last questions:
AWS says this is OK
You cannot directly do this because AWS does not allow you to assign an Elastic IP to a load balancer. What you can do is have a URL record that redirects testdomain.com to https://www.testdomain.com