RDS Proxy vs Application Side Proxy (eg. C3P0) - amazon-web-services

Let's say I want to use RDS Proxy for my Spring Boot application running in EC2/ECS/EKS, I have everything working as expected using standard JDBC connection configuration. Do I still need to configure application side connection pooling using libraries like C3P0 or would that be redundant?
I'm assuming the benefits of using RDS Proxy is to be able to share connection pools across multiple different types of applications (serverless and none serverless) that connect to the same DB.

I'm mostly going to reuse the answer that I just gave to another question, Does RDS proxy affects current application side pooling?:
With a database proxy in the middle, there are two separate legs to a "connection":
First, there is a connection from the application to the proxy. What you called the "application side pooling" is this type of connection. Since there's still overhead associated with creating a new instance of this type of connection, continuing to use a connection pool in your application probably is a good idea.
Second, there is a connection from the proxy to the database. These connections are managed by the proxy. The number of connections of this type is controlled by a proxy configuration. If you set this configuration to 100%, then you're allowing the proxy to use up to the database's max_connections value, and other clients may be starved for connections.
So, the application connection pool is not redundant. When your application wants to use a connection, it needs to get a connection from its local pool. Then, the proxy needs to pair that with a connection to the database. The proxy will reuse connections to the database where possible (this technique also is called multiplexing).
Or, quoting the official docs: "You can open many simultaneous connections to the proxy, and the proxy keeps a smaller number of connections open to the DB instance or cluster. Doing so further minimizes the memory overhead for connections on the database server. This technique also reduces the chance of "too many connections" errors."
Going back to your original question, yes, "share connection pools across multiple different types of applications" is one of the benefits: you don't have to configure your different application connection pools to stay within the database's max_connections value. Other benefits of RDS Proxy, including efficiency, failover, security, etc., are covered in the official docs.

Related

Application ELB - sticky sessions based on consistent hashing

I couldn't find anything in the documentation but still writing to make sure I did not miss it. I want all connections from different clients with the same value for a certain request parameter to end up on the same upstream host. With ELB sticky session, you can have the same client connect to the same host but no guarantees across different clients.
This is possible with Envoy proxy, see: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers#ring-hash
We already use ELB so if the above is possible with ELB then we can avoid introducing another layer in between with envoy.
UPDATE:
Use-case - in a multi-tenant cloud solution, we want all clients from a given customer account to connect to the same upstream host.
Unfortunately this is not possible to be performed in an ALB.
An application load balancer controls all the logic over which host receives the traffic with features such as ELB sticky sessions and pattern based routing.
If there is no work around then you could look at a Classic Loadbalancer which has support for the application setting the sticky session cookie name and value.
From best practice ideally your application should be stateless, is it possible to look at rearchitecting your app instead of trying work around. Some suggestions I would have are:
Using DynamoDB to store any session based data, moving from a disk based session (if that's what your application does).
Any disk based files that need to persist could be shared between all hosts either using EFS for your Linux based hosts, or FSX on Windows.
Medium/Long term persisting files could be migrated to S3, any assets that rarely change could be stored here and then your application could use S3 rather than disk.
It's important to remember that as I stated above, you should keep your application as stateless as you can. Assume that your EC2 instances could fail, by preparing for this it will make it easier to recover.

When using a Google Cloud SQL database, is there a difference between a private IP + SSL, or the cloud proxy sidecar?

When trying to evaluate how to connect to a Cloud SQL database from a Google Kubernetes Engine pod, there are a couple of ways to do this. One is to use a sidecar cloud proxy agent. Another is using a private IP and using a SSL connection between the two. Is there a clear case for either? Or do they both serve the same functionality? Is there one that is considered "best practice"?
Cloud SQL Proxy sidecar
The cloud sql proxy sidecar establishes a TCP connection into a proxy service that is hosted on Google's infrastructure. This then connects you to your cloud SQL instance on the Google network.
Pros
Establishes a secure connection without you having to manage the crypto material in your application
Connects to the instance and you don't have to manage DNS records or IP addresses
Cons
You have to create a secret that stores a service account key.
You have to manage a sidecar instance along side your pod, which if that fails, you no longer can connect to your database
Latency added due to the number of layers you have to the proxy layers
Private IP + SSL
Using a private IP and connecting the instance to your VPC allows you to use an internal IP address, that is not publicly routed, and keeps traffic in your VPC instance. On top of that, setting up SSL only connections to your database to make sure traffic is secure from point to point.
Pros
Low latency connection to the database because its a point to point connection
You manage the keys between the services
No outside dependencies or systems needed to connect between the two
Cons
You have to manage the SSL certificate inside of the connection
You have to verify that the IP and DNS records setup in your cluster are correct
Am I missing something? Do these two indeed provide the same thing? Is there not an absolutely clear winner between the two and you can pick whichever one you see that best fits your style?
Best practice is to use the Proxy. From a secure standpoint they're both good options, but I've found the mess of managing my own SSL keys just a nuisance I didn't need. Also as John mentioned in his comment, if you shift regions, or change IPs for any reason, you have to change the container content rather than just a flag on the proxy startup. You can mitigate that of course using environment variables on the containers, but it's one more thing.
There's a SLIGHT security edge on the proxy IMO as IF your keys get compromised, the window that the ephemeral key generated by the proxy connection is shorter lived than an SSL key generated by yourself (unless you're using the ephermal key calls in the API). So if a vulnerability is found in your app, and a key gets compromised, there's a smaller window that anyone can do malicious things to your DB. But particularly if you're solely on a VPC that's LESS of a concern, but is still greater than zero.

Load balancing mechanism to directly connect backend and clients?

I'm trying to implement a system using multiple servers which communicate with the clients via TCP. As there are going to be multiple backend servers to process clients' requests, I'm wondering if there exists a load balancing mechanism which after the load balancer receives a client request, it connects the client directly to a backend server so that they establish a two-way TCP connection and all future communication between the two is done directly between them and not through the load balancer.
In general terms, no, IP doesn't allow this, individual connections cannot be handed off to other machines. The protocol does not permit it. You can, in the case of high-availability, hand off all connections from one machine to another, but even that is kind of a hack and is done using MAC address shifting.
What you want is probably something like HAProxy which is a TCP/IP load balancer. It's fast enough that being in the middle isn't really a problem, you won't feel a performance hit.
The alternative is to do something in the application layer where your "load balancer" tells clients which server to connect to, but doesn't actually do the connecting. That's done by the client in a secondary request. Sometimes this is done via DNS, sometimes via time-limited tokens to ensure clients respect the routing.

Can AWS Elastic Load Balancer be used to only send traffic to a second server if the first fails

Can an AWS Elastic Load Balancer be setup so it sends all traffic to a main server and if that server fails, only then send traffic to a second server.
Have an existing web app I picked up that was never built to run on multiple servers and the client has become worried about redundancy. They don't want to invest enough to make it run well across multiple servers so I was thinking I could setup a second EC2 server with a MySQL slave and periodically copy files from the primary server to the secondary using rsync. Then have an AWS ELB send traffic to the primary server and only if that fails send it to the second server.
AWS load balancers don't support "backup" nodes that only take traffic when the primary is down.
Beyond that, you are proposing a complicated scenario.
was thinking I could setup a second EC2 server with a MySQL slave
If you do that, you can only fail over once, then you can't fail back, because the master database will then be obsolete. For a configuration like this to work and be useful, your two MySQL servers need to be configured with master/master (circular) replication, so that each is a replica of the other. This is an advanced configuration that requires expertise and caution.
For the MySQL component, an RDS instance with multi-AZ enabled will provide you with hands-off fault tolerance of the database.
Of course, the client may be unwilling to pay for this as well.
A reasonable shortcut for small systems might be EC2 instance recovery which will bring the site back up if the underlying hardware fails. This feature replaces a failed instance with a new instance, reattaches the EBS volumes, and starts it back up. If the system is stable and you have a solid backup strategy for all data, this might be sufficient. Effective redundancy as a retrofit is non-trivial.

Trying to understand how does the AWS scaling work

There is one thing of scaling that I yet do not understand. Assume a simple scenario ELB -> EC2 front-end -> EC2 back-end
When there is high traffic new front-end instances are created, but, how is the connection to the back-end established?
How does the back-end application keep track of which EC2 it is receiving from, so that it can respond to the right end-user?
Moreover, what happen if a connection was established from one of the automatically created instances, and then the traffic is low again and the instance is removed.. the connection to the end-user is lost?
FWIW, the connection between the servers is through WebSocket.
Assuming that, for example, your ec2 'front-ends' are web-servers, and your back-end is a database server, when new front-end instances are spun up they must either be created from a 'gold' AMI that you previously setup with all the required software and configuration information, OR as part of the the machine starting up it must install all of your customizations (either approach is valid). with either approach they will know how to find the back-end server, either by ip address or perhaps a DNS record from the configuration information on the newly started machine.
You don't need to worry about the backend keeping track of the clients - every client talking to the back-end will have an IP address and TCPIP will take care of that handshaking for you.
As far as shutting down instances, you can enable connection draining to make sure existing conversations/connections are not lost:
When Connection Draining is enabled and configured, the process of
deregistering an instance from an Elastic Load Balancer gains an
additional step. For the duration of the configured timeout, the load
balancer will allow existing, in-flight requests made to an instance
to complete, but it will not send any new requests to the instance.
During this time, the API will report the status of the instance as
InService, along with a message stating that “Instance deregistration
currently in progress.” Once the timeout is reached, any remaining
connections will be forcibly closed.
https://aws.amazon.com/blogs/aws/elb-connection-draining-remove-instances-from-service-with-care/