I have a Django server deployed on an Azure medium instance. I just wanted to know the maximum number of concurrent connections that can connect to my server at once. Consider these as REST connections rather than TCP connections because our server has a REST interface.
The number of concurrent connection is not artificially limited for Azure Virtual Machines, the actual number of the concurrent connections would depend upon the performance of your web service, the scalability of Django built-in web server and the reuse of TCP connection pool by your clients. You might want to test this using some web load tests.
Related
I was hoping someone may be able to explain how I would setup a multi-tiered web application. There is a database tier, app tier, web server tier and then the client tier. I'm not exactly sure how to separate the app tier and web server tier since the app tier will be in a private subnet. I would have the client send the request directly to the app server but the private net is a requirement. And having the app server separated from the web server is a requirement as well.
The only idea I have had was to serve the content on the web server and then the client will send all requests to the same web server on another port. Like port 3000, if a request is captured on that port, a node app using express will forward the request to the app tier since the web server can speak to the app server.
I did setup a small proof of concept doing this. The web server serves the content, then I have another express app setup to listen on port 3000, the client sends the request on port 3000 and then it just sends the exact same thing back to the app server.
This is my current setup with the web servers hosting two servers. One to serve the frontend on port 80 and one to receive requests on port 3000. The server listening on port 3000 forwards all requests to the app server ALB(It's basically a copy of all the same routes on the app server but it just forwards the requests instead of performing an action). But is there a way to not have this extra hop in the middle? Get rid of the additional server that is listening on 3000 without exposing the internal ALB?
To separate your web servers and application servers, you can use a VPC with public and private subnets. In fact, this is such a common scenario that Amazon has already provided us with documentation.
As for a "better way to do this," I assume you mean security. Here are some options:
You can (and should) run host based firewalls such as IP tables on your hosts.
AWS also provides a variety of options.
You can use Security Groups, which are statefull firewalls for your hosts
You can also use Network Access Control Lists (ACLs), which are stateless firewalls used to control traffic in and out of subnets.
AWS would also argue that many shops can improve their security posture by using managed services, so that all of the patching and maintenance handled by AWS. For example, static content could be hosted on Amazon S3, with dynamic content provided by microservices leveraging API Gateway. Finally, from a security perspective AWS provides services like Trusted Advisor, which can help you find and fix common security misconfigurations.
Let's say I want to use RDS Proxy for my Spring Boot application running in EC2/ECS/EKS, I have everything working as expected using standard JDBC connection configuration. Do I still need to configure application side connection pooling using libraries like C3P0 or would that be redundant?
I'm assuming the benefits of using RDS Proxy is to be able to share connection pools across multiple different types of applications (serverless and none serverless) that connect to the same DB.
I'm mostly going to reuse the answer that I just gave to another question, Does RDS proxy affects current application side pooling?:
With a database proxy in the middle, there are two separate legs to a "connection":
First, there is a connection from the application to the proxy. What you called the "application side pooling" is this type of connection. Since there's still overhead associated with creating a new instance of this type of connection, continuing to use a connection pool in your application probably is a good idea.
Second, there is a connection from the proxy to the database. These connections are managed by the proxy. The number of connections of this type is controlled by a proxy configuration. If you set this configuration to 100%, then you're allowing the proxy to use up to the database's max_connections value, and other clients may be starved for connections.
So, the application connection pool is not redundant. When your application wants to use a connection, it needs to get a connection from its local pool. Then, the proxy needs to pair that with a connection to the database. The proxy will reuse connections to the database where possible (this technique also is called multiplexing).
Or, quoting the official docs: "You can open many simultaneous connections to the proxy, and the proxy keeps a smaller number of connections open to the DB instance or cluster. Doing so further minimizes the memory overhead for connections on the database server. This technique also reduces the chance of "too many connections" errors."
Going back to your original question, yes, "share connection pools across multiple different types of applications" is one of the benefits: you don't have to configure your different application connection pools to stay within the database's max_connections value. Other benefits of RDS Proxy, including efficiency, failover, security, etc., are covered in the official docs.
I am trying to create a clustered cache service for Cloud Foundry. I understand that I need to implement Service Broker API. However, I want this service to be clustered, and in the Cloud Foundry environment. As you know, container to container connection (TCP) is not supported yet, I don't want to host my backend in another environment.
Basically my question is almost same as this one: http://grokbase.com/t/cloudfoundry.org/vcap-dev/142mvn6y2f/distributed-caches-how-to-make-it-work-multicast
And I am trying to achieve this solution he adviced:
B) is to create a CF Service by implementing the Service Broker API as
some of the examples show at the bottom of this doc page [1] .
services have no inherant network restrictions. so you could have a CF
Caching Service that uses multicast in the cluster, then you would
have local cache clients on your apps that could connect to this
cluster using outbound protocols like TCP.
First of all, where does this service live? In the DEA? Will backend implementation be in the broker itself? How can I implement the backend for scaling the cluster, start the same service broker over again?
Second and another really important question is, how do the other services work if TCP connection is not allowed for apps? For example, how does a MySQL service communicates with the app?
There are a few different ways to solve this, the more robust the solution, the more complicated.
The simplest solution is to have a fixed number of backend cache servers, each with their own distinct route, and let your client applications implement (HTTP) multicast to these routes at the application layer. If you want the backend cache servers to run as CF applications, then for now, all solutions will require something to perform the HTTP multicast logic at the application layer.
The next step would be to introduce an intermediate service broker, so that your client apps can all just bind to the one service to get the list of routes of the backend cache servers. So you would deploy the backends, then deploy your service broker API instances with the knowledge of the backends, and then when client apps bind they will get this information in the user-provided service metadata.
What happens when you want to scale the backends up or down? You can then get more sophisticated, where the backends are basically registering themselves with some sort of central metadata/config/discovery service, and your client apps bind to this service and can periodically query it for live updates of the cache server list.
You could alternatively move the multicast logic into a single (clustered) service, so:
backend caches register with the config/metadata/discovery service
multicaster periodically queries the discovery service for list of cache server routes
client apps make requests to the multicaster service
One difficulty is in implementing the metadata service if you're doing it yourself. If you want it clustered, you need to implement a highly-available-ish consistent-ish datastore, it's almost the original problem you're solving except the service handles replicating data to all nodes in the cluster, so you don't have to multicast.
You can look at https://github.com/cloudfoundry-samples/github-service-broker-ruby for an example service broker that runs as a CF application.
I've written "app" that I would like to distribute for clients as a service. Most of the "app" logic is on database side (PostgreSQL) and it's speed/availability depends mostly on database server. App can be exposed as REST API (node.js) that mostly just queries database for some data and then returns it in JSON format.
So there should be separate database instance (server) per each client to make client apps "independent". When customer will purchase a plan then he will "receive" database server (not exposed to him directly) from the cloud (Amazon RDS, heroku PG or other).
The problem is how to expose REST API to each customer. Normally there should be separate web server per each customer that would connect to single database. But this means that I would have to manage N databases and N web servers for each client, additionally scaling each web server (horizontally) for customer needs (to handle more requests per sec).
So my idea is to create single scalable web server (like single heroku app) that will handle requests from all the clients. So flow would look like this:
Customer makes request to REST API (web server) with his API_TOKEN
One of web server instances (web server is scaled) receives that request.
Customer database address/credentials are "revealed" based on API_TOKEN
web server makes request to customer database, receives result and returns data.
The problem is that database connections should be pooled on web server instances (to make it faster). When there will be 1000 clients then it will be possible that single web server instance will have x*1000 database connections open where x is the number of parallel requests per customer.
On the other hand if I scale web server, then single customer may hit for example i1 then i2 then i3 then i4 (where iX is web server instance selected by load balancer). It will be not efficient because it will "open" connection pool to same database on 4 different web server instances. If those 4 subsequent requests would hit only i1 then they could take advantage of connection pool that would be "open" during first request (and next requests would use already opened connection).
I have two questions:
Is this architecture a good idea? Maybe there are other standard approaches/services to achieve my goal?
Is it possible to somehow configure load balancer to make it so smart that it could forward some requests from same customer to same web server instance (to take advantage of database connections pool on that instance) while still keeping all requests distributed across all instances as evenly as possible (For example if there would be 10 customers c1, c2, c3, c4 ... and 4 web servers instances w1, w2, w3, w4 and each client makes x requests then it could forward c1|c2 to w1, c3|c4 to w2, c5|c6|c7 to w3 and c8|c9|c10 to w4)? Which load balancer (I know HAPROXY, NGINX and some Amazon Elastic Load Blanacer).
There is an option for nginx to have a load-balancing to send a client to webserver based on ip: http://nginx.org/en/docs/http/load_balancing.html#nginx_load_balancing_with_ip_hash, so it looks liek what you want for 2).
And there are serveral discussions about database per client at SO:
What are the advantages of using a single database for EACH client?
https://serverfault.com/questions/107731/one-database-vs-multiple-databases
I am trying to turn the server/client model into a server/server model, so as to have the my 2 computers running the program find each other by perhaps a url or something else like ip address.I was wondering if it was possible for 2 servers to connect via url's. or is ip the only way? examples would be appreciated since this is my second day writing c++.
For HTTP, the server only talks to clients. So, I am not sure what you mean by server to server.
URLs are fine to use to access an HTTP server, but the host name will need to be resolved into an IP address before a network connection can actually be established. You should be able to find libraries that will do those details for you, but it is not hard to manually establish a socket connection to an HTTP server.
There are configurations where there are multiple servers, acting as a single server. These are sometimes referred to as web farms or a HTTP cluster. Typically, there is some sort of load balancer in front of the cluster. Many HTTP load balancers support a server affinity feature to make sure a client is sent to the same server in the cluster for subsequent operations.
In a cluster configuration, servers may need to synchronize shared state, such as file system data or configuration data. This is typically handled by some mechanism that is external to the HTTP server process itself. The HTTP server process may need to cooperate with the synchronization, but this can be as simple as restarting the process.
There is another mode of HTTP server configuration called a reverse proxy configuration. A cluster of HTTP proxy servers sit in front of a single HTTP server. The proxy servers are thought to be cheap and expendable entities that off load work from the HTTP server itself, providing a scalable means to increase HTTP server capacity.
There are many open source HTTP server and proxy projects available as examples of how they are implemented. If you are trying to build your own custom server application, you can have a look at the HTTP examples in Boost asio.