We have an internal installation of PCF.
A developer wants to push a stateless (obeys 12 factor rules) nodejs app which will spawn other app instances i.e leverage nodejs clustering as per https://nodejs.org/api/cluster.html. Hence there would be multiple processes running on each container. Any known issues with this from a PCF perspective? I appreciate it violates the rule/suggestion of one app instance per container but that is just a suggestion :) All info welcome.
Regards
John
When running an application on Cloud Foundry that spawns child processes, the number one thing you need to watch out for is memory consumption. You set a memory limit when you push your application which is for the entire container. That includes the parent process, whatever child processes are spawned and a little overhead (currently init process, sshd process & health check).
Why is this a problem? Most buildpacks make the assumption that only one process will be running and that it will consume as much memory as possible while staying under the defined memory limit. They attempt to configure the software which is running your application to do this. When you spawn child processes, this breaks the buildpack's assumptions and can create scenarios where your application will exceed the defined memory limit. When this happens, even by one byte, the process will be killed and restarted.
If you're concerned with scaling your application, you should not try to spin off child processes in one extra large container. Instead, let the platform help you and scale up the number of application instances. The platform can easily do this and by using multiple smaller containers you can scale just as well. In fact, if you already have a 12-factor app, it should be well positioned to work in this manner.
Good luck!
Related
I came across a questionnaire on Containers and Buildpacks. I could not find the answer to the following questions :
What buildpack is used to stage docker apps?
I would assume a non-docker app would require a buildpack which would produce the container image during the staging process. For a docker based application a buildpack is not relevant since it is already a container image . Is this correct understanding ?
When you allocate 1G to an app, how much memory does the application receive? What
component determines this?
I would assume, the actual application (For example: my_sample_spring_boot_app)would receive less memory than 1G , since some part of teh memory would be utilised for Infra + Stack + runtime etc. Is this correct understanding ?
Could you please help me with some guidance
What buildpack is used to stage docker apps?
I would assume a non-docker app would require a buildpack which would produce the container image during the staging process. For a docker based application a buildpack is not relevant since it is already a container image . Is this correct understanding ?
That sounds like a trick question. When you run Docker images on Cloud Foundry, no buildpacks run.
There are different app lifecycles to run each type of app. So when you have source code to build and run, the buildpacks app lifecycle takes that, runs the buildpacks, and generates a droplet from them. The droplet is then run.
With a Docker image, the docker app lifecycle is used. No build/stage happens since that's assumed to happen externally, and the image is just ready to be run.
When you allocate 1G to an app, how much memory does the application receive? What component determines this?
I would assume, the actual application (For example: my_sample_spring_boot_app)would receive less memory than 1G , since some part of teh memory would be utilised for Infra + Stack + runtime etc. Is this correct understanding ?
Correct. If you allocate a 1G memory limit then everything running in the container must all fit within that 1G limit.
Your application will get most of that memory, but there are a couple of small processes that run in every container: an init process (i.e. PID1), an SSHD (for cf ssh) and in most CF versions there's also an envoy proxy (for TLS termination in the container). I haven't looked recently, but a while back this took up around 24M. If anyone has measured recently, please comment.
Everything has to fit under the memory limit. If your application forks, perhaps because it forks worker processes or if you fork and run sub-processes as part of the application. Everything you run has to all stay under the 1G limit.
Last note. For Java apps, 1G is the total memory size of the container. NOT your JVM heap. That's a common misconception. The entire JVM process and all its different memory regions (heap, metaspace, threads, direct memory, code cache, etc...) have to fit within the 1G limit.
I am reading about the concept of Side Car and Multi Process Application in cloud foundry .
https://docs.cloudfoundry.org/devguide/multiple-processes.html
https://docs.cloudfoundry.org/devguide/sidecars.html
I have few questions which I could not figure out myself.
Q1: When to use a CF Application with Sidecar vs when to use a CF Application with Processes
I understood that the primary difference between sidecar vs multiple process application is related to container. A sidecar process runs in the same container whereas for multi process application all of them run in separate containers.
I could not figure out , in which scenarios we should use sidecar vs in which scenarios we can use multiple process application
Q2: Different Processes in different technology
In an application with multiple processes , if I want to run 2 processes in 2 different technology ( one process on Java another process in Go / anything else), how to do so ?
This question comes to my mind as I see the buildpack configuration goes with the application , not with the process. So I am getting an impression as if only all processes must be in same technology ( or we can provide multiple buildpacks here ?) .
Here is a sample manifest.yml that I am using:
applications:
- name: multi-process1
disk_quota: 1G
path: target/SampleApp-1.jar
instances: 1
memory: 2G
buildpacks:
- java_buildpack
env:
CONFIG_SERVER_PORT: 8080
processes:
- type: 'task'
command: '$PWD/.java-buildpack/open_jdk_jre/bin/java -jar $PWD/BOOT-INF/lib/mycustomlogger-1.jar'
memory: 512MB
instances: 1
health_check:
type: http
- type: 'sampleProcess2'
command: '$PWD/.java-buildpack/open_jdk_jre/bin/java -jar $PWD/BOOT-INF/lib/mycustomlogger-1.jar'
memory: 512MB
instances: 1
health_check:
type: http
- type: 'web'
#command: '$PWD/.java-buildpack/open_jdk_jre/bin/java $PWD/BOOT-INF/classes/com/example/SampleApp'
memory: 1G
health_check:
type: http
Q3: Interacting processes
In this context how can one process call/talk/interact with the other processes within the application. What are the options available here ? I could not find any sample which demonstrate multiple interacting processes within an app, any sample would be really helpful.
Q4 : Difference between Multi Target Application vs Multi Process Application
I came across a concept called Multi Target Application , reference : https://www.cloudfoundry.org/blog/accelerating-deployment-distributed-cloud-applications/
I did not find such possibility in standard Cloud Foundry, but I felt it might be "similar" to Multi Process app ( since they run on independent containers and are not impacting each other).
My questions are:
What are the differences between Multi Target Application vs Multi Process Application?
What are the fundamental Could Foundry concepts on which Multi Target Application is built ?
Any guidance would be really appreciated.
Q1: When to use a CF Application with Sidecar vs when to use a CF Application with Processes
Different process types are helpful when you have well-separated applications. They might talk to each other, they might interact, but it's done through some sort of published interface like a REST API or through a message queue.
A typical example is a work queue. You might have one process that's running your web application & handling traffic, but if a big job comes in then it'll instruct a worker process, running separately, to handle that job. This is often done through a message queue. The advantage is you can scale each individually.
With a sidecar, they are in the same process. This works well for the scenario where you need tight coupling between two or more processes. For example, if you need to share the same file system or if you have proxy process that intercepts traffic.
The two processes are often linked in a way that they are scaled in tandum, i.e. there is a one-to-one relationship between the processes. This relationship is necessary because if you scale up the application instance count, you're scaling up both. You cannot scale them independently.
In an application with multiple processes , if I want to run 2 processes in 2 different technology ( one process on Java another process in Go / anything else), how to do so ?
You're correct. You'd need to rely on multi-buildpack support.
Your application is only staged once and a single droplet is produced. Each process is spun up from the same droplet. Everything you need must be built into that one droplet. Thus you need to push everything all together and you need to run multiple buildpacks.
In your manifest.yml, the buildpacks entry is a list. You can specify the buildpacks you want to run and they will run in that order.
Alternatively, you could pre-compile. If you're using Go. Cross-compilation is often trivial, so you could compile ahead of time and just push the binary you generate with your application.
In this context how can one process call/talk/interact with the other processes within the application. What are the options available here ? I could not find any sample which demonstrate multiple interacting processes within an app, any sample would be really helpful.
I'll split this in two parts:
If you're talking about apps running as a sidecar, it depends on what the sidecar does, but you have options. Basically, anything you can do in a Linux environment, keeping in mind you're running as a non-root user, you can do. Coordinate through a shared file/folder, intercept the network port & proxy to another port, or other ipc.
If you're talking about multiple processes (the CF terminology) such that you have each running in a separate container, then you are more limited. You would need to use some external method to communicate. This could be a service broker, a database (not recommended), or an API (Rest, gRCP or rsocket are all options).
Note that this "external" method of communication doesn't necessarily need to be public, just external to the container. You could use a private service like a broker/DB, or you could use internal routes & container networking.
Q4 : Difference between Multi Target Application vs Multi Process Application
Sorry, I'm not sure about this one. Multi Target apps are not a core concept in CF. I'll leave it for someone else to answer.
I am working on re-designing an existing Spring Boot application which is trying to automate a series of executions of work that the company is currently having done manually by it's staff. It has a main application that works almost as an orchestration application in the sense that it is the service that will call other applications to get the overall piece of work done. It has 7 sub-systems that it invokes, 3 of these systems need to be invoked in some form of order and complete before the other 4 are invoked but they can be invoked asynchronously.
All of these sub-systems have now been moved to Spring Microservices and the application I'm working on must invoke these microservices (some in order and some asynchronously), it is possible that my application will be called more than once at the same time so I need to consider that multiple containers may be needed for each sub-system. I've implemented Open-Feign to invoke each of the Microservices.
They also have the plan in the not so distant future to move this to AWS ECS/Fargate, however for the time being it is going to be run in Linux VM's and the containers are created on the same private network for communication. I'm wondering if I should remove ThreadPoolTaskExecutor completely and just invoke a new container for each simultaneous request to my application, however I've read that threads on a process are still faster and have less overhead than creating a process on a container and considering there's not going to be many containers invoked simultaneously I'm perplexed as to the best approach.
Any advice would be appreciated.
Unless each request scales up the memory consumption linearly in the application by at least an additional 1Gb (then new pod base memory would not be that much compared with it) it is an overkill to spin up a new pod for each request... it is 200Mb of additional memory for each request and I don't see a benefit nor the need.
I'm running a Django app in an EC2 instance, which uses RabbitMQ + Celery for task queuing. Are there any drawbacks to running my RabbitMQ node from the same EC2 instance as my production app?
The answer to this questions really depends on the context of your application.
When you're faced with scenarios you should always consider a few things.
Seperation of concerns
Here, we want to make sure that if one of the systems are not responsible for the running of other systems. This includes things like
If the ec2 instance running all the stuff goes down, will the remaining tasks in queue continue running
if my RAM is full, will all systems remain functioning
Can I scale just one segment of my app without having to redesign infrastructure.
By having rabbit and django (with some kind of service, wsgi, gunicorn, waitress etc) all on one box, you loose a lot of resource contingency.
Although RAM and CPU may be abundant, there is a limit to IO, disk writes, network writes etc. This means that if for some reason you have a heavy write function, all other systems may suffer as a result. If you have a heavy write to RAM funciton, the same applies.
So really the downfalls from keeping things in one system that I can see from your question and my own experience are as follows.
Multiple points of failure. If your one instance of rabbit fails, your queues and tasks stop working.
If your app starts generating big traffic, other systems start to contend for recourses.
If any component goes down, that could mean other downtime of other services.
System downtime means complete downtime of all components.
Lots of headaches when your application demands more resources with minimal downtime.
Lots of web traffic will slow down task running
Lots of task running will slow down web requests
Lots of IO will slow down all the things
The rule of thumb that I usually follow is keep single points of failures far from each other - that way you only need to manage those components. A good use case for this would be to use an EC2 instance for your app, another for your workers and another for your rabbit. That way you can apply smaller/bigger instances for just those components if you need to. You can even create AMIs and create autoscaling groups - if it is your use case.
Here are some articles for reference
Seperation of concern
Modern design architectures
Single points of failure
TLDR; If you can run on one EC2 you should but make it easy to scale today.
Both Joshnidhin and Giannis covered the RAM, IO and CPU aspects.
I have run production apps in single instances with containerization and slept with peace of mind that if tomorrow suddenly lots of people want what I have built, I can scale pretty quickly by deploying those containers on different instances instead of one single instance.
Docker allows you to put a limit on CPU consumption and memory usage for each container hence you can also be sure that they will not step into each other.
If we take EC2 instance out of this question it becomes:
Are there any drawbacks in running RabbitMQ Node on the same server as my productions app?
I would say it depends on various things like, kind of workloads and its composition, complexity of the workload, do you expect growth in usage etc.
If your workload is well behaved and the server is big enough for both (app + task q) then why not as there will be only one server to manage. Make sure to protect these 2 process from each other by limiting their system resource usage.
If your traffic is not well behaved then you might want more the one server. In this case having dedicated servers is better (separation of concerns) as you will have to manage more than one server.
Now back to EC2, all the above still apply. EC2 makes horizontal scaling of applications easier so if you have them on separate instance then you can scale them individually and cost effectively. If not when you scale there will be wastage of resources.
I'm trying with WSO2 products, and I'm thinking about a scenario where bad code could take up all the CPU time (e.g. dead loop or so). I did try it with WSO2 AS with 2 tenants, A and B. And A's bad code does affect B and B's app will have a very long reponse delay or even stuck. Is there a way to restrict the CPU usage of a tenant? Thanks!
At the moment, you will have to setup your environment in what is known as private jet mode, where each tenant gets its own JVM, if you need total isolation.
In a shared environment, we have stuck thread detection which will ensure that critical threads will not run for more than a specified time period. We have plans for CPU usage limiting on per tenant basis. This would be available in a future release.
My suggestion would be to not run two tenants in one application server. Run two separate processes on the same machine. Better yet, run two separate processes in separate OS-level containers (like a jail or an lxc container). Or separate virtual machines if you can't use containers.
Operating systems give you tools for controlling CPU use - rlimit and nice for processes, and implementation-specific facilities for containers and VMs. Because they're implemented in the OS (or virtual machine manager), they are capable of doing this job correctly and reliably. There's no way an application server can do it anywhere near as well.
In any case, having separate applications share an application server and JVM is a terrible idea which should have been put to death in the '90s. There's just no need for it, and it introduces so many potential headaches.