I am creating small app running multiple microservices. I would like to have this app available 24/7, so free dyno hours are not enough for me. If I upgrade to a hobby plan I would get 10 Process Types.
Can I run another microservice on each of the processes (web), or does Heroku give me the ability only to install one web process per dyno, and the other 10 process types are for scaling my app? In other words, If i need 6 microservices running 24/7 should I buy 6 hobby dynos?
You can only have 1 web process type. You can horizontally scale your web process to run on multiple dynos ("horizontal scalability"), however you will need to upgrade to at least standard-1x dyno types to do that (i.e. you can only run 1 web dyno instance if you are using free or hobby dyno types).
However, in addition to your web process, you can instantiate multiple additional process types (e.g. "worker" processes). These will NOT be able to listen on HTTP/S requests from your clients, but can be used for offloading long-running jobs from your web process.
So, if you map each of your 4-6 microservices to a different Process Type in your Procfile, and if your microservices are not themselves web servers, you might be able to make do with hobby dynos.
Heroku's default model is to map a process type to its own dyno: https://devcenter.heroku.com/articles/procfile states
"Every dyno in your application will belong to one of the process
types, and will begin executing by running the command associated with
that process type."
e.g. heroku ps:scale worker=1 for a type of worker.
Other people have written about how to use foreman or Honcho to run multiple python processes in a single dyno, which utilize a secondary Procfile and possibly other slug setup in a post_compile step. Presumably you could do something similar depending on your chosen language and its buildpack; the official buildpack API doesn't list this step though :/. That said, given Heroku's base Ubuntu stacks, you may be able to get away with a web: script.sh to do any setup and exec of processes.
For grouping multiple processes in a Docker container, you could check out https://runnable.com/docker/rails/run-multiple-processes-in-a-container, which relies again on a custom CMD [ "/start.sh" ]. Note that it's contrary to Docker's single-process-per-container philosophy, and can give you more headaches e.g. around forwarding signals to child processes, an ugly Dockerfile to setup all microservices, etc. (If you do decide to use multiple containers, Heroku has a writeup on using Docker Compose for local dev.)
Also, don't forget you're bounded by the performance of your dyno and the process/thread limits.
Of course, multiple processes in a given dyno is generally not recommended for non-toy production or long term dev maintenance. ;)
There's RUNIT buildpack which makes it easy to combine multiple processes within a single dyno - as long as they all fit within your dyno memory limit (512M for a Hobby dyno).
You still have only one HTTP endpoint per Heroku app, which means your microservices will need to communicate via queues, pub/sub or some RPC hub (like deepstream.io).
Related
I am reading about the concept of Side Car and Multi Process Application in cloud foundry .
https://docs.cloudfoundry.org/devguide/multiple-processes.html
https://docs.cloudfoundry.org/devguide/sidecars.html
I have few questions which I could not figure out myself.
Q1: When to use a CF Application with Sidecar vs when to use a CF Application with Processes
I understood that the primary difference between sidecar vs multiple process application is related to container. A sidecar process runs in the same container whereas for multi process application all of them run in separate containers.
I could not figure out , in which scenarios we should use sidecar vs in which scenarios we can use multiple process application
Q2: Different Processes in different technology
In an application with multiple processes , if I want to run 2 processes in 2 different technology ( one process on Java another process in Go / anything else), how to do so ?
This question comes to my mind as I see the buildpack configuration goes with the application , not with the process. So I am getting an impression as if only all processes must be in same technology ( or we can provide multiple buildpacks here ?) .
Here is a sample manifest.yml that I am using:
applications:
- name: multi-process1
disk_quota: 1G
path: target/SampleApp-1.jar
instances: 1
memory: 2G
buildpacks:
- java_buildpack
env:
CONFIG_SERVER_PORT: 8080
processes:
- type: 'task'
command: '$PWD/.java-buildpack/open_jdk_jre/bin/java -jar $PWD/BOOT-INF/lib/mycustomlogger-1.jar'
memory: 512MB
instances: 1
health_check:
type: http
- type: 'sampleProcess2'
command: '$PWD/.java-buildpack/open_jdk_jre/bin/java -jar $PWD/BOOT-INF/lib/mycustomlogger-1.jar'
memory: 512MB
instances: 1
health_check:
type: http
- type: 'web'
#command: '$PWD/.java-buildpack/open_jdk_jre/bin/java $PWD/BOOT-INF/classes/com/example/SampleApp'
memory: 1G
health_check:
type: http
Q3: Interacting processes
In this context how can one process call/talk/interact with the other processes within the application. What are the options available here ? I could not find any sample which demonstrate multiple interacting processes within an app, any sample would be really helpful.
Q4 : Difference between Multi Target Application vs Multi Process Application
I came across a concept called Multi Target Application , reference : https://www.cloudfoundry.org/blog/accelerating-deployment-distributed-cloud-applications/
I did not find such possibility in standard Cloud Foundry, but I felt it might be "similar" to Multi Process app ( since they run on independent containers and are not impacting each other).
My questions are:
What are the differences between Multi Target Application vs Multi Process Application?
What are the fundamental Could Foundry concepts on which Multi Target Application is built ?
Any guidance would be really appreciated.
Q1: When to use a CF Application with Sidecar vs when to use a CF Application with Processes
Different process types are helpful when you have well-separated applications. They might talk to each other, they might interact, but it's done through some sort of published interface like a REST API or through a message queue.
A typical example is a work queue. You might have one process that's running your web application & handling traffic, but if a big job comes in then it'll instruct a worker process, running separately, to handle that job. This is often done through a message queue. The advantage is you can scale each individually.
With a sidecar, they are in the same process. This works well for the scenario where you need tight coupling between two or more processes. For example, if you need to share the same file system or if you have proxy process that intercepts traffic.
The two processes are often linked in a way that they are scaled in tandum, i.e. there is a one-to-one relationship between the processes. This relationship is necessary because if you scale up the application instance count, you're scaling up both. You cannot scale them independently.
In an application with multiple processes , if I want to run 2 processes in 2 different technology ( one process on Java another process in Go / anything else), how to do so ?
You're correct. You'd need to rely on multi-buildpack support.
Your application is only staged once and a single droplet is produced. Each process is spun up from the same droplet. Everything you need must be built into that one droplet. Thus you need to push everything all together and you need to run multiple buildpacks.
In your manifest.yml, the buildpacks entry is a list. You can specify the buildpacks you want to run and they will run in that order.
Alternatively, you could pre-compile. If you're using Go. Cross-compilation is often trivial, so you could compile ahead of time and just push the binary you generate with your application.
In this context how can one process call/talk/interact with the other processes within the application. What are the options available here ? I could not find any sample which demonstrate multiple interacting processes within an app, any sample would be really helpful.
I'll split this in two parts:
If you're talking about apps running as a sidecar, it depends on what the sidecar does, but you have options. Basically, anything you can do in a Linux environment, keeping in mind you're running as a non-root user, you can do. Coordinate through a shared file/folder, intercept the network port & proxy to another port, or other ipc.
If you're talking about multiple processes (the CF terminology) such that you have each running in a separate container, then you are more limited. You would need to use some external method to communicate. This could be a service broker, a database (not recommended), or an API (Rest, gRCP or rsocket are all options).
Note that this "external" method of communication doesn't necessarily need to be public, just external to the container. You could use a private service like a broker/DB, or you could use internal routes & container networking.
Q4 : Difference between Multi Target Application vs Multi Process Application
Sorry, I'm not sure about this one. Multi Target apps are not a core concept in CF. I'll leave it for someone else to answer.
We have an internal installation of PCF.
A developer wants to push a stateless (obeys 12 factor rules) nodejs app which will spawn other app instances i.e leverage nodejs clustering as per https://nodejs.org/api/cluster.html. Hence there would be multiple processes running on each container. Any known issues with this from a PCF perspective? I appreciate it violates the rule/suggestion of one app instance per container but that is just a suggestion :) All info welcome.
Regards
John
When running an application on Cloud Foundry that spawns child processes, the number one thing you need to watch out for is memory consumption. You set a memory limit when you push your application which is for the entire container. That includes the parent process, whatever child processes are spawned and a little overhead (currently init process, sshd process & health check).
Why is this a problem? Most buildpacks make the assumption that only one process will be running and that it will consume as much memory as possible while staying under the defined memory limit. They attempt to configure the software which is running your application to do this. When you spawn child processes, this breaks the buildpack's assumptions and can create scenarios where your application will exceed the defined memory limit. When this happens, even by one byte, the process will be killed and restarted.
If you're concerned with scaling your application, you should not try to spin off child processes in one extra large container. Instead, let the platform help you and scale up the number of application instances. The platform can easily do this and by using multiple smaller containers you can scale just as well. In fact, if you already have a 12-factor app, it should be well positioned to work in this manner.
Good luck!
I use gunicorn for my flask app. So far I have been following the guideline so that the numbers of gunicorn workers per machine is (2 x $num_cores) + 1 (they sit behind a load balancer). I am in the process of dockerising the app and my question is: Should I run one gunicorn worker per container (again, the dockerised is load balanced)? Or is there any point in running multiple gunicorn workers per docker container?
It entirely depends on what type of workers you use and the specifics of your app. Some apps will work well with a large number of processes, some with threads, some with a combination of both, some with gevent coroutines, etc. It's down to you to experiment and test your app under load to determine what works well.
Typically, Python will perform better with processes than threads, and the typical number of processes is either the number of cores or 2x + 1. Gevent coroutines only run within one process (and core) but don't have a limit beyond memory requirements.
I'm using Heroku to host a django application, and I'm using Waitress as my web server.
I run 2 (x2) dynos, And I see in New Relic instance tab that I have 10 instances running.
I was wondering How does Heroku determines the number of web server processes to run on one Dyno when using Waitress?
I know that when using Gunicorn there is a way to set the number of proccess per dyno, but didn't see any way to define it in Waitress.
Thanks!
In Waitress, there is a master process and (by default) 4 worker threads or processes. You can change this if you wish. Here is the docs for these options for waitress-serve
http://waitress.readthedocs.org/en/latest/runner.html#runner
--threads=INT
Number of threads used to process application logic, default is 4.
So if you have 2 dynos, and 5 (4+1) threads on each, then the total would come to 10 instances for this app in the RPM dashboard.
One can add more processes to the dynos as the maximum supported on Heroku 2x dynos is much higher:
2X dynos support no more than 512
https://devcenter.heroku.com/articles/dynos#process-thread-limits
But, you may want to check out some discussion on tuning this vs Gunicorn:
Waitress differs, in that it has an async master process that buffers the
entire client body before passing it onto a sync worker. Thus, the server
is resilient to slow clients, but is also guaranteed to process a maximum
of (default) 4 requests at once. This saves the database from overload, and
makes scaling services much more predictable.
Because waitress has no external dependencies, it also keeps the heroku
slug size smaller.
https://discussion.heroku.com/t/waitress-vs-gunicorn-for-docs/33
So after talking to the New relic support they clarified the issue.
Apparently only processes are counted in the instances tab (threads do not count).
in my Procfile I am also monitoring RabbitMQ workers which add instances to the instance tab, and hence the mismatch.
To quote their answer :
I clarified with our developers how exactly we measure instances for the Instances tab. The Python agent views each monitored process as one instance. Threads do not count as additional instances. I noticed that you're monitoring not only your django/waitress app, but also some background tasks. It looks like the background tasks plus the django processes are adding up to that total of 10 processes being monitored.
After reading a lot of blogposts, I decided to switch from crontab to Celery for my middle-scale Django project. I have a few things I didn't understand:
1- I'm planning to start a micro EC2 instance which will be dedicated to RabbitMQ, would this be sufficient for a small-to-medium heavy tasking? (Such as dispatching periodical e-mails to Amazon SES).
2- Computing of tasks, does compution of tasks occur on the Django server or the rabbitMQ server (assuming the rabbitMQ is on a seperate server)?
3- When I need to grow my system and have 2 or more application servers behind a load balancer, do these two celery machines need to connect to the same rabbitMQ vhost? Assuming application servers are the carbon copy and tasks are same and everything is sync on the database level.
I don't know the answer to this question, but you can definitely configure it to be suitable (e.g. use -c1 for a single process worker to avoid using much memory, or eventlet/gevent pools), see also the --autoscale option. The choice of broker transport also matters here, the ones that are not polling are more CPU effective (rabbitmq/redis/beanstalk).
Computing happens on the workers, the broker is only responsible for accepting, routing and delivering messages (and persisting messages to disk when necessary).
To add additional workers these should indeed connect to the same virtual host. You would
only use separate virtual hosts if you would want applications to have separate message buses.