I have a app which is pushed to cf using cf push. Now I have changed the one of the environment variable and then restage the app using cf restage. What I understand is when we do a restage it will again compile the droplet and build it same can be done with cf push again for the app. So what I want to know is the difference between the 2 commands and how internally cf handles this?
The difference is that one uploads files and one does not.
When you run cf push, the cli is going to take bits off your local file system, upload them, stage your app and if that succeeds, run your app. This is what you want if you have made changes to files in your app and want to deploy them.
When you cf restage, the cli is not going to upload anything. It will just restage your app, and if that succeeds, run the app. This is what you want if there are no app change or you do not have the app source code, yet you want to force the build packs to run again & restart your app using the new droplet.
When you cf restart, the cli won't upload or restage, it will just stop and start the app. This is the fastest option, but only works if you just need to pick up environment changes like a changed service, memory limit or environment variables. It's also good, if you just want to try and have your app placed onto a different Diego Cell.
If you are just making changes to environment variables, you can probably get away with a cf restart, unless those environment variables are being used by one of your buildpacks, like JBP_CONFIG_*.
Hope that helps!
Regarding
So what I want to know is the difference between the 2 commands
cf push internally does the following sequence
uploading : creating a package from your app
staging : creating a container image using the package
starting : starting the container as a long running process, based on the container image
Some use cases where we need to restage
To introduce buildpack changes : CF operators update the buildpacks for various reasons (one example: security patches) effectively updating the runtime dependencies which our apps use.
Introduction of new software components : It might be possible that we need to introduce new / additional components which were not foreseen at the time of initial push. An example can be : introduction of monitoring agent for your app
When the root file system is changed.
In all the 3 scenarios above, our application code is not changing but we need to re-create the container image. In these cases we need to restage. If we compare with cf push , in case of restage we skip the package creation step.
Related
We use Pivotal Cloud Foundry's YML file to set up the environment. Everything is fine. According to DEVOPS if we have to modify/create an environment variable, we have to modify YML and push the app again. I am wondering if it is possible to modify/create an environment variable while the PCF app is running. It will be really cool if it can be done without having to redeploy the app. If it can't be done, is it because of Java's way of handling the environment?
Thanks
Can we change/modify the environment in PCF (Pivotal Cloud Foundy) at Runtime?
Yes and no.
You can modify environment variables associated with an application while the application is running using the cf set-env (to set or update) and cf unset-env (to delete).
This will update the environment variable in Cloud Controller at the time you run the command. However, this will not update the environment variable inside of a running application container. In order for your application to see the change that you made, you must cf restart, cf restage or cf push.
This has nothing to do with language specifics (i.e. it doesn't matter what language you're using). It is a requirement because the container where your application is running gets created with a fixed set of environment variables. When those change, the container must be recreated. That said, even if the container could be changed at runtime, in Linux a process' environment variables cannot be updated externally at runtime either (there are technically some ways to do this, but it's really unlikely you'd do this in practice). The process itself should be restarted for environment variables to change.
If you want to have your configuration update at runtime, you can look at something like Spring Cloud Config server & its refresh capabilities. That said, it turns out that most applications and frameworks assume configuration is read once while the app is starting up, so your application would need to support changing the configuration you want to change at runtime as well.
Let's say I would have one app with two instances.
I would change an environment variable (cf set-env) and not perform a cf restage.
Eventually one of the two instances would crash and restart. Would it take the new environment variable or the old?
In general if an instance crashes (say the app runs out of memory) and is restarted by Diego (the runtime that's actually running the container instances), the restarted instance will still have the environment variables it was originally "desired" (created) with.
If you explicitly cf restart or cf stop && cf start the app it will pickup the new environment variables which out needing to be restaged.
As user152468 said above, if the environment variables are used during the staging process you will need to cf restage the app for them to functionally take effect.
Edge Case Scenario
If the Diego runtime goes away/loses data for some catastrophic reason, the Cloud Controller will re-sync it and recreate the apps that are meant to be running. In this event the behavior is similar to a cf restart and the app will pick up the new environment variables. This is definitely uncommon, but would also count as a "crash" scenario.
EDIT:
After reading tcdowney's answer below, I tried it out. And tcdowney is right. When the app is restarted due to a crash, it will not pick up the new environment variable, so both of your app instances will share the same environment.
In contrast when you do a cf restart my-app then it will pick it up. Sorry for the confusion!
========================================================
It would take the new value of the environment variable. See here: https://docs.run.pivotal.io/devguide/deploy-apps/start-restart-restage.html
Only if the variable is relevant for the buildpack, you will need to restage.
I'm automating app deployment to cloud foundry. So in the start command, I do a db migration. What can happen is that the migration would fail and as the result, the app will be dead. Is there some predefined strategy that can be used to rollback to the last working deployment, or I should manually store the last working version, check for failure and in that case redeploy the stored version?
The typical strategy used to deploy apps on Cloud Foundry is blue/green. This generally works like this:
Push the new app under a new name & host, like my-app-new.
Test the app & make sure it works.
When your satisfied, change the route mapping from the old app to the new app.
Delete the old app & optionally rename the new app.
Step #3 is where the cut-over happens. Prior to that all traffic keeps flowing to the old app.
This is documented more here.
https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html
I'd say this often works well, but sometimes there are problems. Where this breaks down is with steps #1 & #2, if your app cannot have multiple instances of itself running or if migrations to your service are so different that the old app breaks when you update the database. It definitely helps if you keep this strategy in mind as you develop your app.
Aside from that, which has historically been the way to go, you could take a look at the new v3 API functionality. With v3, apps now retain multiple versions of a droplet. With this, you can rollback to a previous version of a droplet.
http://v3-apidocs.cloudfoundry.org/version/3.36.0/index.html#droplets
You can run cf v3-droplets to see the available droplets and cf v3-set-droplet to change the droplet being used.
That said, this will only rollback the droplet. It would not rollback a service like a database schema. If you need to do that, you'd need reverse migrations or perhaps even restore from a backup.
Hope that helps!
I work on very similar automation processes.
Daniel has explained the process very well. I think you're looking for the blue-green deployment methodology
1) Read up on blue green deploy here:
https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html
2) Look at this plugin or implement blue green deploy manually:
https://github.com/contraband/autopilot
3) Blue-green restage plugin (a nice to have, in case you need to restage the app but not cause any downtime to the clients):
https://github.com/orange-cloudfoundry/cf-plugin-bg-restage
It works by creating a temporary app, copying the env vars/routes/code from the working app to he temp app.
The temp app now accepts traffic while the original app is being restaged.
the traffic moves on to the original app after it is restaged and the temporary app is deleted.
is there a possibility that cloud foundry triggers an function if a new application was pushed to the platform.
I would like to trigger same internal functions like registration on the API gateway. I know that I can pull the information from events API https://apidocs.cloudfoundry.org/224/events/list_all_events.html. But, is it also possible by push?
The closest thing I can think of to what you're asking is the profile script.
https://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html#profile
The note about the Java buildpack not supporting .profile scripts is incorrect. It's a platform feature, so all buildpack's support them. The difference with Java apps is that you're probably pushing a JAR or WAR file so it's harder to make sure the file is placed in the correct location. Location of the file is everything.
When your application starts, the platform will first run the .profile script, if it exists, that is packaged with your application. It's a standard shell script and you can do whatever you like in this file.
The only caveat is that your application will not start until this script completes successfully (i.e. exit 0). Thus you have a limited amount of time for that script to run and your application to start. How much time, you ask? That is configured by cf push -t and is in seconds. You can also set it in your manifest.yml with the timeout attribute.
Time (in seconds) allowed to elapse between starting up an app and the first healthy response from the app
This is also something that each application needs to include. I suppose you could also use a custom buildpack to add that file, if you wanted to have it added across multiple applications. There's no easy way to add it for all apps though.
Hope that helps!
As you can see on the official doc of Cloud Foundry
https://docs.cloudfoundry.org/devguide/using-tasks.html
A task is an application or script whose code is included as part of a deployed application, but runs independently in its own container.
I'd like to know if there's a way to run command and manipulate files directly on the main container of an application without using a SSH connection or the manifest file.
Thanks
No. Tasks run in their own container, so they cannot affect other running tasks or running application instances. That's the designed behavior.
If you need to make changes to your application, you should look at one of the following:
Use a .profile script. This will let you execute operations prior to your application starting up. It runs for every application instance that is started (I do not believe it runs for tasks) so the operation will be consistently applied.
While not recommended, you can technically background a task through the .profile script that will continue running for the duration of your app. This is not recommended because there is no monitoring of this script and if it crashes it won't cause your app container to restart.
Integrate with your buildpack. Some buildpack's like the PHP buildpack provide hooks for you to integrate and add behavior to staging. For other buildpacks, you can fork the buildpack and make it do whatever you want. This includes changing the command that's returned by the buildpack which tells the platform how to execute your droplet and ultimately what runs in the container.
You can technically modify a running app instance with cf ssh, but I wouldn't recommend it. It's something you should really only do for troubleshooting or maybe testing. Definitely not production apps. If you feel like you need to modify a running app instance for some reason, I'd suggest looking at the reasons why and look for alternative ways of accomplishing your goals.