Let's say I would have one app with two instances.
I would change an environment variable (cf set-env) and not perform a cf restage.
Eventually one of the two instances would crash and restart. Would it take the new environment variable or the old?
In general if an instance crashes (say the app runs out of memory) and is restarted by Diego (the runtime that's actually running the container instances), the restarted instance will still have the environment variables it was originally "desired" (created) with.
If you explicitly cf restart or cf stop && cf start the app it will pickup the new environment variables which out needing to be restaged.
As user152468 said above, if the environment variables are used during the staging process you will need to cf restage the app for them to functionally take effect.
Edge Case Scenario
If the Diego runtime goes away/loses data for some catastrophic reason, the Cloud Controller will re-sync it and recreate the apps that are meant to be running. In this event the behavior is similar to a cf restart and the app will pick up the new environment variables. This is definitely uncommon, but would also count as a "crash" scenario.
EDIT:
After reading tcdowney's answer below, I tried it out. And tcdowney is right. When the app is restarted due to a crash, it will not pick up the new environment variable, so both of your app instances will share the same environment.
In contrast when you do a cf restart my-app then it will pick it up. Sorry for the confusion!
========================================================
It would take the new value of the environment variable. See here: https://docs.run.pivotal.io/devguide/deploy-apps/start-restart-restage.html
Only if the variable is relevant for the buildpack, you will need to restage.
Related
enter image description here
I deployed a Spring boot Java Application in CF in one Org and Space , Initially i deployed the App with the 2 instances which is described in manifest file later due to the requirement i increased the number of instances to 4 but this i gave in cf push command "cf push appname -i 4 ", in script task bamboo, everything was running as expected but on friday suddenly the instances got decreased to 2 no changes in manifest or script , i checked the logs "cf logs app-name recent and cf events appname" there is no clue for app crash may i know what might be the reason for thi weird behaviour , do i need to check somewhere else, if app restarts suddenly would it take the manifest file ? please help me on this
It is unclear if you're referring to the total instance count or the currently running instance count.
Running Instance Count
If you're currently running instance count does not match the total instance count, then there is some sort of problem with the instances. The app is crashing, you've scaled up/down recently and new instances couldn't be created, or perhaps your platform is undergoing maintenance and app instances are being shuffled around.
In all cases, you'd want to look at cf events and cf logs for clues. The event would tell you if it's an app crash or specific scaling event. If there's nothing in the event's output, it could be platform maintenance. That doesn't generate event entries.
Total Instance Count
If the total instance count does not match what you'd expect, then it was changed by someone or something. The total count is stored in Cloud Controller and would not change unless it was directed.
If your total instance count is changing unexpectedly look at the cf events output. This will include entries that modify the application instance count, like cf push and cf scale. It should also tell you what made the change (the actor), so that can help to track down how it's being changed.
I'm not totally sure this is happening here, but I have seen others hit by this. If you do a one-off cf push or cf scale, that change will stick on the server. However, if you are using a manifest.yml file that contains state, i.e. your old instance count. If you or someone else or your CI then goes and does a cf push with the manifest.yml file that has the old instance count, that will update the instance count on Cloud Controller back to the old instance count.
In short, if you do a one-off cf push or cf scale to change the instance count, you need to make sure your manifest.yml is kept up-to-date & that is checked into CI (if you're doing that).
Lastly, check if you have any autoscaling rules in place. Those could also obviously impact your instance count. If you change your total instance count, you'll probably also want to update your scaling rules.
We use Pivotal Cloud Foundry's YML file to set up the environment. Everything is fine. According to DEVOPS if we have to modify/create an environment variable, we have to modify YML and push the app again. I am wondering if it is possible to modify/create an environment variable while the PCF app is running. It will be really cool if it can be done without having to redeploy the app. If it can't be done, is it because of Java's way of handling the environment?
Thanks
Can we change/modify the environment in PCF (Pivotal Cloud Foundy) at Runtime?
Yes and no.
You can modify environment variables associated with an application while the application is running using the cf set-env (to set or update) and cf unset-env (to delete).
This will update the environment variable in Cloud Controller at the time you run the command. However, this will not update the environment variable inside of a running application container. In order for your application to see the change that you made, you must cf restart, cf restage or cf push.
This has nothing to do with language specifics (i.e. it doesn't matter what language you're using). It is a requirement because the container where your application is running gets created with a fixed set of environment variables. When those change, the container must be recreated. That said, even if the container could be changed at runtime, in Linux a process' environment variables cannot be updated externally at runtime either (there are technically some ways to do this, but it's really unlikely you'd do this in practice). The process itself should be restarted for environment variables to change.
If you want to have your configuration update at runtime, you can look at something like Spring Cloud Config server & its refresh capabilities. That said, it turns out that most applications and frameworks assume configuration is read once while the app is starting up, so your application would need to support changing the configuration you want to change at runtime as well.
I have a app which is pushed to cf using cf push. Now I have changed the one of the environment variable and then restage the app using cf restage. What I understand is when we do a restage it will again compile the droplet and build it same can be done with cf push again for the app. So what I want to know is the difference between the 2 commands and how internally cf handles this?
The difference is that one uploads files and one does not.
When you run cf push, the cli is going to take bits off your local file system, upload them, stage your app and if that succeeds, run your app. This is what you want if you have made changes to files in your app and want to deploy them.
When you cf restage, the cli is not going to upload anything. It will just restage your app, and if that succeeds, run the app. This is what you want if there are no app change or you do not have the app source code, yet you want to force the build packs to run again & restart your app using the new droplet.
When you cf restart, the cli won't upload or restage, it will just stop and start the app. This is the fastest option, but only works if you just need to pick up environment changes like a changed service, memory limit or environment variables. It's also good, if you just want to try and have your app placed onto a different Diego Cell.
If you are just making changes to environment variables, you can probably get away with a cf restart, unless those environment variables are being used by one of your buildpacks, like JBP_CONFIG_*.
Hope that helps!
Regarding
So what I want to know is the difference between the 2 commands
cf push internally does the following sequence
uploading : creating a package from your app
staging : creating a container image using the package
starting : starting the container as a long running process, based on the container image
Some use cases where we need to restage
To introduce buildpack changes : CF operators update the buildpacks for various reasons (one example: security patches) effectively updating the runtime dependencies which our apps use.
Introduction of new software components : It might be possible that we need to introduce new / additional components which were not foreseen at the time of initial push. An example can be : introduction of monitoring agent for your app
When the root file system is changed.
In all the 3 scenarios above, our application code is not changing but we need to re-create the container image. In these cases we need to restage. If we compare with cf push , in case of restage we skip the package creation step.
I created a credhub instance and stored my secrets using below cf command
cf create-service credhub default my-secrets -c "{\"secretMessage\":\"1234567\"}"
My spring boot app picks up the secretMessage value without any issue. And now i executed update-service command and rotated the secretMessage with the below command
cf update-service my-secrets -c "{\"secretMessage\":\"1234567-updated\"}"
But my spring-boot app isn't picking the updated one unless and untill I restart the app in PCF.
Is there any way to pick up the rotated-credentials without restart ?
But my spring-boot app isn't picking the updated one unless and untill I restart the app in PCF.
Is there any way to pick up the rotated-credentials without restart ?
No, that is the expected behavior.
https://docs.pivotal.io/credhub-service-broker/using.html#update
Your service instance is bound to your app. Like all service instances bound to an app, it's providing information through the VCAP_SERVICES environment variable. Environment variables, including VCAP_SERVICES, are only updated when you restart the app.
You might be thinking, wait my VCAP_SERVICES has placeholders from CredHub, shouldn't those automatically update? Unfortunately no. The Diego lifecycle is responsible for looking up CredHub place holders and resolving them. This runs prior to your application starting. It then puts the actual value into VCAP_SERVICES. This is how your app can see the actual value and not the place holders, while simultaneously not having any knowledge about CredHub.
As you can see on the official doc of Cloud Foundry
https://docs.cloudfoundry.org/devguide/using-tasks.html
A task is an application or script whose code is included as part of a deployed application, but runs independently in its own container.
I'd like to know if there's a way to run command and manipulate files directly on the main container of an application without using a SSH connection or the manifest file.
Thanks
No. Tasks run in their own container, so they cannot affect other running tasks or running application instances. That's the designed behavior.
If you need to make changes to your application, you should look at one of the following:
Use a .profile script. This will let you execute operations prior to your application starting up. It runs for every application instance that is started (I do not believe it runs for tasks) so the operation will be consistently applied.
While not recommended, you can technically background a task through the .profile script that will continue running for the duration of your app. This is not recommended because there is no monitoring of this script and if it crashes it won't cause your app container to restart.
Integrate with your buildpack. Some buildpack's like the PHP buildpack provide hooks for you to integrate and add behavior to staging. For other buildpacks, you can fork the buildpack and make it do whatever you want. This includes changing the command that's returned by the buildpack which tells the platform how to execute your droplet and ultimately what runs in the container.
You can technically modify a running app instance with cf ssh, but I wouldn't recommend it. It's something you should really only do for troubleshooting or maybe testing. Definitely not production apps. If you feel like you need to modify a running app instance for some reason, I'd suggest looking at the reasons why and look for alternative ways of accomplishing your goals.