I am looking for a way to execute a script at instance launch in Google cloud platform similar to user data in AWS. I check 'Startup script' but it is executed at every boot. is there any way to achieve it?
Yes, accordingly to the documentation startup script runs on each boot and there's no option available to change this behavior:
Compute Engine lets you create and run your own startup scripts on
your virtual machine (VM) instances to perform automated tasks every
time your instance boots up. Startup scripts can perform actions such
as installing software, performing updates, turning on services, and
any other tasks defined in the script.
To solve this issue you can use this workaround:
Set up a flag, such as a file on the disk, when your startup script runs first time.
Check existence of this flag in your startup script and exit without any action if the flag exists.
Related
I can automatically shutdown a sagemaker instance as indicated here, by using lifecycle configuration
https://stackoverflow.com/questions/53609409/automatically-stop-sagemaker-notebook-instance-after-inactivity
Is there a way to achieve this programmatically, by means of any ''image terminal'' command?
By image terminal, it is meant the Linux shell that can be opened via ''Launcher'' in ''Sagemaker Studio''
My use case is large computational jobs, where the idle inactive time in the automatic solution would be quite expensive.
It would be useful to have a shutdown_instance(), to be added as last command in lengthy shell scripts
I am planning to use Azure VMSS for deploying a set of spring boot apps. I am planning to create a custom linux VM image with all the required softwares/utilities as well as the required directory structure and configure this image in VMSS. We use jenkins as CI/CD tool and Git as source code repo. What is the best way to build and deploy these spring boot apps on VMSS?
I think one way is to write a custom script extension which downloads code from Git repo and then starts these spring boot apps. I believe this script will then get executed every time a new VM is provisioned.
But what about cases where already multiple VMs are running on top of minimum scale instance count. I believe a manual restart will not trigger the CSE script to run on these already running VMs right?
Could anyone advise the best way to handle this?
Also once a VM is deallocated due to auto scale down, what is the best/cost optimal way to back up the log files from VM to storage (blob or file share)?
You could enable Automatically tear down virtual machines after every use in the organization settings/project setting >> agent pool >> VMSS agent pool >> settings. Then, a new VM instance is used for every job. After running a job, the VM will go offline and be reimaged before it picks up another job. The Custom Script Extension will be executed on every virtual machine in the scaleset immediately after it is created or reimaged. Here is the reference document: Create the scale set agent pool.
To back up the log files from VM, you could refer to Troubleshoot and support about related file path on the target virtual machine.
I would like to include an environment variable on a Google VM which is running a JupyterLab notebook - this variable needs to be present before the notebook is started.
So setting it in the terminal or in the notebook does not work.
I have also tried to modify the bashrc with no luck.
In order to have an environment variable set up on you Compute Engine instance from boot you might be interested in checking startup scripts.
Startup scripts are automated tasks that are performed when your instance boots up. To set them it can be done when creating the instance under the automation section; if the instance is already created accessing your instance details in the compute engine console and under custom metadata clicking on Add item.
Steps to create startup scripts can be found here and here.
If you mean google Colab, one solution is using python, for example:
import os
os.environ["BASE_DIR"]="/content/drive/MyDrive/"
Suppose, I have requirement to run startup scripts only on first boot of the vm. My vm can start/stop frequently and I don't want to run script in every boot. As per GCP doc startup scripts runs every time your instance boots up.
Startup scripts run on each boot. There is no "run once" setting/option.
I recommend setting a flag, such as a file on disk after a script runs. Then check for the existence of this flag at the start of your script and exit if it exists.
Depending on your requirement, perhaps a sysprep-specialize-script is what you're looking for?
Order of execution of Windows startup scripts
You can use multiple
startup scripts. Startup scripts stored locally or added directly
execute before startup scripts that are stored in Cloud Storage. The
type of file containing the script also impacts the order of
execution. The following table shows, based on the metadata key, the
order of execution of Windows startup scripts.
Metadata key
Order of execution
sysprep-specialize-script-ps1
First during the initial boot
sysprep-specialize-script-cmd
Second during the initial boot
sysprep-specialize-script-bat
Third during the initial boot
sysprep-specialize-script-url
Fourth during the initial boot
windows-startup-script-ps1
First during each boot after the initial boot
windows-startup-script-cmd
Second during each boot after the initial boot
windows-startup-script-bat
Third during each boot after the initial boot
windows-startup-script-url
Fourth during each boot after the initial boot
As you can see on the official doc of Cloud Foundry
https://docs.cloudfoundry.org/devguide/using-tasks.html
A task is an application or script whose code is included as part of a deployed application, but runs independently in its own container.
I'd like to know if there's a way to run command and manipulate files directly on the main container of an application without using a SSH connection or the manifest file.
Thanks
No. Tasks run in their own container, so they cannot affect other running tasks or running application instances. That's the designed behavior.
If you need to make changes to your application, you should look at one of the following:
Use a .profile script. This will let you execute operations prior to your application starting up. It runs for every application instance that is started (I do not believe it runs for tasks) so the operation will be consistently applied.
While not recommended, you can technically background a task through the .profile script that will continue running for the duration of your app. This is not recommended because there is no monitoring of this script and if it crashes it won't cause your app container to restart.
Integrate with your buildpack. Some buildpack's like the PHP buildpack provide hooks for you to integrate and add behavior to staging. For other buildpacks, you can fork the buildpack and make it do whatever you want. This includes changing the command that's returned by the buildpack which tells the platform how to execute your droplet and ultimately what runs in the container.
You can technically modify a running app instance with cf ssh, but I wouldn't recommend it. It's something you should really only do for troubleshooting or maybe testing. Definitely not production apps. If you feel like you need to modify a running app instance for some reason, I'd suggest looking at the reasons why and look for alternative ways of accomplishing your goals.