I use shell or vps from google cloud, and every time I want to run a process in this shelll when I exit the process in the shell also stops or dies.
I have used the SCREEN and TMUX commands, when I get out of the shell for 20 minutes the process shuts down by itself.
can I get the solution from the masters?
To answer shortly to your question: it's not possible.
With more detail, you need to understand what is Cloud Shell: it's a micro VM that google start freely for you. You have a quota per week and the VM is restarted at least every 12h
Non-interactive usage: Cloud Shell is intended for interactive use only. Non-interactive sessions will be ended automatically after a warning. Note that Cloud Shell sessions are capped at 12 hours, after which sessions are automatically terminated. You can use a new session immediately after.
When you aren't in interaction with the Cloud Shell console, it stops automatically after 20 minutes. Why? To save resource and save money (on Google side), because you don't pay this computing resource.
You have more detail on Cloud Shell environment limitations in the documentation
Anyway, if you want to run a batch job, on a small VM like Cloud Shell, you can use the Free tier compute engine instance for this. (choose correctly the region, else you will pay!)
Related
I have a Cloud Run application built in Python and Docker. The application serves a dashboard that runs queries against data and displays visualizations and statistics. Currently, if I want the app to load quickly I have to set the minimum number of instances to a number greater than 0, I typically use 10. This is great for serving the app immediately, however it can become outdated. I would essentially like to be able to keep a minimum number of instances available to serve the app immediately, but I would like it if they would refresh, or shut down and start up, once every few hours or at least once a day. Is there a way to achieve this?
I have tried looking into Cloud Scheduler to somehow get the Cloud Run application to refresh on a schedule, but I was unclear on how to make the whole thing shut down and reload, especially without serving another revision.
I think about a design but I never tested it. Try to do that.
At startup, store in a global variable the time.Now(). You have the startup time
Implement a Healthcheck probe. The healthcheck answer OK (HTTP 200) if the NOW minus startup time (the global variable) is below X (1 hour for instance). Else it answer KO (HTTP 500)
Deploy your Cloud Run service with the new Health check feature
Like that, after X duration, the instance will autodeclare itself unhealthy and Cloud Run will evict it and create a new one.
It should work. Let me know, I'm interested in the result!
I get disconnect every now and then when running a piece of code in Jupyter Notebooks on Sagemaker. I usually just restart my notebook and run all the cells again. However, I want to know if there is a way to reconnect to my instance without having to lose my progress. At the minute, it shows that there is "No Kernel" at the bottom bar, but my file seems active in the kernel sessions tab. Can I recover my notebook's variables and contents? Also, is there a way to prevent future kernel disconnections?
Note that I reverted back to tornado = 5.1.1, which seems to decrease the number of disconnections, but it still happens every now and then.
Often, disconnections will be caused by inactivity because a job is running for a long time with no user input. If it's pre-processing that's taking a long time, you could increase the instance size of the processing job so that it executes faster, or increase the instance count. If you're using EMR, you can now run an EMR Spark query directly on the EMR cluster since December 2021:
https://aws.amazon.com/about-aws/whats-new/2021/12/amazon-sagemaker-studio-data-notebook-integration-emr/
There's a useful blog here https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/ which is helpful in getting you up and running.
Please let me know if you need more information, or vote for the answer if it's useful. :-)
For me a quick solution was to open a Terminal instead, save the notebook file as a Pytohn file, and run it from the terminal within Sagemaker.
I used the GCP cloudshell but didnt know that it had a limit to use...
I tried to start cloudshell but it says I have to wait until the limit reset.
Is there any ways to access to my files in cloudshell???
Please help me.
Cloud Shell inactivity: If you do not access Cloud Shell for 120 days, your home disk will be deleted. You will receive an email notification before its deletion and simply starting a session will prevent its removal. Please consider a different solution on Google Cloud storage for sensitive data you wish to store long term.
Non-interactive usage: Cloud Shell is intended for interactive use only. Non-interactive sessions will be ended automatically after a warning. Note that Cloud Shell sessions are capped at 12 hours, after which sessions are automatically terminated. You can use a new session immediately after.
Weekly usage: Cloud Shell also has weekly usage limits. If you reach your usage limit, you'll need to wait until the specified time (listed under Usage Quota, found under the three dots menu icon) before you can use Cloud Shell again.
Restoring a session after a service limit violation: If your session is terminated or cannot be established because you exceeded a service limit, Cloud Shell will display an error with a link to a form that allows you to appeal the limit violation. Click the feedback link and submit the form with more information about the tasks you were performing before your session was terminated.
Source: https://cloud.google.com/shell/docs/limitations
I am first time user for google cloud. I am trying to run a script that probably takes 15 hours to run. I set an instance in google compute engine to run tensorflow. The instance also has one GPU. After I ssh into the instance and set everything up, I started to run the script through the shell, but the script seems to stop automatically after 10 mins after I am inactive on my computer. When I click on that shell again, it would automatically reconnect me to it. Wondering what I should do so that it doesn't automatically stop.
I am really new to Google Cloud Shell, and I accidentally closed the tab for the shell...and I cannot find it now.
I know I need to click the highlighted button at the top of the console window to activate Google Cloud Shell, but it is now grey, and no shell is presented on the page (it should be at the bottom).
Can anyone help?
I observed the same issue.
I refreshed the console page ( CTRL + F5 ) and I was able to see the 'Activate Cloud Shell' button again !!
However, if above does not resolves the issue, some tips:
From google docs Google Cloud Shell Limitations
Weekly usage: Cloud Shell also has weekly usage limits. If you reach
your usage limit, you'll need to wait until the specified time (listed
under Usage Quota, found under the three dots menu icon) before you
can use Cloud Shell again.
Also, there are usage limits on this, check your email if you had violated any conditions on shell usage:
Warning: Violating the Terms of Service will result in Cloud Shell
being disabled for your account. This constitutes activity that
adversely impacts Google Cloud Platform services, other customers' or
their end users' use of services, or the Google network used to
provide these services. Coin mining and network scanning using Cloud
Shell are strictly prohibited.
I had the same issue here.
You can install the command line interface in your terminal to access Google Cloud Datalab.
Here you will find the quickstart to configure the environment.
Installing datalab component:
gcloud components install datalab
Connecting with your VM Instance:
datalab connect *instance-name*
Opening the initial page:
http://localhost:8081
See more:
Google Cloud Datalab - Quickstart
It works now. It seems that there was a service problem earlier today, so the page was not well functioning.