I’m very new to cloud and wanted to try using it for machine learning. I set up the notebook to 8vCPUs 30GB RAM, with a T4 GPU. Somehow it runs slower compared to when I would work on colab, even when just loading the libraries. I haven’t reached to training the actual dataset (~200k images) because the site would eventually just error 524. Is there a way to resolve this?
Related
I am planning to migrated from DigitalOcean(DO) to Google Cloud(GCP).
I have taken trial on GCP and hosted Django website on it, but it is running too slow (it take 12-15 seconds to open page). Same website running on DO is very fast(it take hardly 1 second to open page).
Website is hosted on Ubuntu 20 LTS (on DO & GCP) & Apache2 server
On GCP there is no user right now, for testing I am only one user and it is running too slow. I have 2CPU and 8GB memory on VM.
I am not able to find out the issue why it is running slow on GCP and running fast on DO?
Can someone please help to find out solution?
When comparing Google Cloud Platform performance with the local one you should keep in mind that deploying on GCP needs more time to import all the necessary libraries and set up the Django framework.
In general it doesn't make much sense to compare the performance on your local machine with the performance on GCE, as local machines are likely running a different OS than GCE
In addition to that there are various ways to optimize your application’s performance, as the following typical ones:
Scaling configuration, by setting up “min_idle_instances” to be kept running and ready to serve traffic.
Using Warm Up Requests to reduce request and response latency during when your app's code is being loaded to a newly created instance.
I came across PageSpeed Insights, which analyzes the content of a web page, then generates suggestions to make that page faster and could be handy
Heyho!
I'm trying to deploy a machine learning model on Google Cloud Run. On my machine it needs around 2,5 GB, but when I try to run it via GCR I always get an out or memory error even though I put a 4 GB limit.
I also tried it in a to 4 GB limited Virtual Machine and everything works perfectly.
Is there a big overhead I don't know about or is it a bug?
Thanks in advance!
Which service should I use for my Go program that runs ffmpeg in Google Cloud Platform?
The Go program downloads videos from my Google Cloud Storage and runs ffmpeg command to combine them into ~5 minutes mp4 file. I use this program only around 10 times in a month.
I thought of using a compute instance with GPU to get fast ffmpeg processing. However, because of infrequency of the invocation, I don’t want to run the pricey GPU instance for all the time. Cloud Run does not seem to support GPU.
What service would you choose for such workload?
As mentioned in the comments of the question, you can use Cloud Run and you would only pay for the time your code is running.
You can also use a Cloud Function. Here you have a Quickstart
I've run into a bug that seemingly has no explanation as to why its happening.
Running scripts that are in my mounted drive(mounted using gcsfuse) takes forever to run(almost a couple of minutes per command). However, any scripts that I run from outside the mount folder seems to work fine. I can also notice a significant lag in the cursor movement too. Using GCS is a must for me as my dataset is already uploaded there and trying to rsync it to the VM takes forever.
I would appreciate any help regarding this issue.
Here is the cloud monitoring dashboard for some metrics I measured.
I am uploading a file to my Google Cloud Platform VM using scp on Linux. However, after initially uploading it at a speed of 900 kb/s it quickly falls to 20kb/s. My internet upload speed should be around 20mbps. I wanted to upload an SQLite database clocking in at 20gb, but this is unfeasible at this point.
Right now I used 54 minutes to upload a 94 MB file. It cannot be that slow?
I had the same issue multiple times with GCP, the solution I use is to compress all my files, upload it to dropbox and then wget the file from there. The speeds should go back to normal.
This answer should help you ae well, though I don't know if your paticular issue is related to gcp , scp or both.
https://askubuntu.com/questions/760509/how-to-make-scp-go-faster