I cannot access Jupyterlab by web interface (error 524). It still works by ssh. I've followed the support documentations, but nothing works.
My best guess is that the main issue is with the opened ports of docker.
The key problem is probably below:
curl http://127.0.0.1:8080/api/kernelspecs
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused
And the following command simply restarts the service without error (but still inaccessible through web interface)
sudo service jupyter restart
Thanks!
EDIT: to clarify, all help from this article which specifically is supposed to fix error 524, doesn't work at all.
The diagnostic tool give this result, and the --repair doesn't work:
And "Verify that the Jupyter internal API is active" is completely useless as it doesn't explain how to fix the error!!
So I know there is a problem with the Jupyter internal API but no idea how to fix that.
EDIT 2:
On the web console, here is a screenshot:
I have gone through the same error, after upgrading the VM problem got solved all the Jupyter API are healthy try upgrading the VM. Before that take a snapshot of disk(upgrading might erase your VM).
How to upgrade the VM
As I mentioned in the comment a work around to resolve the issue can be by create a new instance keeping the old data.For this you can follow below steps:
Step 1: Create a new storage bucket and a new notebook.
Step 2: Copy the data to the newly created bucket by running the following command in the old notebook terminal.
"gsutil cp -R /home/jupyter/* gs://NEW_STORAGE_BUCKET_PATH"
Step 3: From the new managed notebookâs terminal, run the below command to copy the data to this new notebook .
"gsutil cp gs://NEW_STORAGE_BUCKET_PATH* /home/jupyter/"
I'm trying to run a Pyspark Streaming program on GCP Dataproc. I pip install mmh3 in ssh already, running pyspark then type import mmh3 caused no problem. But when I started running sc.start() and send info over from another ssh terminal, it starts saying the module not found. Any idea why this happened or how to fix it? Thanks.
By installing the package via SSH, you're just install it on the "driver" node. You'll need to install the package for the whole cluster (i.e. all worker nodes) as well. Try following the documentation
Does anyone have experience with debugging Pyspark that runs on AWS EMR using Pycharm?
I couldn't find any good guides or existing threads regrading this one.
I know how to debug Scala-Spark with Intellij against EMR but I have no experince with doing this with Python.
I am aware of being able to connect to remote server using ssh (EMR Master) and maybe with Professional edition I can use the remote deployment feature to run my spark job using Pycharm but I'm not sure if it will work and I want to know if anyone has tried it, before I will go with Pycharm Pro.
I got to debug Pyspark on EMR as I wanted to.
Please look at this Medium blog post that describes how to do so:
https://medium.com/explorium-ai/debugging-pyspark-with-pycharm-and-aws-emr-d50f90077c92
It describes how to use Pycharm Pro - remote deployment feature in order to debug your pyspark program.
I have a website and I want to send a backup file to a my Google VM server on Google Cloud.
At first I thought of using php and curl script but a minute before I started working on it I saw on google site that it is possible to send a file to google vm by gcloud I thought it was a good and simple solution, but when trying to work according to the directory I encountered a strange problem that I found no Answer.
When I try to run the following code:
gcloud compute scp /root/mivzakim.php jtube-get-youtube-1:~
I get this error:
ERROR: (gcloud.compute.start-iap-tunnel) Python version 2.7.6 does not support SSL/TLS SNI needed for certificate verification on WebSocket connection.
ssh_exchange_identification: Connection closed by remote host
lost connection
ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [ 1].
I couldn't understand the connection to Python and I couldn't quite understand the meaning of this error,
if anyone could help me, he would be blessed :) Thank you!
Unable to deploy my app as I started getting below error since today morning.
I have tried gcloud info --run-diagnostics and gcloud components reinstall without much help.
I tried to deploy it using the old Google App Engine Launcher for Windows but faced the same error.
Earlier it worked till yesterday night (IST) using gcloud. Please help!
I am on latest gcloud sdk and have updated all its components. I use Win10. I tried rebooting my laptop as well.
C:\gaurav\coding\python\myapp\myapp\dist>gcloud app deploy --project=myproject --version 1 --verbosity=info ./app.yaml
INFO: Refreshing access_token
ERROR: gcloud crashed (SSLHandshakeError): [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
If you would like to report this issue, please run the following command:
gcloud feedback
To check gcloud for common problems, please run the following command:
gcloud info --run-diagnostics
C:\gaurav\coding\python\myapp\myapp\dist>
Diagnostics Output.
C:\gaurav\coding\python\myapp\myapp\dist> gcloud info --run-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
ERROR: Reachability Check failed.
Cannot reach https://accounts.google.com (SSLHandshakeError)
Cannot reach https://cloudresourcemanager.googleapis.com/v1beta1/projects (SSLHandshakeError)
Cannot reach https://www.googleapis.com/auth/cloud-platform (SSLHandshakeError)
Network connection problems may be due to proxy or firewall settings.
Do you have a network proxy you would like to set in gcloud (Y/n)? n
ERROR: Network diagnostic (0/1 checks) failed.
C:\gaurav\coding\python\myapp\myapp\dist>
Although gcloud info --run-diagnostics complains that the three URLs are not reachable. I am able to open them from web browser.
I found that when using Fiddler (for viewing network traffic) and have decrypting https traffic enabled, then I received the SSLHandshakeError.
Stopping the tool (or choosing not to decrypt https traffic) and then running the gcloud resulted in success.
According to the comments, also a problem with other web debugging proxies such as Charles.
A problem in recent GAE and GCloud SDK versions is the presence of invalid SSH certificates, see, for example, Google App Engine SSL Certificate Error and issue 38338974.
You could try to use my suggested solution in the above-mentioned post and replace your SDK's certificate file with a valid one (will have to locate a good one for the gcloud SDK, my answer was for the GAE SDK).
You might also be able to use the gcloud config command to set the core custom_ca_certs_file configurable property to point to a file with up to date certificates, if you have one. I didn't try it, YMMV.
Upgrading to Python 2.7.9 on MacOS High Sierra solved the issue for me.
I had this issue upon install of the google cloud SDK on MacOS Mojave. I am not behind a corporate proxy, and all the answers on the web seemed to indicate that this was the issue. I noticed in the install.sh script that it takes an environment variable CLOUDSDK_PYTHON for the python executable. So, I fixed this by exporting the path to my python 3 executable.
In my case:
export CLOUDSDK_PYTHON=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6
The install worked as expected after this.
On MacOS Catalina, the solution was to completely uninstall gcloud and reinstall it.
I had the same issue, downloaded the root/intermediate cert from one of the google url you get when you run the command: gcloud info --run-diagnostics and append to the cacerts.txt file that is being used. In my case it was the following one: google-cloud-sdk/lib/third_party/httplib2/python2/httplib2/cacerts.txt
For me it was a conflict in the python versions. gcloud was calling a different version. The solution was to set CLOUDSDK_PYTHON to point to the correct python (python2 in this case).
I tried to deploy gcloud and I was getting this error. Here is how I fixed it:
Access to Google cloud using Firefox
Download certificate. pem file and chain file
Append them on cert.pem located at:
C:\Program Files\google-cloud-sdk\lib\third_party\certifi\cert.pem
(using this you are creating custom CA cert file based on your proxy)
Update cert file as mentioned here: https://cloud.google.com/sdk/gcloud/reference/config/set
(Absolute path to a custom CA cert file.)
Make sure you have cert.pem at the end of path.
I fixed the problem by installing gcloud from apt-get. The guide is at this link