I am following tutorial how to run Jupyter notebook on Google Cloud Platform (https://towardsdatascience.com/running-jupyter-notebook-in-google-cloud-platform-in-15-min-61e16da34d52). I am stuck at "Step 8: Set up the VM server". I have created Jupyter configuration file by typing jupyter notebook
--generate-config
in SSH session. After checking if it was created with
ls ~/.jupyter/jupyter_notebook_config.py
I get message No such file or directory. I really don't understand what is going on. I have never created VM before and I am a biologist (who tries to become a data scientist, lost in IT terminology), all I want to do is merge my dataframes on the cloud as I am lacking memory in my laptop. Can you please help me?
Related
I cannot access Jupyterlab by web interface (error 524). It still works by ssh. I've followed the support documentations, but nothing works.
My best guess is that the main issue is with the opened ports of docker.
The key problem is probably below:
curl http://127.0.0.1:8080/api/kernelspecs
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused
And the following command simply restarts the service without error (but still inaccessible through web interface)
sudo service jupyter restart
Thanks!
EDIT: to clarify, all help from this article which specifically is supposed to fix error 524, doesn't work at all.
The diagnostic tool give this result, and the --repair doesn't work:
And "Verify that the Jupyter internal API is active" is completely useless as it doesn't explain how to fix the error!!
So I know there is a problem with the Jupyter internal API but no idea how to fix that.
EDIT 2:
On the web console, here is a screenshot:
I have gone through the same error, after upgrading the VM problem got solved all the Jupyter API are healthy try upgrading the VM. Before that take a snapshot of disk(upgrading might erase your VM).
How to upgrade the VM
As I mentioned in the comment a work around to resolve the issue can be by create a new instance keeping the old data.For this you can follow below steps:
Step 1: Create a new storage bucket and a new notebook.
Step 2: Copy the data to the newly created bucket by running the following command in the old notebook terminal.
"gsutil cp -R /home/jupyter/* gs://NEW_STORAGE_BUCKET_PATH"
Step 3: From the new managed notebook’s terminal, run the below command to copy the data to this new notebook .
"gsutil cp gs://NEW_STORAGE_BUCKET_PATH* /home/jupyter/"
Just started an EC2 instance (P3) and am trying to configure Jupyter on it. Followed the instructions here: https://docs.aws.amazon.com/dlami/latest/devguide/setup-jupyter.html
I am able to get the server up and running, but when I try to create a notebook, none of the kernels work since the kernelspecs can't be found. I thought this was odd since earlier in the log, it said that Jupyter "Found new kernels in environments". Here is what I'm seeing: Kernelspec error pic
While using Vertex AI notebook instance kernel on GCP, the notebook gets detached everytime my system sleeps.
How can I keep my notebook running even if my system shuts down?
The Jupyter community has discussed this issue for quite a while now. There is no fix as such but there is a workaround to buffer the output and then display it when the notebook is opened again.
This answer is adopted from the comment from this Stack thread. I’ve also seen this workaround being suggested in a Jupyter Github issue.
The workaround would be to install the “Screen” utility (terminal multiplexer) on the GCE instance where JupyterHub is hosted, launch a new terminal session from JupyterHub and execute the notebook using the below “nbconvert” command.
jupyter nbconvert --to notebook --inplace --execute /home/path/to/notebook.ipynb
This way the terminal session could be preserved even if the personal computer is shut down and allow it to be resumed with the screen -r command.
I am running a virtual machine in Google Cloud. I have installed the default Debian OS, and configured the desktop environment for remote connection, as explained here: https://cloud.google.com/solutions/chrome-desktop-remote-on-compute-engine
I have been able to connect to the instance via Chrome Remote Desktop, however periodically I have the problem that the Remote Desktop says the vm instance is online, however if I try to connect to it I get:
Looking at the Google Cloud console, the instance is clearly running. Normally if I restart the instance the problem is solved, however I have processes running on the instance that I do not want to stop.
UPDATE:
Following the advice from Serhii Rohoza I ran
sudo systemctl status chrome-remote-desktop
The status looked normal, listing:
Active: active (exited) since...
I then ran
sudo systemctl restart chrome-remote-desktop
and this solved the problem, I could log into remote desktop again, but it seemed the VM instance had restarted, which is a big problem since I am running processes on it that should not shut down. I guess this is a problem to send to Google Cloud Services support.
UPDATE 2:
I'm still running into this problem. I normally have a Jupyter Notebook running on the VM - this Notebook must keep running. When I saw the message saying that the remote computer is offline, I logged in via ssh and checked if the Jupyter Notebook is running:
jupyter notebook list
This returned:
http://localhost:8888/?token=9110bf40789971b5e252a272e9497039b4f3b45e506348df :: /home/qgenixtech
So the Notebook was running. I then ran:
sudo systemctl restart chrome-remote-desktop
and after that again:
jupyter notebook list
and then it shows no Notebooks running. So the restart command closed down the Notebook (and also all other open windows on the desktop).
UPDATE 3:
I spoke to a support technician at Google. The problem is on the Remote Desktop side, not the virtual machine. According to the technician this is a known problem, by he didn't have a solution for it. He referred me to these two links from Google Support:
https://support.google.com/chrome/thread/10213547?hl=en
https://support.google.com/chrome/thread/3333421?hl=en
The next option for me is to look at something like X2go
To solve your issue you should follow documentation Troubleshooting and check status of the Chrome Remote Desktop service with command:
sudo systemctl status chrome-remote-desktop
and check log messages at /tmp/chrome_remote_desktop_DATE_TIME_*.
To investigate why your VM instance was restarted you should look for some clues at the logs:
Go to Compute Engine -> VM instances -> click on NAME_OF_YOUR_VM -> find section Logs -> click on Stackdriver Logging. More information you can find in the documentation Viewing logs (Classic)
Go to Compute Engine -> VM instances -> click on NAME_OF_YOUR_VM -> find section Logs -> click on Serial port 1 (console). More information in the documentation Viewing Serial Port Output
You can contact with Google Cloud Support as well.
In addition, have a look at the documentation Setting instance availability policies.
same issue. when checking logs i see:
2021-01-05 14:29:38,319:INFO:Starting Xvfb on display :20
xdpyinfo: unable to open display ":20".
2021-01-05 14:29:40,837:INFO:X server is active.
restarting service or even VM doesn't work.
i need to delete connection on "client" and re-auth with /headless link
I'm trying to restart a Jupyter Lab server (not just the kernels) running in the background of an AWS SageMaker notebook instance. I have already tried the following:
Killing the server by it's process ID
pgrep doesn't show me the process
pkill can't find the process
ps aux shows the process ID as constantly changing
Stopping the server through jupyter notebook stop
I get an SSL error and nothing happens
The only thing I've been able to do is reboot the entire instance, which isn't a great option as it can take awhile to become available again.
Edit 1:
The main reason I am trying to do this is that after installing the tqdm package and trying to use tqdm.notebook in Jupyter Lab, in order for it to display correctly I need to enable/install notebook and lab extensions. In order for these to take effect the server then needs to be restarted.
Try this:
Left hand navbar, Commands
Navigate to the Help section on the popout menu
Reset Application State
Both classic Jupyter and Jupyter lab live within the same process.
sudo initctl restart jupyter-server --no-wait is what AWS suggest in https://forums.aws.amazon.com/thread.jspa?messageID=917594󠁚
Assuming it runs on port 8888:
jupyter lab stop 8888 && jupyter lab