I am working on a project in an EC2 isntance using jupyter notebook. It seems like jupyter notebook does not allow multiple ssh to its server at the same time, I have to log out if other people want to connect to jupyter notebook through the instance. Is it possible to make multiple access to jupyter notebook from the same instance?
Create/update jupyter_notebook_config.py file in ~/.jupyter folder with the below code. All jupyter notebook command calls post this file creation will infer configuration from the file.
from notebook.auth import passwd
c = get_config()
c.NotebookApp.open_browser=False
c.NotebookApp.ip='*' # Allows access from anywhere
c.NotebookApp.port=8885 # Jupyter runs in port 8885
c.NotebookApp.password=passwd('jupyter') # Password to access notebooks
Make sure you've opened 8885 (or any other port of your choice mentioned in the configuration file) allows connection requests from outside (update EC2 security groups appropriately). Not sure if you've tried this or what the issue at your end is but this works for me.
The juypter notebook is installed on a server in this case an EC2 machine.
Any number of people can SSH into this machine if they have the credentials using putty or some ssh client, this has no connection with jupyter notebook.
(assuming that the SSH port 22 is open for the other users and they are able to connect)
When you launch the jupyter notebook using the jupyter notebook command -> you start a local instance of the jupyter notebook on the default port (maybe 8888)
you will have a URL for this notebook interface and you can work on it.
Important to note -> This is a local instance of your notebook. It is not public and can only be accessed on your OS username as a localhost.
If other OS users run the jupyter notebook command they will get their local version of the notebook on a different port (maybe 8889 by default as the port number 8888 is already in use by you )
You can make your notebook public and then you will get a public URL for your notebook (serverip:8888 or the port you have specified)
This public link can be shared with others. Now multiple people have visibility to your notebook and can edit code in your notebook.
p.s -> for public notebooks the port in which you are running the notebook needs to accept connections from AWS end. This can be configured in AWS console under the security groups tab
Related
Let me give the necessary context:
Security group details for Instance:
Jupyter Notebook Config File Snapshot:
When I run jupyter notebook this is what I get
And I cant connect to the Instance as I am getting this error:
Not sure what I am missing out or doing wrong. Any help would be appreciated.
You can run it in a headless mode, without browser:
jupyter notebook --no-browser
Then you can access it from your local workstation. Preferably over ssh tunnel for security reasons, rather then directly.
GCP has finally released managed Jupyter notebooks. I would like to be able to interact with the notebook locally by connecting to it. Ie. i use PyCharm to connect to the externaly configured jupyter notebbok server by passing its URL & token param.
Question also applies to AWS Sagemaker notebooks.
AWS does not natively support SSH-ing into SageMaker notebook instances, but nothing really prevents you from setting up SSH yourself.
The only problem is that these instances do not get a public IP address, which means you have to either create a reverse proxy (with ngrok for example) or connect to it via bastion box.
Steps to make the ngrok solution work:
download ngrok with curl https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip > ngrok.zip
unzip ngrok.zip
create ngrok free account to get permissions for tcp tunnels
run ./ngrok authenticate with your token
start with ./ngrok tcp 22 > ngrok.log & (& will put it in the background)
logfile will contain the url so you know where to connect to
create ~/.ssh/authorized_keys file (on SageMaker) and paste your public key (likely ~/.ssh/id_rsa.pub from your computer)
ssh by calling ssh -p <port_from_ngrok_logfile> ec2-user#0.tcp.ngrok.com (or whatever host they assign to you, it;s going to be in the ngrok.log)
If you want to automate it, I suggest using lifecycle configuration scripts.
Another good trick is wrapping downloading, unzipping, authenticating and starting ngrok into some binary in /usr/bin so you can just call it from SageMaker console if it dies.
It's a little bit too long to explain completely how to automate it with lifecycle scripts, but I've written a detailed guide on https://biasandvariance.com/sagemaker-ssh-setup/.
On AWS, you can use AWS Glue to create a developer endpoint, and then you create the Sagemaker notebook from there. A developer endpoint gives you access to connect to your python or Scala spark REPL via ssh, and it also allows you to tunnel the connection and access from any other tool, including PyCharm.
For PyCharm professional we have even tighter integration, allowing you to SFTP files and debug remotely.
And if you need to install any dependencies on the notebook, apart from doing it directly on the notebook, you can always choose new>terminal and you will have a connection to that machine directly from your jupyter environment where you can install anything you want.
There is a way to SSH into a Sagemaker notebook instance without having to use a third party reverse proxy like ngrok, nor setup an EC2 bastion, nor using AWS Systems Manager, here is how you can do it.
Prerequisites
Use your own VPC and not the VPC managed by AWS/Sagemaker for the notebook instance
Configure an ingress rule in the security group of your notebook instance to allow SSH traffic (port 22 over TCP)
How to do it
Create a lifecycle script configuration that is executed when the instance starts
Add the following snippet inside the lifecycle script :
INSTANCE_IP=$(/sbin/ifconfig eth2 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
echo "SSH into the instance using : ssh ec2-user#$INSTANCE_IP" > ~ec2-user/SageMaker/ssh-instructions.txt
Add your public SSH key inside /home/ec2-user/.ssh/authorized_keys, either manually with the terminal of jupyterlab UI, or inside the lifecycle script above
When your users open the Jupyter interface, they will find the ssh-instructions.txt file which gives the host and command to use : ssh ec2-user#<INSTANCE_IP>
If you want to SSH from a local environment, you'll probably need to connect to your VPN that routes your traffic inside your VPC.
GCP's AI Platform Notebooks automatically creates a persistent URL which you can use to access your notebook. Is that what you were looking for?
Try using CreatePresignedNotebookInstanceUrl to access your notebook instance using an url.
I am a beginner for Amazon Ec2 and recently I successfully ssh to EC2 instance. yet when I tried to activate jupyter before ssh:
jupyter notebook --no-browser --port=8888
I get the message:
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8888/?token=????????????????????
I copied the URL as instructed to the browser (chrome and safari), but it did not work. How could I proceed to ssh jupyter notebook? Thanks!
I hope you just didn't copied the link as it is (locahost), it is running on ec2, not on your computer. So change the server name to IP address of your EC2 instance (assuming you allowed traffic on correct ports)
There are a few guides how to access jupyter notebooks on remote servers, e. g. see
https://jupyter-notebook.readthedocs.io/en/stable/public_server.html#notebook-public-server
If you are just playing along and don't care about security in this case, you may just update the binding IP in your jupyter_notebook_config.py :
c.NotebookApp.ip = '*'
You can start the jupyter server using the following command:-
jupyter notebook --ip=*
If you want to keep it running even after the terminal is closed then use:-
nohup jupyter notebook --ip=* > nohup_jupyter.out&
Remember to open the port 8888 in the AWS EC2 security group inbound to Anywhere (0.0.0.0/0, ::/0)
Then you can access jupyter using http://:8888
Hope this helps.
I am trying to open on my local browser the Jupyter notebook opened through the AWS instance.
I am using m4.2xlarge instance from the ami-d36386aa, Ireland region.
I then typed inside the instance command line:
jupyter notebook --no-browser --ip=xx.xx.xx.xx
Where the ip is the public ip from AWS.
run Jupyter notebook
given output:
http://ec2-xx-xxx-x-xxx.eu-west-1.compute.amazonaws.com:8888/?token=....
I copied the output from (2) to my browser but nothing is happening.
Any suggestions?
I am running Spark EMR on Amazon EC2 and I am trying to tunnel Jupyter Notebook and Zeppelin, so I can access them locally.
I tried running the below command with no success:
ssh -i ~/user.pem -ND 8157 hadoop#ec2-XX-XX-XXX-XX.compute-1.amazonaws.com
What exactly is tunnelling and how can I set it up so I can use Jupyter Notebook and Zeppelin on EMR?
Is there a way to I set up a basic configuration to make this work?
Many thanks.
Application ports like 8890, for Zeppelin on the master node, are not exposed outside of the cluster. So, if you are trying to access the notebook from your laptop, it will not work. SSH tunneling is a way to access these ports via SSH, securely. You are missing at least one step outlined in Set Up an SSH Tunnel to the Master Node Using Dynamic Port Forwarding. Specifically, "After the tunnel is active, configure a SOCKS proxy for your browser."