Docker with tcp:// and --tls in WebStorm IDE - webstorm

I'd like to use the WebStorm IDE (Windows) to manage my Docker images. For the local installation it works perfectly fine, but I have problems with my secured instance.
I can connect with the following command:
docker -H tcp://<<URL>>:<<PORT>> --tls --tlskey <<path_to_key>>/key.pem --tlscert <<path_to_cert>>/cert.pem info
so I guess I just need the option to enable tls.
Does WebStorm or IntelliJ have this option?

As of 2019.1 (maybe earlier), it is possible (but appearently undocumented) to connect to a TLS-enabled docker daemon. Simply use https://<host>:2376 instead of tcp://<host>:2376 and make sure you either set the certificate directory or have your certificates in ~/docker/. The files must be named ca.pem, cert.pem and key.pem.
Update: As I also just found out, in 2019.1 remote docker interpreters only work with Dockerfiles, not with docker-compose.

There is no such option in PhpStorm 2017.3 EAP #PS-173.3415.21 (which should be pretty much the same), and also the docs for 2017.2 state, that you need to either use Unix socket or tpc without tls (https://www.jetbrains.com/help/phpstorm/docker.html#docker_settings).

Related

AWS Docker Interpreter with Pycharm

I'm having difficulty to set this up correctly, and burning through AWS server time while I try to make it work. I have segmentation code that is heavily memory intensive that I'd like to temporarily spin up an AWS server with 192GB of ram. I understand that this is possible using docker, but the instructions on pycharm are non-existent with respect to the docker instructions necessary to tie it together (it references existing code as opposed to showing how to assemble it from scratch). What would the docker run command on the server look like to enable a connection to the 2375 port?
EDIT: I am using Pycharm Professional
UPD: Checking PyCharm options I found that there is an option to use Docker Machines. This seem to be exactly what you need. With Docker Machines you can make Docker spin up an EC2 instance for you with proper security out-of-the-box. Read official documentation on how to get started here and AWS driver options to learn how to set EC2 instance type, AMI, and other options here .
Original post:
To enable this feature you have to run Docker daemon with '-H' option:
sudo dockerd -H tcp://0.0.0.0:2375
You may read more on that in the Docker docs: https://docs.docker.com/engine/reference/commandline/dockerd/ .
Beware though, for EC2 you may also need to open that port using security group https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/authorizing-access-to-an-instance.html .
I also want to add that what you want to achieve isn't good from security perspective. Exposing docker socket like that is like an invitation for bad guys to throw a party at your EC2 instance. But since you mentioned that this is temporary...

Get err_connection_refused accessing django running on wsl2 from Windows but can curl from Windows terminal

I got the err_connection_refused when trying to accessing django running on wsl2 (http://localhost:8000) from Windows but when I use curl http://localhost:8000 from Windows terminal bash, it's working fine. I have tried to add a new firewall inbound rule for port 8000 but it's still not working. Is there anything else I need to take care of.
Thanks a lot
Seems like a forwarding problem. WSL2's interface is NAT'd, whereas WSL1 was bridged by default. WSL seems to do some "auto-forwarding" of ports, but only on localhost. However, sometimes this auto-forwarding mechanism seems to "break down". The main culprit seems to be hibernation or Windows Fast Startup (which are both closely-related features).
Does the problem resolve if you do a wsl --shutdown and then restart the WSL2 session? If so, try disabling Windows' Fast Startup. I already had Fast Startup disabled due to a different (non-WSL issue) on my system, so that could be related to why I am not able to reproduce.
Along the same lines, do you Hibernate instead of powering off? In that case, a wsl --shutdown might resolve as well.
For future readers, note that the above two points seem to resolve the issue for most people who have upvoted and responded in the comments. However, if that does not work for you, the following were my original "additional suggestions":
For some additional ideas, see this github issue. There are some suggestions on services that might be needed. (Side question - Are you running Windows Home or Professional?)
Is there any chance that your Windows hosts file (e.g. c:\windows\system32\drivers\etc\hosts) points localhost to an IP other than 127.0.0.1? If I attempt to access via my local Windows IP address, rather than 127.0.0.1 or localhost, I get an ERR_CONNECTION_REFUSED as well.
Since you were looking at the firewall rules, maybe look at a forwarding rule instead of just an inbound allow?
If all else fails, try exporting/backing up the WSL2 session (see wsl --export), then import it in as a new WSL1 session. See if it works there.
On my WSL2/Ubuntu 20.04 system, I attempted to reproduce (but haven't been able to yet) with the following steps:
mkdir -p ~/src/dj-test
cd ~/src/dj-test
python3 -m venv dj
source dj/bin/activate
pip install Django
django-admin startproject config .
python manage.py runserver
(although I used activate.fish since I'm running the fish shell)
From Vivaldi web browser in Windows, accessed localhost:8000, which returned "The install worked successfully! Congratulations! ..."
curl under Powershell Core worked as well.

VS Code integration with C++ development-environment inside Docker

I would like to run VSCode on my host machine, but (using its features / extensions) fire up tools from within the dev-env living inside my Docker container.
I have set up a docker image as a development environment for C++. Let's call it dev-env.
It is linux-based and contains required libraries, crosscompilation toolchains and various tools we use for building and testing our software (cmake, ninja, cppcheck, clang-tidy etc.)
I have a GIT repository on a host machine, which I mount inside a docker.
So my usual workflow would be to run docker:
host$
host$ docker run -v path/to/my/codebase/on/host:path/inside/docker -h dev-env --rm -it image_name bash
docker#
docker# cd build; cmake ..
etc...
And as such, I can build, test and run my tools inside my unified development environment inside the docker.
Now, the goal is to take it out of the terminal to the world of IDE.
I happen to use VS Code.
On host machine, I open my codebase folder in VSCode. Since it's mapped inside the docker, any changes I make locally will be available inside dev-env as well.
But if I now run anything from VSCode (CMake configure, build, etc.) it will of course call the tools from within my host machine - which of course will not work, and is not what I want.
With tasks defined in tasks.json I could probably manage with having them run something like docker exec CONTAINER my_command
It gets more complicated with extensions:
What I would like is to have the e.g. VSCode CMake Tools extension configured in such a way, that when I run Cmake Configure (in a VSCode running on my host machine), it will actually run cmake commands from within Docker container, using cmake installed inside Docker, not from my host machine.
Temporary solution: Forwarding display through X / VNC
So Installing VSCode inside the Docker, running x/vnc server inside the Docker, exposing port and connecting to it from the host machine.
Yes, it is possible, I have it running here. But it has many limitations and problems, of which the most painful is the lag/delay.
This is bad solution in general, so I would strongly push for avoiding this.
Another solution that I can think about:
VSCode instance running as a server inside the docker.
VSCode instance on your host connecting to the server instance.
You do all the work inside your host VSCode, but anytime you run a command, it is executed by a server instance, which runs everything inside Docker.
I guess this would require support from VSCode (or maybe an extension).
VSCode Live Share extension is not made exactly for that, but it's functionalities might do the job. I have not tested it yet.

Is it possible to SSH in AWS instances using any IDEs such PYCHARM?

I am stuck in a technical issue on a project and I think you the forum could help me out.
I have an EC2 Instance Type:p2.xlarge running on AWS, I cloned a repository in this instance which requires pytorch and cuda dependencies(this point has been taken care of).
Now, The issue is that I wanna work & run this code-base(which is is AWS instance now) somehow in my local pyCHARM IDE. In short, I didn't have proper resources on my laptop to run the repository, so I have to run in an AWS instance but for debugging purposes the local IDE would be a great option.
Is it possible to do that?. In other words, we can do SSH into AWS instance and run code, but all will be done through command line, if we could SSH through PYCHARM and can see the code in AWS here in local machine within PYCHARM and change, debug or run it as it was local but actually it gets executed in the instance.
Please suggest a solution to it.
Thanks in advance.
EDIT-1:
After following, #Cromulent suggestion, I have arrived here
Setting the remote:
Upload happening within the local & remote repo.
I still didn't understand the requirement of syncing the local and remote folders, when I only want to open the remote folder in my PYCHARM IDE and work on it.
I think after this setup, I have to change the code in local copy and the PYCHARM will sync the code in remote copy. How will I be running(using resources-GPUs of the remote Instance, not my local machine.) the remote code in PYCHARM in this scenario, I am just syncing it, for running again I have to ssh through command line and run the script(This does not serve the purpose)?
EDIT-2:
After #Cromulent suggestions.
Actually, it did work, but still, I am not able to run the remote code locally.
I am getting the below error while running any remote script. If I run the same script using ssh in the terminal, the scripts run normally. I tried to fix the problem using this post on StackOverflow, but it didn't work too.
ssh://ubuntu#ec2-52-41-247-169.us-west-2.compute.amazonaws.com:22/home/ubuntu/anaconda3/bin/python -u <08ad9807-3477-4916-96ce-ba6155e3ff4c>/home/ubuntu/InsightProject/scripts/download_flownet2.py
/home/ubuntu/anaconda3/bin/python: can't open file '<08ad9807-3477-4916-96ce-ba6155e3ff4c>/home/ubuntu/InsightProject/scripts/download_flownet2.py': [Errno 2] No such file or directory
The below is the screenshot for the above problem:
PyCharm Professional supports remote Python interpreters (either the globally installed Python interpreter or a virtualenv). It works by creating an SSH connection to the server and then running the code on the remote host. The results are then displayed locally in PyCharm Professional. You can also do remote debugging as well.
You MUST be using the professional version of PyCharm though. The free community version does not support this feature.
You can find the documentation here:
https://www.jetbrains.com/help/pycharm/configuring-remote-interpreters-via-ssh.html
One more solution is to deploy a Jupyter Notebook on your remote server. Then you will be able to use it from PyCharm Professional Edition.
Don't forget to make rules for the jupyter ports (e.g. allow all 8888) in your AWS console and in your instance.
To configure a remote interpreter for your notebook do this (source):
Open the Jupyter Notebook page of the Settings/Preferences dialog.
On this page, select or clear the Markdown cells rendering enabled option, and specify the username and password. Note that for the
single-user notebooks these fields are optional - leave them blank.
Fill in the username (for JupyterHub) and password.
Click the link Configure remote interpreter. You'll find yourself at the Project Interpreter page.
Configure the remote interpreter, as described in the section Configuring Python Interpreter.
You will want to configure a remote interpreter.
I tried the above approach but it didn't work for me. I have edited my post so that I can get additional input from the community, but I didn't any after the first answer was posted.
My friend actually figured out a secondary way to fix the issue. He actually uses "NOMACHINE" on the local machine and open connection to the remote desktop. Then you can directly install PYCHARM in the remote machine and work in there. I hope this will help others.
The solution is in his blog post. (Thanks to Shaobo Guan)
Another solution would be to use VNC instead of NoMachine

Remote builds with IntelliJ

I am having the following problem:
a) I have a UNIX build environment set up on a remote server I can ssh into
b) I have IntelliJ on Windows
What I would like to do is to edit the files through a remote connection
(similar to the functionality that exists for Emacs) and issue shell commands
such as 'make' and running simple 'test-scripts' which exist on the server..
Are the functions to do this integrated into IntelliJ or will I need a plug-in?
There is a SSH plugin for Intellij IDEA. It is available in https://plugins.jetbrains.com/idea/plugin/1203-ssh
For anyone still looking for a way to edit files remotely while running Intellij locally, version 2016.3 Ultimate Edition has a Remote Server Configuration option. It allows to access files remotely using either FTP, SFTP or FTPS.
More details on how to configure the remote servers can be found in Intellij's Help Pages.