Running a Qt GUI in a docker container - c++

So, I have a C++ GUI based on Qt5 which I want to run from inside a Docker container.
When I try to start it with
docker run --rm -it my_image
this results in the error output
qt.qpa.xcb: could not connect to display localhost:10.0
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
So I searched for how to do this. I found GUI Qt Application in a docker container, and based on that called it with
QT_GRAPHICSSYSTEM="native" docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix my_image
which resulted in the same error.
Then I found Can you run GUI applications in a Docker container?.
The accepted answer in there seems to be specific to certain applications such as Firefox?
Scrolling further down I got a solution that tells me to set X11UseLocalhost no in sshd_config and then call it like
docker run -v $HOME:/hosthome:ro -e XAUTHORITY=/hosthome/.Xauthority -e DISPLAY=$(echo $DISPLAY | sed "s/^.*:/$(hostname -i):/") my_image
this produces a slight variation of the error above:
qt.qpa.xcb: could not connect to display 127.0.1.1:13.0
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.
Following another answer, I added ENV DISPLAY :0 to my Dockerfile and called it with
xhost +
XSOCK=/tmp/.X11-unix/X0
docker run -v $XSOCK:$XSOCK my_image
This time, the first line of my error was qt.qpa.xcb: could not connect to display :0.
Then I tried another answer, added
RUN export uid=0 gid=0 && \
mkdir -p /home/developer && \
echo "developer:x:${uid}:${gid}:Developer,,,:/home/developer:/bin/bash" >> /etc/passwd && \
echo "developer:x:${uid}:" >> /etc/group && \
mkdir /etc/sudoers.d/ && \
echo "developer ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/developer && \
chmod 0440 /etc/sudoers.d/developer && \
chown ${uid}:${gid} -R /home/developer
to my Dockerfile and called docker run -ti --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix my_image, again same error.
I also tried several of the ways described in http://wiki.ros.org/docker/Tutorials/GUI, same error.
Am I doing something wrong? Note that I'm working on a remote machine via SSH, with X11 forwarding turned on of course (and the application works just fine outside of Docker). Also note that what I write is a client-server-application, and the server part which needs no GUI elements but shares most of the source code works just fine from it's container.
I hope for a solution that doesn't require me to change the system as the reason I use Docker in the first place is for users of my application to get it running without much hassle.

You have multiple errors that are covering each other. First of all, make sure you have the correct libraries installed. If your docker image is debian-based, it usually looks like a line
RUN apt-get update && \
apt-get install -y libqt5gui5 && \
rm -rf /var/lib/apt/lists/*
ENV QT_DEBUG_PLUGINS=1
Note the environment variable QT_DEBUG_PLUGINS. This will make the output much more helpful and cite any missing libraries. In the now very verbose output, look for something like this:
Cannot load library /usr/local/lib/python3.9/site-packages/PySide2/Qt/plugins/platforms/libqxcb.so: (libxcb-icccm.so.4: cannot open shared object file: No such file or directory)
The bolded part will be the missing library file; you can find the package it's in with your distribution's package manager (e.g. dpkg -S libxcb-iccm.so.4 on debian).
Next, start docker like this (can be one line, separated for clarity):
docker run \
-e "DISPLAY=$DISPLAY" \
-v "$HOME/.Xauthority:/root/.Xauthority:ro" \
--network host \
YOUR_APP_HERE
Make sure to replace /root with the guest user's HOME directory.
Advanced graphics (e.g. games, 3D modelling etc.) may also require mounting of /dev with -v /dev:/dev. Note that this will give the guest root access to your hard disks though, so you may want to copy/recreate the devices in a more fine-grained way.

On host system allow X connection from docker xhost +local:root
And start your container
docker run -it \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
--name my_app \
--rm \
<your_image>

Related

CLion Full Remote Mode with FreeBSD as the remote host

Currently, the Full Remote Mode of CLion only supports Linux as a remote host OS. Is it possible to have a FreeBSD remote host?
Yes, you can!
However, note that I'm recalling these steps retrospectively, so probably I have missed one step or two. Should you encounter any problem, please feel free to leave a comment below.
Rent a FreeBSD server, of course :)
Update your system to the latest release. Otherwise, you may get weird errors like "libdl.so.1" not found when installing packages. The one I'm using is FreeBSD 12.0-RELEASE-p3.
Create a user account. Don't forget to make it a member of wheel, and uncomment the %wheel ALL=(ALL) ALL line in /usr/local/etc/sudoers.
Set up SSH. This step is especially tricky, because we need to use both public-key and password authentication.
Due to a known bug, in some cases, the remote host must use password authentication, or you'll get an error when setting up the toolchain. You can enable password authentication by setting PasswordAuthentication yes in /etc/ssh/sshd_config, followed by a sudo /etc/rc.d/sshd restart.
It appears that CLion synchronizes files between the local and remote host with rsync and SSH. For some reasons I cannot explain, this process will hang forever if the host server doesn't support passphrase-less SSH key login. Follow this answer to create an SSH key as an additional way of authentication.
CLion assumes the remote host OS to be Linux, so we must fix some incompatibilities between GNU/Linux and FreeBSD.
Install GNU utilities with sudo pkg install coreutils.
Rename the BSD utility stat with sudo mv /usr/bin/stat /usr/bin/_stat.
Create a "new" file /usr/bin/stat with the content in Snippet 1. This hack exploits the fact that CLion sets the environment variable JETBRAINS_REMOTE_RUN to 1 before running commands on the remote server.
Do sudo chmod a+x /usr/bin/stat to make it executable.
Again, rename the BSD utility ls with sudo mv /bin/ls /bin/_ls.
Create a "new" file /bin/ls with the content in Snippet 2, like before.
Lastly, sudo chmod a+x /bin/ls.
Install the dependencies with sudo pkg install rsync cmake gcc gdb gmake.
Now you can follow the official instructions, and connect to your shiny FreeBSD host!
Snippet 1
#!/bin/sh
if [ -z "$JETBRAINS_REMOTE_RUN" ]
then
exec "/usr/bin/_stat" "$#"
else
exec "/usr/local/bin/gnustat" "$#"
fi
Snippet 2
#!/bin/sh
if [ -z "$JETBRAINS_REMOTE_RUN" ]
then
exec "/bin/_ls" "$#"
else
exec "/usr/local/bin/gls" "$#"
fi
Additionally you need to fix one more incompatibility between GNU/Linux and FreeBSD.
Check gtar is installed if no pkg install gtar
Rename the BSD utility tar with mv /usr/bin/tar /usr/bin/_tar
Create a "new" file /usr/bin/tar with the content in Snippet 3, like before.
Lastly, sudo chmod a+x /usr/bin/tar
Snippet 3
#!/bin/sh
if [ -z "$JETBRAINS_REMOTE_RUN" ]
then
exec "/usr/bin/_tar" "$#"
else
exec "/usr/local/bin/gtar" "$#"
fi
Starting CLion 2020.1 the instruction regarding gnustat and "ls" is not relevant anymore. Because CLion 2020.1 includes the proper fixes in jsch-nio library (https://github.com/lucastheisen/jsch-nio/commit/410cf5cbb489114b5da38c7c05237f6417b9125b)
Starting CLion 2020.2 doesn't use tar --dereference option, so the instruction regarding gtar (gnutar) is also not relevant anymore.

Qt app UI elements randomly rendered as blank/black in Docker

I prepared Dockerfile to build Docker image of my Qt application. To run the application I use X - I enable access to X server (xhost +local:root), then I use the following command to run it:
docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" \
--volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" mindforger:latest mindforger
The problem is that some Qt UI elements (menu items, dialogs, ...) are rendered blank in black (randomly) - check the screenshot below:
I'm on Ubuntu 16.04.5 with Docker 18.06 and Qt 5.01.
I had the same problem, and I couldn't solve it formally. But we found an alternative workaround to not display the error:
In our case, we have four QComboBoxes in a window. Our problem was that after starting the app, the second (sometimes the first) combo box you clicked displayed a black popup. So what we did was initializing the window with two dummy combo boxes, calling the showPopup method, and then hiding both the popups and the combo boxes. So the user can't notice the error. I hope you can make something similar with your app.
I had the same problem and found this solution which you have to run after starting the docker container. Once the specific docker container is started, run the following command within the docker container.
export QT_GRAPHICSSYSTEM="native"
Once this is pasted and executed within the docker terminal session, run the desired QT application and this black box problem should go away. You could even paste this option inside your docker's .bashrc if you didn't want to run it every time manually.
None of the posted solutions solved the identical problem that I encountered. However, this did fix it:
QT_GRAPHICSSYSTEM=raster
In my case (Qt5 application) I resolved this by adding the param --shm-size 128M, mounting /dev/shm:/dev/shm should work too.
I ran into this problem trying to get Google Earth Pro to work in Docker -- several dialog boxes and a few menus would be black-on-black or random pixels.
That specific graphics issue was solved by adding --ipc host to the create (or run) command.
Here is the distilled create command that I used; I didn't have to disable or configure shm (and none of the shm suggestions I tried helped):
docker create \
--ipc host \
-e DISPLAY -e XAUTHORITY \
-h "$HOSTNAME" \
-u "$(id -u):$(id -g)" \
--device /dev/dri/card0 \
-v /dev/dri/card0:/dev/dri/card0 \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v "$XAUTHORITY:$XAUTHORITY" \
my-qt-app
A small break-down:
--ipc host - This is what fixed the "black menus/dialogs"
-v "$XAUTHORITY:$XAUTHORITY", -v /tmp/.X11-unix:/tmp/.X11-unix and -e XAUTHORITY/-e DISPLAY is required or the app doesn't launch and complains about not finding a display server and complete failure without the -v $XAUTHORITY:XAUTHORITY
-v /dev/card0:/dev/card0 (in my script, I find those somewhat naively using findfind /dev/dri -maxdepth 1 -type c) is required to avoid Mesa errors, however, the application still works (poorly).
-u ... My implementation "hides" docker from the user, so it creates the container as the user, sets it up with a containerized "fake" version of their user account and uses it to launch the app, which the docker container is configured to use when it runs it. ...avoiding root GUI apps.

Keep running golang project even if console closed

How to keep golang project running even if console (putty) is closed. I have REST API developed in golang and hosted on AWS and using putty to connect and run the project
following command are used to install and run the project ( myapi )
go install myapi
myapi
when I close putty it stops working.
You have a number of options to keep your process running. The easiest of which is to use the nohup command.
$ nohup myapi &
The above command should run your application and print the output to a file called nohup.out. This file will be located in the directory where you run the command. Another option is to use screen or tmux.
If you want to start running your project in a more production ready way, you should look into service managers like systemd.
You can use something like supervisord
Run your program as a non-privileged user and use the setcap utility to grant it the needed permissions.
For example, to allow binding to a low port number (like 80) run will need to run setcap once on the executable:
sudo setcap 'cap_net_bind_service=+ep' /opt/yourGoBinary
You may need to install setcap: sudo aptitude install libcap2-bin
Alternatively
Debian comes with a tool called start-stop-daemon which is a standard way for starting daemons in init.d scripts. It can also also put the process in background for you if the program does not do it on its own. Have a look at the --background option.
Use /etc/init.d/skeleton as the basis of your init script, but change the do_start() function as follows:
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
--background --exec $DAEMON --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
--background --exec $DAEMON -- $DAEMON_ARGS \
|| return 2
Also above added the --make-pidfile option which creates the PID file for you.
In case you need to switch to a different user in a secure way, there is also --chuid option.
On Ubuntu and RHEL/CentOS/SL 6.X the simplest way is to write an upstart job configuration file. Just put exec /usr/sbin/yourprogram in the /etc/init/yourprogram.conf configuration file. With upstart there is no need to force the program in background. Do not add expect fork or expect daemon which you need with traditional daemons. With upstart it is better if the process does not fork.

How to customize the docker run command on Elastic Beanstalk?

Here's the thing, I need to tell Docker to not containerize the container’s networking, because it needs to connect to a MongoDB that is inside a VPN (enterprise private DB).
There is a Docker command that let's me do exactly that: --net=host. Reference here.
So, for example, when running the container on my local machine, I will do something like:
docker run --rm -it --net=host [image-name]:[version] bash -il
And that command will do the trick. Thanks to that, I can connect to the "private" MongoDB.
So, my question is: Is there a way customize the docker run command of a Single Docker Environment on Elastic Beanstalk so I can add the --net=host?
I have tried using the container_commands into the config.yml file to add that instruction there, but I don't think that does what I need, here is a snippet:
container_commands:
00-test_command:
command: bundle exec thin --net=host
01-networking-fix:
command: "docker run --rm -it --net=host [image-name]:[version] bash -il"
I ended up fixing it with two container commands
container_commands:
00_fix_networking:
command: sed -i 's/docker run -d/docker run --net=host -d/' /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh
01_fix_docker_ip:
command: sed -i 's/server $EB_CONFIG_NGINX_UPSTREAM_IP/server localhost/' /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh
Update:
I also had to fix the Upstart script. Unfortunately, I didn't write down what I did because I didn't end up needing to alter the docker run command. You would do a files directive for (I think) /etc/init/docker. AWS edits the Nginx configuration in the same manner as in 01flip.sh in that file as well.
Explanation:
In the 64bit Amazon Linux 2015.03 v2.0.2 running Docker 1.7.1 platform version, the file you need to edit is /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh. This file is now far more complex than Samar's version so I didn't want to put the actual contents in there. However, the change is basically the same. There's the line that starts with
docker run -d
I fixed it with a container command:
container_commands:
00_fix_networking:
command: sed -i 's/docker run -d/docker run --net=host -d/' /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh
This successfully adds the --net=host argument but now there's another problem. The system ends up with an invalid Nginx directive. Using --net=host means that when you run docker inspect <container id> there is no IP address in the NetworkSettings. AWS uses this to create the server directive for Nginx and ends up generating server :<some port you chose> (before adding --net=host it would look like server <ip>:<port>). I needed to patch that file, too. It's generated in /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh.
01_fix_docker_ip:
command: sed -i 's/server $EB_CONFIG_NGINX_UPSTREAM_IP/server localhost/' /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh
While elastic beanstalk is generally well suited for applications that work with standard set of configurations, its difficult to customize and keep things updated along with the updates AWS provides to EB stacks. Having said that, I've done something like below which is a bit hacky but works fine.
files:
"/opt/elasticbeanstalk/hooks/appdeploy/pre/04run.sh":
mode: "000755"
owner: root
group: root
encoding: plain
content: |
#script content of original 04run.sh along with modification on docker run cmd
# eg. I injected multi-ports here
docker run -d \
"${EB_CONFIG_DOCKER_ENV_ARGS[#]}" \
"${EB_CONFIG_DOCKER_VOLUME_MOUNTS[#]}" \
"${EB_CONFIG_DOCKER_ENTRYPOINT_ARGS[#]}" \
"${PORT_ARGS[#]}" \
$EB_CONFIG_DOCKER_IMAGE_STAGING \
"${EB_CONFIG_DOCKER_COMMAND_ARGS[#]}" 2>&1 | tee /tmp/docker_run.log | tee $EB_CONFIG_DOCKER_STAGING_APP_FILE
This is not very neat, at least I have to make sure that it does not break with updates on elastic beanstalk. The above one is for docker 1.5 stack but you can do something similar with the version you're running.
Note that the latest version of the AWS stack (with Docker 1.7.1) has a slightly different pre-deploy setup. You'll need to update the file at the location: /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh
commands:
00001_add_privileged:
cwd: /tmp
command: 'sed -i "s/docker run -d/docker run --privileged -d/" /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh'
or, for example, if you want to pass args to your Docker image:
commands:
00001_modify_docker_run:
cwd: /tmp
command: 'sed -i "s/\$EB_CONFIG_DOCKER_IMAGE_STAGING/\$EB_CONFIG_DOCKER_IMAGE_STAGING -gzip -enable-url-source/" /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh'

Starting Mesos slave in Docker on Amazon Linux results in cgroup error

I'm taking a docker-compose setup that's working on my Mac, and trying to get it running on a couple of AWS EC2 instances. The Mesos master Docker container came up fine, as did Zookeeper/Marathon, but the Mesos slave is having trouble:
$ sudo docker run --name mesos-slave1 -p 5051:5051 \
-e "MESOS_LOG_DIR=/var/log" -e "MESOS_MASTER=zk://10.x.x.x:2181/mesos" \
-e "MESOS_HOSTNAME=172.17.42.1" -e "MESOS_PORT:5051" \
-e "MESOS_ISOLATOR=cgroups/cpu,cgroups/mem" -e "MESOS_CONTAINERIZERS=docker,mesos" \
-e "MESOS_EXECUTOR_REGISTRATION_TIMEOUT:5mins" \
redjack/mesos-slave:0.21.0
I0708 19:26:09.559125 1 logging.cpp:172] INFO level logging started!
I0708 19:26:09.569294 1 main.cpp:142] Build: 2014-11-22 05:29:57 by root
I0708 19:26:09.569327 1 main.cpp:144] Version: 0.21.0
I0708 19:26:09.569340 1 main.cpp:147] Git tag: 0.21.0
I0708 19:26:09.569350 1 main.cpp:151] Git SHA: ab8fa655d34e8e15a4290422df38a18db1c09b5b
Failed to create a containerizer: Could not create DockerContainerizer: Failed to find a mounted cgroups hierarchy for the 'cpu' subsystem; you probably need to mount cgroups manually!
I've done various searches, and have tried things like mounting /sys and similar approaches, with no luck. I've even tried running docker in privileged mode just as a sanity check. My docker-compose.yml specifies:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /usr/local/bin/docker:/usr/local/bin/docker
- /sys/fs/cgroup:/sys/fs/cgroup
And I started there, but mounting those on the EC2 instance doesn't work either. Since it works on my Mac, there's clearly some difference between OS X and Amazon Linux that's connected with the issue, but I haven't been able to determine a work-around. In case it's handy for OS identification purposes, on the EC2 instance it says:
$ uname -a
Linux ip-10-x-x-x 3.14.35-28.38.amzn1.x86_64 #1 SMP Wed Mar 11 22:50:37 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
I may end up just installing Mesos directly on the EC2 instance, but it would be very convenient to avoid that by using the Docker container, of course.
If anyone's stumbled across this and found a solution, please share! I'll post back if I figure something out.
I was just about to post this question when I found one more thing to try, and it looks like it did it, so I figured I'd go ahead and post anyway after putting the work in, and hopefully the info may save someone else the time I spent trying to figure this out. Looks like the key is to mount /cgroup as a volume, which is presumably a difference between OS X and Amazon Linux (aka Centos variant). The final docker incantation that seems to be working:
$ sudo docker run --privileged=true --name mesos-slave1 -p 5051:5051 \
-e "MESOS_LOG_DIR=/var/log" -e "MESOS_MASTER=zk://10.x.x.x:2181/mesos" \
-e "MESOS_HOSTNAME=172.17.42.1" -e "MESOS_PORT:5051" \
-e "MESOS_ISOLATOR=cgroups/cpu,cgroups/mem" \
-e "MESOS_CONTAINERIZERS=docker,mesos" \
-e "MESOS_EXECUTOR_REGISTRATION_TIMEOUT:5mins" \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /usr/local/bin/docker:/usr/local/bin/docker -v /sys:/sys \
-v /cgroup:/cgroup redjack/mesos-slave:0.21.0
I'm still experimenting so I can't say for sure whether the privileged mode is required, and whether some of the other volumes are really needed (the docker stuff). But if this saves someone else some time, that's great.