In a Dockerfile, the common way to copy a directory as a non-root user (e.g $UID 1000) is the following:
COPY --chown=1000:1000 /path/to/host/dir/ /path/to/container/dir
However, I want to use variables instead. For example
ARG USER_ID=1000
ARG GROUP_ID=1000
COPY --chown=${USER_ID}:${GROUP_ID} /path/to/host/dir/ /path/to/container/dir
But this is not possible. There exist a workaround?
Note I know that a possible workaround could be to copy the directory as root and then run chown on the directory (variables works fine with RUN). However, the size of the image will grow just for the use of chown in a separate command.
You can create a user before running the --chown;
mkdir -p test && cd test
mkdir -p path/to/host/dir/
touch path/to/host/dir/myfile
Create your Dockerfile:
FROM busybox
ARG USER_ID=1000
ARG GROUP_ID=1000
RUN addgroup -g ${GROUP_ID} mygroup \
&& adduser -D myuser -u ${USER_ID} -g myuser -G mygroup -s /bin/sh -h /
COPY --chown=myuser:mygroup /path/to/host/dir/ /path/to/container/dir
Build the image
docker build -t example .
Or build it with a custom UID/GID:
docker build -t example --build-arg USER_ID=1234 --build-arg GROUP_ID=2345 .
And verify that the file was chown'ed
docker run --rm example ls -la /path/to/container/dir
total 8
drwxr-xr-x 2 myuser mygroup 4096 Dec 22 16:08 .
drwxr-xr-x 3 root root 4096 Dec 22 16:08 ..
-rw-r--r-- 1 myuser mygroup 0 Dec 22 15:51 myfile
Verify that it has the correct uid/gid:
docker run --rm example ls -lan /path/to/container/dir
total 8
drwxr-xr-x 2 1234 2345 4096 Dec 22 16:08 .
drwxr-xr-x 3 0 0 4096 Dec 22 16:08 ..
-rw-r--r-- 1 1234 2345 0 Dec 22 15:51 myfile
Note: there is an open feature-request for adding this functionality:
issue #35018 "Allow COPY command's --chown to be dynamically populated via ENV or ARG"
In my case, I used my UID and GID numbers and it works as I do have the same non-root account in the DEV and PROD environments.
COPY --chown=1000:1000 /path/to/host/dir/ /path/to/container/dir
And you can find the user and group IDs with the linux command: id
There is a python project which is deployed by using virtualenv. Simplified version of the build script is next:
virtualenv --system-site-packages runtime
source ./runtime/bin/activate
pip install -r requirements.txt --index-url=...
deactivate
After build we'll end with some python binaries in runtime/bin/:
[....#... project1]$ ll runtime/bin/python*
lrwxrwxrwx 1 root root 9 лис 15 13:26 runtime/bin/python -> python2.7
lrwxrwxrwx 1 root root 9 лис 15 13:26 runtime/bin/python2 -> python2.7
-rwxr-xr-x 1 root root 4864 лис 15 13:26 runtime/bin/python2.7
-rwxr-xr-x 1 root root 2350 лис 15 13:26 runtime/bin/python-config
For convenience there is a symlink to runtime/bin/python2.7 in the other directory: bin; the bin directory located on the same level as runtime:
[...#... project1]$ ll
total 96
drwxr-xr-x 16 root root 4096 лис 15 19:27 bin
....
drwxr-xr-x 9 root root 4096 лис 15 13:26 runtime
...
[...#... project1]$ ll bin/python
lrwxrwxrwx 1 root root 24 лис 15 19:27 bin/python -> ../runtime/bin/python2.7
And here are the strange things I can see in sys.path from linked and original binaries:
[...#... project1]$ bin/python -c "import sys, pprint;pprint.pprint(sys.path)"
['',
'/media/ephemeral0/project1/src/project1_config',
'/media/ephemeral0/project1/src/project1.api.activities',
'/media/ephemeral0/project1/src/project1_comments',
'/media/ephemeral0/project1/src/project1_oauthorizer',
'/media/ephemeral0/project1/src/project1_social',
'/media/ephemeral0/project1/src/project1_bulk',
'/media/ephemeral0/project1/src/project1_core',
'/media/ephemeral0/project1/src/project1_tags',
'/media/ephemeral0/project1/src/project1_messages',
'/media/ephemeral0/project1/src/project1_sync',
'/media/ephemeral0/project1/src/project1_contacts',
'/media/ephemeral0/project1/src/project1_search',
'/media/ephemeral0/project1/src/project1_deals',
'/media/ephemeral0/project1/src/project1_wufoo',
'/media/ephemeral0/project1/src/project1_mailchimp',
'/media/ephemeral0/project1/src/project1_insights',
'/media/ephemeral0/project1/src/project1_marketing',
'/media/ephemeral0/project1/runtime/lib64/python27.zip',
'/usr/lib64/python2.7/site-packages/amqp-1.4.6-py2.7.egg',
'/usr/lib64/python2.7/site-packages/Pillow-2.6.1-py2.7-linux-x86_64.egg',
'/usr/lib/python2.7/site-packages/pyrasite-2.0-py2.7.egg',
'/usr/lib/python2.7/site-packages/pyOpenSSL-0.14-py2.7.egg',
'/usr/lib/python2.7/site-packages/cryptography-0.7.2-py2.7-linux-x86_64.egg',
'/usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg',
'/usr/lib/python2.7/site-packages/enum34-1.0.4-py2.7.egg',
'/usr/lib/python2.7/site-packages/pyasn1-0.1.7-py2.7.egg',
'/usr/lib/python2.7/site-packages/cffi-0.8.6-py2.7-linux-x86_64.egg',
'/usr/lib/python2.7/site-packages/pycparser-2.10-py2.7.egg',
'/media/ephemeral0/project1/runtime/lib64/python2.7',
'/media/ephemeral0/project1/runtime/lib64/python2.7/plat-linux2',
'/media/ephemeral0/project1/runtime/lib64/python2.7/lib-tk',
'/media/ephemeral0/project1/runtime/lib64/python2.7/lib-old',
'/media/ephemeral0/nimble/runtime/lib64/python2.7/lib-dynload',
'/usr/lib64/python2.7',
'/usr/lib/python2.7',
'/media/ephemeral0/project1/runtime/lib/python2.7/site-packages',
'/usr/lib64/python2.7/site-packages',
'/usr/lib/python2.7/site-packages']
[...#... project1]$ runtime/bin/python2.7 -c "import sys, pprint;pprint.pprint(sys.path)"
['',
'/media/ephemeral0/project1/src/project1_config',
'/media/ephemeral0/project1/src/project1.api.activities',
'/media/ephemeral0/project1/src/project1_comments',
'/media/ephemeral0/project1/src/project1_oauthorizer',
'/media/ephemeral0/project1/src/project1_social',
'/media/ephemeral0/project1/src/project1_bulk',
'/media/ephemeral0/project1/src/project1_core',
'/media/ephemeral0/project1/src/project1_tags',
'/media/ephemeral0/project1/src/project1_messages',
'/media/ephemeral0/project1/src/project1_sync',
'/media/ephemeral0/project1/src/project1_contacts',
'/media/ephemeral0/project1/src/project1_search',
'/media/ephemeral0/project1/src/project1_deals',
'/media/ephemeral0/project1/src/project1_wufoo',
'/media/ephemeral0/project1/src/project1_mailchimp',
'/media/ephemeral0/project1/src/project1_insights',
'/media/ephemeral0/project1/src/project1_marketing',
'/media/ephemeral0/project1/runtime/lib64/python27.zip',
'/media/ephemeral0/project1/runtime/lib64/python2.7',
'/media/ephemeral0/project1/runtime/lib64/python2.7/plat-linux2',
'/media/ephemeral0/project1/runtime/lib64/python2.7/lib-tk',
'/media/ephemeral0/project1/runtime/lib64/python2.7/lib-old',
'/media/ephemeral0/project1/runtime/lib64/python2.7/lib-dynload',
'/usr/lib64/python2.7',
'/usr/lib/python2.7',
'/media/ephemeral0/project1/runtime/lib/python2.7/site-packages',
'/usr/lib64/python2.7/site-packages/amqp-1.4.6-py2.7.egg',
'/usr/lib64/python2.7/site-packages/Pillow-2.6.1-py2.7-linux-x86_64.egg',
'/usr/lib/python2.7/site-packages/pyrasite-2.0-py2.7.egg',
'/usr/lib/python2.7/site-packages/pyOpenSSL-0.14-py2.7.egg',
'/usr/lib/python2.7/site-packages/cryptography-0.7.2-py2.7-linux-x86_64.egg',
'/usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg',
'/usr/lib/python2.7/site-packages/enum34-1.0.4-py2.7.egg',
'/usr/lib/python2.7/site-packages/pyasn1-0.1.7-py2.7.egg',
'/usr/lib/python2.7/site-packages/cffi-0.8.6-py2.7-linux-x86_64.egg',
'/usr/lib/python2.7/site-packages/pycparser-2.10-py2.7.egg',
'/usr/lib64/python2.7/site-packages',
'/usr/lib/python2.7/site-packages']
... order of the paths is different! For example: position of the '/media/ephemeral0/project1/runtime/lib/python2.7/site-packages',.
How is it possible? There are no .pth files in the bin (symlink's location) directory and I can not understand who is culprit.
I just created a VM vagrant with centos, installed python2.7 and pip using Miniconda, installed pymqi using pip, created a test python file to see if my pymqi installation is correct :
import pymqi
print "hello..."
but I got this :
[vagrant#localhost projects]$ python test.py
Traceback (most recent call last):
File "test.py", line 1, in <module>
import pymqi
File "/home/vagrant/miniconda2/lib/python2.7/site-packages/pymqi/__init__.py", line 109, in <module>
import pymqe, CMQC, CMQCFC, CMQXC
ImportError: libmqic_r.so: cannot open shared object file: No such file or directory
I looked for that file :
[vagrant#localhost projects]$ find /opt/mqm/ -name 'libmqic_r.so'
/opt/mqm/lib/compat/libmqic_r.so
/opt/mqm/lib/libmqic_r.so
/opt/mqm/lib64/compat/libmqic_r.so
/opt/mqm/lib64/libmqic_r.so
Thank you, your help is appreciated.
I found the solution :
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mqm/lib64
As a general rule, using the LD_LIBRARY_PATH variable is a bad practice. You'd better just create the appropriate symlink to the 64bit version of the shared objects.
For fome reason, when you install the IBM MQSeries Client, only 32bit mq libraries are linked into /usr/lib/:
[root#host ~]# ll /usr/lib/libmq*
lrwxrwxrwx 1 root root 26 Jan 25 12:49 /usr/lib/libmqicb_r.so -> /opt/mqm/lib/libmqicb_r.so
lrwxrwxrwx 1 root root 24 Jan 25 12:49 /usr/lib/libmqicb.so -> /opt/mqm/lib/libmqicb.so
lrwxrwxrwx 1 root root 25 Jan 25 12:49 /usr/lib/libmqic_r.so -> /opt/mqm/lib/libmqic_r.so
lrwxrwxrwx 1 root root 23 Jan 25 12:49 /usr/lib/libmqic.so -> /opt/mqm/lib/libmqic.so
lrwxrwxrwx 1 root root 25 Jan 25 12:49 /usr/lib/libmqiz_r.so -> /opt/mqm/lib/libmqiz_r.so
lrwxrwxrwx 1 root root 23 Jan 25 12:49 /usr/lib/libmqiz.so -> /opt/mqm/lib/libmqiz.so
lrwxrwxrwx 1 root root 25 Jan 25 12:49 /usr/lib/libmqjx_r.so -> /opt/mqm/lib/libmqjx_r.so
lrwxrwxrwx 1 root root 26 Jan 25 12:49 /usr/lib/libmqmcs_r.so -> /opt/mqm/lib/libmqmcs_r.so
lrwxrwxrwx 1 root root 24 Jan 25 12:49 /usr/lib/libmqmcs.so -> /opt/mqm/lib/libmqmcs.so
lrwxrwxrwx 1 root root 25 Jan 25 12:49 /usr/lib/libmqmzse.so -> /opt/mqm/lib/libmqmzse.so
While 64bit libs are not:
[root#host ~]# ll /usr/lib64/libmq*
ls: /usr/lib64/libmq*: No such file or directory
You can fix by just executing
[root#host ~]# ln -s /opt/mqm/lib64/libmq* /usr/lib64/
Please check if you have installed MQSeriesClient or else .so files is not in LIB path
I want to create a shell script, that iterates through folders and deletes folders that match [versionnumber-n] where n > 0
the version number is in a file that's content is like:
MAVEN_VERSION=1.2.7.0-SNAPSHOT
Here's an example:
The file listing is like
drwxrwxr-x 4 jenkins jenkins 4096 Jul 29 10:54 ./
drwxrwxr-x 20 jenkins jenkins 4096 Jul 4 09:20 ../
drwxr-xr-x 2 jenkins jenkins 4096 Jul 23 12:35 1.2.6.0-SNAPSHOT/
drwxr-xr-x 2 jenkins jenkins 4096 Jul 28 23:13 1.2.7.0-SNAPSHOT/
-rw-rw-r-- 1 jenkins jenkins 403 Jul 29 10:11 maven-metadata-local.xml
-rw-r--r-- 1 jenkins jenkins 403 Jul 28 23:13 maven-metadata-mtx-snapshots.xml
-rw-r--r-- 1 jenkins jenkins 40 Jul 28 23:13 maven-metadata-mtx-snapshots.xml.sha1
-rw-r--r-- 1 jenkins jenkins 403 Jul 28 23:13 maven-metadata.xml
-rw-r--r-- 1 jenkins jenkins 32 Jul 28 23:13 maven-metadata.xml.md5
-rw-r--r-- 1 jenkins jenkins 40 Jul 28 23:13 maven-metadata.xml.sha1
-rw-r--r-- 1 jenkins jenkins 186 Jul 28 23:13 resolver-status.properties
Where I want the script to delete the folder 1.2.6.0-SNAPSHOT/ but not 1.2.7.0-SNAPSHOT/. If there where folders like 1.2.5.0-SNAPSHOT/ 1.2.4.0-SNAPSHOT/ them too.
What I have at this point:
.*(?!1.2.7.0)(-SNAPSHOT)
Which unfortunately matches both folders (in the example above)
edit: just hit submit too early ...
With Bash you can just use negation with extended pathname expansion.
shopt -s extglob
rm -fr /dir/1.2.!(7).0-SNAPSHOT
Dry run example:
$ ls -1
1.2.10.0-SNAPSHOT
1.2.5.0-SNAPSHOT
1.2.6.0-SNAPSHOT
1.2.7.0-SNAPSHOT
a
$ echo rm -fr 1.2.!(7).0-SNAPSHOT
rm -fr 1.2.10.0-SNAPSHOT 1.2.5.0-SNAPSHOT 1.2.6.0-SNAPSHOT
See Extended Pattern Matching and Filename Expansion.
How I did it in the end:
if [ -z "$MAVEN_VERSION_SERVER" ]
then
echo "\$MAVEN_VERSION_SERVER NOT set! \n exiting ..."
else
find /var/lib/jenkins/.m2/repository/de/db/mtxbes -mindepth 1 -type d -regex '.*SNAPSHOT' -not -name $MAVEN_VERSION_SERVER | xargs -d '\n' rm -fr
fi
(the $MAVEN_VERSION_SERVER gets set and read with groovy scripts before)
mercurial-server runs on Ubuntu 12.04 LTS
myserver#ip:/etc$ hg --version
Mercurial Distributed SCM (version 2.0.2)
myserver#ip:/etc$ dpkg -s mercurial-server
Package: mercurial-server
Version: 1.2-1
....
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ ls -ltr
total 12
-rw-r--r-- 1 root root 180 Oct 10 2011 logging.rc
-rw-r--r-- 1 root root 139 Oct 10 2011 access.rc
-rw-r--r-- 1 root root 74 Mar 13 22:14 check.rc
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ cat check.rc
[hooks]
pretxncommit.author_check = /SOURCE/mercurial-server/validate.sh
#manually added here too
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ cat ~hg/repos/hgadmin/.hg/hgrc
# WARNING: when these hooks run they will entirely destroy and rewrite
# ~/.ssh/authorized_keys
[extensions]
hgext.purge =
[hooks]
changegroup.aaaab_update = hg update -C default > /dev/null
changegroup.aaaac_purge = hg purge --all > /dev/null
changegroup.refreshauth = python:mercurialserver.refreshauth.hook
pretxncommit.author_check = /SOURCE/mercurial-server/validate.sh
myserver#ip:/etc/mercurial-server/remote-hgrc.d$ cat /SOURCE/mercurial-server/validate.sh
#!/bin/bash
echo "REMUSR:$REMOTE_USER"
echo "ATHR:`hg tip --template "{author}\n"`b"
exit 1
myserver#ip:~$ sudo -u hg cat ~hg/.ssh/authorized_keys
no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,command="/usr/share/mercurial-server/hg-ssh root/user1/user1.pub" ssh-rsa AAAAB3xOMN8ZiF user1#server.com
no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,command="/usr/share/mercurial-server/hg-ssh users/user2/user2.pub" ssh-rsa AAAAB3N..0HchQQw== user2#server.com
After this from a local machine(Windows) I cloned a testproject ,changed,commited,push and it was successfull without any error or message.I tried this with both the initial user/key and a user/key added via hgadmin push
D:\hg\testproj>hg push
pushing to ssh://hg#myserver.com/testproj
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
Works with
$ cat check.rc
[hooks]
pretxnchangegroup.author_check = /SOURCE/mercurial-server/validate.sh
Not working with pretxncommit