Using regex with run-parts on Alpine 3.9 - regex

I'm creating a Docker image FROM alpine:3.9.2 and I need to run run-parts. I used the script below in the past on ubuntu:16.04 without problems.
run-parts --verbose --regex '\.sh$' "$DIR"
However, this time around, I get errors on the options I pass to it. I.e.
run-parts: unrecognized option: verbose
run-parts: unrecognized option: regex
From my understading Alpine 3.9.2 uses run-parts 4.8.6 https://pkgs.alpinelinux.org/package/edge/main/x86/run-parts which should come from the debianutils https://manpages.debian.org/testing/debianutils/run-parts.8.en.html and supports both verbose and regex.
Am I missing anything here?
how can I run all the files ending with .sh on Alpine 3.9.2?

There is very cut version of run-parts in alpine image by default. It is busybox one:
/ # which run-parts
/bin/run-parts
/ # run-parts --help
BusyBox v1.29.3 (2019-01-24 07:45:07 UTC) multi-call binary.
Usage: run-parts [-a ARG]... [-u UMASK] [--reverse] [--test] [--exit-on-error] DIRECTORY
Run a bunch of scripts in DIRECTORY
-a ARG Pass ARG as argument to scripts
-u UMASK Set UMASK before running scripts
--reverse Reverse execution order
--test Dry run
--exit-on-error Exit if a script exits with non-zero
It can only run a bunch of scripts in directory.
If you want to use uncut run-parts from the debianutils package, you need to install it first to alpine image:
/ # apk add --no-cache run-parts
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/1) Installing run-parts (4.8.6-r0)
Executing busybox-1.29.3-r10.trigger
OK: 6 MiB in 15 packages
Now, there is a full version of run-parts in alpine instance:
/ # which run-parts
/usr/bin/run-parts
/ # run-parts --help
Usage: run-parts [OPTION]... DIRECTORY
--test print script names which would run, but don't run them.
--list print names of all valid files (can not be used with
--test)
-v, --verbose print script names before running them.
--report print script names if they produce output.
--reverse reverse execution order of scripts.
--exit-on-error exit as soon as a script returns with a non-zero exit
code.
--lsbsysinit validate filenames based on LSB sysinit specs.
--new-session run each script in a separate process session
--regex=PATTERN validate filenames based on POSIX ERE pattern PATTERN.
-u, --umask=UMASK sets umask to UMASK (octal), default is 022.
-a, --arg=ARGUMENT pass ARGUMENT to scripts, use once for each argument.
-V, --version output version information and exit.
-h, --help display this help and exit.

Related

Does Dockerfile support inline comments? [duplicate]

I am writing a Dockerfile. Is there a way to make comments in this file?
Does Docker have a comment option that takes the rest of a line and ignores it?
You can use # at the beginning of a line to start a comment (whitespaces before # are allowed):
# do some stuff
RUN apt-get update \
# install some packages
&& apt-get install -y cron
#'s in the middle of a string are passed to the command itself, e.g.:
RUN echo 'we are running some # of cool things'
As others have mentioned, comments are referenced with a # and are documented here. However, unlike some languages, the # must be at the beginning of the line. If they occur part way through the line, they are interpreted as an argument and may result in unexpected behavior.
# This is a comment
COPY test_dir target_dir # This is not a comment, it is an argument to COPY
RUN echo hello world # This is an argument to RUN but the shell may ignore it
It should also be noted that parser directives have recently been added to the Dockerfile which have the same syntax as a comment. They need to appear at the top of the file, before any other comments or commands. Originally, this directive was added for changing the escape character to support Windows:
# escape=`
FROM microsoft/nanoserver
COPY testfile.txt c:\
RUN dir c:\
The first line, while it appears to be a comment, is a parser directive to change the escape character to a backtick so that the COPY and RUN commands can use the backslash in the path. A parser directive is also used with BuildKit to change the frontend parser with a syntax line. See the experimental syntax for more details on how this is being used in practice.
With a multi-line command, the commented lines are ignored, but you need to comment out every line individually:
$ cat Dockerfile
FROM busybox:latest
RUN echo first command \
# && echo second command disabled \
&& echo third command
$ docker build .
Sending build context to Docker daemon 23.04kB
Step 1/2 : FROM busybox:latest
---> 59788edf1f3e
Step 2/2 : RUN echo first command && echo third command
---> Running in b1177e7b563d
first command
third command
Removing intermediate container b1177e7b563d
---> 5442cfe321ac
Successfully built 5442cfe321ac
Use the # syntax for comments
From: https://docs.docker.com/engine/reference/builder/#format
# My comment here
RUN echo 'we are running some cool things'
Dockerfile comments start with #, just like Python.
kstaken has good examples:
# Install a more-up-to date version of MongoDB than what is included in the default Ubuntu repositories.
FROM ubuntu
MAINTAINER Kimbro Staken
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
RUN echo "deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen" | tee -a /etc/apt/sources.list.d/10gen.list
RUN apt-get update
RUN apt-get -y install apt-utils
RUN apt-get -y install mongodb-10gen
#RUN echo "" >> /etc/mongodb.conf
CMD ["/usr/bin/mongod", "--config", "/etc/mongodb.conf"]
Docker treats lines that begin with # as a comment, unless the
line is a valid parser directive. A # marker anywhere else in a line
is treated as an argument.
example code:
# this line is a comment
RUN echo 'we are running some # of cool things'
Output:
we are running some # of cool things
Format
Here is the format of the Dockerfile:
We can use # for commenting purpose, as for example #COMMENT
#FROM microsoft/aspnetcore
FROM microsoft/dotnet
COPY /publish /app
WORKDIR /app
ENTRYPOINT ["dotnet", "WebApp.dll"]
From the above file when we build the docker, it skips the first line and goes to the next line because we have commented it using #
# this is comment
this isn't comment
is the way to do it. You can place it anywhere in the line and anything that comes later will be ignored

How to use scl command as a script shebang?

If I want to run a specific command (with arguments) under Software Collections, I can use this command:
scl enable python27 "ls /tmp"
However, if I try to make a shell script that has a similar command as its shebang line, I get errors:
$ cat myscript
#!/usr/bin/scl enable python27 "ls /tmp"
echo hello
$ ./myscript
Unable to open /etc/scl/prefixes/"ls!
What am I doing wrong?
You should try using -- instead of surrounding your command with quotes.
scl enable python27 -- ls /tmp
I was able to make a python script that uses the rh-python35 collection with this shebang:
#!/usr/bin/scl enable rh-python35 -- python
import sys
print(sys.version)
The parsing of arguments in the she-bang command is not really defined. From man execve:
The semantics of the optional-arg argument of an interpreter script vary across implementations. On Linux, the entire string following the interpreter name is passed as a single argument to the interpreter, and this string can include white space. However, behavior differs on some other systems. Some systems use the first white space to terminate optional-arg. On some systems, an interpreter script can have multiple arguments, and white spaces in optional-arg are used to delimit the arguments.
No matter what, argument splitting based on quote sis not supported. So when you write:
#!/usr/bin/scl enable python27 "ls /tmp"
It's very possible that what gets invoked is (using bash notation):
'/usr/bin/scl' 'enable' 'python27' '"ls' '/tmp"'
This is probably why it tries to open the "ls file at /etc/scl/prefixes/"ls
But it is just as likely that the shebang evaluates to:
'/usr/bin/scl' 'enable python27 "ls /tmp"'
And that would fail since it wont be able to find a command named enable python27 "ls /tmp" for scl to execute.
There's a few workarounds you can use.
You can call your script via scl:
$ cat myscript
#!/bin/bash
echo hello
$ scl enable python27 ./myscript
hello
You can also use the heredoc notation, but it might lead to subtle issues. I personally avoid this:
$ cat ./myscript
#!/bin/bash
scl enable python27 -- <<EOF
echo hi
echo \$X_SCLS
EOF
$ bash -x myscript
+ scl enable python27 --
hi
python27
You can see one of the gotcha's already: I had to write \$X_SCLS to access the environment variable instead of just $X_SCL.
Edit: Another option is two have two scripts. One that has the actual code, and the second that simply does scl enable python27 $FIRST_SCRIPT. Then you wont have to remember to enter scl ... manually.
The software collections documentation may also be helpful. In particular you can try
cat myscript.sh | scl enable python27 -
As well as permanently enabling a software collection
source scl_source enable python27
./myscript.sh

C++ linux install executable file on deploy environment

Hello every one i need to deploy linux(centos) c++ project with make file or script. By one makefile or script install dependency and project executable binary.
my dependency applications libboost-devel,gcc-g++ and pcre. my excuteble binary file is run_excute
Yip sure - put the below commands into a file. At the top of the file add:
#!/bin/bash
Save the file - lets say you call it install; on the command line type:
chmod +x ./install
Then to build and install your program type:
sudo ./install
Alternatively, if you've got some time on your hands:
http://www.rpm.org/max-rpm/ch-rpm-build.html
As an example the basic rpm build process for fedora is:
Step 1: setup your machine to do packaging:
dnf install #development-tools fedora-packager rpmdevtools
rpmdev-setuptree
Step 2: source and Makefile
Place these in ~/rpmbuild/SOURCES
Step 3: Create a spec file
In ~/rpmbuild/SPECS create file called myname.spec. It should contain something like:
Summary: My program description
Name: myname
Version: 0.0.0
Release: 0
License: GPLv2
Group: Applications/Databases
Source: https://xyz.tar.gz
URL: http://myurl
BuildRequires: libicu-devel
BuildRequires: pcre-devel
%description
A couple of lines describing the package
%prep
%setup -q
%build
cd %{myname}/source
make %{?_smp_mflags}
%install
%make_install
%files
%{_bindir}/*
%changelog
* Tue Nov 10 2015 Yours Truly <me#somewhere.com> - 0.0.0-0
- Some change comments
Step 4: create the source and binary rpm
cd ~/rpmbuild/SPECS
rpmbuild -ba myname.spec
Step 5: use the rpm
cd ~/rpmbuild/RPMS/x86_64
rpm -Uvh ./myprogram-version-release.a.whole.lot.of.stuff.rpm
To install the dependencies use yum, so:
sudo yum install libboost-devel
sudo yum group install "Development Tools"
sudo yum install pcre-devel
To build the application, move to the directory with the makefile in it and do:
make
sudo make install
Finally to run the application
./run_excute
or if your lucky
run_excute
will work.

phpunit with HHVM 3.11

Following the accepted answer to Running phpunit tests using HHVM (HipHop), I attempted to run some tests:
unit-tests/ [develop] > hhvm $(which phpunit) --colors -c phpunit.xml --testsuite all .
/usr/bin/env php -d allow_url_fopen=On -d detect_unicode=Off /usr/local/Cellar/phpunit/4.3.4/libexec/phpunit-4.3.4.phar $*
It appears that this is a command to run the tests (which it does), but I'm confused about
why it's printing this command instead of just running the tests
whether executing that command even uses HHVM, since it starts with /usr/bin/env php...
Does anyone have any insight into this? Thanks so much!
What happened here is that you installed PHPUnit using homebrew, so it created a wrapper script for the actual PHAR file. That wrapper script is a Bash script that runs the PHPUnit PHAR and that script is what you're trying to get HHVM to run. Since it's not a PHP or Hack script, the Bash script is outputted directly.
Instead, you probably want to try to execute $(brew --prefix phpunit)/libexec/phpunit*.phar
e.g.: hhvm $(brew --prefix phpunit)/libexec/phpunit*.phar --colors -c phpunit.xml --testsuite all .
The wildcard is so that you don't need to specify the version of PHPUnit being using.

Running the "exec" command in Jenkins "Execute Shell"

I'm running Jenkins on a Linux host. I'm automating the build of a C++ application. In order to build the application I need to use the 4.7 version of g++ which includes support for c++11. In order to use this version of g++ I run the following command at a command prompt:
exec /usr/bin/scl enable devtoolset-1.1 bash
So I created a "Execute shell" build step and put the following commands, which properly builds the C++ application on the command prompt:
exec /usr/bin/scl enable devtoolset-1.1 bash
libtoolize
autoreconf --force --install
./configure --prefix=/home/tomcat/.jenkins/workspace/project
make
make install
cd procs
./makem.sh /home/tomcat/.jenkins/workspace/project
The problem is that Jenkins will not run any of the commands after the "exec /usr/bin/scl enable devtoolset-1.1 bash" command, but instead just runs the "exec" command, terminates and marks the build as successful.
Any ideas on how I can re-structure the above so that Jenkins will run all the commands?
Thanks!
At the begining of your "Execute shell" script, execute source /opt/rh/devtoolset-1.1/enable to enable the devtoolet "inside" of your shell.
Which gives:
source /opt/rh/devtoolset-1.1/enable
libtoolize
autoreconf --force --install
./configure --prefix=/home/tomcat/.jenkins/workspace/project
make
make install
cd procs
./makem.sh /home/tomcat/.jenkins/workspace/project
I needed to look up what scl actually does.
Examples
scl enable example 'less --version'
runs command 'less --version' in the environment with collection 'example' enabled
scl enable foo bar bash
runs bash instance with foo and bar Software Collections enabled
So what you are doing is running a bash shell. I guess, that the bash shell returns immediately, since you are in non-interactive mode. exec runs the the command within the shell without creating a new shell. That means if the newly opened bash ends it also ends your shell prematurely. I would suggest to put all your build steps into a bash script (e.g. run_my_build.sh) and call it in the following way.
exec /usr/bin/scl enable devtoolset-1.1 run_my_build.sh
This kind of thing normally works in "find" commands, but may work here. Rather than running two, or three processes, you run one "sh" that executes multiple things, like this:
exec sh -c "thing1; thing2; thing3"
If you require each step to succeed before the next step, replace the semi-colons with double ampersands:
exec sh -c "thing1 && thing2 && thing3"
I have no idea which of your steps you wish to run together, so I am hoping you can adapt the concept to fit your needs.
Or you can put the whole lot into a script and exec that.