I'm trying to update a Docker image based on the official CentOS7 image. It is used as a builder for Node.js projects.
I need to add the systemd-devel package for compiling some dependencies, but this fails with the following error:
fakesystemd-1-17.el7.centos.noarch has installed conflicts systemd: fakesystemd-1-17.el7.centos.noarch
Thanks
fakesystemd is a special package in the CentOS Docker image that satisfies the dependency to Systemd without actually installing Systemd (after all, you don't usually need an init system within a container). yum info fakesystemd tells a bit more:
Minimal docker-specific package to satisfy systemd Provides: without installing systemd in Docker images. It is intended strictly for use in Docker images/containers. It doesn't provide any functionality from systemd package - it only contains few important directories and files. fakesystemd is definitely not applicable for full bootable operation system!
To install the real systemd in the image you need to run yum swap command in this form:
yum swap -- remove fakesystemd -- install systemd systemd-libs
You need to swap the fakesystemd package with the "real" systemd package, and can then also install systemd-devel:
RUN yum swap -y fakesystemd systemd && \
yum install -y systemd-devel
Related
How can I install both of them on CentOS 7? They both depend on each other. The machine is not connected to the internet.
[efe#localhost Desktop]$ sudo rpm -i glibc-2.17-317.el7.x86_64.rpm
warning: glibc-2.17-317.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
error: Failed dependencies:
glibc-common = 2.17-317.el7 is needed by glibc-2.17-317.el7.x86_64
[efe#localhost Desktop]$ sudo rpm -i glibc-common-2.17-317.el7.x86_64.rpm
warning: glibc-common-2.17-317.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
error: Failed dependencies:
glibc = 2.17-317.el7 is needed by glibc-common-2.17-317.el7.x86_64
EDIT1: I have installed 2.17-292 for everything on x86_64 and it is successful. However, for i686 (yes, I need both unfortunately) I have tried to combine all packages into single command, lots of other dependencies appeared. I have tried to install 2.17-292 like I did for x86_64, this is the final output with errors. Before this last command, dependency errors appeared all the time, this is the first different error.
[efe#localhost packages]$ sudo rpm -i glibc-2.17-292.el7.i686.rpm glibc-common-2.17-292.el7.i686.rpm glibc-headers-2.17-292.el7.i686.rpm glibc-devel-2.17-292.el7.i686.rpm nss-softokn-freebl-3.44.0-8.el7_7.i686.rpm nspr-4.25.0-2.el7_9.i686.rpm nss-util-3.53.1-1.el7_9.i686.rpm libselinux-2.5-15.el7.i686.rpm libsepol-2.5-10.el7.i686.rpm pcre-8.32-17.el7.i686.rpm libgcc-4.8.5-44.el7.i686.rpm libstdc++-4.8.5-44.el7.i686.rpm
warning: glibc-2.17-292.el7.i686.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
file /usr/bin/ldd from install of glibc-common-2.17-292.el7.i686 conflicts with file from package glibc-common-2.17-292.el7.x86_64
file /usr/lib/locale/locale-archive.tmpl from install of glibc-common-2.17-292.el7.i686 conflicts with file from package glibc-common-2.17-292.el7.x86_64
file /usr/share/gcc-4.8.2/python/libstdcxx/v6/printers.pyc from install of libstdc++-4.8.5-44.el7.i686 conflicts with file from package libstdc++-4.8.5-36.el7.x86_64
file /usr/share/gcc-4.8.2/python/libstdcxx/v6/printers.pyo from install of libstdc++-4.8.5-44.el7.i686 conflicts with file from package libstdc++-4.8.5-36.el7.x86_64
EDIT2: I have entered a command as suggested by Florian Weimer.
[efe#localhost i686]$ sudo yum update *.rpm
Loaded plugins: fastestmirror, langpacks
Examining glibc-2.17-292.el7.i686.rpm: glibc-2.17-292.el7.i686
Package glibc.i686 not installed, cannot update it. Run yum install to install it instead.
Examining glibc-common-2.17-292.el7.i686.rpm: glibc-common-2.17-292.el7.i686
Package glibc-common.i686 not installed, cannot update it. Run yum install to install it instead.
Examining glibc-devel-2.17-292.el7.i686.rpm: glibc-devel-2.17-292.el7.i686
Package glibc-devel.i686 not installed, cannot update it. Run yum install to install it instead.
Examining glibc-headers-2.17-292.el7.i686.rpm: glibc-headers-2.17-292.el7.i686
Package glibc-headers.i686 not installed, cannot update it. Run yum install to install it instead.
Examining libgcc-4.8.5-44.el7.i686.rpm: libgcc-4.8.5-44.el7.i686
Marking libgcc-4.8.5-44.el7.i686.rpm as an update to libgcc-4.8.5-36.el7.x86_64
Examining libselinux-2.5-15.el7.i686.rpm: libselinux-2.5-15.el7.i686
Marking libselinux-2.5-15.el7.i686.rpm as an update to libselinux-2.5-14.1.el7.x86_64
Examining libsepol-2.5-10.el7.i686.rpm: libsepol-2.5-10.el7.i686
Package libsepol.i686 not installed, cannot update it. Run yum install to install it instead.
Examining libstdc++-4.8.5-44.el7.i686.rpm: libstdc++-4.8.5-44.el7.i686
Marking libstdc++-4.8.5-44.el7.i686.rpm as an update to libstdc++-4.8.5-36.el7.x86_64
Examining nspr-4.25.0-2.el7_9.i686.rpm: nspr-4.25.0-2.el7_9.i686
Marking nspr-4.25.0-2.el7_9.i686.rpm as an update to nspr-4.19.0-1.el7_5.x86_64
Examining nss-softokn-freebl-3.44.0-8.el7_7.i686.rpm: nss-softokn-freebl-3.44.0-8.el7_7.i686
Marking nss-softokn-freebl-3.44.0-8.el7_7.i686.rpm as an update to nss-softokn-freebl-3.36.0-5.el7_5.x86_64
Examining nss-util-3.53.1-1.el7_9.i686.rpm: nss-util-3.53.1-1.el7_9.i686
Marking nss-util-3.53.1-1.el7_9.i686.rpm as an update to nss-util-3.36.0-1.el7_5.x86_64
Examining pcre-8.32-17.el7.i686.rpm: pcre-8.32-17.el7.i686
Package pcre.i686 not installed, cannot update it. Run yum install to install it instead.
Resolving Dependencies
--> Running transaction check
---> Package libgcc.x86_64 0:4.8.5-36.el7 will be updated
--> Processing Dependency: libgcc(x86-64) for package: redhat-lsb-core-4.1-27.el7.centos.1.x86_64
Loading mirror speeds from cached hostfile
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=stock error was
14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error"
One of the configured repositories failed (Unknown),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled
yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid>
or
subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/7/x86_64
You can upgrade all the packages at the same time, using a command like this one:
sudo rpm -U glibc-2.17-317.el7.x86_64.rpm glibc-common-2.17-317.el7.x86_64.rpm
Or you can use yum update, like this:
sudo yum update --disablerepo="*" *.rpm
This has the advantage that packages not installed are not automatically installed by the upgrade (unless dependencies require it). --disablerepo="*" ensures that yum update does not try to download anything.
jlehtone in forums.centos.org solved my situation. I just needed to write
sudo rpm -i glibc-2.17-292.el7.i686.rpm nss-softokn-freebl-3.44.0-8.el7_7.i686.rpm nspr-4.25.0-2.el7_9.i686.rpm nss-util-3.53.1-1.el7_9.i686.rpm
Link to forums.centos.org
Background
I have a CI pipeline for a C++ library I've been developing. So far, I can distribute this lib to Linux and Windows systems. Since I use GitLab to build, test and package my lib, I'd like to have my Windows builds running faster and I have no clue on how to do that.
Currently, I use the following script for my Windows builds:
.windows_template:
tags:
- windows
before_script:
- choco install cmake.install -y --installargs '"ADD_CMAKE_TO_PATH=System"'
- choco install python --pre -y
- choco install git -y
- $env:ChocolateyInstall = Convert-Path "$((Get-Command choco).Path)\..\.."; Import-Module "$env:ChocolateyInstall\helpers\chocolateyProfile.psm1"; refreshenv
- python -m pip install --upgrade pip
- pip install conan monotonic
The problem
Any build with the script above can take up to 10 minutes; worse: I have two stages, each one taking the same amount of time. This means that my whole CI pipeline will take 20 minutes to finish because of slowness in Windows builds.
Ideal solution
EVERYTHING in my before_script can be cached or stored as an image. I only need some hints on how to do it properly.
Additional information
I use the following tools for my builds:
CMake: to support my building process;
Python3: to test and build packages;
Conan (requires Python3): to support the creation of packages with several features, as well as distribute them;
Git: to download Googletest in CMake configuration step This is already provided in the cookbooks - I might just remove this extra installation step in my before_script;
Googletest (requires Python3): testing library;
Visual Studio DEV Tools: to compile the library This is already in the cookbooks.
Installing packages like this (whether it's OS packages though apt-get install... or pip, or anything else) is generally against best practices for CI/CD jobs because every job that runs will have to do the same thing, costing a lot of time as you run more pipelines, as you've seen already.
A few alternatives are to search for an existing image that has everything you need (possible but not likely with more dependencies), split up your job into pieces that might be solved by an image with just one or two dependencies, or create a custom docker image to use in your jobs. I answered a similar question with an example a few weeks ago here: "Unable to locate package git" when running GitLab CI/CD pipeline
But here's an example Dockerfile with Windows:
# Dockerfile
FROM mcr.microsoft.com/windows
RUN ./install_chocolatey.sh
RUN choco install cmake.install -y --installargs '"ADD_CMAKE_TO_PATH=System"'
RUN choco install python --pre -y
RUN choco install git -y
...
The FROM line says that our new image extends the mcr.microsoft.com/windows base image. You can extend any image you have access to, even if it already extends another image (in fact, that's how most images work: they start with something small, like a base OS installation, then add things needed for that package. PHP for example starts on an Ubuntu image, then installs the necessary PHP packages).
The first RUN line is just an example. I'm not a Windows user and don't have experience installing Chocolatey, but you'd do here whatever you'd normally do to install it locally. The rest are for installing whatever else you need.
Then run
docker build /path/to/dockerfile-dir -t mygroup/mytag:version
The path you supply needs to be the directory that contains the Dockerfile, not the Dockerfile itself. The -t flag sets the image's tag after it's built (though you can do that with a separate command, docker tag too).
Next, you'll have to log into whichever registry you're using (Docker Hub (https://docs.docker.com/docker-hub/repos/), Gitlab Container Registry (https://docs.gitlab.com/ee/user/packages/container_registry/), a private registry your employer may support, or any other option.
docker login my.docker.hub.com
Now you can push the image to the registry:
docker push my.docker.hub.com/mygroup/mytag:version
You'll have to review the information in the docs about telling your Gitlab runner or pipelines how to authenticate with the registry (unless it's Public on Docker Hub or you use the Gitlab Container Registry) https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-an-image-from-a-private-container-registry
Once all that's done, you can use your new image in your CI jobs, and everything we put into the image will be ready to use:
.windows_template:
image: my.docker.hub.com/mygroup/mytag:version
tags:
- windows
...
I'm trying to build a basic Docker container based on a tutorial. I am on Windows 10 Home version 2004, and I am using the standard command line. I've created the following Docker file to facilitate this, with the only change from the tutorial's version being my older version of gcc:
FROM gcc:6.3.0
RUN apt-get -qq update
RUN apt-get -qq upgrade
RUN apt-get -qq install cmake
RUN apt-get install libboost-all-dev=1.62.0.1
RUN apt-get -qq install build-essential libtcmalloc-minimal4 && \
ln -s /usr/lib/libtcmalloc_minimal.so.4 /usr/lib/libtcmalloc_minimal.so
Once the script gets to the step where it tries to install libboost-all-dev I get the following output:
Reading package lists...
Building dependency tree...
Reading state information...
E: Version '1.62.0.1' for 'libboost-all-dev' was not found
The command '/bin/sh -c apt-get install libboost-all-dev=1.62.0.1' returned a non-zero code: 100
and the build stops.
I've tried updating the build script to use the current version of Boost (1.74.0) as well and get the same issue. I'm not really finding any solutions in my research online and the output is not very helpful in trying to figure out what the issue is. Could anyone with more experience with installing Boost as part of the Docker process point me in the right direction?
The package manager will only be able to install versions of Boost that it knows exist, based on the enabled package manager repositories. There is typically only one version of Boost in the default repositories. In my experience, this applies to any Linux OS that supplies Boost, not only those that are run within a Docker container.
The Docker image you started with, gcc:6.3.0, appears to have only Boost version 1.55.0.2, so requesting any other version will yield the same error.
If you want a different version of Boost in your image, you can follow the typical steps for installing a different version of Boost outside a Docker container. These steps are well-documented on Stack Overflow, or you might find a repository such as this to enable in your package manager to directly install it from apt-get.
In the dockerfiles I have seen, and the in the best practices for writing a docker file: https://docs.docker.com/engine/reference/builder/#copy, when apt-get is used to install some packages, apt-get update is always run first. I have a concern on this because the app we build in the corresponding docker container would depend on these installed packages, if there is some inconsistency in the newest version of the installed packages, the software we build will not work right any more. Why we do not specify a version of the packages, but use apt-get update instead?
From the man page for apt-get:
update is used to resynchronize the package index files from their
sources. The indexes
of available packages are fetched from the location(s) specified in
/etc/apt/sources.list. For example, when using a Debian archive, this command retrieves
and scans the Packages.gz files, so that information about new and updated packages is
available. An update should always be performed before an upgrade or dist-upgrade.
Please be aware that the overall progress meter will be incorrect as the size of the
package files cannot be known in advance.
You can try running apt-get install without running update on a docker image but you'll probably find that a lot of things will fail to install because the package indexes are out of date.
Once you update the package data, then you can specify a specific version for packages when you run install e.g.
apt update && apt install -y \
git=1:2.7.4-0ubuntu1.4
Example with docker container:
> sudo docker run -it ubuntu:16.04 /bin/bash
# root#513eb786d86d:/# apt install git
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package git
root#513eb786d86d:/# apt install git=1:2.7.4-0ubuntu1.4
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package git
root#513eb786d86d:/# apt update
...
root#513eb786d86d:/# apt install git=1:2.7.4-0ubuntu1.4
# works this time!
I'm using azk and my system depends on extra packages. I'd be able to install them using (since I'm using an Ubuntu-based image):
apt-get -yq update && apt-get install -y libqtwebkit-dev qt4-qmake
Can I add this steps to provision? In the Azkfile.js, it would look like:
// ...
provision: [
"apt-get -yq update",
"apt-get install -y libqtwebkit-dev qt4-qmake",
"bundle install --path /azk/bundler",
"bundle exec rake db:create",
"bundle exec rake db:migrate",
]
Or it's better to create a new Docker image?
Provision steps are run in a separated container, so all the data generated inside of it is lost after the provision step, unless you persist them. That's why you probably have bundle folders as persistent folders.
Since that, you should use a Dockerfile in this case. It'll look like this:
FROM azukiapp/ruby:2.2.2 # or the image you were using previously
RUN apt-get -yq update && \
apt-get install -y libqtwebkit-dev qt4-qmake && \
apt-get clean -qq && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* # Keeping the image as small as possible
After that, you should edit your Azkfile.js and replace the image property of your main system to use the created Dockerfile (you can check azk docs here):
image: { dockerfile: './PATH_TO_DOCKERFILE' },
Finally, when you run azk start, azk will build this Dockerfile and use it with all your dependencies installed.
Tip: If you want to force azk to rebuild your Dockerfile, just pass -B flag to azk start.
As it looks like you're using a Debian-based Linux distribution, you could create (https://wiki.debian.org/Packaging) your own Debian virtual package (https://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html#s-virtual) that lists all the packages it depends on. If you just do that one thing, you can dpkg -i (or apt-get install if you host a custom debian repository yourself) your custom package and it will install all the dependencies you need via apt.
You can then move on to learning about postinst and prerm scripts in Debian packages (https://www.debian.org/doc/manuals/debian-faq/ch-pkg_basics.en.html#s-maintscripts). This will allow you to run commands like bundle and gem as the last step of the package installation and the first step of package removal.
There are a few advantages to doing it this way:
1. If you host a package repository somewhere you can use a pull method of dependency installation in a dynamic scaling environment by simply having the host apt-get update && apt-get install custom-dependencies-diego
2. Versioning your dependency list - Using dpkg -l you can tell what version everything is on a given host, including the version of your dependency virtual package.
3. With prerm scripts, you can ensure that removing your virtual package will also have the effect of removing the changes your installation scripts made so you can get a host back to a "clean" state".
The disadvantage of doing it this way is that it's debian/apt specific. If you wanted to deploy to Slack or RHEL you'd have to change things a bit. Changing to a new distro wouldn't be particularly hard, but it's definitely not as portable as using Bash, for example.