I need to disable the shared library while building dpdk libraries, how can I do that in latest LTS dpdk branch (20.X)?
In previous build I just add CONFIG_RTE_BUILD_SHARED_LIB=n in the mk files, what should I do now?
Related
I have a Linux build machine with a cross build toolchain (armhf) for my ARM target device. Both systems are Debian based.
I am struggling how to properly include prebuilt external libraries in my toolchain. When not cross compiling, I would just install the dev package versions with apt on my build machine.
So the first idea was to download the dev packages of the libraries (and all its dependencies) on the target device and extract these to a library directory on the build machine, which I can then include to my build.
Then I realized that it is kind of a second sysroot and that tools like pkg-config have trouble finding some libraries in this setup.
So the second idea was to backup the original sysroot and extract all dev packages to the sysroot.
This allows me to build my application and other libraries.
But is this a valid way of including external libraries?
Are there better ways to use the prebuilt dev packages in the cross build environment?
I am using conan in an enterprise environment where the operating system is rather old and has an old version of glibc (2.11). As a result, a lot of the pre-built binaries on conan.io do not end up working in my environment. However, conan doesn't know that and will happily download and install them on my system, resulting in link-time errors.
I have found that if I build from source I can get the libraries to work.
My desired behavior would be as follows:
The first time using conan install to install the library (e.g. it is not in my cache) then conan will build from source and place it in my cache, then use it.
On subsequent invocations of conan install, conan finds the cached library and uses that without having to rebuild from source.
I am invoking conan install as part of an automated build script, so I would like to not have to modify the invocation depending on if this is the first time the library is installed or not (but modifying a configuration file is fine).
I have had troubles obtaining this behavior in practice. Here are the challenges I have run into:
If I use conan install --build=thelibrary then conan will rebuild that library from source every time I invoke conan install --build=thelibrary, even if it is already present in my cache.
If I use conan install --build=missing, then Ican trick conan into building the library by setting some build options that do not have a pre-built binary associated with them.
This is fragile, as it only works for projects with enough build options that it is not tractable to create pre-built options for all combinations.
It also doesn't work if all the build options I need correspond to a pre-built binary.
Here is what I am looking for (and I assume exists but am not able to find):
Some setting I can place in my conanfile.txt (or some other configuration file) that tells conan to ignore pre-built binaries for a given library or libraries and instead build from source, but use the cached version if it is available.
This ideally should work without me having to tinker with build options.
I don't necessarily want to build all libraries from source, just the ones that won't run on my ancient OS, but if I have to settle for "all-or-nothing" I will take "all".
Is this possible with conan?
glibc version is an old headache for Conan, because it's not part of settings, thus is not counted as part of package ID. The Conan Docker images are running Ubuntu, some of them are old, others are new. But there is a specific Docker image running CentOS6, which was created because of glibc 2.12 and could help with package generation.
For your specific case, we have few options:
Add glibc as part of settings, so Conan won't replace your package because of its package ID. As you should have more coworkers, you can use conan config command for settings distribution.
# ~/.conan/settings.yml
glibc: [None, 2.11, ...]
Adding it, you can update you profile too, making glibc=2.11 as a default setting.
Another alternative is package revisions feature, where you can lock a specific binary package for usage, which means, you want use that specific package. You just need to upload your generated package with glibc and use its binary package revision, e.g. lib/1.0#conan/stable#RREV:PACKAGE_ID#PREV
Also, answering your question:
Some setting I can place in my conanfile.txt (or some other configuration file) that tells conan to ignore pre-built binaries for a given library or libraries and instead build from source, but use the cached version if it is available.
Your cache is Conan first option, it will look for a pre-built package there first, if it's not available, it will look into your remotes, following a sorted order. Your request is not possible, first, because conanfile.txt doesn't support build policies, second, because conanfile.py only supports build all from sources, or build only missing.
My propose is, install an Artifactory instance, build what you need, upload your custom packages, and make it as your default remote.
I don't necessarily want to build all libraries from source, just the ones that won't run on my ancient OS, but if I have to settle for "all-or-nothing" I will take "all".
You can associate some package reference to a remote, running conan remote command. Let's say you want to download zlib/1.2.11 built with glibc-2.11 and it's available only in your organization remote:
$ conan remote add_ref zlib/1.2.11#org/stable my_org_repo
$ conan remote list_ref # only to validate, not mandatory
zlib/1.2.11#org/stable: my_org_repo
Now your specific package is associated to your organization. Conan still will look for that package your local cache first, but when not found, it will try to find at your Artifactory.
As you can see, your case could be solved easier using a new setting, instead of trying to hack build policies. As another alternative, you can replace glib setting by distro and its version.
I am trying to compile a single codebase with references to both protobuffers 3.4.1 and 2.6.1. Now the 2.6.1 variant is globally defined as I am using ubuntu xenial, also:
$ protoc --version
yields:
libprotoc 2.6.1
The requirement for protobuffer version 3.4.1 comes from Google Cartographer (https://github.com/googlecartographer/cartographer) while the requirement for 2.6.1 comes from rotors simulator (https://github.com/ethz-asl/rotors_simulator) as it relies on Gazebo-7 (which uses protobuffer 2.6.1). In order to compile Google Cartographer I have added the binaries (added them in a proto3 folder, see below) to the installation by adapting the CMakeList.txt (see original file here: https://raw.githubusercontent.com/googlecartographer/cartographer/master/CMakeLists.txt) for Google Cartographer by adding the following lines:
set(CMAKE_PREFIX_PATH CMAKE_PREFIX_PATH "${CMAKE_SOURCE_DIR}/proto3")
...
install(DIRECTORY proto3/ DESTINATION .)
So the binaries of the protobuffer 3.4.1 are added to the install folder. I am utilizing catkin-tools (https://catkin-tools.readthedocs.io/en/latest/) to build the whole workspace. Now in a CMakelist.txt for Rotors Simulator I have the following line:
find_package(Protobuf 2.6.1 REQUIRED HINTS "/usr")
But at the moment while trying to compile it does not seem to be able to find the protobuffer 2.4.1 as it returns the following:
Could not find a configuration file for package "Protobuf" that is
compatible with requested version "2.6.1".
The following configuration files were considered but not accepted:
/home/jochem/catkin_ws/install/lib/cmake/protobuf/protobuf-config.cmake,
version: 3.4.1
As a side-note, if I compile the packages separately I am able to compile and install the packages. This is done with the following commands:
catkin build cartographer_ros
and
catkin build rotors_gazebo_plugins
I am at the moment trying to adapt the package of rotors_gazebo_plugins but am so far unsuccessful at making sure the correct protobuffer library is selected, am I missing something by defining references to a local protobuffer version?
You will find it possible to build a single executable that references 2 versions of the same library on mac, quite difficult on windows, and pretty much impossible on unix. This is because the symbol names are not distinct between the two libraries, so if you load both libraries, there is no way to know which library should service which call.
If you are building 2 different executables in one makefile package, then you just need to set the right libraries to load in the link stage. In linux, libraries are usually installed on your system with a version-number suffix, and a symlink that publishes the latest version without the version number. Normally you simply link to the unsuffixed latest version, but in your case, in your link command you will need to explicitly add the suffix.
If you really do need to link this cobble-together into a single executable, on unix you can do a lot with objcopy --redefine-syms to rename all the entrypoints in one of the libraries, and all the references in the dependant code all after compilation, but before linking. Note that the intended end result is that both libraries will run independently and will not be aware of each other.
If you will be able to wrap up at least one of the libs (i.e. either Cartographer or Rotors or both) into a separate shared library, and if the protobuff is only used internally in each of them, you still might be able to use them both in a single executable by building the shared libs with -fvisibility=hidden gcc flag (to switch the default visibility to hide symbols) and only exporting the symbols that are needed (that the app is using) via __attribute__((visibility("default"))).
This way I recall I was able to use two completely different Boost versions in the past, in the same app (by the shared lib not exporting the boost symbols linked in statically).
I am using cpprestsdk/casablanca in my project. cpprestsdk is dependent on Boost library. Several weeks ago I downloaded cpprest's source and built *.so library with Boost 1.65.1 and some version of OpenSSL. My system is Arch Linux.
Due to a recent events with Meltdown and Spectre exploits, I've made a full system upgrade (kernel 4.14, latest versions of libraries) and now I have Boost 1.66.0 and a more recent version of OpenSSL.
When I try to compile my project, linker states that
libboost_[random, system, etc. - there are many of them].so.1.65.1, needed by /usr/local/lib/libcpprest.so, not found (try using -rpath or -rpath-link)
It also shows warning about SSL because latest version of cpprestsdk is incompatible with OpenSSL 1.1+ so you have to build it (cpprest) with the previous.
libcrypto.so.1.0.0, needed by /usr/local/lib/libcpprest.so, may conflict with libcrypto.so.1.1.
Obviously, there is no boost 1.65.1 installation on my system, so there is no point in using -rpath.
I figured that I should build another version of boost, but several sources claim that it is bad idea to store multiple versions of boost on one system. I am not sure how to correctly store custom boost version either.
I can rebuild libccprest with the current library but it'll only work till the next update.
I guess that in order to completely build my project, I should embed specific versions of Boost, SSL and other dependencies into my project.
What are general solutions for such a problem? How do you manage and deploy custom (non-system-side) versions of shared libraries? Every "big" library has it's own installation system/scripts so I have no idea how to integrate it to my own project's build system. I've never encounter issues with libraries before so I am not sure what path to choose. I am using Makefile to build my project.
I am trying to install Poco on Mac. I downloaded the basic edition from here. As per the instructions, I did configure, then did make. It took a long time, but it succeeded (my make version is 3.81).
After this, when I did sudo make install, it finished pretty quickly. But as per the documentation, which states that I should have the libraries installed in /usr/local, I don't see them. The /user/local/include/Poco has all necessary header files, and /user/local/lib has lot of dynamic libraries like libPocoFoundationd.30.dylib, etc, but I don't see the libraries which I need to use.
How do I get them? My system is OS X 10.10, Yosemite.
If you have the entries like libPoco*.30.dylib (dynamic library binary) and libPoco*.dylib (link to the library), that's it. If you are looking for the static libraries, they are not built by default. To build static libraries, do
configure --static
and, after make install, in /usr/local/lib you will find libPoco*.a etc.