I'm making a conda package that is C++ code that is dependent on libboost and libopencv, and is then exposed to Python wrappers. I have libboost and libopencv installed on my system in a particular way.
I want to redistribute these libraries along with my package (this is for internal use only). I use plain make.
I'm not sure how to package my code in a way that shared libraries are copied and used from within the package. So that even if I send my package to a computer with no dependencies installed, it will still run. I want to prioritize my shared libraries even if there are other system libraries installed. Think of it as a standalone package.
I'm not sure if I have to use CMake to build my stuff in the right way - does anyone have any example that I can use?
Related
I am using conan in an enterprise environment where the operating system is rather old and has an old version of glibc (2.11). As a result, a lot of the pre-built binaries on conan.io do not end up working in my environment. However, conan doesn't know that and will happily download and install them on my system, resulting in link-time errors.
I have found that if I build from source I can get the libraries to work.
My desired behavior would be as follows:
The first time using conan install to install the library (e.g. it is not in my cache) then conan will build from source and place it in my cache, then use it.
On subsequent invocations of conan install, conan finds the cached library and uses that without having to rebuild from source.
I am invoking conan install as part of an automated build script, so I would like to not have to modify the invocation depending on if this is the first time the library is installed or not (but modifying a configuration file is fine).
I have had troubles obtaining this behavior in practice. Here are the challenges I have run into:
If I use conan install --build=thelibrary then conan will rebuild that library from source every time I invoke conan install --build=thelibrary, even if it is already present in my cache.
If I use conan install --build=missing, then Ican trick conan into building the library by setting some build options that do not have a pre-built binary associated with them.
This is fragile, as it only works for projects with enough build options that it is not tractable to create pre-built options for all combinations.
It also doesn't work if all the build options I need correspond to a pre-built binary.
Here is what I am looking for (and I assume exists but am not able to find):
Some setting I can place in my conanfile.txt (or some other configuration file) that tells conan to ignore pre-built binaries for a given library or libraries and instead build from source, but use the cached version if it is available.
This ideally should work without me having to tinker with build options.
I don't necessarily want to build all libraries from source, just the ones that won't run on my ancient OS, but if I have to settle for "all-or-nothing" I will take "all".
Is this possible with conan?
glibc version is an old headache for Conan, because it's not part of settings, thus is not counted as part of package ID. The Conan Docker images are running Ubuntu, some of them are old, others are new. But there is a specific Docker image running CentOS6, which was created because of glibc 2.12 and could help with package generation.
For your specific case, we have few options:
Add glibc as part of settings, so Conan won't replace your package because of its package ID. As you should have more coworkers, you can use conan config command for settings distribution.
# ~/.conan/settings.yml
glibc: [None, 2.11, ...]
Adding it, you can update you profile too, making glibc=2.11 as a default setting.
Another alternative is package revisions feature, where you can lock a specific binary package for usage, which means, you want use that specific package. You just need to upload your generated package with glibc and use its binary package revision, e.g. lib/1.0#conan/stable#RREV:PACKAGE_ID#PREV
Also, answering your question:
Some setting I can place in my conanfile.txt (or some other configuration file) that tells conan to ignore pre-built binaries for a given library or libraries and instead build from source, but use the cached version if it is available.
Your cache is Conan first option, it will look for a pre-built package there first, if it's not available, it will look into your remotes, following a sorted order. Your request is not possible, first, because conanfile.txt doesn't support build policies, second, because conanfile.py only supports build all from sources, or build only missing.
My propose is, install an Artifactory instance, build what you need, upload your custom packages, and make it as your default remote.
I don't necessarily want to build all libraries from source, just the ones that won't run on my ancient OS, but if I have to settle for "all-or-nothing" I will take "all".
You can associate some package reference to a remote, running conan remote command. Let's say you want to download zlib/1.2.11 built with glibc-2.11 and it's available only in your organization remote:
$ conan remote add_ref zlib/1.2.11#org/stable my_org_repo
$ conan remote list_ref # only to validate, not mandatory
zlib/1.2.11#org/stable: my_org_repo
Now your specific package is associated to your organization. Conan still will look for that package your local cache first, but when not found, it will try to find at your Artifactory.
As you can see, your case could be solved easier using a new setting, instead of trying to hack build policies. As another alternative, you can replace glib setting by distro and its version.
I have build an application in C++ which is linked with 3rd party shared libraries such as opencv. Now I would require to package this application and redistribute as tar files to users, with out having them to install and compile the 3rd party dependencies. Compiling libraries such as opencv in linux/Ubuntu is such a painful process.
Now I will want to find exactly what all specific modules of a library is linked to executable and include them in the distribution tar. I dont want to include the whole library as the size will of the tar will blow up.
Will it be sufficient enough just to include libraries detected by the ldd command? Any guidance or tip-off/starting point would be helpful
By its definition "ldd - print shared object dependencies". Besides, I personally confirm that it works as I always use it in professional projects.
Also you can check the same question and answers here.
https://unix.stackexchange.com/questions/120015/how-to-find-out-the-dynamic-libraries-executables-loads-when-run
The ldd command can be used to show what libraries an executable (or library) is linked to.
I tip that it works for me (after adding all dependencies with ldd) is to install a fresh linux in virtualBox and try the distribution tar as I'd be the final user. That way you can check that everything is ok.
I am trying to install Poco on Mac. I downloaded the basic edition from here. As per the instructions, I did configure, then did make. It took a long time, but it succeeded (my make version is 3.81).
After this, when I did sudo make install, it finished pretty quickly. But as per the documentation, which states that I should have the libraries installed in /usr/local, I don't see them. The /user/local/include/Poco has all necessary header files, and /user/local/lib has lot of dynamic libraries like libPocoFoundationd.30.dylib, etc, but I don't see the libraries which I need to use.
How do I get them? My system is OS X 10.10, Yosemite.
If you have the entries like libPoco*.30.dylib (dynamic library binary) and libPoco*.dylib (link to the library), that's it. If you are looking for the static libraries, they are not built by default. To build static libraries, do
configure --static
and, after make install, in /usr/local/lib you will find libPoco*.a etc.
I have written one module in cpp which uses openssl for SSL communication. I decided to use this module in android application by writing JNI layer above this one. Now the main problem is I need openssl "static" libraries which are built using ndk.
I got the openssl package from this link git repository
This works great if I am using as it is for building shared libraries out of it, but I want to build it for static libraries. I am very new to make files. Can some one please guide me how can I change the makefiles in this openssl package which I got it from above given link to get the static libraries.. ?
thanks in advance.
First of all, your JNI can be linked to shared OpenSSL libraries. You only need to load these libraries from your Java code before you load the JNI library that uses them.
Second, OpenSSL libraries (shared objects) are installed on any Android, so you probably do not have to build them at all. The versions may vary, but in Android 4.01 tree, the version is more advanced than that in eighthave git.
Finally, if your NDK is installed in $(NDK_ROOT) directory, you can simply run the following command:
$(NDK_ROOT)/ndk_build APP_MODULES="ssl crypto" BUILD_SHARED_LIBRARY="$(NDK_ROOT)/build/core/build-static-library.mk"
(from the root directory of eighthive git clone) ... and automagically you will find the static libraries obj/local/armeabi/libcrypto.a and obj/local/armeabi/libssl.a
I've been working on various open-source projects, which involve the following C++ libraries (& others):
MuPDF
Boost
FreeType
GTKmm
hummus PDF libraries
LibTiff
LibXML2
Wt xpdf
xpdf
Poppler
ZLib
It often takes a long time to configure these libraries, when setting them up on a clean machine. Is there a way to automate the grabbing of all dependencies on a windows machine?
The closest I've found is CMake, which checks to make sure you have the dependencies installed/extracted before generating your project files. But I haven't found anything for Windows which can parse the list of dependencies and then download+install the required versions.
Please recommend a package manager for Windows with up-to-date C++ libraries.
Vcpkg, a Microsoft open source project, helps you get C and C++ libraries on Windows.
Take a look at the Hunter package manager when you already use CMake to setup your project. It automatically downloads and builds your dependencies whith only a few lines of extra cmake code. Hunter is based on cmake export and import targets.
For example if you want to use the GoogleTest library in your cmake based project you would add the following lines to your root CMakeLists.txt
# file root CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
# To get hunter you need to download and include a single cmake file
# see documentation for correct name
include("../gate.cmake")
project(download-gtest)
# set the location of all your hunter-packages
set( HUNTER_ROOT_DIR C:/CppLibraries/HunterLibraries )
# This call automaticall downloads and compiles gtest the first time
# cmake is executed. The library is then cached in the HUNTER_ROOT_DIR
hunter_add_package(GTest)
# Now the GTest library can be found and linked to by your own project
find_package(GTest CONFIG REQUIRED)
add_executable(foo foo.cpp)
target_link_libraries(foo GTest::main)
Not all the libraries you list are available as "hunter-packages" but the project is open source so you can create hunter-packages for your dependencies and commit them to the project. Here is a list of libraries that are already available as hunter packages.
This will not solve all your problems out of the box because you have to create hunter-packages for your dependencies. But the existing framework already does a lot of the work and it is better to use that instead of having a half-assed selfmade solution.
Biicode is a new dependency manager for C++. It also has a few libraries that you listed. Biicode automatically scans your source files for dependencies, downloads and builds them. See here for a very cool example that includes Freeglut.
What I've found:
Closest thing to what I'm looking for:
NuGET
Unfortunately it doesn't have any of the libraries I require in its repository.
So I ended getting most of the libraries from the KDE4windows project and custom building the rest.
Npackd is a package manager for Windows. There is a default repository for C++ libraries and also a third party repository for Visual Studio 2010 64 bit libraries. Boost and zlib are already in the default repository. If you decide to use Npackd, you could file an issue if you need other libraries.
Windows does not have a package manager. Go to the libraries' website and download the Windows builds if they provide any.
There are some alternatives, but not without drawbacks:
Cygwin: provides a nice package manager, but all binaries are built for Cygwin, which means they run slower than their native equivalent, any apps using them will link to the Cygwin DLL, and you're stuck with that license. Also the use of the native Win32 API is sometimes troublesome due to incompatibility with the POSIX emulation offered. Only for GCC.
MinGW-get: is a package manager for the MinGW.org compiler. These are native Win32 binaries, but only for use with MinGW's GCC.
There is no package manager or slightly equivalent thing for anything Visual Studio or MinGW-w64 related.
There is no package management on Windows. On Windows developers typically use full-blown everything-and-the-kitchen-sink development environments and produce monolithic applications themselves, shipped with all dependencies.