I am trying to incorporate a libtool-based package into a project of my own, perhaps in a non-standard way. Here is my goal:
Build external project:
./configure --prefix=$HOME/blah --etcetera && make && make install
Build my own project which depends upon the external project's shared libraries and executables at runtime:
gcc -I$HOME/blah/include -L$HOME/blah/lib -o $HOME/blah/bin/program
Package everything into a single "localized" tarball... that is, while I have everything in $HOME/blah on the build host I want the ability to extract the tarball to any arbitrary directory (on some other host) without having to futz with my environment. The intent is to allow multiple versions of my project to coexist side-by-side without any nasty "cross-pollination".
I know that I can use -rpath '$ORIGIN/../lib' for my project to ensure that the right shared libraries always get loaded at runtime. However, it seems that libtool insists on assigning its own -rpath setting based on the exact path of $HOME/blah/lib, which breaks if I happen to untar everything into a different directory (say, for example, $HOME/blah.2011-06-02).
Is there a way around this limitation? I see a rather lengthy rpath discussion between debian and libtool folks on the topic, but it's somewhat old and inconclusive beyond "we disagree".
Among the options presented here on Rpathissue on the debian Wiki, using chrpath in your 'install' step or some post-processing script sounds like a viable option. (It's available on a bunch of distros via your favorite package manager.)
It doesn't require patching libtool which is a plus IMO.
Note that it has some limitations: can only save the new rpath if it's shorter (or same length) as the original one.
The other (pragmatic) option is to remove the rpath (chrpath can do that), and just have a wrapper script that sets LD_LIBRARY_PATH to whatever is necessary for your app. That has a chance of being slightly more portable too (if you handle the other shared library path environment vars some OSes have).
Related
I've a question regarding compilation of a dependency (namely poppler). Poppler is a dependency of my app which is an electron app.
It's working fine on windows (because npm package node-poppler ships the poppler windows build). But on a mac / linux I have to install poppler dependencies by myself (e.g. via homebrew) and then also everything works well.
Now I want to ship the app including poppler and therefore I've started to compile poppler by myself. This is working particularly and all dependencies that it needs I've added via homebrew and finally the build is running and it compiles successfully after long try and error.
But my problem now is, hat if I remove all homebrew dependencies after I've compiled poppler it stops working because there are "links ? (dont know the exact term)" to these dependencies.
e.g.
cmd: './extlib/darwin/poppler/poppler-gitlab/cmake-build-release/utils/pdfinfo -v',
stdout: '',
stderr: 'dyld[93398]: Library not loaded: /usr/local/opt/fontconfig/lib/libfontconfig.1.dylib\n' +
' Referenced from: <0062B574-27F1-35D4-BEF8-81E2F2B5EEDB> /Users/bernhard´/Coding/Sides/backend/extlib/darwin/poppler/poppler-gitlab/cmake-build-release/libpoppler.126.0.0.dylib\n' +
" Reason: tried: '/usr/local/opt/fontconfig/lib/libfontconfig.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/fontconfig/lib/libfontconfig.1.dylib' (no such file), '/usr/local/opt/fontconfig/lib/libfontconfig.1.dylib' (no such file), '/usr/local/lib/libfontconfig.1.dylib' (no such file), '/usr/lib/libfontconfig.1.dylib' (no such file, not in dyld cache)\n"
After a lot of reading I've found out that it might be possible to compile it static (even unsure if I'm using the right term here).
I'd like to know what I neet to do, that I can ship the app and what's required, to build poppler in a way that is has no dependencies anymore or has it's dependencies in a place that I have control over and can ship it with my app.
Actually I'm building using following CMAKE options:
-G Ninja -DCMAKE_BUILD_TYPE=Release -DC_MAKE_PROGRAM=/Applications/CLion.app/Contents/bin/ninja/mac/ninja -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_VERBOSE_MAKEFILE=ON -DPOPPLER_QT5=OFF -DPOPPLER_ENABLE_CMS=none -DPOPPLER_ENABLE_SPLASH=OFF -DPOPPLER_ENABLE_UTILS=OFF -DPOPPLER_ENABLE_GLIB=OFF -DPOPPLER_ENABLE_CPP=OFF -DPOPPLER_ENABLE_XPDF_HEADERS=OFF -DPOPPLER_ENABLE_ZLIB=OFF -DPOPPLER_ENABLE_LIBCURL=OFF -DPOPPLER_ENABLE_LIBSYSTEMD=OFF -DPOPPLER_ENABLE_OPENJPEG=OFF -DPOPPLER_ENABLE_TIFF=OFF
Thanks for any help and sorry for maybe misusing terms as I'm mostly a frontend dev.
You have to compile your program with an RPATH option for your shipped libraries. The reason why you can't run it without those deleted dependencies is simple. (Regarding RPATH check out the ld man pages.)
Your application seems to be linking against the same version of the dependency as the one in your homebrew directory. Thus this shared object is compatible with your executable.
Your linker is only configured to look for the shared objects within certain directories. Thus when you delete the directory, your linker doesn't know where to find these shared objects anymore (because they do not exist).
On linux distributions you could run your application for example with: LD_LIBRARY_PATH=./path_to_your/library/ && ./your_executable. And it would suddenly work.
RPATH solves this by telling the linker where to look for the linked dependencies.
I would like to give you a general warning here. This is not the way to go when it comes to shipping your executable (and especially libraries) on different distros. It's better to make a package, using the available -dev packages for the given distribution. - Because otherwise you are slowly introducing dependency hell for the developers who wish to use it!
To answer your question regarding static linking: You will increase the size of your executable, so keep that in your mind if you wish to take this path instead
I have been developing a research code using CMake to generate the Makefiles for a c++ code on an Ubuntu machine. I link in several shared libraries which are rather involved to setup and build on a machine (one in particular has a dozen or so version specific dependencies which themselves are non trivial to build). Some are also custom builds of the library (bug fixes).
I am more familiar with the windows environment with the CLR, but, what I am hoping to achieve is building the binary and including all of the shared libraries along with it. Hopefully the linker on the other environment (redhat EL6) would then be able to use those shared objects at runtime.
Since the linker doesn't look in the applications path, I assume I would also need to bring the shared libraries into a user specific library path for it to find.
Is there a nice way using Cmake (perhaps Cpack), to build the binary and 'package' all of the shared objects with it for the other machine? Then I could (even if manually) install the shared libraries for my user only, and run the binary there.
I'm hoping the answer is not using static libraries, as that has given me a lot of trouble for these dependencies in the past.
I'm a linux noob, so if my issues is in lack of understanding a better approach I am all ears :)
One approach is to set the "rpath" of the binary, which hardcodes some additional search paths. One can set it to $ORIGIN, which means that libraries in the same directory as the binary itself will be used first.
IF(UNIX)
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_RPATH}:\$ORIGIN/../bin:\$ORIGIN")
ENDIF()
This CMake code takes effect at install time (make install), so first you need to set up the INSTALL commands of your CMake setup. INSTALL both your binary and all extra shared objects. Finally, CPack internally runs an "install", so once make install works, you can use CPack to automatically build a TGZ, or even an RPM if you're willing to invest the time to get it set up.
Here's an old answer of mine which talks about a similar but not identical issue, linked for completeness:
https://stackoverflow.com/a/22209962/1401351
So I recently got fed up with Windows and installed Linux Mint. I am trying to get a project to build I have in Code::Blocks. I have installed Code::Blocks but I need glew(as well as a few other libraries). I found it in the software manager and installed it. I've managed to locate and include the header files. But I feel like the next step should be relatively straightforward and all over the internet but (perhaps due to lack of proper terminology) I have been as of yet unable to locate an answer.
Do I need to locate the files on my system and link to each library manually? This is what I did on windows but I just downloaded the binaries and knew where they were. I found one library from the software manager and linked to it manually but it just feels like I'm doing it the wrong way. Since it's "installed" on the system is there some quick way to link?
You should use two flags for linker '-l' and '-L'. You can set these flags somewhere in project properties.
The first one '-l' tells linker to link with particular library. For example glew, probably in /usr/lib is a file named libglew.so, when you link your program with '-lglew' flag, it will link it with glew library. Linker looks for libraries in few standard places: /usr/lib, /usr/local/lib and few extra. If you have your libs in nonstandard place, use '-L' flag to point these dirs.
Many linux distributions provide two kinds of packages with libraries, regular ones just with runtime, and devel ones (usually prefixed or suffixed with dev or devel) with header files and development version of libraries.
use build systems, Luke!
the typical way to develop/build software in *nix world is 3 steps:
configure stage -- before building smth you have to realize in what environment you are going to build your software... is everything that required is installed... it wouldn't be good if at compile stage (after few hours of compilation) you (or user who build your soft) got an error: unable to #include the 'xxx.h'. the most popular build systems are: cmake, my favorite after autotools. yout may try also scons or maybe crazy (b)jam...
compile stage -- usually just make all
install stage -- deploy just built software into the system. or other way: build packages for target distro (.deb/.rpm/&etc)
at configuration stage using test scripts (don't worry there are plenty of them for various use cases) you can find all required headers/libraries/programs/compiler options/whatever you need to compile your package... and yes: do not use hardcoded paths in your Makefiles (or whatever you use to make your binaries)
Answer to this question really depends on what you want to achieve. If you want just to build you app by yourself then you can just write path to libraries in your makefile, or your code editor settings. You may not even have to do that as if libraries installed by your linux distribution package manager, headers usually go to /usr/include and libraries to /usr/lib or /urs/lib64 etc. That locations are standard and you do not need to specify them explicitly. Anyway you need to specify libraries you want to link to.
If you want to create application that can be build by others, or by you on many different configurations/environments using something like cmake would be very helpful.
I want to use a certain version of Qt4 in my project. I'm using debian and there is already an older version of Qt4 installed. When I'm using the find_package command in my CMakeLists file, of course, the system library is found, because the file /usr/share/cmake-2.8/Modules/FindQt4.cmake is used by cmake.
What I've done so far to link the newer Qt4, is to edit the paths with ccmake manually. The problem is I'm not allowed to install the newer Qt4 version in the directories of the system. Is there any easier solution to tell cmake don't use the system library just use another version. Of course I could create my own module and give the find_package command the path to my own module, but I think this is annoying and there have to be an more easier solution.
What I've also looked for, are there some environment variables which are used by the FindQt4.cmake module, but there aren't. - So, is there a general solution to avoid system libraries and to use libraries which installed in not system directories without doing some dirty tricks?
The only real way to use different versions of libraries on Linux is to use static linking. If you require the ability to use different versions of the same library simultaneously, you'll need to use an OS that supports that paradigm. Solaris (and its derivatives) is particularly good at that (compile with -L to point to the library to compile against and -R to indicate the path to that library at runtime). I believe BSD is the same.
Barring that, you'll want to create a chroot (a poor man's BSD jail) to deploy your application in. You'll install a copy of all the dependencies to the chroot, (use ldd to find them out) and the copy of Qt4 you wish to use.
Don't know about system libraries in general. For Qt, you set the search path to qmake and qt will do the rest. Qmake path is set through QT_QMAKE_EXECUTABLE, I use Qt commercial and this is how I set the path in Cmake.
set(QT_QMAKE_EXECUTABLE $ENV{HOME}/QtCommercialSDK/Desktop/483/gcc-64/bin/qmake)
I'm not new in C++ although I'm new in Linux. I'm using CMake to precompile a cross-platform game engine with some third-party components, but I have a lot of doubts about using libraries. My question is how to work with third-party libraries and where to put them. Apt installs libs in their official place (/usr/local, /usr/lib/ ..) but I develop in Windows using local libs that are in a folder in my project dir.
Also, I need a good tutorial to know the rules of how libraries work. For example: when trying to compile my project, luabind is asking for liblua.s0.1, but AFAIK there is no way to generate this library with the source provided by Lua (at least doing make, make install).
I know, this question is fuzzy but I haven't enough experience to be more concise.
Update: After reading some answers, a more concise question is the following. If I install all third-party libraries, how can I distribute my program? How do I manage dependencies without using a large readme?
Where to put libraries
The best solution is to use your Linux distribution's packaging system (apt-get, yum, or similar) to install libraries from distro-provided packages wherever possible.
If the distro's packaged libraries aren't of a recent enough version, or if you need some nonstandard build options, or if you need a library that your distro doesn't provide, then you can build and install it yourself. You have two main options for where to put the library:
/usr/local (libraries under /usr/local/lib, headers under /usr/local/include). This installs the libraries systemwide and is probably the simplest solution, since you should then be able to build against them without taking any extra steps. Do NOT install libraries directly under /usr, since that will interfere with your distro's packaging system.
Under your project directory, as you did under Windows. This has the advantages of not requiring root access and not making systemwide changes, but you'll have to update your project's include paths and library paths, and you'll have to put any shared library files someplace where the dynamic linker can find them (using LD_LIBRARY_PATH or ld.so.conf - see the link for more details).
How libraries work
See David A. Wheeler's excellent Programming Library HOWTO. I'd recommend reading that then posting any specific questions as new topics.
How to distribute your program
Traditionally, Unix / Linux programs do not include copies of their dependencies. It's instead up to the end user or developer to install those dependencies themselves. This can require a "large README," as you said, but it has a few advantages:
Development libraries can be installed, managed, and updated via the distro's package manager, instead of each source copy having its own set of libraries to track.
There's only one copy of any given library on a system, so there's only one place that needs updating if, for example, a security flaw is found. (For example, consider the chaos that resulted when zlib, a very widely used compression library, was found to have a security flaw, so every application that included an affected version needed to be updated.)
If your program is popular enough (and is open source or at least freely available), then package maintainers for various Linux distributions may want to package it and include it in their distro. Package maintainers really don't like bundled libraries. See, for example, Fedora's page on the topic.
If you're distributing your program to end users, you may want to consider offering a package (.dpkg or .rpm) that they could simply download and install without having to use source. Ideally, from the end user's perspective, the package would be added to distros' repositories (if it's open source or at least freely available) so that users can download it using their package managers (apt-get or yum). This can all get complicated, because of the large number of Linux distros out there, but a Debian/Ubuntu compatible .dpkg and a Red Hat/CentOS/Fedora-compatible .rpm should cover a good percentage of end users. Building packages isn't too hard, and there are good howtos online.
for the first part of your question regarding Windows: there's no real standard place for libraries/headers on Windows, so the easy solution is: create your own. Simply provide a single lib/ and include/ on your system and have all your projects use it (by setting the path in a cmake file that you include everywhere). Put all third party libs in there, for example:
your projects:
d:/projects/projectA
d:/projects/projectB
third party stuff:
d:/api/lib/lua.lib
d:/api/include/lua/....
(you can even use symlinks aka 'directory junctions' if you have different version)
and the corresponding cmake file:
include_directories( d:/api/include )
link_directories( d:/api/lib )
Okay, so this is one of the basic questions and while I myself might not come across very clear on this, here goes:
While building a project, your compiler will need to find the header files of the libraries. The headers must be in the include path.
after compilation is done, the linker will look for the library binaries (files.so or something like that). These must be in the Library path.
That's the basics.
If you have some specific libaries, you can add them to your own project-specific lib/ and include/ directories and add them to the include path and the library path respectively.
Adding these dirs to these paths can be done in many ways, depending upon how you are building the project. I'm sure there is something called LD_PATH involved in all this... But I don't really know the specifics involved with CMake.
A little googling can help you do the above with CMake.
Hope that helps,
jrh
If you are installing the libraries with a package manager, they will probably all end up in the right place. If not you can get the compiler to search for the by providing the an additional search path using the -L <path> flag. You should be able to pass this extra flag to CMake.
Incidentally the -I <path> can be used to add an extra directory to search for include files.