I'm trying to do a Linux release for a piece of Qt software, but I'm a clueless Linux newbie. I've built my application with the Qt libraries statically linked, and made .deb and .rpm packages, but when I test them I get warnings about missing libraries on some distributions. For example, my .deb version runs fine on Debian and Ubuntu, but it doesn't start on Linux Mint KDE due to missing libraries. Running ldd I find there are four missing libraries:
libpng16.so.16 => not found
libicui18n.so.57 => not found
libicuuc.so.57 => not found
libicudata.so.57 => not found
Clearly I need to somehow ensure all the necessary library are present, however the list of libraries required seems to vary. For example Linux Mint Cinnamon lists 33 libraries, while Ubuntu Unity lists 34 (libpng12.so.0 is the extra one). I also assume that some of those libraries are standard Linux libraries that would come with all distributions so I wouldn't need to include them.
I don't really know what I'm doing, so a few questions:
What is the best way to determine the list of libraries I need to include with my .deb and .rpm packages? If ldd is the best option, do I need to include all the listed libraries or are there some that are bundled as standard with all Debian/rpm distributions?
What is the best way to to include those libraries? Do you include them in the package or specify them in the dependency list?
Any other general advice would be appreciated because I'm not really sure I'm going about this the right way.
Thanks.
Don't think about library dependencies; think about package dependencies.
When you create your (deb, rpm) package, you can list other packages that it depends on. Figure out which package has your desired library in it, and add that package as a dependency.
Then, when the user installs your package, their package manager will also, if it needs to, install the packages that it depends on. Presto: you have the libraries you need.
Related
I have been developing a research code using CMake to generate the Makefiles for a c++ code on an Ubuntu machine. I link in several shared libraries which are rather involved to setup and build on a machine (one in particular has a dozen or so version specific dependencies which themselves are non trivial to build). Some are also custom builds of the library (bug fixes).
I am more familiar with the windows environment with the CLR, but, what I am hoping to achieve is building the binary and including all of the shared libraries along with it. Hopefully the linker on the other environment (redhat EL6) would then be able to use those shared objects at runtime.
Since the linker doesn't look in the applications path, I assume I would also need to bring the shared libraries into a user specific library path for it to find.
Is there a nice way using Cmake (perhaps Cpack), to build the binary and 'package' all of the shared objects with it for the other machine? Then I could (even if manually) install the shared libraries for my user only, and run the binary there.
I'm hoping the answer is not using static libraries, as that has given me a lot of trouble for these dependencies in the past.
I'm a linux noob, so if my issues is in lack of understanding a better approach I am all ears :)
One approach is to set the "rpath" of the binary, which hardcodes some additional search paths. One can set it to $ORIGIN, which means that libraries in the same directory as the binary itself will be used first.
IF(UNIX)
SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_RPATH}:\$ORIGIN/../bin:\$ORIGIN")
ENDIF()
This CMake code takes effect at install time (make install), so first you need to set up the INSTALL commands of your CMake setup. INSTALL both your binary and all extra shared objects. Finally, CPack internally runs an "install", so once make install works, you can use CPack to automatically build a TGZ, or even an RPM if you're willing to invest the time to get it set up.
Here's an old answer of mine which talks about a similar but not identical issue, linked for completeness:
https://stackoverflow.com/a/22209962/1401351
So I recently got fed up with Windows and installed Linux Mint. I am trying to get a project to build I have in Code::Blocks. I have installed Code::Blocks but I need glew(as well as a few other libraries). I found it in the software manager and installed it. I've managed to locate and include the header files. But I feel like the next step should be relatively straightforward and all over the internet but (perhaps due to lack of proper terminology) I have been as of yet unable to locate an answer.
Do I need to locate the files on my system and link to each library manually? This is what I did on windows but I just downloaded the binaries and knew where they were. I found one library from the software manager and linked to it manually but it just feels like I'm doing it the wrong way. Since it's "installed" on the system is there some quick way to link?
You should use two flags for linker '-l' and '-L'. You can set these flags somewhere in project properties.
The first one '-l' tells linker to link with particular library. For example glew, probably in /usr/lib is a file named libglew.so, when you link your program with '-lglew' flag, it will link it with glew library. Linker looks for libraries in few standard places: /usr/lib, /usr/local/lib and few extra. If you have your libs in nonstandard place, use '-L' flag to point these dirs.
Many linux distributions provide two kinds of packages with libraries, regular ones just with runtime, and devel ones (usually prefixed or suffixed with dev or devel) with header files and development version of libraries.
use build systems, Luke!
the typical way to develop/build software in *nix world is 3 steps:
configure stage -- before building smth you have to realize in what environment you are going to build your software... is everything that required is installed... it wouldn't be good if at compile stage (after few hours of compilation) you (or user who build your soft) got an error: unable to #include the 'xxx.h'. the most popular build systems are: cmake, my favorite after autotools. yout may try also scons or maybe crazy (b)jam...
compile stage -- usually just make all
install stage -- deploy just built software into the system. or other way: build packages for target distro (.deb/.rpm/&etc)
at configuration stage using test scripts (don't worry there are plenty of them for various use cases) you can find all required headers/libraries/programs/compiler options/whatever you need to compile your package... and yes: do not use hardcoded paths in your Makefiles (or whatever you use to make your binaries)
Answer to this question really depends on what you want to achieve. If you want just to build you app by yourself then you can just write path to libraries in your makefile, or your code editor settings. You may not even have to do that as if libraries installed by your linux distribution package manager, headers usually go to /usr/include and libraries to /usr/lib or /urs/lib64 etc. That locations are standard and you do not need to specify them explicitly. Anyway you need to specify libraries you want to link to.
If you want to create application that can be build by others, or by you on many different configurations/environments using something like cmake would be very helpful.
I'm not new in C++ although I'm new in Linux. I'm using CMake to precompile a cross-platform game engine with some third-party components, but I have a lot of doubts about using libraries. My question is how to work with third-party libraries and where to put them. Apt installs libs in their official place (/usr/local, /usr/lib/ ..) but I develop in Windows using local libs that are in a folder in my project dir.
Also, I need a good tutorial to know the rules of how libraries work. For example: when trying to compile my project, luabind is asking for liblua.s0.1, but AFAIK there is no way to generate this library with the source provided by Lua (at least doing make, make install).
I know, this question is fuzzy but I haven't enough experience to be more concise.
Update: After reading some answers, a more concise question is the following. If I install all third-party libraries, how can I distribute my program? How do I manage dependencies without using a large readme?
Where to put libraries
The best solution is to use your Linux distribution's packaging system (apt-get, yum, or similar) to install libraries from distro-provided packages wherever possible.
If the distro's packaged libraries aren't of a recent enough version, or if you need some nonstandard build options, or if you need a library that your distro doesn't provide, then you can build and install it yourself. You have two main options for where to put the library:
/usr/local (libraries under /usr/local/lib, headers under /usr/local/include). This installs the libraries systemwide and is probably the simplest solution, since you should then be able to build against them without taking any extra steps. Do NOT install libraries directly under /usr, since that will interfere with your distro's packaging system.
Under your project directory, as you did under Windows. This has the advantages of not requiring root access and not making systemwide changes, but you'll have to update your project's include paths and library paths, and you'll have to put any shared library files someplace where the dynamic linker can find them (using LD_LIBRARY_PATH or ld.so.conf - see the link for more details).
How libraries work
See David A. Wheeler's excellent Programming Library HOWTO. I'd recommend reading that then posting any specific questions as new topics.
How to distribute your program
Traditionally, Unix / Linux programs do not include copies of their dependencies. It's instead up to the end user or developer to install those dependencies themselves. This can require a "large README," as you said, but it has a few advantages:
Development libraries can be installed, managed, and updated via the distro's package manager, instead of each source copy having its own set of libraries to track.
There's only one copy of any given library on a system, so there's only one place that needs updating if, for example, a security flaw is found. (For example, consider the chaos that resulted when zlib, a very widely used compression library, was found to have a security flaw, so every application that included an affected version needed to be updated.)
If your program is popular enough (and is open source or at least freely available), then package maintainers for various Linux distributions may want to package it and include it in their distro. Package maintainers really don't like bundled libraries. See, for example, Fedora's page on the topic.
If you're distributing your program to end users, you may want to consider offering a package (.dpkg or .rpm) that they could simply download and install without having to use source. Ideally, from the end user's perspective, the package would be added to distros' repositories (if it's open source or at least freely available) so that users can download it using their package managers (apt-get or yum). This can all get complicated, because of the large number of Linux distros out there, but a Debian/Ubuntu compatible .dpkg and a Red Hat/CentOS/Fedora-compatible .rpm should cover a good percentage of end users. Building packages isn't too hard, and there are good howtos online.
for the first part of your question regarding Windows: there's no real standard place for libraries/headers on Windows, so the easy solution is: create your own. Simply provide a single lib/ and include/ on your system and have all your projects use it (by setting the path in a cmake file that you include everywhere). Put all third party libs in there, for example:
your projects:
d:/projects/projectA
d:/projects/projectB
third party stuff:
d:/api/lib/lua.lib
d:/api/include/lua/....
(you can even use symlinks aka 'directory junctions' if you have different version)
and the corresponding cmake file:
include_directories( d:/api/include )
link_directories( d:/api/lib )
Okay, so this is one of the basic questions and while I myself might not come across very clear on this, here goes:
While building a project, your compiler will need to find the header files of the libraries. The headers must be in the include path.
after compilation is done, the linker will look for the library binaries (files.so or something like that). These must be in the Library path.
That's the basics.
If you have some specific libaries, you can add them to your own project-specific lib/ and include/ directories and add them to the include path and the library path respectively.
Adding these dirs to these paths can be done in many ways, depending upon how you are building the project. I'm sure there is something called LD_PATH involved in all this... But I don't really know the specifics involved with CMake.
A little googling can help you do the above with CMake.
Hope that helps,
jrh
If you are installing the libraries with a package manager, they will probably all end up in the right place. If not you can get the compiler to search for the by providing the an additional search path using the -L <path> flag. You should be able to pass this extra flag to CMake.
Incidentally the -I <path> can be used to add an extra directory to search for include files.
My program is written in C++, using GCC on Ubuntu 9.10 64 bit. If depends on /usr/lib64/libstdc++.so.6 which actually points to /usr/lib64/libstdc++.so.6.0.13. Now I copy this program to virgin Ubuntu 7.04 system and try to run it. It doesn't run, as expected. Then I add to the program directory the following files:
libstdc++.so.6.0.13
libstdc++.so.6 (links to libstdc++.so.6.0.13)
Execute command:
LD_LIBRARY_PATH=. ./myprogram
Now everything is OK. The question: how can I write installation script for such program? myprogram file itself should be placed to /usr/local/bin. What can I do with dependencies? For example, on destination computer, /usr/lib64/libstdc++.so.6 link points to /usr/lib64/libstdc++.so.6.0.8. What can I do with this?
Note: the program is closed-source, I cannot provide source code and makefile.
If you're working on Ubuntu, making a .deb (Debian Package) seems to way to go. Here is a link to get you started.
Your package will state it depends on some other packages (typically the packages that includes libstdc++.so.6.0.13 - i guess the package name is something like libstdc++) and dependencies will be installed when you install your own package using dpkg -i <yourpackage>.deb.
Afterwards, you'll be able to uninstall it using dpkg -r <yourpackage>.
Anyway, never ship such standards files with your own archive. Dependencies exists for this exact purpose.
Hope it helps.
The real problem is that you try to install a binary that use newer versions os common libraries that the ones available on Ubuntu 9.10. The best option should be to make a specific target for the old Ubuntu 7.10 an compile it with the old libraries (that's a backport).
Then you should make two (or more) .deb packages, one for Ubuntu 9.10 and one for Ubuntu 7.10.
Another possibility is to continue doing what you are doing now : set LD_LIBRARY_PATH to point to the desired version of libstdc++ and other necessary libraries. You just set this environment variable in a launcher shell script. In you script you check if the new libraries are available or not and you set your LB_LIBRARY_PATH (say to /usr/local/lib/myprogram/) only if needed. But as others poster pointed out : that's a very bad practice. Anyway, never try to put these provided libraries at their standard place in Ubuntu 9.10, you would risk broking the target system and causing update problems for users or your program if these libraries are officially backported someday.
But if you choose to include your own set of system libraries there is still another way to go than the above one: just link these libraries statically. If a program is the only user of a library as it will probably be in the above scenario you'll lose all advantages of using a shared dynamic library, then why bother with using it at all ? And with stacically linked libraries you won't have to install them.
I have a c++ code that runs perfect on my linux machine (Ubuntu Karmic).
When I try to run it on another version, I have all sort of shared libraries missing.
Is there any way to merge all shared libraries into single executable?
Edit:
I think I've asked the wrong question. I should have ask for a way to static-link my executable when it is already built.
I found the answer in ermine & statifier
There are 3 possible reasons you have shared libraries missing:
you are using shared libraries which do not exist by default on the other distribution, or you have installed them on your host, but not the other one, e.g. libDBI.so
you have over-specified the version at link time, e.g. libz.so.1.2.3 and the other machine has an API compatible (major version 1) but different minor version 2.3, which would probably work with your program if only it would link
the major version of the library has changed, which means it is incompatible libc.so.2 vs libc.so.1.
The fixes are:
don't link libraries which you don't need that may not be on different distros, OR, install the additional libraries on the other machines, either manually or make them dependencies of your installer package (e.g. use RPM)
don't specify the versions so tightly on the command line - link libz.so.1 instead of libz.so.1.2.3.
compile multiple versions against different libc versions.
What you are describing is the use of static libraries instead of shared libraries.
There have been several technical solutions to the original problem noted here, e.g.
compile multiple versions against
different libc versions.
or
install the additional libraries on the
other machines
but if you're in the position of an ISV, there is really just one sane solution:
Get a clean install of an older system, (e.g. Ubuntu 6.x if you're targeting desktops, perhaps as far back as Red Hat 9 if you're targeting servers) and build your software on that. Generally libraries (and definitely libc) are backwards compatible, so you you won't have problems running on newer systems.
Of course if you have non-standard or recent-version lib dependencies this doesn't completely solve the problem. In that case, as other's have suggested, if you want to be robust it's better to dlopen() and report the problems (or run with reduced functionality).
I am not too sure, but you may want to create your executable by statically linking all the libraries.
One alternative is to dynamically load shared libraries using dlopen() and if it fails to load, exit gracefully with the message that the dependent library is required for the executable to work.
The user then may install the appropriate library.
Another possible solution is using statifier (http://statifier.sf.net) or Ermine (http://magicErmine.com)
Both of them are able pack dynamic executable and all of it's needed libraries into one self-containing executable