I am currently trying to setup a project in C++, b that uses the luabind library. Unfortunately on my distro, namely Arch, this library isn't in the official repos and the one in the AUR is out of date and fails to compile.
Considering that I need the library only for this project I thought that I could make a sandboxed environment similar to python's virtualenv by building the library then installing(copying) the include files and resulting binaries in 2 sub-directories of my project called include and lib, respectively which I'll add to the linking and include paths when building. I understand why distributing the libraries with your project is bad: security and bug fixes in the meantime for example. However distributing DLLs is almost universally done on Windows(which I might do if I cross-compile) and many projects such as games on Linux tend to package their libraries to avoid inconsistencies between disrtos. Moreover if ever need a patched or forked version of a lib I doubt I'll ever find it in any official repo.
So my question is:
Is what I described above a common practice? Should I do it like this?
If not, what is the most commonly-agreed-upon solution to this problem?
NOTE: I use Cmake for build automation, if it matters
EDIT: This question slightly overlaps with mine.
Your approach is interesting, but it is not necessary for you to devise a working system because it has already been done, and luckily, you are only one step away from the solution !
Using CMake, it is easy to automate the building and linking of external source code, using the ExternalProject module.
See http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html for useful information.
This approach has several advantages:
you do not have to include the library's source code in your repository
you can point to the specific version/git tag of the library that you know works with your software OR the latest release if you are certain it will not break compatibility
you do not have to write a complete CMakeLists.txt file to build a possibly complex code base
you can eventually configure the external project to build as a static library so you will not have to distribute shared libraries
you can even completely bypass this if not necessary, by trying to detect a working version of the library on your system with the usual find_package call, and only fall back to building it as an external project if not found
Related
When I'm using CMake and library which also uses CMake, I add the library directory in my CMake project to allow building the library alongside my project. For example:
# add SFML library dependencies
add_subdirectory("third_party/lib/SFML")
include_directories("third_party/lib/SFML/include")
target_link_libraries(${CMAKE_PROJECT_NAME} sfml-system sfml-window sfml-graphics)
Then CMake automatically matches project Debug builds to use library Debug build, and project Release builds to use library Release build. In some cases it is useful build targets to be controlled separately for the main project and every one of the libraries used by it. For example, if I'm not interested in debugging inside the library code I will want to build in Debug mode only my code and to link it against Release version of the library, because I don't want to sacrifice additional performance. In other cases maybe I want to debug only inside one of the used libraries, if I have suspicious for bug inside it, but again for performance reasons I want to link release versions for all other libraries. Is it possible and what is the best way to achieve this behavior?
With both imported targets and dependent targets from the same build tree, you will always get the behavior you described, that each configuration uses its own matching build of the library. Messing with this means fiddling with CMake's internals, so I'd advise against it.
If you want to link against a specific version of the library, the most robust way is to use find_library. Note that this will only work if the library dependency is already available in its binary form at configure time. That is, you can no longer build the dependency as part of the dependent project.
If that is not an option, consider using ExternalProject_Add to build the dependency and specify the location of the dependency binary manually.
All in all, your current approach is the most convenient one, so only change this if performance of the dependency's debug build is a real problem. Also note that while mixing debug and release builds is mostly fine for C libraries, it can easily break for C++ libraries, especially if you have standard library types on the interfaces.
When we build the opencv library, either in a dynamic way or in a static way, we will produce several separated libraries. Take the 2.48 version for example, we will have
opencv_core248 opencv_imgproc248 zlib IlmImf comctl32 opencv_highgui248 libpng
and so on. So my question here is clearly there are some dependencies between these libraries, for example, if I want to invoke opencv_core248 and opencv_imgproc24 library in a project, I have link errors. Then, if I add zlib library, the compilation error will be solved.
Question: I want to ask a very general question: how can I know the dependencies between all the libraries insider opencv? Are there some documents I can follow? Thanks.
I don't think there is a document listing all the dependencies between the OpenCV libraries.
However I can suggest two methods to find out these dependencies:
Using Dependency Walker, a free tool allowing to analyze executables and DLLs. For instance, if you open opencv_calib3dXXX.DLL (where XXX represents your OpenCV version), you'll see that it requires opencv_coreXXX.dll, opencv_flannXXX.dll, opencv_imgprocXXX.dll and opencv_features2dXXX.dll and some system DLLs.
Using the project structure generated by CMake, a free tool for cross-platform compilation which is used for compliing OpenCV from sources. For instance, if I generate the project structure for VS2010 and open it, I can right-click on the project associated to opencv_calib3d and view the project dependencies.
I mentionned the tools I know and use for Windows, but equivalent tools must also exist for other platforms.
So I recently got fed up with Windows and installed Linux Mint. I am trying to get a project to build I have in Code::Blocks. I have installed Code::Blocks but I need glew(as well as a few other libraries). I found it in the software manager and installed it. I've managed to locate and include the header files. But I feel like the next step should be relatively straightforward and all over the internet but (perhaps due to lack of proper terminology) I have been as of yet unable to locate an answer.
Do I need to locate the files on my system and link to each library manually? This is what I did on windows but I just downloaded the binaries and knew where they were. I found one library from the software manager and linked to it manually but it just feels like I'm doing it the wrong way. Since it's "installed" on the system is there some quick way to link?
You should use two flags for linker '-l' and '-L'. You can set these flags somewhere in project properties.
The first one '-l' tells linker to link with particular library. For example glew, probably in /usr/lib is a file named libglew.so, when you link your program with '-lglew' flag, it will link it with glew library. Linker looks for libraries in few standard places: /usr/lib, /usr/local/lib and few extra. If you have your libs in nonstandard place, use '-L' flag to point these dirs.
Many linux distributions provide two kinds of packages with libraries, regular ones just with runtime, and devel ones (usually prefixed or suffixed with dev or devel) with header files and development version of libraries.
use build systems, Luke!
the typical way to develop/build software in *nix world is 3 steps:
configure stage -- before building smth you have to realize in what environment you are going to build your software... is everything that required is installed... it wouldn't be good if at compile stage (after few hours of compilation) you (or user who build your soft) got an error: unable to #include the 'xxx.h'. the most popular build systems are: cmake, my favorite after autotools. yout may try also scons or maybe crazy (b)jam...
compile stage -- usually just make all
install stage -- deploy just built software into the system. or other way: build packages for target distro (.deb/.rpm/&etc)
at configuration stage using test scripts (don't worry there are plenty of them for various use cases) you can find all required headers/libraries/programs/compiler options/whatever you need to compile your package... and yes: do not use hardcoded paths in your Makefiles (or whatever you use to make your binaries)
Answer to this question really depends on what you want to achieve. If you want just to build you app by yourself then you can just write path to libraries in your makefile, or your code editor settings. You may not even have to do that as if libraries installed by your linux distribution package manager, headers usually go to /usr/include and libraries to /usr/lib or /urs/lib64 etc. That locations are standard and you do not need to specify them explicitly. Anyway you need to specify libraries you want to link to.
If you want to create application that can be build by others, or by you on many different configurations/environments using something like cmake would be very helpful.
I have just downloaded the boost libraries from the boost website and extracted them to my desktop. I was hoping to just have a quick look at them and have them installed on my machine and perhaps use them in the future when I am more accustomed to C++.
When I extracted it, I was confused with all of the extracted files. There is all of the headers in the boost directory but tutorials mention running bootstrap.bat (I'm using Windows).
So I am asking this: do I simply extract the headers to my compilers include directory like normal to get boost up and running or do I need to do something else?
As I understand it from searching about, apparently "most" of boost is just templates and can be used simply by including the headers, but what about the rest?
Am I totally barking up the wrong tree?
Thanks for any help
Since you mentioned you run Windows, take a look at this automated installer:
► http://www.boostpro.com/download/
Also, some general advice:
do I simply extract the headers to my compilers include directory
No! Do not pollute your compiler's includes with third-party includes; make a separate directory specifically for a particular library. You'll then need to tell your specific IDE in what directory it can find the library headers.
I usually use boostpro's installer, it is less work. I vaguely remember having to set up the BOOST_ROOT environment variable on one of my systems to use it.
The libraries that contained compiled source should be included in the installer.
If you don't use the installer (or don't set up your build correctly), and try to use the libraries that need it you will likely get some linker errors when you try and compile your program. Usually if you take those linker errors and plop them in google it tells you pretty quick which libraries you need to include in your build system. I use CMake for that and have been very happy..
Just add the root boost directory to include paths of your compiler/IDE (so if you have Boost extracted to C:\Boost, the compiler will use that path, not C:\Boost\boost).
Don't do any copying of the boost folder to your compiler's include directory, because it may make upgrading Boost harder in the future.
Also if you plan to use any of boost's compiled libraries, add Boost's lib directory to compiler's library search paths. Configuring the compiling step is a matter of putting the right toolset parameter to boost's build tool. If you have your command line environment configured properly, bootstrap should run and compile the builder without any problems, and the Boost builder should properly detect your toolset, so no parameters will be necessary.
As you do such configuration only once every time you do a clean install of your favorite compiler, it's not as hard or daunting as it seems.
I'm not new in C++ although I'm new in Linux. I'm using CMake to precompile a cross-platform game engine with some third-party components, but I have a lot of doubts about using libraries. My question is how to work with third-party libraries and where to put them. Apt installs libs in their official place (/usr/local, /usr/lib/ ..) but I develop in Windows using local libs that are in a folder in my project dir.
Also, I need a good tutorial to know the rules of how libraries work. For example: when trying to compile my project, luabind is asking for liblua.s0.1, but AFAIK there is no way to generate this library with the source provided by Lua (at least doing make, make install).
I know, this question is fuzzy but I haven't enough experience to be more concise.
Update: After reading some answers, a more concise question is the following. If I install all third-party libraries, how can I distribute my program? How do I manage dependencies without using a large readme?
Where to put libraries
The best solution is to use your Linux distribution's packaging system (apt-get, yum, or similar) to install libraries from distro-provided packages wherever possible.
If the distro's packaged libraries aren't of a recent enough version, or if you need some nonstandard build options, or if you need a library that your distro doesn't provide, then you can build and install it yourself. You have two main options for where to put the library:
/usr/local (libraries under /usr/local/lib, headers under /usr/local/include). This installs the libraries systemwide and is probably the simplest solution, since you should then be able to build against them without taking any extra steps. Do NOT install libraries directly under /usr, since that will interfere with your distro's packaging system.
Under your project directory, as you did under Windows. This has the advantages of not requiring root access and not making systemwide changes, but you'll have to update your project's include paths and library paths, and you'll have to put any shared library files someplace where the dynamic linker can find them (using LD_LIBRARY_PATH or ld.so.conf - see the link for more details).
How libraries work
See David A. Wheeler's excellent Programming Library HOWTO. I'd recommend reading that then posting any specific questions as new topics.
How to distribute your program
Traditionally, Unix / Linux programs do not include copies of their dependencies. It's instead up to the end user or developer to install those dependencies themselves. This can require a "large README," as you said, but it has a few advantages:
Development libraries can be installed, managed, and updated via the distro's package manager, instead of each source copy having its own set of libraries to track.
There's only one copy of any given library on a system, so there's only one place that needs updating if, for example, a security flaw is found. (For example, consider the chaos that resulted when zlib, a very widely used compression library, was found to have a security flaw, so every application that included an affected version needed to be updated.)
If your program is popular enough (and is open source or at least freely available), then package maintainers for various Linux distributions may want to package it and include it in their distro. Package maintainers really don't like bundled libraries. See, for example, Fedora's page on the topic.
If you're distributing your program to end users, you may want to consider offering a package (.dpkg or .rpm) that they could simply download and install without having to use source. Ideally, from the end user's perspective, the package would be added to distros' repositories (if it's open source or at least freely available) so that users can download it using their package managers (apt-get or yum). This can all get complicated, because of the large number of Linux distros out there, but a Debian/Ubuntu compatible .dpkg and a Red Hat/CentOS/Fedora-compatible .rpm should cover a good percentage of end users. Building packages isn't too hard, and there are good howtos online.
for the first part of your question regarding Windows: there's no real standard place for libraries/headers on Windows, so the easy solution is: create your own. Simply provide a single lib/ and include/ on your system and have all your projects use it (by setting the path in a cmake file that you include everywhere). Put all third party libs in there, for example:
your projects:
d:/projects/projectA
d:/projects/projectB
third party stuff:
d:/api/lib/lua.lib
d:/api/include/lua/....
(you can even use symlinks aka 'directory junctions' if you have different version)
and the corresponding cmake file:
include_directories( d:/api/include )
link_directories( d:/api/lib )
Okay, so this is one of the basic questions and while I myself might not come across very clear on this, here goes:
While building a project, your compiler will need to find the header files of the libraries. The headers must be in the include path.
after compilation is done, the linker will look for the library binaries (files.so or something like that). These must be in the Library path.
That's the basics.
If you have some specific libaries, you can add them to your own project-specific lib/ and include/ directories and add them to the include path and the library path respectively.
Adding these dirs to these paths can be done in many ways, depending upon how you are building the project. I'm sure there is something called LD_PATH involved in all this... But I don't really know the specifics involved with CMake.
A little googling can help you do the above with CMake.
Hope that helps,
jrh
If you are installing the libraries with a package manager, they will probably all end up in the right place. If not you can get the compiler to search for the by providing the an additional search path using the -L <path> flag. You should be able to pass this extra flag to CMake.
Incidentally the -I <path> can be used to add an extra directory to search for include files.