My Home directory is shared among several linux computers via Network File System.
I would like to install some C C++ library from source under my Home directory, and wish they can be used under all the linux computers.
Do I have to install different versions of the library under different directories of my Home for different computer?
Assuming I have a C C++ program that calls these libraries, how do I specify different include and link files and directories for different computer in Makefile? Is it to determine the directories based on the hostname of the computer?
Is it possible to combine the different versions of the .a and .so files and header files of the libary for different linux computers so that the include and link files and directories of the libary are the same for all the computers and I don't have to specify different directories for different computer in the Makefile of my C C++ program?
Thanks and regards!
This is easy and common.
By C C++ I assume you mean you have libraries compiled with a C compiler, and those compiled with a C++ compiler.
If the version of the compiler you are using is the same, then you do not need different libraries for each version. If they are different it may still be possible to use the same C libraries, but C++ becomes more problematic.
If the files are in your home directory, the easiest thing to do in your Makefile is make all of the paths relative to $HOME. This environment variable should be set correctly on each system.
If you need to reference different libraries on the different machines, the most straightforward way would be to put them in a directory with the same name as the hostname. Something like this:
CXXFLAGS=-I$(HOME)/app/$(HOST)/include
You could do something more fancy by extracting the gcc version number and using that, but its probably overkill for just a couple of machines.
Related
I need to create a portable linux program that uses a lot of additional libraries defined from yum (CentOS).
It is forbidden to install new packages on portable machines. There are no necessary libraries there.
How to assemble my program and all packages into a single folder through the gcc compiler? When I move this folder to another machine, my program should start and run successfully.
My program is ONLY allowed to use dynamic libraries. Static libraries are STRICTLY prohibited.
When trying to replace rpath with /usr/lib64/ with my libraries that are stored in my directory, after transferring to another machine, additional libraries give an error (glibc version conflict).
This sounds like a doomed project, for anything non-trivial.
Static libraries are not the issue though. Since they're just collections of .o files, you can unpack them. You can then state that you have just linked object files. Stupid rules give stupid results.
I am ignoring software licensing here, though, but that seems implied by the question. You don't need a license for libraries installed via yum, since YOU aren't shipping them. But you absolutely need licenses when you are shipping these libraries in one form or another as part of your product. And given the stupid rules, (L)GPL is likely out of the question, so you will need to obtain commercial licenses for all 7 libraries.
Let's say I have created and compiled a simple program using the MinGW 64 (g++ compiler). Running this program on my computer and looking in Process Explorer for what DLL files the program is using I find (among many others):
libgcc_s_seh-1.dll
libstdc++6.dll
libwinpthread-1.dll
These are the only ones that reside under my MinGW installation folder. The rest of the DLL files used reside under C:\Windows.
Question 1:
Are the MinGW DLL files the MinGW C++ runtime libraries (so to speak)? Do they serve the same purpose as for example msvcrXXX.dll (XXX = version of Microsoft runtime library).
Question 2:
If I want to run the application on a different computer which does not have MinGW installed, is it sufficient to include those DLL files listed above (i.e. placing them in the same folder as my executable) to have it run on the other computer (we assume the other computer is also a 64-bit Windows machine). If yes, does this mean we basically ship the MinGW C++ runtime with our executable. If no, why?
libstdc++6.dll is the C++ standard library, like you said.
libwinpthread-1.dll is for C++11 threading support. MinGW-W64 has two possible thread variants: Either use the native Windows functions like CreateThread, but C++11 stuff like std::thread won´t be available then; or include this library and use the C++11 classes (too).
Note that to switch the thread model, you´ll need to reinstall MinGW. Just removing the DLL and not using the C++11 stuff won´t work, the DLL will be required nonetheless with your current install.
libgcc_s_seh-1.dll is something about C++ exception handling.
Yes, it should be sufficient to deliver the DLLs too
(or use static linking and deliver only your program file).
For complicated projects where you're not exactly sure which DLL files need to be included to distribute your application, I made a handy dandy Bash script (for MSYS2 shells) that can tell you exactly what DLL files you need to include. It relies on the Dependency Walker binary.
#!/usr/bin/sh
depends_bin="depends.exe"
target="./build/main.exe" # Or wherever your binary is
temp_file=$(mktemp)
output="dll_list.txt"
MSYS2_ARG_CONV_EXCL="*" `cygpath -w $depends_bin` /c /oc:`cygpath -w $temp_file` `cygpath -w $target`
cat $temp_file | cut -d , -f 2 | grep mingw32 > $output
rm $temp_file
Note that this script would need to be modified slightly for use in regular MSYS (the MSYS2_ARG_CONV_EXCL and cygpath directives in particular). This script also assumes your MinGW DLL files are located in a path which contains MinGW.
You could potentially even use this script to automatically copy the DLL files in question into your build directory as part of an automatic deploy system.
You may like to add the options -static-libgcc and -static-libstdc++ to link the C and C++ standard libraries statically and thus remove the need to carry around any separate copies of those.
I used ntldd to get a list of dependencies.
https://github.com/LRN/ntldd
I'm using msys2 so i just installed it with pacman. Use that and then copy all the needed dependencies
There are several major challenges to distributing compiled software:
Compiling the code for all target processors (remember, when it comes to compiled code, you need to produce separate downloads/distributions for each type of instruction set architecture).
Ensuring that the builds are reproducible, consistent, and can be easily correlated with a specific version of the code (and versions of the dependencies).
Ensuring that the build output is self-contained and includes all of its dependencies within it (so that it is not dependent on any other installations that happen to exist on just your system).
Making sure that your code is built and distributed regularly, with updates distributed automatically so that -- in the event of security issues -- you can push out new patched versions.
For convenience and to increase reach, it is nice for non-savvy users to have a prebuilt version that they can install. However, I would recommend sharing the source code as a first step.
Most of these requirements are fairly non-trivial to hit and often require automating not only build process, but also automating the instantiation / configuration of VMs in which the build should take place. However, there are open source projects that can help... for example, check out Gitian.
In terms of bullet point #3, the key thing here is to use static linking... while this does make the binary you distribute much larger (because its dependencies are now baked into the output), it also makes your binary isolated from the version of the libraries on the system (avoiding "dependency hell").
Point #4 is very tricky, but thankfully there are also opensource tools to help here, as well such as cloudup, which provides a way to add auto-updating capability to your application distribution.
I'm totally spinning my wheels with getting a couple of 3rd party libraries to work with my c++ programs. I'm looking for some general advice (40,000 foot level) about the general steps that one needs to take when implementing libraries.
First, some specifics: I am using code::blocks in Windows as my IDE. I like this IDE and really don't want to switch to anything else if I don't have to (I've tried visual c++ and also some things in linux). The libraries that I am trying to use are GMP and crypto++.
OK. What I think I know is this: After downloading the library, I unzip the file to a folder. I've been unzipping directly to C:\ with each zip file extracted to its own folder (e.g. c:\cryptopp and c:\gmp). I think that the next step is to build the library, but this is where I get totally stuck. How is this done? There are no executable files among those extracted. From what I can tell, I believe that I do this in code::blocks, but I have no idea how?
Finally, assuming that I can get this done, which I believe creates the .lib files, the last step before actually using the library in my code, is to link into the library. This part, I believe that I understand.
So, my question is broad: do I understand this process overall? And if so, how do I go about building these libraries, if in fact that it the thing that I am missing.
Thanks very much for indulging my ignorance. I'm totally rudderless right now and despite hours and hours on google, I'm making no progress. Also, feel free to correct anything that I have stated as fact that is not correct. Thanks a lot!
Usually libraries have a special file called makefile in them, and are built with a utility called Make (or one of it's variations, whatever works uder windows).
Usually all you have to do is to run Make in the directory where you have unpacked the source files, and it will do the rest itself.
If those libraries you mention (GMP and crypto++; disclaimer: I'm not familiar with either of them) don't have project files for code::blocks then you may still be able to compile them under Windows with MinGW.
If you have installed MinGW you use the MinGW shell to navigate to the appropriate directories which would be /c/cryptopp/ and /c/gmp in your examples - The MinGW shell works like a Unix shell, and has different naming conventions.
Then you need to compile the libraries. Check whether there's a Makefile in those directories, if there isn't you can check whether there's a configure script, which will generate the Makefile. If you have the Makefile you can type make which will compile the libraries with MinGW's port of the GCC compiler.
When the compilation is complete you should have a library in the form of a .a file (say libcryptopp.a) that you can link to your project. In code::blocks you would set the linker path (the -L command line option in GCC) to C:\cryptopp\bin or wherever the library has been compiled, and then add libcryptopp.a to the list of libraries you want to link (this is associated with the -l option in GCC). The convention is to leave out the lib prefix and the .a extension, so you would just add cryptopp to your library list. In the end your linker options should look like -LC:\cryptopp\bin -lcryptopp along with the
Also, to be able to use the libraries you need to add the path to the headers directory to the include path of your project. This is associated to the -I command line option in GCC, so your compiler's command line options would have something like -IC:\cryptopp\include somewhere.
Of course, all of the above assumes that you use code::blocks with GCC. If you're using it with VisualC++ then the principles are the same, but the specific steps differ.
I am currently working in Ubuntu Linux. I am working with
a .hpp file and .cpp file. From these two I am
creating an .a file (like a dll in order to use and work with
my application on any computer that has linux installed).
I mention the fact that both .hpp and.cpp are in folder 1.
I would like to ask :
If I include in .cpp a header from a folder like:
#include "/home/tests/folder1/folder2/header.h"
will this work correctly after i create the .a using ar rcs and send my .a on another computer?
does the path to a specific header from a folder influence the .a created?
If I had to download for example gsoap in order
to accomplish my task, after I've created
the .a file that contains a lot of .xml and .cpp/.h files
from gsoap and from my .h and .cpp file do I need
to create a makefile in order to download gsoap
on the computer where I want to use my .a (or dll in windows)
application?
"Any computer that has linux installed" isn't going to work. Linux encases a wide variety of platforms and architectures, unlike Windows which generally encases only two (and the 64-bit versions are backward compatible with the 32-bit versions, so the .dlls always work).
As already mentioned elsewhere, a .a is a static library and is equivalent to a Windows .lib, not a .dll. The Linux equivalent to a Windows .dll is a .so "shared object".
No. #includes are resolved by the pre-processor in a step prior to compilation. The contents of the file are literally inserted into the copy of your source file in memory, then the whole lot is compiled. The string with the folder does not exist in your actual compiled module.That said, writing absolute paths is very bad form. It means you cannot move your development environment/directory anywhere. Use relative paths: they should be relative to your current working directory and/or to your defined include path. Read your toolchain's documentation for more information.
If you statically link gsoap, then you don't have to do anything. It's compiled into your project.If you want to dynamically link it, then your .a should not contain any .cpp files from gsoap. The target computer must have gsoap shared libraries installed, and this will be a required dependency that your installer or your user must resolve.Makefiles do not download dependencies. Package managers do.
Actually, a ".a" file is an archive file. Linux chose that format for its library files, so you can compare it to ".lib" (".so" is the rough equivalent of ".dll").
There are a number of stages of compilation: preprocessing, compiling, assembling and linking.
Preprocessing effectively answers your first question because code in the .h/.hpp file is inserted into the .cpp file, meaning that when your code is compiled, all code that is necessary to compile the .cpp file successfully is in that one file.
Compiling turns your code into assembly instructions for the specific computer that you're using. This means that if your code was built to run on a PowerPC computer (Mac), your code would use machine instructions that any PPC computer could use (meaning that Intel, AMD, SPARC, Alpha, etc. computers couldn't use your code). This answers your question about moving a ".a" file to another computer - you can use it as long as the computer's processor AND operating system is compatible (you may have a 64-bit processor, but that doesn't mean 32-bit Windows will let you use it to its full capacity).
Assembling converts the primitive text-based assembly instructions into machine instructions that the processor can understand. This creates an object file (.obj on Windows, .o on Linux). This file is what goes in the library (.lib on Windows, .a on Linux). There are other names for machine instructions such as "machine code" and "object code", and any one of them can be used to describe the same thing.
Linking is the last stage. It takes the necessary code from libraries and the various necessary object files and turns them into an actual binary (.exe file on Windows, Linux doesn't need an extension because of how it is designed). This is your application.
Because linking is the last stage, the gsoap library (for example) must be specified in addition to your library or else the linker will say it couldn't find certain "symbols". However, as with your library, the gsoap library must be on that computer to be able to use it. Installing it with the package manager is preferred when possible, but if you can't do that, you need to compile it on that computer. If you're moving from a PPC computer to an Intel/AMD computer, you would also need to re-compile your library as well as gsoap (if you couldn't install gsoap via package manager).
"does the path to a specific header from a folder influence the .a created?" - Only may be debug information. Nothing that would prevent it from working if you copy it to another place.
*.a is a static library. It is like *.lib in windows - not like *.dll
You can move any static and dynamic libraries (*.a/*.so in linux, *.lib/*.dll in windows) to any folder/computer/planet you like and use it there while dependencies are satisfied (available all the necessary dynamic and static libs, software, hardware that your library depend on). Of course running the code using your library will require the CPU architecture you've compiled for and all dynamic libs your code uses directly or indirectly.
Not directly related to the question asked: don't #include files by absolute paths. Ever. Define and use include directories. It is a matter of style and readability. Includes like "/home/user/working_dir/blabla.h" or "D:/working_dir/blabla.h" or "..\..\some\directory\blabla.h" is ugly and unmaintainable. Includes like <blabla.h> or <blabla/defs.h> is perfect for library APIs and like "blabla.h" or "subdir/blabla.h" is OK for internal headers.
I'm not new in C++ although I'm new in Linux. I'm using CMake to precompile a cross-platform game engine with some third-party components, but I have a lot of doubts about using libraries. My question is how to work with third-party libraries and where to put them. Apt installs libs in their official place (/usr/local, /usr/lib/ ..) but I develop in Windows using local libs that are in a folder in my project dir.
Also, I need a good tutorial to know the rules of how libraries work. For example: when trying to compile my project, luabind is asking for liblua.s0.1, but AFAIK there is no way to generate this library with the source provided by Lua (at least doing make, make install).
I know, this question is fuzzy but I haven't enough experience to be more concise.
Update: After reading some answers, a more concise question is the following. If I install all third-party libraries, how can I distribute my program? How do I manage dependencies without using a large readme?
Where to put libraries
The best solution is to use your Linux distribution's packaging system (apt-get, yum, or similar) to install libraries from distro-provided packages wherever possible.
If the distro's packaged libraries aren't of a recent enough version, or if you need some nonstandard build options, or if you need a library that your distro doesn't provide, then you can build and install it yourself. You have two main options for where to put the library:
/usr/local (libraries under /usr/local/lib, headers under /usr/local/include). This installs the libraries systemwide and is probably the simplest solution, since you should then be able to build against them without taking any extra steps. Do NOT install libraries directly under /usr, since that will interfere with your distro's packaging system.
Under your project directory, as you did under Windows. This has the advantages of not requiring root access and not making systemwide changes, but you'll have to update your project's include paths and library paths, and you'll have to put any shared library files someplace where the dynamic linker can find them (using LD_LIBRARY_PATH or ld.so.conf - see the link for more details).
How libraries work
See David A. Wheeler's excellent Programming Library HOWTO. I'd recommend reading that then posting any specific questions as new topics.
How to distribute your program
Traditionally, Unix / Linux programs do not include copies of their dependencies. It's instead up to the end user or developer to install those dependencies themselves. This can require a "large README," as you said, but it has a few advantages:
Development libraries can be installed, managed, and updated via the distro's package manager, instead of each source copy having its own set of libraries to track.
There's only one copy of any given library on a system, so there's only one place that needs updating if, for example, a security flaw is found. (For example, consider the chaos that resulted when zlib, a very widely used compression library, was found to have a security flaw, so every application that included an affected version needed to be updated.)
If your program is popular enough (and is open source or at least freely available), then package maintainers for various Linux distributions may want to package it and include it in their distro. Package maintainers really don't like bundled libraries. See, for example, Fedora's page on the topic.
If you're distributing your program to end users, you may want to consider offering a package (.dpkg or .rpm) that they could simply download and install without having to use source. Ideally, from the end user's perspective, the package would be added to distros' repositories (if it's open source or at least freely available) so that users can download it using their package managers (apt-get or yum). This can all get complicated, because of the large number of Linux distros out there, but a Debian/Ubuntu compatible .dpkg and a Red Hat/CentOS/Fedora-compatible .rpm should cover a good percentage of end users. Building packages isn't too hard, and there are good howtos online.
for the first part of your question regarding Windows: there's no real standard place for libraries/headers on Windows, so the easy solution is: create your own. Simply provide a single lib/ and include/ on your system and have all your projects use it (by setting the path in a cmake file that you include everywhere). Put all third party libs in there, for example:
your projects:
d:/projects/projectA
d:/projects/projectB
third party stuff:
d:/api/lib/lua.lib
d:/api/include/lua/....
(you can even use symlinks aka 'directory junctions' if you have different version)
and the corresponding cmake file:
include_directories( d:/api/include )
link_directories( d:/api/lib )
Okay, so this is one of the basic questions and while I myself might not come across very clear on this, here goes:
While building a project, your compiler will need to find the header files of the libraries. The headers must be in the include path.
after compilation is done, the linker will look for the library binaries (files.so or something like that). These must be in the Library path.
That's the basics.
If you have some specific libaries, you can add them to your own project-specific lib/ and include/ directories and add them to the include path and the library path respectively.
Adding these dirs to these paths can be done in many ways, depending upon how you are building the project. I'm sure there is something called LD_PATH involved in all this... But I don't really know the specifics involved with CMake.
A little googling can help you do the above with CMake.
Hope that helps,
jrh
If you are installing the libraries with a package manager, they will probably all end up in the right place. If not you can get the compiler to search for the by providing the an additional search path using the -L <path> flag. You should be able to pass this extra flag to CMake.
Incidentally the -I <path> can be used to add an extra directory to search for include files.