I was wondering why executables (written in c++) have to be recompiled from sources on every linux machine, even if the machines are software and hardware the same?
I had a search engine, written in c++, and I have to recompile it every time I want to move it on a new linux machine to make it work.
Any ideas?
If you are asking why an executable compiled on Linux-X won't run on Linux-Y, then the reason is probably that dynamic libraries (.so) are missing or could not be found.
EDIT: oh sorry, looks like I didn't read your question well enough. Removed the sarcasm.
It normally shouldn't be necessary to recompile. Many applications are distributed as executables and they work fine.
What errors do you get when you just copy the executable and run it on a different machine?
Maybe the problem is with the way you're copying the executable, it might be corrupting it.
The recompilation is ensuring you get optimal performance on your machine, because each time the configuration script is running to find dependencies and settings. This also ensures the openness of software as its source is always available and is modifiable by an appropriate agents - that is us.
This is not necessarily the case. Consider how Ubuntu packages are installed for example - https://askubuntu.com/questions/162477/how-are-packages-actually-installed-via-apt-get-install
Apparently these are not compiled from source on the destination machine, but installed as pre-built binaries. Having said that, it is generally a good idea to build binaries from source on the machine that they will run on as you will avoid potential problems such having incompatible shared libraries (such as libc) which can occur when building something on Linux X and running on Linux Y.
Related
Let's say I have created and compiled a simple program using the MinGW 64 (g++ compiler). Running this program on my computer and looking in Process Explorer for what DLL files the program is using I find (among many others):
libgcc_s_seh-1.dll
libstdc++6.dll
libwinpthread-1.dll
These are the only ones that reside under my MinGW installation folder. The rest of the DLL files used reside under C:\Windows.
Question 1:
Are the MinGW DLL files the MinGW C++ runtime libraries (so to speak)? Do they serve the same purpose as for example msvcrXXX.dll (XXX = version of Microsoft runtime library).
Question 2:
If I want to run the application on a different computer which does not have MinGW installed, is it sufficient to include those DLL files listed above (i.e. placing them in the same folder as my executable) to have it run on the other computer (we assume the other computer is also a 64-bit Windows machine). If yes, does this mean we basically ship the MinGW C++ runtime with our executable. If no, why?
libstdc++6.dll is the C++ standard library, like you said.
libwinpthread-1.dll is for C++11 threading support. MinGW-W64 has two possible thread variants: Either use the native Windows functions like CreateThread, but C++11 stuff like std::thread won´t be available then; or include this library and use the C++11 classes (too).
Note that to switch the thread model, you´ll need to reinstall MinGW. Just removing the DLL and not using the C++11 stuff won´t work, the DLL will be required nonetheless with your current install.
libgcc_s_seh-1.dll is something about C++ exception handling.
Yes, it should be sufficient to deliver the DLLs too
(or use static linking and deliver only your program file).
For complicated projects where you're not exactly sure which DLL files need to be included to distribute your application, I made a handy dandy Bash script (for MSYS2 shells) that can tell you exactly what DLL files you need to include. It relies on the Dependency Walker binary.
#!/usr/bin/sh
depends_bin="depends.exe"
target="./build/main.exe" # Or wherever your binary is
temp_file=$(mktemp)
output="dll_list.txt"
MSYS2_ARG_CONV_EXCL="*" `cygpath -w $depends_bin` /c /oc:`cygpath -w $temp_file` `cygpath -w $target`
cat $temp_file | cut -d , -f 2 | grep mingw32 > $output
rm $temp_file
Note that this script would need to be modified slightly for use in regular MSYS (the MSYS2_ARG_CONV_EXCL and cygpath directives in particular). This script also assumes your MinGW DLL files are located in a path which contains MinGW.
You could potentially even use this script to automatically copy the DLL files in question into your build directory as part of an automatic deploy system.
You may like to add the options -static-libgcc and -static-libstdc++ to link the C and C++ standard libraries statically and thus remove the need to carry around any separate copies of those.
I used ntldd to get a list of dependencies.
https://github.com/LRN/ntldd
I'm using msys2 so i just installed it with pacman. Use that and then copy all the needed dependencies
There are several major challenges to distributing compiled software:
Compiling the code for all target processors (remember, when it comes to compiled code, you need to produce separate downloads/distributions for each type of instruction set architecture).
Ensuring that the builds are reproducible, consistent, and can be easily correlated with a specific version of the code (and versions of the dependencies).
Ensuring that the build output is self-contained and includes all of its dependencies within it (so that it is not dependent on any other installations that happen to exist on just your system).
Making sure that your code is built and distributed regularly, with updates distributed automatically so that -- in the event of security issues -- you can push out new patched versions.
For convenience and to increase reach, it is nice for non-savvy users to have a prebuilt version that they can install. However, I would recommend sharing the source code as a first step.
Most of these requirements are fairly non-trivial to hit and often require automating not only build process, but also automating the instantiation / configuration of VMs in which the build should take place. However, there are open source projects that can help... for example, check out Gitian.
In terms of bullet point #3, the key thing here is to use static linking... while this does make the binary you distribute much larger (because its dependencies are now baked into the output), it also makes your binary isolated from the version of the libraries on the system (avoiding "dependency hell").
Point #4 is very tricky, but thankfully there are also opensource tools to help here, as well such as cloudup, which provides a way to add auto-updating capability to your application distribution.
I realize this must be a somewhat naive question, but I have written C++ program for a client. He needs the program installed on his machine, but I don't want to give him the code obviously.
How can I protect the code so he doesn't have access to the source code? any suggestions to help me get started would be appreciated.
thanks!
Compile the program, and give him the compiled version? Like most computer programs?
Beyond that, I refer you to Protecting executable from reverse engineering?
You don't have to give your customer the source code of your program. Generally speaking, he should only need the executable program.
C++ is a compiled language. That means that after compilation, the compiler will generate a binary file which contains machine code - for example, a dll, a lib or an exe file under Windows. In windows, all you have to do is deliver the exe's and associated dll's, if they are not already present on the client's machine. There can be different versions of the binaries (depending on platforms, e.g. 32bit vs 64bit compilations) so you might have to run more compilations and let an installer utility handle the distribution.
Can I create a Python program, send it to a remote computer, and run it there without that computer having Python installed? I've heard that you cannot, as Python needs to be interpreted. If this is true, then it seems very odd as it would be hard to distribute your program unless everyone decides to install Python.
Also, what about C and C++? Can they be run on a remote computer without having the language installed? (I think you can, as it is a compiled language).
I'm not exactly sure and would like clarification.
EDIT:
I'm getting some mixed answers on this and am not sure where to go. I see that I can include the Python library in the program and I can use py2exe.
However, I'm confused on C and C++. Do I have to include their libraries in the program? Can they only be run on certain machines? Does the compiler allow it to run on all machines?
Look at py2exe and py2app for Windows and Mac. Macs running OSX and most modern Linuces have Python installed, however.
C/C++ apps are normally compiled to executables which work on one machine/OS architecture (e.g. 32-bit Windows, or 64-bit OSX); such an executable can run on some but not all machines. For example, 64-bit Windows or OSX can run programs built either for the 32-bit or 64-bit flavor of their respective OSes.
python is interpreted, so it won't run without python. However, that doesn't mean that python has to be installed, you can include a copy in your program directory or even bundle your program and the python runtime into a single file.
C and C++ compilers toolchains generate machine code (in most cases, C interpreters do exist, as do C and C++ -> p-code and bytecode compilers). But most C and C++ programs use shared libraries, and will not run unless the shared library is present (again, doesn't have to be installed, can be placed in the program directory). There's also usually a build option (static linking) to include all necessary libraries in the main program file.
But the result is still limited to a particular combination of OS and CPU architecture. Getting a program to run on more than a single platform always requires platform-specific runtime support.
You can use py2exe for distributing Python programs to Windows.
http://www.py2exe.org/
If a you have written a program in any language, and that program is not compiled to machine code, something on the user's computer must convert it to machine code before it can be run.
In the case of JavaScript, that "something" is often a web browser. In the case of Python, that is often a stand-alone interpreter, though it is possible to compile it:
Is it feasible to compile Python to machine code?
However, to be clear: just because your program is not compiled to imachine code does not mean that it will be interpreted. Programs written in C# are usually compiled to MSIL, which is compiled to machine code the first time the program is run. Java programs are also compiled when they are first run.
I will give a practical application of sending code to a remote machine to run. This is typically done in the BOINC project, a community GRID computing initiative which has produced gems such as SETI#Home. The applications typically are compiled C++ versions with multi-platform binaries for x86-linux, AMD64-linux, win32, win64 and Mac OS Universal Binaries (with ppc,x86 and 64-bit). This is a lot of variety for distribution, but a modern make system can easily automate all that (e.g. CMake).
A lot of people prefer the WORA method (write once run anywhere) and stick with VM based language like Java or Python. In this case the boinc projects distribute a version of the VM as well as the code to run on it. Java VM's being encumbered with licensing issues, Python VM is much nicer. Boinc is attempting to embed the Python VM in various BOINC clients to make the distribution of Python based GRID applications easier.
I hope this gives you an idea about application distribution and helps you make an informed decision.
There is a py2exe that can produce an executable that will run on another computer without that user installing the normal Python package.
Yes, C and C++ are (at least normally) implemented as compilers that can produce standalone executables.
Edit: In a typical case, a C or C++ implementation will link the functions from the standard library that are used in the program into the executable. This can (and often does) include quite a bit that's not used directly, but still doesn't normally include (anywhere close to) the entire standard library.
In most cases you can also produce an executable that depends on an implementation of the standard library already being present on the target machine in the form of a shared library, DLL, etc. (different OSes use different names). This reduces the size of the executable, but increases the headaches involved in distribution; I use it for code I'm compiling on my own machine, but generally avoid it when/if I'm distributing an executable to anybody else. Given current hard drive prices, the savings in disk space is rarely worth the headache.
Look into Pyinstaller for standalone executables with no python integration needed. Well, apart from the crucial libraries so it can run!
It's recently updated, well maintained and even supports cython integration though that can get complex. You can compress the files to be smaller or if you have multiple executables, you can link them to one file to reduce size.
You can also of course create a single executable with python installed. Don't use anaconda though (use default python 3.6) to ensure your program is very small in size.
Hope this helps.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I used to program in Windows with Microsoft Visual C++ and I need to make some of my portable programs (written in portable C++) to be cross-platform, or at least I can release a working version of my program for both Linux and Windows.
I am total newcomer in Linux application development (and rarely use the OS itself).
So, today, I installed Ubuntu 10.04 LTS (through Wubi) and equipped Code::Blocks with the g++ compiler as my main weapon. Then I compiled my very first Hello World linux program, and I confused about the output program.
I can run my program through the "Build and Run" menu option in Code::Blocks, but when I tried to launch the compiled application externally through a File Browser (in /media/MyNTFSPartition/MyProject/bin/Release; yes, I saved it in my NTFS partition), the program didn't show up.
Why? I ran out of idea.
I need to change my Windows and Microsoft Visual Studio mindset to Linux and Code::Blocks mindset.
So I came up with these questions:
How can I execute my compiled linux programs externally (outside IDE)?
In Windows, I simply run the generated executable (.exe) file
How can I distribute my linux application?
In Windows, I simply distribute the executable files with the corresponding DLL files (if any)
What is the equivalent of LIBs (static library) and DLLs (dynamic library) in linux and how to use them?
In Windows/Visual Studio, I simply add the required libraries to the Additional Dependencies in the Project Settings, and my program will automatically link with the required static library(-ies)/DLLs.
Is it possible to use the "binary form" of a C++ library (if provided) so that I wouldn't need to recompile the entire library source code?
In Windows, yes. Sometimes precompiled *.lib files are provided.
If I want to create a wxWidgets application in Linux, which package should I pick for Ubuntu? wxGTK or wxX11? Can I run wxGTK program under X11?
In Windows, I use wxMSW, Of course.
If question no. 4 is answered possible, are precompiled wxX11/wxGTK library exists out there? Haven't tried deep google search.
In Windows, there is a project called "wxPack" (http://wxpack.sourceforge.net/) that saves a lot of my time.
Sorry for asking many questions, but I am really confused on these linux development fundamentals.
Any kind of help would be appreciated =)
Thanks.
How can I execute my compiled linux
programs externally (outside IDE)? In
Windows, I simply run the generated
executable (.exe) file
On Linux you do the same. The only difference is that on Linux the current directory is by default not in PATH, so typically you do:
./myapp
If you add current dir to the path
PATH=".:$PATH"
then windows-like way
myapp
will do, but this is not recommended due to security risks, at least in shared environments (you don't want to run /tmp/ls left by somebody).
How can I distribute my linux application?
In Windows, I simply distribute the executable files with the corresponding DLL files (if any)
If you are serious about distributing, you should probably learn about .deb (Ubuntu, Debian) and .rpm (RedHat, CentOS, SUSE). Those are "packages" which make it easy for the user to install the application using distribution-specific way.
There are also a few installer projects which work similarly to windows installer generators, but I recommend studying the former path first.
What is the equivalent of LIBs (static library) and DLLs (dynamic library) in linux and how to use them?
.a (static) and .so (dynamic). You use them in more or less the same way as on Windows, of course using gcc-specific compilation options. I don't use Code::Blocks so I don't know how their dialogs look like, in the end it is about adding -llibrary to the linking options (guess what: on windows it is about adding /llibrary ;-))
Is it possible to use the "binary form" of a C++ library (if provided) so that I wouldn't need to recompile the entire library source code?
Yes. And plenty of libraries are already present in distributions.
Note also that if you use .deb's and .rpm's for distribution, you can say "my app needs such and such libraries installed" and they will be installed from the distribution archives. And this is recommended way, in general you should NOT distribute your copy of the libraries.
If I want to create a wxWidgets application in Linux, which package should I pick for Ubuntu? wxGTK or wxX11? Can I run wxGTK program under X11?
Try wxGTK first, dialogs may look better, gnome themes should be used etc.
If question no. 4 is answered possible, are precompiled wxX11/wxGTK library exists out there? Haven't tried deep google search.
Try
apt-cache search wx
(or spawn your Ubuntu Software Center and search for wx)
In short: you will find everything you need in distribution archives.
Navigate to the folder with your compiled program and execute ./program
Send the program, plus any .so files
.a is static library, .so is shared libraries.
Yes, but often you need to compile it yourself first.
Not sure about wxWidgets distributions, though.
Since Ubuntu comes with wxGTK packages you should definitely build against them. For development you should use a debug version though, so it might be good to build yourself, but for deployment building against the packages the system provides seems better.
wxX11 is a worse choice than wxGTK, use it only for systems where wxGTK doesn't exist or requires newer GTK libraries than are available.
Why not just stick with what you know and develop in .NET? Ubuntu comes native with Mono. You could keep using Visual C++ or step up to C# and make your life a whole lot easier.
A piece of general advice to Linux newcomers, but who are technically minded to begin with, is: You should learn to use your chosen distribution properly.
In your case, that means learning how to acquire the right development packages provided by Ubuntu. For instance, some other people are advising you to download the source for libraries you are going to use, but the better way is to use Ubuntu's package system to download the libraries you want to program against, together with the headers for that library (often put in a separate package) as well as the debug symbols for the library (also often in a separate package).
Look in the System->Administration menu in Ubuntu for the Synaptic tool, which allows you to search the package repositories on the Internet. You'll almost certainly find packages for the libraries you need, as well as all tools.
1, Unix generally doesn't have a particular extension for an executable - so myprog.exe would just be myprog.
You might have to set it to be executable if the IDE doesn't do this automatically, type "chmod +x myprog"
5, For wxWindows I would download the source and build it, check the build instructions but it's probably just a matter of "configure;make;make install". Generally in Unix you build libs form source so that they can correctly find all the components on your machine - you also have the source of examples etc.
I just added some information to rlbond's answer.
It is depens on Linux version. If you use a Ubuntu - create a deb-package. (http://ubuntuforums.org/showthread.php?t=51003)
Can I run wxGTK program under X11?
Yes, if you have wxGTK package installed :)
This is not really going to answer your questions, but I think is a valid recommendation.
You have two issues you are trying to deal with:
The Linux environment.
Making sure your program is
portable.
If I were you I would load CodeBlocks on Windows and run against either Cygwin or Mingw, that will help you make sure your code is portable across platforms. You are familiar with the environment and would gain maximum productivity getting over the OS hurdle.
Once you are satisfied with the above then take your code and move it to Linux. At that point any porting effort should be trivial.
When you say your program didn't show up I assume you mean that it was there in the file browser but when you double clicked it you got a busy cursor for a moment and then nothing happened?
If so then it means that the program failed to run, probably because it couldn't find the dynamic libraries it's linked against. To diagnose the problem you can run it from a terminal and then you'll be told what the problem is.
You might want to read the manual page for ld.so i.e. type
man ld.so
into a terminal. This tells you where the Linux dynamic library linker looks for libraries at run-time. It also refers you to another useful tool called ldd which I recommend becoming familiar with if your are doing Linux development.
I've got a C++ project where we have loads and loads of dependencies. The project should work on Linux and Windows, so we've ported it to CMake. Most dependencies are now included right into the source tree and build alongside the project, so there are no problems with those.
However, we have one binary which depends on Fortran code etc. and is really complicated to build. For Linux, it's also not available as a package, but only as precompiled binaries or with full source (needs a BLAS library installed and several other dependencies). For windows, the same library is available as binary, building for Windows seems even more complicated.
The question is, how do you handle such dependencies? Just check in the binaries for the supported platforms, and require the user to set up his build environment otherwise (that is, manually point to the binary location), or would you really try to get them compiled along (even if it requires installing like 10 libraries -- BLAS libraries are the biggest pain here), or is there some other recommended way to handle that?
If the binary is independant of the other part of your build process, you definitively should check-in it. But as you cannot include every version of the binary (I mean for every platform and compile flags the user might use) the build from source seems mandatory.
I have done something similar. I have checked-in the source code archives of the libraries/binaries I needed. Then I wrote makefile/scripts to build them according to the targeted platform/flags in a specific location (no standard OS location) and make my main build process to point to the right location. I have done that to be able to handle the correct versions and options of the libraries/binaries I needed. It's quite a hard work to make things works for different platforms but it's worth the time !
Oh, and of course it's easier if you use crossplatform build tools :)
One question to you. Does the users need to modify this binary, or are they just happy it's there so the can use/access it? If they don't need to modify it, check in the binaries.
I would agree, check in the binaries for each platform if they are not going to be modified very often. Not only will this reduce build times, but it will also reduce frustration from unnecessary compilations.