I'm setting up Autotools for a large scientific code written primarily in C++, but also some CUDA. I've found an example for compiling & linking CUDA code to C code within Autotools, but I cannot duplicate that success with C++ code. I've heard that this is much easier with CMake, but we're committed to Autotools, unfortunately.
In our old hand-written Makefile, we simply use a make rule to compile 'cuda_kernels.cu' into 'cuda_kernels.o' using nvcc, and add cuda_kernels.o to the list of objects to be compiled into the final binary. Nice, simple, and it works.
The basic strategy with Autotools, on the other hand, seems to be to use Libtool to compile the .cu files into a 'libcudafiles.la', and then link the rest of the code against that library. However, this fails upon linking, with a whole bunch of "undefined reference to ..." statements coming from the linker. This seems like it might be a name-mangling issue with g++ vs. the nvcc compiler (which would explain why it works with C code), but I'm not sure what to do at this point.
All .cpp and .cu files are in the top/src directory, and all the compilation is done in the top/obj directory. Here's the relevant details of obj/Makefile.am:
cuda_kernals.cu.o:
$(NVCC) -gencode=arch=compute_20,code=sm_20 -o $# -c $<
libcudafiles_la_LINK= $(LIBTOOL) --mode=link $(CXX) -o $# $(CUDA_LDFLAGS) $(CUDA_LIBS)
noinst_LTLIBRARIES = libcudafiles.la
libcudafiles_la_SOURCES = ../src/cuda_kernels.cu
___bin_main_LDADD += libcudafiles.la
___bin_main_LDFLAGS += -static
For reference, the example which I managed to get working on our GPU cluster is available at clusterchimps.org.
Any help is appreciated!
libtool in conjunction with automake currently generates foo.lo (libtool-object metadata) files, the non-PIC (static) object foo.o, and the PIC object .libs/foo.o.
For consistent .lo files, I'd use a rule like:
.cu.lo:
$(LIBTOOL) --tag=CC --mode=compile $(NVCC) [options...] -c $<
I have no idea if, or how, -PIC flags are handled by nvcc. More options here. I don't know what calls you are making from the program, but are you forward declaring CUDA code with C linkage? e.g.,
extern "C" void cudamain (....);
It seems others have run up against the libtool problem. At worst, you might need a 'script' solution that mimics the .lo syntax and file locations, as described on the clusterchimps site.
Related
What I'm trying to do is create a c++ file from an object file but I cannot figure out a way to do so.
INCLUDEDIR = ../headers
CXXFLAGS = -std=c++11 -I $(INCLUDEDIR) -Wall -Wfatal-errors -O2
all:primeFactors.o
primeFactors.o: primeFactors.cpp $(INCLUDEDIR)/primeFactors.h
g++ $(CXXFLAGS) -c $< -o $#
When I try to build this I get
make: *** No rule to make target 'primeFactors.cpp', needed by
'primeFactors.o'. Stop.
which I understand but when I take out the primeFactor.cpp argument I then get told there is nothing to be done with the make file. So is there a way to do this?
In general; no, you cannot do that. An object file (.o file) is the result of the code passing through the compiler front-end (to parse the language) the optimizer (to make it run fast) and the compiler back-end (to produce code that will run on your CPU).
This process is not reversible. You cannot go from compiled code back to source.
can you create a c++ file from an .o object file with makefile?
A makefile will allow you to do that only if you have an underlying tool to do it. make, which uses makefiles to do its job, does not have any built-in mechanisms to pull that off.
Your makefile has a rule for building primeFactors.o.
primeFactors.o: primeFactors.cpp $(INCLUDEDIR)/primeFactors.h
It says that primeFactors.cpp and $(INCLUDEDIR)/primeFactors.h are needed to build primeFactors.o. If you don't have those files, or no rules to build them, there is no way for make to build primeFactors.o.
How this simple make script knows that some of cpp files is changed? Does it means that for each .o file it will look for corresponding .cpp one? What if extension will be different - for example .c
hellomake: hellomake.o hellofunc.o
gcc -o hellomake hellomake.o hellofunc.o -I.
UPD:
According to my understanding scrip I provide should not look to c and cpp files. And when I asked to build project second time MAKE told me "make: 'hellomake' is up to date.
But I was surprised when I have changed hellomake.cpp MAKE has decided do rebuild project. Why?
GNU make has many builtin rules. Run make -p to find them. And use the existing rules in your Makefile, see this or that or this
Obvious documentation links were already provided. I just wanted to comment on your example. You told make the following:
The file hellomake relies on hellomake.o and hellofunc.o, ie. both are prerequisites of hellomake. If any prerequisite changes since the last build, hellomake will be rebuilt. How (re-) building is done is the second line, ie. the gcc invocation.
To answer your question: The snippet provided, will not look for any cpp files. You need different rules in addition for that, ie. something like
%.o: %.cpp
gcc -I. -c #< -o $#
In case you are searching for a rather generic Makefile to start with, I'd recommend this one. It has been the basis for many of my Makefiles in use.
I generated a makefile out of a codeblocks project (written in c++11), so I can use Atom as IDE. But it does not update the object files when I i.e. change the default constructors parameter in the header file, which is really annoying. It just links the existing object files again. But even if I make a little change to the .cpp file, it recompiles the object without recognizing the changes in the header file. The only quick fix I found is to delete the object file manually, so it really generates it completely new. The header part I currently often change looks like this:
VRParticles(): VRParticles(123){}
The whole makefile is available here (generated using cbp2make): https://github.com/Pfeil/polyvr/blob/master/Makefile
(Please note that I am just a fairly new contributor and not responsible for the way this is programmed ;) )
I use the makefile mostly with one of those two commands:
make -j 3 build_debug
make debug
Note that everything compiles fine when I delete VRParticles.o or do make clean and make debug.
Please note that my experience with makefiles is very low. The makefile basically works like this (remember the link to the full version above):
OBJ_DEBUG = # all object files
build_debug: before_debug out_debug after_debug
debug: before_build build_debug after_build
out_debug: before_debug $(OBJ_DEBUG) $(DEP_DEBUG)
$(LD) $(LIBDIR_DEBUG) -o $(OUT_DEBUG) $(OBJ_DEBUG) $(LDFLAGS_DEBUG) $(LIB_DEBUG)
$(OBJDIR_DEBUG)/src/addons/Bullet/Particles/VRParticles.o: src/addons/Bullet/Particles/VRParticles.cpp
$(CXX) $(CFLAGS_DEBUG) $(INC_DEBUG) -c src/addons/Bullet/Particles/VRParticles.cpp -o $(OBJDIR_DEBUG)/src/addons/Bullet/Particles/VRParticles.o
I'd really like give more information, but I have no idea what else could be important, so please ask if you need more. My question basically is how I need to modify the makefile (I guess this file contains the issue) so the object files get updated if needed. Without recompiling everything.
I'm on Linux (Ubuntu 14.04 LTS).
If we look at your dependencies for VRParticles.o:
VRParticles.o : src/addons/Bullet/Particles/VRParticles.cpp
You are telling make that the object file only depends on VRParticles.cpp. So when you update VRParticles.h, that doesn't matter - you never listed VRParticles.h as a dependency! Thankfully, gcc can generate those dependencies for you automatically:
$(CC) $(other flag stuff) -MP -MMD -MF $(#:.o=.d) -o $# -c $<
That will create a file VRParticles.d which will have make-style rules for dependencies, in this case something like:
VRParticles.o : VRParticles.d
So at that point, all we need is to include them:
DEPENDENCY_FILES = $(....)
-include $(DEPENDENCY_FILES)
I was trying to use Boost.Tokenizer library. In my PROG.cpp, I have the following:
#include <boost/tokenizer.hpp>
And my Makefile was initially something like
CXX = g++-4.8
CXXFLAGS = ## some irrelevant flags
LDFLAGS = ## some irrelevant flags
SOURCES = PROG.cpp
OBJECTS = $(SOURCES:.cpp=.o)
TARGETS = PROG
$(TARGETS) : $(OBJECTS)
$(CXX) $(CXXFLAGS) -o $# $^ $(LDFLAGS)
## other targets
It won't compile, since boost/tokenizer.hpp cannot be found:
fatal error: boost/tokenizer.hpp: No such file or directory
Then I manually added the boost include path to CXXFLAGS:
-I/opt/local/include/
(which is the path from MacPorts.)
Then I tried to include the Tokenizer library, but in /opt/local/lib/ I have libboost_atomic-mt.dylib, libboost_chrono-mt.dylib, etc., but nothing like tokenizer. I was rather confused at the time. I supposed that still wouldn't work since the library was not linked against. Surprisingly, the program built, linked, and ran perfectly.
So I'm really confused now. Here are some questions:
(1) I did not link against boost explicitly, so boost is treated like standard library by the linker?
(2) If boost is treated like standard, why the headers are not standard?
(3) Why there are libboost_atomic-mt.dylib, libboost_chrono-mt.dylib, etc. but not tokenizer? Which dynamic library does tokenizer belong to?
I'm not very familiar with g++ linking mechanism; speaking of boost, this is my very first program with boost. So I'd really appreciate detailed explanation. Thanks in advance!
For reference, this is what I extracted by gcc -print-search-dirs:
install: /usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/
programs: =/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/libexec/gcc/x86_64-apple-darwin12.3.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/bin/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/bin/
libraries: =/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/lib/x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../../x86_64-apple-darwin12.3.0/lib/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../x86_64-apple-darwin12.3.0/4.8.0/:/usr/gcc-4.8.0/lib/gcc/x86_64-apple-darwin12.3.0/4.8.0/../../../:/lib/x86_64-apple-darwin12.3.0/4.8.0/:/lib/:/usr/lib/x86_64-apple-darwin12.3.0/4.8.0/:/usr/lib/
Most of the boost libraries, are just header files, if you look in the .hpp files you will not see just the declaration of the classes, like you would expect in a header file, but actually the entire implementation. This is why for 90% of the boost libraries, you don't need to worry about linking, only inclusion.
However for a few libraries, serialiser, a few others, there is just too much polluting code for the header inclusion method to be reasonable. I'm sure there is a better, more rigid definition about when the implementation is included in the header and when it isn't.
http://www.boost.org/boost-build2/doc/html/bbv2/faq/header-only-libraries.html
Here is another question about it:
Why are not all boost libraries header-only?
p.s.
Generally it is better to keep the boost library separate and in your makefile do something like:
For compilation:
CXXFLAGS += -I/path/to/boost/include
For Linking:
LDPATH += -L/path/to/boost/lib
This makes it easier to upgrade your boost version as you just have to change the path in one place.
I am running Windows 7 with gcc/g++ under Cygwin. What would be the Makefile format (and extension, I think it's .mk?) for compiling a set of .cpp (C++ source) and .h (header) files into a static library (.dll). Say I have a variable set of files:
file1.cpp
file1.h
file2.cpp
file2.h
file3.cpp
file3.h
....
What would be the makefile format (and extension) for compiling these into a static library? (I'm very new to makefiles) What would be the fastest way to do this?
The extension would be none at all, and the file is called Makefile (or makefile) if you want GNU Make to find it automatically.
GNU Make, at least, lets you rely on certain automatic variables that alone give you control over much of the building process with C/C++ files as input. These variables include CC, CPP, CFLAGS, CPPFLAGS, CXX, CXXFLAGS, and LDFLAGS. These control the switches to the C/C++ preprocessor, compiler, and the linker (the program that in the end assembles your program) that make will use.
GNU Make also includes a lot of implicit rules designed to enable it automatically build programs from C/C++ source code, so you don't [always] have to write your own rules.
For instance, even without a makefile, if you try to run make foobar, GNU Make will attempt to first build foobar.o from foobar.c or foobar.cpp if it finds either, by invoking appropriate compiler, and then will attempt to build foobar by assembling (incl. linking) its parts from system libraries and foobar.o. In short, GNU Make knows how to build the foobar program even without a makefile being present -- thanks to implicit rules. You can see these rules by invoking make with the -p switch.
Some people like to rely on GNU Make's implicit rule database to have lean and short makefiles where only that specific to their project is specified, while some people may go as far as to disable the entire implicit rule database (using the -r switch) and have full control of the building process by specifying everything in their makefile(s). I won't comment on superiority of either strategy, rest assured both do work to some degree.
There are a lot of options you can set when building a dll, but here's a basic command that you could use if you were doing it from the command line:
gcc -shared -o mydll.dll file1.o file2.o file3.o
And here's a makefile (typically called Makefile) that will handle the whole build process:
# You will have to modify this line to list the actual files you use.
# You could set it to use all the "fileN" files that you have,
# but that's dangerous for a beginner.
FILES = file1 file2 file3
OBJECTS = $(addsuffix .o,$(FILES)) # This is "file1.o file2.o..."
# This is the rule it uses to assemble file1.o, file2.o... into mydll.dll
mydll.dll: $(OBJECTS)
gcc -shared $^ -o $# # The whitespace at the beginning of this line is a TAB.
# This is the rule it uses to compile fileN.cpp and fileN.h into fileN.o
$(OBJECTS): %.o : %.cpp %.h
g++ -c $< -o $# # Again, a TAB at the beginning.
Now to build mydll.dll, just type "make".
A couple of notes. If you just type "make" without specifying the makefile or the target (the thing to be built), Make will try to use the default makefile ("GNUMakefile", "makefile" or "Makefile") and the default target (the first one in the makefile, in this case mydll.dll).