I'm new to makefile, and I'm writing a simple C++ shared library.
Is there a way of finding a library's path dynamically by the makefile itself? What I want is something like this: (in my makefile)
INCLUDE_DIRS := `which amplex-gui`
LIBRARY_DIRS := `which amplex-gui`
amplex-gui is a library I use in my code, and I need to put its lib and include directories in my makefile. I want to figure out its path dynamically because each user might install it in a different path on their machine. Therefore, I need my makefile to dynamically parse the which command (or perhaps the $PATH environment variable) to find that path. How can I go about doing this?
Remember that backquoting is shell syntax. Make doesn't do anything special with backquotes. If you're using GNU make, you can use $(shell which amplex-gui) to get equivalent behavior as backquotes.
Regarding your comment above, I'm not sure exactly what you mean by "nest commands", but you can definitely use the shell's $() syntax within a make shell function. However, as with all strings that make expands, you need to double the dollar signs to quote them so that they are passed to the shell. So for example:
INCLUDE_DIRS := $(shell echo $$(dirname $$(dirname $$(which amplex-gui))))
Of course you can also use make functions; unfortunately the make dir function is annoying in that it leaves the final slash, so it cannot be used multiple times directly. You have to put a patsubst in there, like:
INCLUDE_DIRS := $(dir $(patsubst %/,%,$(dir $(shell which amplex-gui))))
Finally, if you have a sufficiently new version of GNU make there's the abspath function, so you could do something like this:
INCLUDE_DIRS := $(abspath $(dir $(shell which amplex-gui))../..)
Related
I want to use an awk script to figure out the which modules have to be compiled before I can compile a FORTRAN source file. My project is structured in a way that I can obtain the filenames that provide the modules by running
awk '$1=/use/{print gensub(",","","g", $2) ".o"}' file.f90
on the file I want to compile.
However, my make command
%.o: $(shell awk '$$1=/use/{print gensub(",","","g", $$2) ".o"}' /path/to/%.f90)
fails with
awk: fatal: cannot open file `/path/to/%.f90' for reading: No such file or directory
So %.f90 does not get expanded. Why is that the case and how can I solve that issue?
Variables and functions in targets and prerequisites are expanded when the makefile is parsed, not when make is running the makefile. But, pattern rules are only expanded when make is running the makefile, trying to build a target that matches the pattern. So at the time these variables and functions are expanded, you only have the literal pattern string not its expansion into a real filename.
See How make reads a makefile in the docs.
There are a number of ways to do this. One option is using secondary expansion. However note you'll have to double-escape the $ you are escaping!!
.SECONDEXPANSION:
%.o: $$(shell awk '$$$$1=/use/{print gensub(",","","g", $$$$2) ".o"}' /path/to/$$*.f90)
ETA
You could alternatively not use .SECONDEXPANSION at all and instead use eval like this:
%.o:
...
SRCS := $(wildcard *.f90)
OBJS := $(SRCS:%.f90=%.o)
$(foreach O,$(OBJS),\
$(eval $O: $(shell awk '$$1=/use/{print gensub(",","","g", $$2) ".o"}' $(O:%.o=%.f90))))
Since you didn't give an actual example I just made up SRCS and OBJS variables. Maybe you have similar variables already.
My C++ project has source files organized in nested subdirectories of ./src. I have a pattern rule in my makefile which compiles all of the .cpp source files into objects:
$(OBJDIR)/%.o: %.cpp makefile
$(CXX) -c $< -o $#
Since I am using this pattern rather than writing a compilation rule for each source file, I need to tell make to look recursively through ./src for these prerequisites. Right now I have:
VPATH := $./src/:./src/folder1:./src/folder2:./src/folder3
This works, but it feels pretty inelegant and also causes bugs when I inevitably forget to add in a new folder.
Hoping someone has a better solution!
You can automate the building of the VPATH variable like yours by searching for subdirectories and replacing spaces with colons:
space :=
space +=
VPATH := $(subst $(space),:,$(shell find src -type d))
This assumes that you have no spaces in your directories or filenames.
With this approach, it is not clear to me what you would do if two source files in two different subdirectories have the same name -- but that seems to be more related to your overall setup than to your question about the VPATH specifically.
For the $(space) variable trick, see the nifty Escaping comma and space in GNU Make blog post.
gmake itself does not have any functions for recursive directory traversal, so you have to resort to $(shell ...):
VPATH := $(shell find src -type d -print | tr '\012' ':' | sed 's/:$$//')
Tweak the shell script to get the right semantics. You want to use the := operator, in order to evaluate this one time.
I would like to extract all the *.cc and *.h filenames from a Makefile that also contains unused targets. Example Makefile here : https://gist.github.com/berceanu/7554a9c4371b807e425259c7e99b5de9
I've tried running make -Bnd and looking at the pruned files but I don't know if this misses anything.
make -Bnd | grep "Pruning file" | sort | uniq
Expected result: list of all *.h and *.cc files used by make run on the above Makefile.
Your approach trying to extract this information from the Makefile is probably the wrong way. make doesn't know which header files are actually used. make only knows which header files you've explicitely told make about in the dependencies, and that's not very reliable. The information in the Makefile could be wrong in two ways. It could contain unused targets (as you have noticed) or unused header files. It could miss header files that are actually included but not mentioned in the Makefile. Even worse, what if the actual inclusion of header files depends on macros, like #ifdef XYZ_FEATURE #include "additionalHeaderFile.h" #endif.
There are at least three ways how you could generate the desired list, that is, list of .cc and .h files actually used during compilation:
(blindly trusts the Makefile) make -n --print-data-base
(follows the real files of the build, but also a bit difficult to parse) strace -f make
(relies on a feature present in GCC, clang, armcc and possibly other compilers for Makefile generation, very reliable) Add CPPFLAGS:=-MMD to the Makefile, run make clean, then make, then use cat *.d to get the list of all .cc and .h files used to build run. You could even do that without changing the Makefile: make clean; make CPPFLAGS:=-MMD && cat *.d | sed -e 's/\\//g' -e 's/:/ /g' -e 's/ \+/\n/g' | sort -u.
Also, the Makefile that you've shared in the gist has a large number of issues.
The default target should, by convention, be named all 1 2 3. That doesn't need to be the name of the binary, it would just be a .PHONY target.
The variable with the list of object files should be named OBJS or OBJECTS, not OBJ. The name OBJ is misleading, because it's singular.
Instead of rm use $(RM) which implies -f and thus will not trouble in case a file doesn't exist. (As a side-effect, the Makefile would become more portable, as not all platforms use rm for deleting files.)
clean is not a file and thus should be a .PHONY target.
clean should use :: instead of : for its recipe so that in future, when the Makefile is bigger and split into multiple files, each file can have their own clean target without problems.
Variables that are not rule-specific should be expanded at the moment of definition, not when they are referenced, and thus defined using := instead of =.
Instead of C++ use CXX which is already defined.
Instead of putting the options into C++/CXX, use LDFLAGS, because you're linking.
You should have a source file that has the same basename as the binary. Then you can use the built-in rule for linking.
Explicit dependencies in the Makefile are a pain for maintenance. Every time an #include statement for a project header file is added, removed, or changed, the Makefile would have to be updated, and that's easy to forget, and it's a pain, especially when the #include statement is in a header file. Even with due diligence, this is also an invisible merge conflict. Instead of having explicit dependencies in the Makefile, you should use CPPFLAGS+=-MMD at the start of your Makefile and append -include $(wildcard *.d) at the end of your Makefile, and add *.d to the list of files to delete in clean. You could then remove all dependency rules from the Makefile (except for the one for the linkage).
Naming the binary run is not a good idea. Users that see that your Makefile has a run target would expect this to run the actual program, not to link it.
It is better to have each object mentioned on a line of its own. That significantly reduces merge conflicts in projects when multiple developers change the object list at the same time.
Your actual Makefile should look like this, with the binary renamed from run to program:
LDFLAGS:=-Wno-deprecated -lm
CPPFLAGS+=-MMD
BINARY:=program
OBJECTS:= \
$(BINARY).o \
binomh.o \
# More objects here
.PHONY: all
all: $(BINARY)
$(BINARY): $(OBJECTS)
.PHONY: clean
clean::
$(RM) \
$(BINARY) \
*.[adios] \
-include $(wildcard *.d)
That Makefile will do the "same" thing as your Makefile, but it is almost maintenance-free. No need to update dependencies, as they are automatically taken from the dependency files generated by the C Preprocessor. The *.[adios] will also remove files created in case you added -save-temps to any of the CFLAGS, CXXFLAGS, or CPPFLAGS.
This type of Makefile is known to work for GCC, clang, AOCC (AMD Optimizing C Compiler), and armcc. It probably works for a number of other compilers and preprocessors as well, especially when they are based on, or try to be compatible with, GCC or clang/LLVM.
BTW in case you're interested in knowing that this works for you, with high confidence, besides experience: I've taken your Makefile and added the following lines to it in order to reproduce your source code structure. The header files would just be empty files. The C++ source files would be lists of #include statements taken from the dependencies in your Makefile.
%.cc:
grep '^$*\.o.*:' $(MAKEFILE_LIST) | sed -e 's/.*://' -e 's/.*$*\.cc//' -e 's/ \([^ ]\+\)/#include "\1"\n/g' >$#
%.h:
touch $#
Instead of -Bnd, I would suggest using --dry-run --print-data-base to dump the full database of targets, their dependencies, rules, variables, etc..
I use GNU make for building reports (LaTeX for source, python for figures, etc.). For targets, I use extensively pattern matching, for example:
all : sample1_test1.png sample2_test1.png sample2_test.png
sample%_test1.png : sample%.dat
python gen_figure.py $< $# --test 1
sample%_test2.png : sample%.dat
python gen_figure.py $< $# --test 2
Now, to simplify the rules I would like to use multiple pattern groups (like regex groups) and use the matches separately in the build rule, for example:
all : sample1_test1.png sample2_test1.png sample2_test.png
sample(?P<Sample>[0-9]+)_test(?P<Test>[0-9]+).png : sample$(<Sample>).dat
python gen_figure.py $< $# --test $(<Test>)
(the syntax is arbitrary, but the point is that I can define two different match groups called Sample and Test and use them as parameters to my script).
How would I achieve this in make or another build system (waf, scons etc.)?
To do it in GNU make, you can use one of two different "metaprogramming" models supported by GNU make:
Auto-generated include files. In your main makefile, add -include generated.mk then write a makefile rule with the target generated.mk (probably listing Makefile as a prerequisite), where the recipe generates the appropriate targets based on the list of targets. You have the full power of the shell to construct your target lists however you want. Every time you modify the makefile, the included file will be rebuilt then GNU make will automatically re-exec itself so you don't have to do anything extra.
Use GNU make's $(eval ...) function, probably combined with $(call ...) and $(foreach ...), to automatically evaluate rules. To do this you define a "template" for the rule using define ... enddef, with variables installed where you want to provide arguments, then use $(call ...) to instantiate them, use $(eval ...) on the result of the call, and do it in a loop for each target. Something like: $(foreach T,$(TARGETS),$(eval $(call DEFINERULE,$(T))))
Here's an example of method 1. Suppose you have this predefined content in your makefile:
TESTS := sample1_test1.png sample2_test1.png sample2_test.png
Then you can use this makefile to get something like the above:
all: $(TESTS)
-include generated.mk
generated.mk : Makefile
#rm -f '$#'
#for t in $(TESTS); do \
eval `echo "$$t" | sed 's/^sample\([0-9]*\)_test\([0-9]*\).*/sample=\1 test=\2/'`; \
echo "$$t : sample$$sample.dat ; python gen_figure.py \$$< \$$# --test $$test" >> '$#'; \
done
Note I just wrote this off the top of my head but I think it will work.
I'm using make to build a C++ project. During the course of the project, I wanted to make some changes to the Makefile. Unfortunately, ever since I executed make once, it keeps using that particular version of the Makefile and just doesn't do anything with the changes at all.
I have run make clean, I have renamed the makefile, I've searched for other Makefiles which might be used instead, all to no avail. There is no mention of any caching mechanism in the man pages for make, nor anywhere on Google.
Does anyone have any idea why make isn't using the new version and what I can do about it? I'm compiling on a Ubuntu 12.04.2 LTS (x86_64) box, with (GNU) make version 3.81.
Update:
Some additional information. It seems make is using the current version of the makefile after all. If I change something in the main target, it's working just fine. But if I change something in the obj/%.o target, it just keeps running the same command, no matter what changes I make to that target.
Full Makefile an be found here: http://pastebin.com/WK43NRcL
CC_FILES = $(shell find -name "*.cc" -exec echo "{}" +;)
That find command is incorrect, shouldn't it be looking in the src directory? And why use echo to print the name when that's what find does anyway?
That means your list of CC_FILES and so also list of OBJ_FILES is empty.
I think you want:
CC_FILES := $(shell find src -name "*.cc")
Note that this uses := not = because otherwise the shell function gets run every time you reference the CC_FILES variable. Using := means it is run and evaluated only once.
However, since it seems all your .cc files are in the same directory you don't need a recursive find, just do:
CC_FILES := $(wildcard src/*.cc)
As you've realised, your patsubst is broken, you can just do:
OBJ_FILES := $(patsubst src/%.cc,obj/%.o,$(CC_FILES))
(Again, use := here)
Also:
obj/%.o: obj src/%.cc
$(CXX) $(CFLAGS) -c -o $# $<
I think you need to read what the $< variable expands to, because the rule above isn't going to do what you expect.
This makefile is full of errors, you need to use echo in pattern rules to print out the values of variables, so you can verify they have the values you expect. (As another option for debugging, set SHELL=bash -x so every shell command is echoed)
make does not somehow magically keep track of your old makefile; it will use whatever file is first in the list of files it looks for, in the current directory.
To find out which Makefile is actually used, see this question: Getting the name of the makefile from the makefile
Since you're using GNU make, check its excellent manual on what filenames it looks for, and in which order.