Object file target not matching implicit rule with pattern - build

I'm using GNU makefiles to build a C project. I want to keep all build artifacts on an isolated build tree in order to minimize clutter. The project looks like this:
$prefix/
include/$tree/program.h
source/$tree/program.c
build/
objects/$tree/program.o
dependencies/$tree/program.d
Where $prefix represents the directory of the project and $tree represents an arbitrary folder structure.
I wanted to match source files in the source/ directory to their object and dependency file counterparts in the build/ tree. So, I wrote the following rules:
# Remove the built-in rules
%.o : %.c
$(objects_directory)/%.o : $(source_directory)/%.c $(dependencies_directory)/%.d
$(compiler_command_line) $(compiler_option_output) $# $<
$(build_directory)/$(target) : $(objects)
$(compiler_command_line) $(compiler_option_output) $# $^
Make correctly figures out the compilation target and the object files needed to build it. However, make stops at this point with the error:
No rule to make target 'build/objects/project/program.o', needed by 'build/program.dll'.
So why is this happening, and how do I fix it?
I investigated the problem by running make --print-data-base, the output of which included:
# Not a target:
build/objects/project/program.o:
# Implicit rule search has been done.
# File does not exist.
# File has not been updated.
Which suggests that the prerequisite is not matching the implicit rule as intended. However, I verified that it does match when I tried to work my way backwards by writing:
object := build/objects/project/program.o
$(object:$(objects_directory)/%.o=$(source_directory)/%.c)
$(object:$(objects_directory)/%.o=%)
These lines result in source/project/program.c and project/program, which means the stem is being correctly computed.
I have studied the GNU make documentation and I don't remember reading anything that suggests that this kind of pattern matching can't happen in implicit rule definitions.
Here are the variable definitions:
include_directory := include
source_directory := source
build_directory := build
objects_directory := $(build_directory)/objects
dependencies_directory := $(build_directory)/dependencies
sources := $(wildcard $(source_directory)/**/*.c)
objects := $(sources:$(source_directory)/%.c=$(objects_directory)/%.o)
# Take the name of the project's directory
target := $(notdir $(CURDIR)).dll
compiler_dependency_file = $(patsubst $(source_directory)/%.c,$(dependencies_directory)/%.d,$<)
compiler_options = -I $(include_directory) -MMD -MF $(compiler_dependency_file)
CC = gcc
compiler_command_line = $(CC) $(compiler_options) $(CFLAGS)
compiler_option_output = -o

It turns out it wasn't the pattern matching. The root of the problem was in the dependency prerequisite of the implicit rule.
The dependency file isn't supposed to be a prerequisite in the first place; it should be one of the targets that gets generated along with the object file.
As I read once more ยง 4.14 Generating Prerequisites Automatically of the manual, the answer jumped out at me:
The purpose of the sed command is to translate (for example):
main.o : main.c defs.h
into:
main.o main.d : main.c defs.h
While my build system makes no use of sed, the fact that main.d was on the left-hand side of the example rule felt strange. In my code, it was on the right-hand side.
When I put my rule in the left-hand side, it worked and the problem was solved. The erroneous recipe was essentially treating one of its byproducts as a prerequisite.

Related

Makefile - replace in wildcard

In my project all .cpp files are stored in
Classes/
Classes/Something/
Classes/Something/Else
...
I want to compile all .cpp file separetly to Bin/ directory, replacing / with _, so that:
Classes/First.cpp -> Bin/Classes_First.o
Classes/Foo/Bar.cpp -> Bin/Classes_Foo_Bar.o
Now I wanted to create rules for compiling:
Bin/%.o: $(subst _,/,%.cpp)
$(COMPILER)g++ $(COMPILE_FLAGS) -c -o $# $^
I tried:
make Bin/Classes_Test.o
But compilation failed.
So I created debugging pattern:
%.cpp:
#echo CPP: $#
Now it printed:
CPP: Classes_Test.cpp
Why?!
So I changed my pattern to:
Bin/%.o: $(subst _,/,Test1_Test2.cpp)
and I saw:
CPP: Test1/Test2.cpp
I'm a little bit confused why subst does not work if I use wildcard as source...
This is an evaluation order issue.
When make parse the makefile it evaluates the $(subst) call but the argument to $(subst) at that point is the literal string %.cpp which has nothing to substitute in it and so does not do anything.
At target evaluation/execution time the % in the target pattern and prereq pattern are filled out but the $(subst) has long-since gone away.
To do this you will need to manually (in one way or another) map the output files to the input files. You can do that and keep the %.o pattern rule target for the actual recipe to run though (so you just need to generate a bunch of Bin/Test1_Test2.o: Test1/Test2.cpp lines).
Alternatively, I believe you can might be able to use secondary expansion to do this:
.SECONDEXPANSION:
Bin/%.o: $$(subst _,/,%.cpp)

Make cannot find target with wildcard pattern

CXXSRC = $(shell find source -iname "*.cpp")
CXXSRCFN = $(notdir $(CXXSRC))
CXXOBJ = $(CXXSRCFN:%.s=output/obj/%.cpp.o)
OUTPUT = output/kernel.elf
.PHONY: builduserspace clean all
all: $(OUTPUT)
#$(QEMU) -vga std -serial file:"output/serialout.log" -m 20 -hda output/disk.img -rtc base=localtime
$(OUTPUT): $(CXXOBJ)
# Linking object files...
#$(LD) $(LDFLAGS) -o output/temp.elf output/obj/Start.s.o $(shell find output/obj -name "*.o" ! -name "Start.s.o") -Lgcc
# Performing objcopy...
#$(OBJCOPY) -O elf32-i386 output/temp.elf output/kernel.elf
%.cpp.o: %.cpp
#echo $(notdir $<)
#$(CXX) $(CXXFLAGS) -o $(notdir $#) $<
That's the makefile. Here's the situation, imagine this directory structure:
output
|
|-- obj
source
|
|-- file1.cpp
|-- file2.cpp
|-- subdirectory
|
|-- file3.cpp
Say I run make in the root folder (where output and source are). The target output is output/kernel.elf.
Essentially, I want all the files in the folder 'source' to be compiled into object files and placed into the folder output/obj.
I managed to get the variables about right; CXXSRC is simply the list of all source files to be compiled; CXXOBJ is the list of outputs.
However, make: * No rule to make target output/obj/file.cpp.o', needed byoutput/kernel.elf'. Stop.
After some trial and error, I managed to narrow down the issue: If I modify the target to this:
output/obj/%.cpp.o: source/subdirectory/%.cpp
It works fine (ie. it errors on the other files in my tree, because not all files are in subdirectory)
Obviously this would defeat the purpose of the % wildcard operator, so how do I fix this problem?
Thanks.
First SO post, take it easy on me (:
The short answer is, you can't. In a pattern rule the pattern (%) must be identical (string-wise) between the target and the prerequisite. It can't mean "something sort of, but not exactly, the same".
I think it's a bit unpleasant to compile source files from multiple different directories and put the output into a single directory. Every single time I've seen that done it's become a big problem (because people sometimes use the same source file name and then you have a mess).
If you really want to do it there's no choice but to declare multiple rules with the different source directories. There are ways to do this in the makefile without writing them all by hand, but they're somewhat more advanced.

Structuring Makefiles with multiple directories

I am trying to compile my project which has the following structure
Project:
MakeFile
Executable
Source1
.cxx
.h
Source2
.cxx
.h
Build
*.o
And I'm having difficulty writting a Makefile to compile. I currently have commands like:
Src1 = $(wildcard $(SRCDIR1)/*.cxx)
Obj1 = $(patsubst $(SRCDIR1)/%.cxx, $(OBJDIR)/%.o, $(Src1))
But then I have difficulty making the compile rules for the object files a) Because I can no longer do:
$(Obj1): %.cxx
$(CXX) $(CFLAGS) -c $(#:.o=.cxx) -o $#
Because the '$#' command now includes the path of the build directory and b) because the prerequisites now include the build path and I should have a source path. I have read large bits of the make manual to try and find a solution but no luck.
Any help towards a solution appreciated!
Jack
From personal experience, after playing around a bit with "raw" Makefiles, I'd really recommend using some tool building the Makefiles for you, like automake or cmake.
You'll still have to specify all the source files manually - but at least I prefer that to manually fiddling around with the Makefiles.
One option I prefer is building an isomorphic directory structure in the build directory. That is, a source file ${src_dir}/project_x/main.cxx builds into ${build_dir}/project_x/main.o. This way you are protected from name clashes when there are source files with the same name in different source directories. The compiler rule would look something like:
${obj_dir}/%.o : ${src_dir}/%.cxx # % includes directory name, e.g. project_x/main
#-mkdir -p ${#D}
${CXX} -c -o $# ${CPPFLAGS} ${CXXFLAGS} $<
Notice how in the above it creates the target directory on the fly. This is a bit simplistic, in a real-world build system object files depend (using order-only dependency) on its directory, so that make automatically creates the destination directory if it does not exist instead of speculatively trying to create them for each target object file even if it already exists.

Which build system will do this the most 'naturally'?

Instead of the flat structure my code currently has, I want to organize it into modules contained in sub-folders (and perhaps sub-sub folders if the modules get big enough).
Each module will have one or more translation units which will each produce a .o file.
The final target would be to mash up all these object files into a static library (for now).
I am using plain 'make' and it is already complicated enough.
Is there a system in which the specified model comes naturally or with much less effort compared to writing makefiles by hand ?
(If you are going to recommend cmake, I need some hints as I have already tried and could not come up with a good solution.)
Some paraphrased bits from my current project's makefile that may help you out with good old fashioned GNU make:
SOURCEDIR := dir1 dir2/subdir1 dir3 dir4 dir5/subdir1 dir6/subdir1
SOURCES := $(foreach srcdir,$(SOURCEDIR),$(wildcard $(srcdir)/*.c))
OBJECTS := $(patsubst %.c,build/%.o,$(SOURCES))
OBJDIRS := $(addprefix build/,$(SOURCEDIR))
MAKEDEPS := $(patsubst %.c,build/%.d,$(SOURCES))
all: example
$(OBJDIRS):
-mkdir -p $#
build: $(OBJDIRS)
build/%.o : %.c | build
cc -MMD -c -o $# $<
example: $(OBJECTS)
cc -o $# $(OBJECTS)
-include $(MAKEDEPS)
In essence, it builds all of the source files found in the designated directories into object files located in subdirectories of the build directory in a hierarchy that parallels their source directory layout (important if multiple source files have the same name) and then links the results into an executable example.
As a bonus, dynamic dependency generation and inclusion via the MAKEDEPS variable and clang's -MMD flag.
It really depends upon your purposes: Build packages are generally intended for the audience rather than the performer. Often, they take into consideration the disparate environments into which people deploy. I played around with 'tup,' which seemed more a way of generating an executable as quickly as possible after an edit. 'Premake' seems to shoot at multiple platforms, but I found specifying compiler options no more enlightened than with Cmake.
It looks as though you've found a good Makefile tutor, so I'll leave my observations at that.

Testing C++ program with Testing classes over normally used classes

This will probably be a bot of a waffly question but ill try my best.
I have a simple c++ program that i need to build testing for. I have 2 Classes i use besides the one i actually am using, these are called WebServer and BusinessLogicLayer.
To test my own code i have made my own versions of these classes that feed dummy data to my class to test it functionality.
I need to know a way of somehow, via a makefile for instance, how to tell the source code to use the test classes over the normally used classes. The test classes are in a different "tester" c++ file, and the tester c++ file also has its own header file.
Regards
Paul
P.S. This is probably a badly worded question, but i dont know any better way to put my question.
You can define abstract base classes that declare the public interfaces for your components, then wire the objects together at runtime (in main() or something else fairly high up in the food chain). In testing, you simply wire up different objects.
To build debug and release versions of a program with source code in directories ${SRC_DIR_1} and ${SRC_DIR_2}:
CXX := g++
CPPFLAGS = ...
CXXFLAGS = ...
SRC_DIR_1 := ...
SRC_DIR_2 := ...
ifeq (${debug},1)
BIN_DIR := ./obj_debug
CXXFLAGS += -g
else
BIN_DIR := ./obj_release
CXXFLAGS += -DNDEBUG
endif
# Make sure that the directory exists.
TEMP := ${shell test -d ${BIN_DIR} || mkdir ${BIN_DIR}}
# Tell make to search each directory
VPATH := ${SRC_DIR_1}:${SRC_DIR_2}
# You can modify this to use a suffix other than .cc
${BIN_DIR}/%.o : %.cc
${CXX} ${CPPFLAGS} ${CXXFLAGS} -c $< -o $#
# Build the requested version of the program.
ifeq (${debug},1)
default: program_name_debug
else
default: program_name
endif
tidy::
#${RM} -r ./obj_*
In the Project Configuration dialog, specify the target name as "program_name, program_name_debug". (Replace program_name with the name of your program.)
To build the program, invoke "make debug=X" with X replaced by either 0 or 1.
Reference
Why does your tester code have a header file of its own? Your test code should have the same interface as the code it emulates, and using the same header file prevents a lot of headaches.
Anyway, this will work:
real_program: WebServer.o BusinessLogicLayer.o
test_program: tester.o
real_program test_program: MyClass.o OtherThings.o
$(LINK) $^ -o $#