I have below Makefile which runs Google's protoc on some proto files, generate the code and build the library
Everything works fine except the fact that it generates code every make run, despite there were no changes to the proto files or whatever. How do I prevent that?
GEN_FLAG:=$(PROJECT_ROOT)/build/.etcd_protocols_gen
ETCD=$(PROJECT_ROOT)/ext/etcd/
PROTOCOLS=$(ETCD)proto/
PROTO_FILES=$(shell find $(PROTOCOLS) -name \*.proto)
FILES=$(wildcard *.cc)
OBJS=$(patsubst %.cc,%.o,$(FILES))
LIB=$(BUILD_LIB_DIR)/libetcdclient.so
DIRS=proto
.PHONY: all
all: etcd-gen $(LIB)
-include $(FILES:.cc=.d)
.PHONY: etcd-gen
etcd-gen:
$(Q)echo "Generating ETCD gRPC C++ sources"
#ln -sf $(PROTOBUF_DIR)/libprotoc.so.16.0.0 $(PROTOBUF_DIR)/libprotoc.so.16
#ln -sf $(PROTOBUF_DIR)/libprotobuf.so.16.0.0 $(PROTOBUF_DIR)/libprotobuf.so.16
#LD_LIBRARY_PATH=$(PROTOBUF_DIR) $(PROTOBUF_DIR)/protoc -I $(PROTOCOLS) --cpp_out=. $(PROTO_FILES)
#LD_LIBRARY_PATH=$(PROTOBUF_DIR) $(PROTOBUF_DIR)/protoc -I $(PROTOCOLS) --grpc_out=. --plugin=protoc-gen-grpc=$(PROJECT_ROOT)/ext/grpc/bin/grpc_cpp_plugin $(PROTO_FILES)
#touch $(GEN_FLAG)
%.o: %.cc
#echo "CC $<"
$(Q)$(CC) $(CFLAGS) -c -o $# $< -ldl
$(Q)$(CC) -MM $(CFLAGS) -MF $*.d -MT $*.o $*.cc
$(LIB): $(OBJS)
#echo "LD $#"
$(Q)$(CXX) $(CXXFLAGS) -shared -Wl,-soname,$(notdir $(LIB)) -o $(LIB) $(OBJS) $(LFLAGS) -L$(PROTOBUF_DIR) -l:libprotobuf.so.16
clean:
$(Q)rm -f *.d *.o *.pb.cc *.pb.h
$(Q)rm -f $(BUILD_LIB_DIR)/libetcdclient.so
$(Q)rm -f GEN_FLAG
make considers the target to be the file to be generated. Any comparison of timestamps to determine whether or not to rebuild that file is therefore a comparison of those of the dependencies versus that of the target.
In your case make is checking the timestamp of a non-existing file etcd-gen, and since it doesn't exist or bears no relationship to the sources anyway (the sources aren't listed as dependencies) a rebuild is triggered each time.
What you could do instead is re-structure your makefile so your proto sources are dependencies of the $(GEN_FLAG) and the contents of $(GEN_FLAG) is a target. Then you can declare $(GEN_FLAG) a dependency of etcd-gen.
Related
I am a cuda programmer and new in vscode and Makefile environment.
For a better programming, I use multiple .cu files for my functions. Thus, for the pull_model and build_meshgrid functions, the makefile becomes:
# Target rules
all: build
build: kernel
check.deps:
ifeq ($(SAMPLE_ENABLED),0)
#echo "Launch file will be waived due to the above missing dependencies"
else
#echo "Launch file is ready - all dependencies have been met"
endif
kernel: kernel.o Engine/Engine.o Engine/pull_model.o Engine/build_meshgrid.o
$(EXEC) $(NVCC) $(ALL_LDFLAGS) $(GENCODE_FLAGS) -o $# $+ $(LIBRARIES)
$(EXEC) mkdir -p bin/$(TARGET_ARCH)/$(TARGET_OS)/$(BUILD_TYPE)
$(EXEC) cp $# bin/$(TARGET_ARCH)/$(TARGET_OS)/$(BUILD_TYPE)
%.o: %.cu
$(EXEC) $(NVCC) $(INCLUDES) $(ALL_CCFLAGS) $(GENCODE_FLAGS) -o $# -c $<
run: build
$(EXEC) ./kernel
testrun: build
clean:
rm -f kernel *.o Engine/*.o
rm -rf bin/$(TARGET_ARCH)/$(TARGET_OS)/$(BUILD_TYPE)/kernel
clobber: clean
I need to reduce the kernel somehow, like this:
kernel: kernel.o Engine/*.o
$(EXEC) $(NVCC) $(ALL_LDFLAGS) $(GENCODE_FLAGS) -o $# $+ $(LIBRARIES)
$(EXEC) mkdir -p bin/$(TARGET_ARCH)/$(TARGET_OS)/$(BUILD_TYPE)
$(EXEC) cp $# bin/$(TARGET_ARCH)/$(TARGET_OS)/$(BUILD_TYPE)
So, all .cu files under Engine folder are targed.
Best Regards!
Well, you can't use Engine/*.o because when you run your makefile, no object files exist and so this wildcard will expand to nothing, then nothing will be built (except kernel.o).
It's a catch-22 because you can't just tell make "build all the object files that you would build if you knew which objects to build!"
If what you want is to build object files for all source files in the Engine directory, you can tell make to do that:
SRCS := $(wildcard Engine/*.c)
OBJS := $(SRCS:%.c=%.o)
kernel: kernel.o $(OBJS)
....
I am very new to C++ makefiles and I have the makefile below which I mostly copied from somewhere on the internet. My contribution to the makefile is the "test" target, which is the problem.
My code is organized in include (for headers), src (for sources) and test (for test sources) directories. For the standard target I want to build all files in the src folder which also includes the main.cpp file. To make the test build I want to build all files in the src folder except main.cpp and additionally all files in the test folder. I hoped that my test target would achieve that, but it fails. It just build the files from the standard target.
How can I correct my mistakes.
Thank you
The makefile I use:
APP=myappname
SRC_DIR=src
INC_DIR=include
OBJ_DIR=obj
BIN_DIR=bin
TEST_SRC_DIR=test
MAIN_CPP_NAME=main.cpp
CC=g++
LD=g++
CFLAGS=-O2 -c -Wall -std=c++17
LFLGAS=
DFLAGS=-g3 -O0 -DDEBUG
INCFLAGS=-I$(INC_DIR)
SOURCES=$(wildcard $(SRC_DIR)/*.cpp)
HEADERS=$(wildcard $(INC_DIR)/*.hpp)
OBJECTS=$(SOURCES:$(SRC_DIR)/%.cpp=$(OBJ_DIR)/%.o)
DEPENDS=$(OBJ_DIR)/.depends
.PHONY: all
all: $(BIN_DIR)/$(APP)
.PHONY: debug
debug: CFLAGS+=$(DFLAGS)
debug: all
.PHONY: test
test: SOURCES=$(wildcard $(SRC_DIR)/*.cpp)
test: SOURCES=$(filter-out $(SRC_DIR)/$(MAIN_CPP_NAME), $(SOURCES))
test: OBJECTS=$(SOURCES:$(SRC_DIR)/%.cpp=$(OBJ_DIR)/%.o)
test: TEST_SOURCES=$(wildcard $(TEST_SRC_DIR)/*.cpp)
test: OBJECTS+=$(TEST_SOURCES:$(TEST_SRC_DIR)/%.cpp=$(OBJ_DIR)/%.o)
test: all
$(BIN_DIR)/$(APP): $(OBJECTS) | $(BIN_DIR)
$(LD) $(LFLGAS) -o $# $^
$(OBJ_DIR)/%.o: | $(OBJ_DIR)
$(CC) $(CFLAGS) $(INCFLAGS) -o $# $<
$(DEPENDS): $(SOURCES) | $(OBJ_DIR)
$(CC) $(INCFLAGS) -MM $(SOURCES) | sed -e 's!^!$(OBJ_DIR)/!' >$#
ifneq ($(MAKECMDGOALS),clean)
-include $(DEPENDS)
endif
$(BIN_DIR):
mkdir -p $#
$(OBJ_DIR):
mkdir -p $#
.PHONY: clean
clean:
rm -rf $(BIN_DIR) $(OBJ_DIR)
You're trying to write a sophisticated makefile without understanding the basics, which is almost impossible. Let's try a very simple approach.
First, suppose the source files are
src/main.cpp
src/func.cpp
test/unit.cpp
test/global.cpp
Now we need a list of the object files that test will need.
SOURCES:=$(wildcard $(SRC_DIR)/*.cpp) # this will be src/main.cpp src/func.cpp
NON_MAIN_SOURCES:=$(filter-out $(SRC_DIR)/$(MAIN_CPP_NAME), $(SOURCES)) # this will be src/func.cpp
TEST_DIR_SOURCES:=$(wildcard $(TEST_SRC_DIR)/*.cpp) # this will be test/unit.cpp test/global.cpp
TEST_SOURCES:=$(notdir $(NON_MAIN_SOURCES) $(TEST_DIR_SOURCES)) # this will be func.cpp unit.cpp global.cpp
TEST_OBJECTS:=$(patsubst %.cpp,$(OBJ_DIR)/%.o,$(TEST_SOURCES)) # this will be obj/func.o obj/unit.o obj/global.o
(Note that you must avoid name collisions between src/ and test/, e.g. having a src/func.cpp and also a test/func.cpp. This is a consequence of your directory structure; there's no way for the makefile to work around it.)
Now we must tell Make how to build those object files. We can start with a rule for the sources in src:
$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp
$(CC) $(CFLAGS) $(INCFLAGS) -o $# $<
(I have omitted the $(OBJ_DIR) prerequisite for simplicity. For now, you will be responsible for making sure that obj/ exists.)
Now for the sources in test. A crude but effective approach is to add a rule for them:
$(OBJ_DIR)/%.o: $(TEST_DIR)/%.cpp
$(CC) $(CFLAGS) $(INCFLAGS) -o $# $<
There are more sophisticated ways, but you must master the basics first.
Finally, the rule to build bin/mytestname:
$(BIN_DIR)/mytestname: $(TEST_OBJECTS)
$(LD) $(LFLAGS) -o $# $^
That's enough for now.
Edit
After writing this I noticed some things and went back over the file. I changed the line:
$(CXX) -MT $(BUILD_DIR)/$(#:.d=.o) -MM $(CXXFLAGS) $^ > $#
To the following:
$(CXX) -MM $(CXXFLAGS) $^ > $#
Since it seemed kind of redundant to first take the generated dependency files, name them into .o files and then turn them back into .d files.
Now the target actually makes, but it will still not remake when the header files are changed.
Original
I have this makefile I created as part of a course, and everything works fine when running make, building for both the host and target. The problem arises when a change occurs in a header file(example: part1.h), this implementation was made specifically to take this into account, and it seemed to work.
A few weeks on now, and I finally realised it doesn't work as intended, as it will not recompile after a change to any of the header files. (part1.h or part2.h).
I took a look into the dependency files generated by the target:
$(BUILD_DIR)/%.d: %.cpp
An example of part1.d has the following contents:
build/host/build/host/part1.o: part1.cpp part1.h
To me, the double directory listing is an error, but so I tried changing the line:
$(CXX) -MT $(BUILD_DIR)/$(#:.d=.o) -MM $(CXXFLAGS) $^ > $#
To the following, removing the BUILD_DIR variable:
$(CXX) -MT $(#:.d=.o) -MM $(CXXFLAGS) $^ > $#
Doing this solves the double directory listing problem, but gives me a linker error:
build/host/main.o: file not recognized: File format not recognized
I'm trying to find some solution to this, which will make sure that the program is recompiled when changes are made to the header files.
I'm trying to figure out what I'm missing, I've looked at the related literature and everything seems to match up.
SOURCES=main.cpp part1.cpp part2.cpp
OBJECTS=$(addprefix $(BUILD_DIR)/,$(SOURCES:.cpp=.o))
DEPS=$(addprefix $(BUILD_DIR)/,$(SOURCES:.cpp=.d))
EXE=prog
CXXFLAGS=-I.
#Make sure that default choice for make is the host
ifndef (${ARCH})
ARCH=host
endif
#Making for host
ifeq (${ARCH},host)
CXX=g++
BUILD_DIR=build/host
BIN_DIR=bin/host
endif
# Making for target
ifeq (${ARCH},target)
CXX=arm-devkit-g++
BUILD_DIR=build/target
BIN_DIR=bin/target
endif
$(BIN_DIR)/$(EXE): $(DEPS) $(OBJECTS)
$(CXX) $(CXXFLAGS) -o $# $(OBJECTS)
#Generate dependency files. Place them in build/host or build/target depending on choice
$(BUILD_DIR)/%.d: %.cpp
$(CXX) -MT $(BUILD_DIR)/$(#:.d=.o) -MM $(CXXFLAGS) $^ > $#
#Generate object files. Place them in build/host or build/target, depending on choice.
$(BUILD_DIR)/%.o: %.cpp
$(CXX) -o $# -c $^
#Create the build dir,
${BUILD_DIR}:
mkdir -p ${BUILD_DIR}
#Create the bin dir
${BIN_DIR}:
mkdir -p ${BIN_DIR}
.PHONY:clean
clean:
rm -f $(BUILD_DIR)/*.o $(BIN_DIR)/$(EXE) $(BUILD_DIR)/*.d
ifneq ($(MAKECMDGOALS),clean)
-include $(DEPS)
endif
If you generate the dependencies as part of the compilation not only do you avoid having to take the extra step, you also rely on the compiler to generate the correct paths. This entails a few other changes to your makefile but I found it simpler to just rewrite it, make has a number of features to simplify some of the other operations.
arch ?= host
build_dir := build/$(arch)
bin_dir := bin/$(arch)
ifeq ($(arch),target)
prefix := arm-devkit-
endif
sources := main.cpp part1.cpp part2.cpp
objects := $(sources:%.cpp=$(build_dir)/%.o)
deps := $(objects:.o=.d)
target := $(bin_dir)/prog
CXX := $(prefix)$(CXX)
CPPFLAGS := -MMD -MP
$(target): CC := $(CXX)
$(target): $(objects) | $(bin_dir)
$(LINK.o) $^ $(LDLIBS) -o $#
$(build_dir)/%.o: %.cpp | $(build_dir)
$(COMPILE.cpp) $(OUTPUT_OPTION) $<
$(build_dir) $(bin_dir): ; mkdir -p $#
.PHONY: clean
clean: ; $(RM) $(target) $(objects) $(deps)
-include $(deps)
I'm trying to write a Makefile for my project that automatically generates the dependencies between my source files.
The Makefile is as follows:
CC = g++
CFLAGS = -std=c++11 -Wfatal-errors -fdiagnostics-color=always
LDFLAGS = -lm -lfftw3
SRCDIR = src
OBJDIR = obj
DEPDIR = dep
SRCS = $(wildcard $(SRCDIR)/*.cpp)
OBJS = $(patsubst $(SRCDIR)/%.cpp,$(OBJDIR)/%.o,$(SRCS))
DEPS = $(patsubst $(SRCDIR)/%.cpp,$(DEPDIR)/%.d,$(SRCS))
PROG = whistle_recognition
$(PROG): $(OBJS)
$(CC) $(CFLAGS) -o$(PROG) $(OBJS) $(LDFLAGS)
.PHONY: clean run
$(DEPS): $(DEPDIR)/%.d : $(SRCDIR)/%.cpp
$(CC) $(CFLAGS) -MM $< -MF $#
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp
$(CC) $(CFLAGS) -c $< -o $#
clean:
rm -r $(OBJS) $(DEPS) $(PROG)
run: $(PROG)
./$(PROG)
-include $(DEPS)
Generating the dependencies works fine. However, Make ignores them.
Suppose I have a file foo.cpp that includes the header file quux.h .
If foo.cpp changes Make rebuilds foo.o. However if quux.h changes Make thinks that foo.o is up to date. How can I fix this?
Edit:
Heres more info: output of make -d whistle_recognition | grep signalscanner.d:
Reading makefile 'dep/signalscanner.d' (search path) (don't care) (no ~ expansion)...
Considering target file 'dep/signalscanner.d'.
File 'dep/signalscanner.d' does not exist.
Finished prerequisites of target file 'dep/signalscanner.d'.
Must remake target 'dep/signalscanner.d'.
g++ -std=c++11 -Wfatal-errors -fdiagnostics-color=always -MM src/signalscanner.cpp -MF dep/signalscanner.d
Putting child 0xcda970 (dep/signalscanner.d) PID 2404 on the chain.
Live child 0xcda970 (dep/signalscanner.d) PID 2404
Successfully remade target file 'dep/signalscanner.d'.
Reading makefile 'dep/signalscanner.d' (search path) (don't care) (no ~ expansion)...
Considering target file 'dep/signalscanner.d'.
Finished prerequisites of target file 'dep/signalscanner.d'.
Prerequisite 'src/signalscanner.cpp' is older than target 'dep/signalscanner.d'.
No need to remake target 'dep/signalscanner.d'.
output of make -p whistle_recognition | signalscanner.o(shortened):
[...]
obj/signalscanner.o: src/signalscanner.cpp
[...]
signalscanner.o: src/signalscanner.cpp src/signalscanner.h src/frequencyanalyzer.h src/freqanalyzer_test.h src/wav_file.h src/signalscanner_test.h
And theres the problem: g++ doesn't include the obj/-prefix to the targets... Is there a way to fix this via pattern substitution?
People often have such rules for dependency generation, but they are really unnecessary.
The first time a project is built no dependencies are necessary since it builds all sources anyway. It is only the subsequent builds that require the dependencies from the previous build to detect what needs to be rebuilt.
The dependencies are just a by-product of compilation.
The generated dependencies contain paths to corresponding .o files. Since .o output paths were not specified when generating dependencies, those paths are incorrect.
The following solution puts .d files along with corresponding .o files and those .d files contain the correct paths to .o. It also does compilation and dependencies in one pass.
Fixes:
Remove these lines:
$(DEPS): $(DEPDIR)/%.d : $(SRCDIR)/%.cpp
$(CC) $(CFLAGS) -MM $< -MF $#
-include $(DEPS)
Update these lines:
DEPS = $(OBJS:%.o=%.d)
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp
$(CC) -c $(CFLAGS) -MD -MP -o $# $<
Add these lines at the bottom:
ifneq ($(MAKECMDGOALS),clean)
-include $(DEPS)
endif
I have been creating a library. When I compile it as a static library, it works fine. Now I want to turn it into a shared library. The library is created and in the proper place, but when I try to compile the client code, the linking phase says that it can't find the library.
I already tried to rename it to al or dylib but that doesn't help either. When I put the -v flag on the linking, I can see that my library path is there. I also tried different paths. I used a relative path, but even with a full path it doesn't find it.
The Makefile from the library:
.SUFFIXES:
.SUFFIXES: .o .cpp
.SUFFIXES: .o .d
CC := g++
LNK:= g++
CXXFLAGS_RELEASE = -fPIC -shared -O2 -Wall -fmessage-length=0
CXXFLAGS_DEBUG = -fPIC -shared -g -Wall -fmessage-length=0 -D _DEBUG
CXXFLAGS = $(CXXFLAGS_DEBUG)
OBJDIR:= obj
SRCDIR:= src
HDIR:= include
INCLUDE_PATHS:= -Iinclude -Iinclude/interfaces -Iinclude/support
CPP_FILES := propertyfile/propertyfile.cpp \
propertyfile/propertyitem.cpp \
propertyfile/propertyfactory.cpp \
helper/string_helper.cpp
OBJ := $(patsubst %.cpp,$(OBJDIR)/%.o, $(CPP_FILES))
SRC := $(patsubst %.cpp,$(SRCDIR)/%.o, $(CPP_FILES))
LIBS:=
TARGET:= libsupport.so
all: $(TARGET)
$(TARGET): $(OBJ)
$(LNK) -o $(TARGET) $(OBJ) -shared
#cp $(TARGET) ../lib
#cp -r include ..
clean:
rm -f $(OBJ) $(ASM) $(TARGET)
-include $(patsubst %.cpp,$(OBJDIR)/%.d, $(CPP_FILES))
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp $(OBJDIR)/%.d
#mkdir -p `dirname $#`
$(CC) $(CXXFLAGS) -c $< -o $# $(INCLUDE_PATHS)
$(OBJDIR)/%.d: $(SRCDIR)/%.cpp
#mkdir -p `dirname $#`
$(CC) $(CXXFLAGS) -MM -MT $# -MF $(OBJDIR)/$*.d -c $< $(INCLUDE_PATHS)
And here is the Makefile for the application:
.SUFFIXES:
.SUFFIXES: .o .cpp
CC := g++
LD := g++
CXXFLAGS_RELEASE = -O2 -Wall -fmessage-length=0
CXXFLAGS_DEBUG = -g -Wall -fmessage-length=0 -D _DEBUG
CXXFLAGS = $(CXXFLAGS_DEBUG)
OBJDIR:= obj
SRCDIR:= src
INCLUDE_PATHS:= -Iinclude -I../include
LIBS:= -L /cygdrive/d/src/c/lib -lsupport
CPP_FILES := nohupshd.cpp \
daemon.cpp \
task.cpp
OBJ := $(patsubst %.cpp,$(OBJDIR)/%.o, $(CPP_FILES))
SRC := $(patsubst %.cpp,$(SRCDIR)/%.o, $(CPP_FILES))
TARGET:= nohupshd
all: $(TARGET)
$(TARGET): $(OBJ)
$(LD) -o $(TARGET) $(OBJ) $(LIBS)
clean:
rm -f $(OBJ) $(ASM) $(TARGET)
-include $(patsubst %.cpp,$(OBJDIR)/%.d, $(CPP_FILES))
$(OBJDIR)/%.o: $(SRCDIR)/%.cpp $(OBJDIR)/%.d
#mkdir -p `dirname $#`
$(CC) $(CXXFLAGS) -c $< -o $# $(INCLUDE_PATHS)
$(OBJDIR)/%.d: $(SRCDIR)/%.cpp
#mkdir -p `dirname $#`
$(CC) $(CXXFLAGS) -MM -MT $# -MF $(OBJDIR)/$*.d -c $< $(INCLUDE_PATHS)
After some experimenting I found a solution on how to compile a shared library under cygwin.
Apparently the compiler is looking for a DLL file even though it is inside cygwin. so the first step is to add your path, where the library is going to be to the PATH variable.
export PATH=$PATH:/cygdrive/d/src/c/lib
Apparently when linking against a shared library, the linker seems to look for a DLL file by default. I don't know why, because inside cygwin I would expect it to look for a .so file just like on other UNIX systems.
However, there are two solutions to this, which both work.
First, you can create a link to your .so library with the name .dll
ln -s /cygdrive/d/src/lib/libsupport.so libsupport.dll
In this case the makefile doesn't have to be changed and -lsupport will find the library while linking. I prefer this solution.
Second, you can specify the linker option with the full name.
LIBS:= -L /cygdrive/d/src/c/lib -l:libsupport.so
then you don't have to create a link.
So the crucial thing seems to be that the shared library must be in the PATH under cygwin. Using LD_LIBRARY_PATH doesn't help in that case as you can link the executable, but when trying to run it, it will not find it.
ldd nohupshd.exe
libsupport.so => not found
UPDATE: For some reason when I checked with ldd, my library was suddenly gone from the list. I found out that cygwin uses the name to differentiate between MS Windows and Unix shared libraries. So in order to make it work, the name of the library must be cyg.so to make it work, otherwise the exectuable seems to be some Windows build. In this case you don't need to create the link named x.dll as the shared library stays inside the Unix environment.
$(LNK) -o cyg$(TARGET).so $(OBJ) -shared
When using eclipse for debugging, the path to the shared library must also be in the windows path environment variable. Otherwise the debug session immediately terminates without an error.