Makefile to build shared library - c++

I've been building a C++11 library, and the number of header/source files has grown to the point where compiling programs invoking it, entails passing 20+ .cpp files to g++. I've been reading up on shared libraries and it seems to be the best solution.
However, as headers/source change frequently, I'm hoping to create a makefile that would automatically generate all the .so files from the headers and source.
To better demonstrate what I'm trying to do, I'll take one of my sub-libraries, Chrono and show how I would do this manually.
I first create the object files like so,
$ g++ -std=c++11 -fPIC -g -c -Wall ../src/Chrono/cpp/DateTime.cpp
$ g++ -std=c++11 -fPIC -g -c -Wall ../src/Chrono/cpp/Schedule.cpp
$ g++ -std=c++11 -fPIC -g -c -Wall ../src/Chrono/cpp/Duration.cpp
$ g++ -std=c++11 -fPIC -g -c -Wall ../src/Chrono/cpp/DayCount.cpp
So that I now have DateTime.o, Schedule.o, Duration.o, and DayCount.o in the current directory. I then create the .so file,
$ g++ -shared -Wl,-soname,libChrono.so.1 -o libChrono.so.1.0.1 DateTime.o Schedule.o Duration.o DayCount.o -lc
I then go,
$ rm ./*.o && ldconfig -n ./
So that my working directory now contains, libChrono.so.1.0.1 and the symlink libChrono.so.1.
There are quite a few subdirectories I need to do this for, so you can see that this quickly grows inefficient whenever changes to headers/source are made. I would be grateful if anyone can help me design a makefile that accomplishes all this simply by invoking make.
Thanks!
UPDATE:
Based on goldilock's advice and some digging, I managed to bang together:
CXX=g++
CFLAGS=-std=c++11
TARGET=./lib/libChrono.so.1.0.1
CHRONODIR=./src/Chrono
CHRONOSRC=$(wildcard $(CHRONODIR)/cpp/*.cpp)
CHRONOOBJ=$(join $(addsuffix ../obj/, $(dir $(CHRONOSRC))), $(notdir (CHRONOSRC:.cpp=.o)))
all: $(TARGET)
#true
clean:
#-rm -f $(TARGET) $(CHRONOOBJ)
./lib/libChrono.so.1.0.1: $(CHRONOOBJ)
#echo "======================="
#echo "Creating library file $#"
#echo "======================="
#$(CXX) -shared -Wl,-soname,$(join $(basename $#), .1) -o $# $^ -l
#echo "-- $# file created --"
$(CHRONODIR)/cpp/../obj/%.o : $(CHRONOSRC)
#mkdir -p $(dir $#)
#echo "============="
#echo "Compiling $<"
#$(CXX) $(CFLAGS) -fPIC -g -Wall -c $< -o $#
4 .o files are produced in lib/ but I get multiple definition complaints from ld. Before I was compiling the object files separately, but this unwinds CHRONOOBJ on one line. Any ideas?

Fortunately you included the origin of your problem:
I've been building a C++11 library, and the number of header/source files has grown to the point where compiling programs invoking it, entails passing 20+ .cpp files to g++.
Because this reveals a potential XY problem. The straightforward solution to this is to put object files into an archive (aka. a static library) and use that.
GNU make has an implicit rule for creating C++ .o files. It amounts to this:
%.o: %.cpp
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $<
Meaning, if you make DateTime.o in a directory with a makefile that doesn't redefine this, it will make DateTime.o. You may want to add things to $(CXXFLAGS) however, e.g.:
CXXFLAGS += -Wall -Wextra --std=c++11
If you intend to stick with the shared lib route, -fPIC can go there too. That one line could be your entire makefile.
However, you also want to put these together, so you must first declare all the objects and a rule for combining them:
OBJS = DateTime.o Schedule.o Duration.o
libChrono.a: $(OBJS)
ar crvs $# $^
This last line (see man ar) creates the archive (libChrono.a) containing all the objects in $(OBJS). You can then use this with whatever program by placing it in the same directory (or a directory in the library path) and linking -lChrono. Only the necessary parts will be extracted and compiled in. This saves you having to maintain a shared lib in a system directory.
If you still think you need a shared lib, $# and $^ are automatic variables; you can use similar methodology to create a .so, something along the lines of:
SO_FLAGS = -shared
libChrono.so.1.0.1: $(OBJS)
$(CXX) $(SO_FLAGS) -Wl,-soname,libChrono.so.1 -o $# $^ -lc
If that is your first rule, make will take care of everything: building first the objects and then the library. Notice this one has excluded your normal $(CXXFLAGS) to duplicate exactly the compiler line from the question.

Related

When do files in C++ with direct & indirect dependencies have to be recompiled and when is a new linking of the executable sufficient? [duplicate]

I have the following makefile that I use to build a program (a kernel, actually) that I'm working on. Its from scratch and I'm learning about the process, so its not perfect, but I think its powerful enough at this point for my level of experience writing makefiles.
AS = nasm
CC = gcc
LD = ld
TARGET = core
BUILD = build
SOURCES = source
INCLUDE = include
ASM = assembly
VPATH = $(SOURCES)
CFLAGS = -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions \
-nostdinc -fno-builtin -I $(INCLUDE)
ASFLAGS = -f elf
#CFILES = core.c consoleio.c system.c
CFILES = $(foreach dir,$(SOURCES),$(notdir $(wildcard $(dir)/*.c)))
SFILES = assembly/start.asm
SOBJS = $(SFILES:.asm=.o)
COBJS = $(CFILES:.c=.o)
OBJS = $(SOBJS) $(COBJS)
build : $(TARGET).img
$(TARGET).img : $(TARGET).elf
c:/python26/python.exe concat.py stage1 stage2 pad.bin core.elf floppy.img
$(TARGET).elf : $(OBJS)
$(LD) -T link.ld -o $# $^
$(SOBJS) : $(SFILES)
$(AS) $(ASFLAGS) $< -o $#
%.o: %.c
#echo Compiling $<...
$(CC) $(CFLAGS) -c -o $# $<
#Clean Script - Should clear out all .o files everywhere and all that.
clean:
-del *.img
-del *.o
-del assembly\*.o
-del core.elf
My main issue with this makefile is that when I modify a header file that one or more C files include, the C files aren't rebuilt. I can fix this quite easily by having all of my header files be dependencies for all of my C files, but that would effectively cause a complete rebuild of the project any time I changed/added a header file, which would not be very graceful.
What I want is for only the C files that include the header file I change to be rebuilt, and for the entire project to be linked again. I can do the linking by causing all header files to be dependencies of the target, but I cannot figure out how to make the C files be invalidated when their included header files are newer.
I've heard that GCC has some commands to make this possible (so the makefile can somehow figure out which files need to be rebuilt) but I can't for the life of me find an actual implementation example to look at. Can someone post a solution that will enable this behavior in a makefile?
EDIT: I should clarify, I'm familiar with the concept of putting the individual targets in and having each target.o require the header files. That requires me to be editing the makefile every time I include a header file somewhere, which is a bit of a pain. I'm looking for a solution that can derive the header file dependencies on its own, which I'm fairly certain I've seen in other projects.
As already pointed out elsewhere on this site, see this page:
Auto-Dependency Generation
In short, gcc can automatically create .d dependency files for you, which are mini makefile fragments containing the dependencies of the .c file you compiled.
Every time you change the .c file and compile it, the .d file will be updated.
Besides adding the -M flag to gcc, you'll need to include the .d files in the makefile (like Chris wrote above).
There are some more complicated issues in the page which are solved using sed, but you can ignore them and do a "make clean" to clear away the .d files whenever make complains about not being able to build a header file that no longer exists.
You could add a 'make depend' command as others have stated but why not get gcc to create dependencies and compile at the same time:
DEPS := $(COBJS:.o=.d)
-include $(DEPS)
%.o: %.c
$(CC) -c $(CFLAGS) -MM -MF $(patsubst %.o,%.d,$#) -o $# $<
The '-MF' parameter specifies a file to store the dependencies in.
The dash at the start of '-include' tells Make to continue when the .d file doesn't exist (e.g. on first compilation).
Note there seems to be a bug in gcc regarding the -o option. If you set the object filename to say obj/_file__c.o then the generated _file_.d will still contain _file_.o, not obj/_file_c.o.
This is equivalent to Chris Dodd's answer, but uses a different naming convention (and coincidentally doesn't require the sed magic. Copied from a later duplicate.
If you are using a GNU compiler, the compiler can assemble a list of dependencies for you. Makefile fragment:
depend: .depend
.depend: $(SOURCES)
rm -f ./.depend
$(CC) $(CFLAGS) -MM $^>>./.depend;
include .depend
There is also the tool makedepend, but I never liked it as much as gcc -MM
You'll have to make individual targets for each C file, and then list the header file as a dependency. You can still use your generic targets, and just place the .h dependencies afterwards, like so:
%.o: %.c
#echo Compiling $<...
$(CC) $(CFLAGS) -c -o $# $<
foo.c: bar.h
# And so on...
Basically, you need to dynamically create the makefile rules to rebuild the object files when the header files change. If you use gcc and gnumake, this is fairly easy; just put something like:
$(OBJDIR)/%.d: %.c
$(CC) -MM -MG $(CPPFLAGS) $< | sed -e 's,^\([^:]*\)\.o[ ]*:,$(#D)/\1.o $(#D)/\1.d:,' >$#
ifneq ($(MAKECMDGOALS),clean)
include $(SRCS:%.c=$(OBJDIR)/%.d)
endif
in your makefile.
Over and above what #mipadi said, you can also explore the use of the '-M' option to generate a record of the dependencies. You might even generate those into a separate file (perhaps 'depend.mk') which you then include in the makefile. Or you can find a 'make depend' rule which edits the makefile with the correct dependencies (Google terms: "do not remove this line" and depend).
Simpler solution: Just use the Makefile to have the .c to .o compilation rule be dependent on the header file(s) and whatever else is relevant in your project as a dependency.
E.g., in the Makefile somewhere:
DEPENDENCIES=mydefs.h yourdefs.h Makefile GameOfThrones.S07E01.mkv
::: (your other Makefile statements like rules
::: for constructing executables or libraries)
# Compile any .c to the corresponding .o file:
%.o: %.c $(DEPENDENCIES)
$(CC) $(CFLAGS) -c -o $# $<
None of the answers worked for me. E.g. Martin Fido's answer suggests gcc can create dependency file, but when I tried that it was generating empty (zero bytes) object files for me without any warnings or errors. It might be a gcc bug. I am on
$ gcc --version gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
So here's my complete Makefile that works for me; it's a combination of solutions + something that wasn't mentioned by anyone else (e.g. "suffix replacement rule" specified as .cc.o:):
CC = g++
CFLAGS = -Wall -g -std=c++0x
INCLUDES = -I./includes/
# LFLAGS = -L../lib
# LIBS = -lmylib -lm
# List of all source files
SRCS = main.cc cache.cc
# Object files defined from source files
OBJS = $(SRCS:.cc=.o)
# # define the executable file
MAIN = cache_test
#List of non-file based targets:
.PHONY: depend clean all
## .DEFAULT_GOAL := all
# List of dependencies defined from list of object files
DEPS := $(OBJS:.o=.d)
all: $(MAIN)
-include $(DEPS)
$(MAIN): $(OBJS)
$(CC) $(CFLAGS) $(INCLUDES) -o $(MAIN) $(OBJS) $(LFLAGS) $(LIBS)
#suffix replacement rule for building .o's from .cc's
#build dependency files first, second line actually compiles into .o
.cc.o:
$(CC) $(CFLAGS) $(INCLUDES) -c -MM -MF $(patsubst %.o,%.d,$#) $<
$(CC) $(CFLAGS) $(INCLUDES) -c -o $# $<
clean:
$(RM) *.o *~ $(MAIN) *.d
Notice I used .cc .. The above Makefile is easy to adjust for .c files.
Also notice importance of these two lines :
$(CC) $(CFLAGS) $(INCLUDES) -c -MM -MF $(patsubst %.o,%.d,$#) $<
$(CC) $(CFLAGS) $(INCLUDES) -c -o $# $<
so gcc is called once to build a dependency file first, and then actually compiles a .cc file. And so on for each source file.
I believe the mkdep command is what you want. It actually scans .c files for #include lines and creates a dependency tree for them. I believe Automake/Autoconf projects use this by default.

Makefile with multiple separate *.cpp files to output separate *.exe files in different dir

I am stuck, writing my Makefile.
Directory structure:
.\
Makefile
.\src\*.cpp(s)
.\bin
Desire: What I want to achieve with one Makefile.
Run: make
Output (Terminal):
g++ -g -Wall -c -o src/program1.o src/program1.cpp
g++ -g -Wall -c -o src/program2.o src/program2.cpp
g++ -g -Wall -c -o src/program3.o src/program3.cpp
g++ -g -Wall -c -o src/program4.o src/program4.cpp
Output (in /bin/)
program1.exe
program2.exe
program3.exe
program4.exe
EDIT:
CXX = g++
CXXFLAGS = -Wall -g3 -O0
SRC := ${wildcard src/*.cpp}
OBJS := $(SRC:.cpp=.o)
BIN := $(SRC:src/%.cpp=bin/%)
.PHONY: all
all: $(BIN)
$(BIN): $(OBJS)
$(CXX) -c $(CXXFLAGS) -o $(OBJS)
bin/%: src/%.o
$(CXX) -o $# $^
Error:
g++: warning: linker input file unused because linking not done
The introductory parts of the GNU make manual describe that all: $(BIN) creates a target all that depends on a target bin. That means make will try to create bin. Then you have $(BIN): $(OBJS) which says bin depends on all the object files, so make will try to create all the object files. Then there's a recipe for that rule that says, after you've created the object files run this command, which links together all the object files into a single program (bin).
So make is doing exactly what you asked it to do.
The problem is that is apparently not what you want it to do.
In your question you write, then take the original filenames of each *.cpp and add that to the executable which I don't fully understand, but I assumed that you want to link all the objects into a single executable, which is what your makefile does.
But then later you write: How can I output to bin directory and generate the correct executables?, but you never define what "correct executables" means, and this makes it sound like you want to turn each individual object file into its own executable; that's clearly not what your makefile does.
So before you can tell make what you want, first you have understand clearly what you want so you can write it in your makefile. And if you need us to help you write it into your makefile, you need to explain it clearly in your question so we can understand it.
Cheers!
ETA
OK so you want every source file to compile into an object file, then every object file to compile to a separate binary.
First compute the names of all the binaries you want to build:
SRCS := $(wildcard src/*.cpp)
BINS := $(SRCS:src/%.cpp=bin/%)
Now make a rule that depends on all the binaries:
all: $(BINS)
Now make a pattern rule that tells make how to build each one of those binaries:
bin/% : src/%.o
$(CXX) $(CXXFLAGS) -o $# $^ $(LDLIBS)
Now you're actually done, because make already has a built-in rule that knows how to build a .o file into the same directory where the .c file lives, so it can figure out how to build the src/x.o files on its own.
Try something like:
SRC:=${wildcard src/*.cpp}
OBJ:=$(patsubst %.cpp,%.o,${patsubst src/%,bin/%,${SRC}}}
to get the list of the object files, and the rule:
obj/%.o : src/%.cpp
${CXX} -o $# -c $<
for compiling into the right location.
EDIT You have now clarified that each file is a separate main.
SRC:=${wildcard src/*.cpp}
BIN:=$(patsubst %.cpp,,${patsubst src/%,bin/%,${SRC}}}
to get the list of the object files, and the rule:
bin/% : src/%.cpp
${CXX} -o $# $<
will write each output as an executable in bin. To kick it off:
all : ${BIN}

How to improve my makefile in C to move object files and compile even if there are errors?

I created the following makefile:
#COMPILER
CC=gcc
CFLAGS=-I./include/ -L. -Wl,-rpath=. -Wall
CORFLAGS=-I./include/ -c -ansi -pedantic-errors -Wall -Wextra -g
COFLAGS=-I./include/ -Wall -Werror -fpic -c
CSOFLAGS=-shared
#vpath
vpath %.h ./include/
vpath %.c ./test/
vpath %.c ./source/
vpath %.o ./obj/
#PATH
SOURCE=./source/
OUT=-o ./obj/$#
TEST=./test/
OBJPATH=./obj/
#LISTS
CFILESWP=$(wildcard ./source/*.c)
TFILESWP=$(wildcard ./test/*.c)
CFILES=$(notdir $(CFILESWP))
TFILES=$(notdir $(TFILESWP))
TOFILES=$(TFILES:.c=.o)
OFILES=$(CFILES:.c=.o)
OFILESWP=$(addprefix ./obj/,$(OFILES))
NAMES=$(TOFILES:_test.o=)
HFILES=$(CFILES:.c=.h)
.PHONY: clean debug release all
debug: CSOFLAGS+=-g
debug: libds.so
release: CSOFLAGS+=-O2
release: libds.so
test: $(NAMES)
all: libds.so $(NAMES)
%: %_test.c libds.so
$(CC) $(CFLAGS) -o $# $< -lds -g
#SHARED LIBRARY
libds.so: $(OFILES)
$(CC) $(CSOFLAGS) -o libds.so $(OFILES)
#OBJFILES
%.o: %.c %.h
$(CC) $(COFLAGS) -o $# $< -g
#CLEAN
clean:
rm -f *.o $(OBJPATH)*.o
rm -f $(NAMES) libds.so
My make file creates a shared library which called libds.so on make command and creates compiled executables on make test command.
it takes source files called TARGET.c from /source/ directory a.k.a stack.c, queue.c, cbuffer.c and compiles them togeher with their test files from /test/ directory aka stack_test.c, queue_test.c, TARGET_NAME_test.c.
All the .h files are located in the /include/ directory.
and there is also a /obj directory which should contains all the object files which created after running the makefile.
How can I make this makefile better?
How can I move all .o files to /obj directory after each run of make?
Is it possible to create each "project" without the need to compile ALL the targets?
I mean, can I write make, which will create the shared library, and then write create stack and it'll create only executable of stack which compiles /source/stack.c, test/stack_test, include/stack.h and all other associated .h files which appear to be inside the code of the source files.
Can I somehow force the makefile to run and compile only the projects that can be compiled and not to stop the "making", the compilation of the files just because several projets that have syntax errors inside of them or some other errors?
For example:
If I have the following projects: stack.c, queue.c, cbuffer.c
and cbuffer cannot be compiled because something is wrong with its code.
I want to be able to run make and make test and compile the other projects that can be compiled like stack and queue and just show me the compilation error of cbuffer but not to stop the make process.
Thanks.

Is it possible to use gcc/g++/nvcc automatic dependency -M in a single pass of a Makefile without saving dependencies to a file?

I was trying to come up with a solution for automatic dependency using gcc/g++/nvcc and a Makefile.
I thought I'd come up with a solution, to call gcc -M $(SRC FILES) in a Makefile before any compilation targets, with the assumption that Make would now have updated rules for the compilation targets.
An example of the Makefile I've thought would work is as follows:
PROG = main.out
SRC = $(wildcard *.cc)
OBJ = $(SRC:.cc=.o)
all: $(PROG) | deps
$(PROG): $(OBJ)
g++ -o $# $^
$(OBJ): $(SRC)
g++ -c $<
.PHONY: deps
deps:
g++ -M $(SRC)
Now I'm wondering if the call to
g++ -M $(SRC)
Just causes the dependencies to be printed to stdout and infact the Makefile is still none the wiser to the automatic dependencies.
Ideally I'm looking for a solution that will run in a single pass of a Makefile and use gcc/g++/nvcc automatic dependency flags, and preferably one that doesn't require saving the dependencies to a whole bunch of files.
You can do something like below to get both .o and .d files:
g++ -c main.cpp -o main.o -MP -MMD -MF main.d
So define your dependency files (e.g. DEPFILES) in your Makefile and generate .d like the above command, then include DEPFILES by -, which tells GNU Make to include the dep file if it exists.
-include $(DEPFILES)

makefile picks the same source file again and again for different objects

I am trying to write a makefile, suppose I have 4 source files in a directory src/app/:
src/app/file1.cpp
src/app/file2.cpp
src/app/file3.cpp
src/app/file4.cpp
and I want to create different object files (i.e. file1.o, file2.o etc.) into another directory called obj/. In the makefile, I am defining the variables like this:
$(SRC) := $(wildcard src/app/*.cpp)
$(OBJ) := $(addprefix obj/,$(notdir $(SRC:.cpp=.o)))
and my commands are like this:
all: $(OBJ)
$(OBJ): $(SRC)
$(CC) $(CFLAGS) -I/src/app/app.h -c $< -o $#
so when I run make, I see these operations:
g++ -g -Wall -I/src/app/app.h -c src/app/file1.cpp -o obj/file1.o
g++ -g -Wall -I/src/app/app.h -c src/app/file1.cpp -o obj/file2.o
g++ -g -Wall -I/src/app/app.h -c src/app/file1.cpp -o obj/file3.o
g++ -g -Wall -I/src/app/app.h -c src/app/file1.cpp -o obj/file4.o
You can see, the object files have different names but the source file is the same. How do I fix it ?
Please note that I can't use %.o: %.cpp since I have other targets in my makefile for different purposes.
If you can't use
obj/%.o: src/app/%.cpp
then the second simplest way is to add $(OBJ): to the front:
$(OBJ): obj/%.o: src/app/%.cpp
It's called "static pattern rules" - look it up in the GNU Make manual.
I have found a way to solve this (thanks to Zereges for the idea):
$(RGAOBJ): $(RGASRC)
$(CC) $(CFLAGS) -I/src/app/app.h -c \
$(addprefix src/app/,$(notdir $(#:.o=.cpp))) -o $#
I think there is a better way to do.
So any other pointer will be helpful.
The $< variable picks up the first dependency. It is used with generic targets which have variable targets and produce different results. The compile target would match a pattern as target and have a generic dependency. The first target doesn't change in your case.
Your constraint of not being able to use a generic rule is entirely defeating the purpose of the $< variable. A somewhat naive work-around is to translate the destination name back into the source, e.g.,
$(OBJ): $(SRC)
$(CC) $(CFLAGS) -I/src/app/app.h -c $(#:%.o=%.cpp) -o $#
The problem with that approach is that no all object files depend on all source files. As a result, every time one of the source files is changed all object files are produced. For tiny projects and source that may be viable. On anything which I'd consider a real project neither of these conditions holds.
My understanding on your statement on not being able to use pattern rules is that you already have rules for the same pattern but you need to build some targets with some special rules. A work around for that situation is to build targets with a custom suffix which actually build something different and just use a marker file to prevent unnecessary rebuilds. For example:
all: $(OBJ:%=%.custom)
%.o.custom: %.cpp
$(CC) $(CFLAGS) -W -I/src/app/app.h -c $< -o $(#:%.custom=%) && touch $#