Let's say I'm considering a directory structure that looks something like the following for a C project using the bazel build system.
/util
util.c
util.h
/error
error.c
error.h
/math
math.c
math.h
/linalg
matrix.c
matrix.h
And, the following additional requirements:
Each directory should be built as a cc_library that contains or relinks its child cc_libraries.
cc_libraries (or their children) will depend on sibling cc_libraries. For example, matrix.c may #include error.h.
Are there ways to preserve this structure when using bazel that don't result in difficult to maintain BUILD files? Are nested packages usable?
/util
BUILD
util.c
util.h
/error
BUILD
error.c
error.h
/math
BUILD
math.c
math.h
/linalg
BUILD
matrix.c
matrix.h
Or, should the folder structure simply take a shallower form?
/util
BUILD
util.h
util.c
/error
BUILD
error.c
error.h
/math
BUILD
math.c
math.h
/linalg
BUILD
matrix.c
matrix.h
I think either approach could work, but I prefer the first.
There's no rule dictating what can (or cannot) depend on what, as long as it's visible to the rule and no cycles are formed. A heuristic like "deps only go sideways or down" doesn't prevent, but can help in avoiding cycles.
Nothing looks particularly difficult to maintain from this example, just specify your deps as needed:
cc_library(
name = "linalg",
hdrs = [ "matrix.h" ],
srcs = [ "matrix.c" ],
visibility = [ "//visibility:public" ],
deps = [ "//util/error" ],
)
Etc.
Related
I am having difficulties with re-writting my C++ library into template form; main problem concerns re-designing the Makefile.
In the previous state, when it was non-template I had:
a header file lib.h in which (protected by include guards) we have the declarations of classes and overloaded operators. This file does not include any other libraries at all.
an implementation file lib.cpp in which, on top of the file, I include many headers from the standard library (cmath, iostream, etc) as well as I include the header file for this custom library: #include "lib.h"
a Makefile with commands for build/install the library:
CC = g++
CFLAGS = -O2
SRC = lib.cpp
HDR = $(SRC:.cpp=.h)
OBJ = $(SRC:.cpp=.o)
LIB = $(OBJ:.o=.a)
.PHONY: all install clean uninstall
# =========================================================
# Build
# =========================================================
all: $(LIB)
# Create an object file
$(OBJ):
$(CC) $(CFLAGS) -c -o $# $(SRC)
# Create a static library file
$(LIB): $(OBJ)
ar rcs $# $<
# =========================================================
# Install
# =========================================================
install: ~/lib/lib/$(LIB) ~/lib/include/$(HDR)
# Create top-level directory for the libraries
~/lib:
mkdir -p $#;
# Create top-level directory for the static library files
~/lib/lib:
mkdir -p $#;
# Create top-level directory for the headers
~/lib/include:
mkdir -p $#;
# Copy the library file into the right directory
~/lib/lib/$(LIB): $(LIB) ~/lib/lib
cp $< $#
# Copy the header file into the right directory
~/lib/include/$(HDR): $(HDR) ~/lib/include
cp $< $#
following the installation my CI (GH Actions) would compile a small test program, which includes my library header (#include <lib.h>) with the following command:
g++ -O0 -Wall --std=c++14 test.cpp -I ~/lib/include/ ~/lib/lib/lib.a -o test
This setup worked fine.
Problems emerge now, when I want to re-write my class to be template.
Following the information in this post I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc). With that change I needed to adjust the compilation process and do not provide my implementation file in the command line (as it is already included in the header and I have to avoid re-definition errors). That works fine. The core of the problem is now in the Makefile, in the command to build an object file. As I try to compile the implementation file of my library, it includes the header and the header includes the implementation again... I red about the problem in this port and they suggest to remove the header inclusion from the implementation file. So I did that, I commented the #include "lib.h" and tried to run:
g++ -O2 -c -o lib.o lib.cpp
And I end up with a lot of error: use of undeclared identifier errors...
How can I build the library properly with make? My constraint is that the library stays in two separate files: header and implementation. What I want in the end is to be able to #include <lib.h> in my further programs.
Is it even possible to create an archived object file? What about shared object library (.so)
I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc).
I can't say I think much of the advice you followed there. Code that is intended to be used as or in a header should be named appropriately (.h, .hpp, or whatever convention your project follows), and code that is suitable for direct compilation pretty much never belongs in a header. Perhaps your conversion involves changing everything of the latter kind into to former kind, so maybe you want to rename lib.cpp to lib_impl.h, or something similar, and skip trying to compile it at all. Maybe.
Note well, however, that if you name and structure the implementation code as a header, then it needs its own include guards. Note also that it must not, then, contain any external, non-template functions. If it did, then no two separate translation units contributing to the same program could both include the header, as that would result in duplicate function definitions, include guards notwithstanding. And if there are no external, non-template functions (and no external object definitions) then it is pointless to try to compile to an object file, as there would be no accessible entry point to any function within.
How can I build the library properly with make?
This isn't really about make in particular. That just provides automation. The issue is with the structure of the library itself and your expectations.
My constraint is
that the library stays in two separate files: header and
implementation.
This constraint makes sense only if the implementation contains any instantiated templates for external objects or functions, or any external non-template functions or objects. These are the things that would contribute to a buildable object file that you could directly or indirectly link to applications. If converting to a template library means that there are no longer any such entities in your library then the constraint is arbitrary, as the converted result is a header-only library. Nevertheless, you can split up your headers any way you like, as long as each resulting one is structured as a header, with only contents appropriate for a header, and with its own inclusion guards.
On the other hand, if your converted implementation code does contain any of those things then they must not be #included into any header, as discussed above.
What I want in the end is to be able to #include <lib.h> in my further programs.
If the converted implementation code is suitable for use as or in a header, then you are already there, but it would be much better style to rename your lib.cpp as a header. If you want to be able to include that header directly into code other than the main library header then it needs its own inclusion guards -- that will take care of the duplicate declarations. Do note, however, that those errors arise from the fact that you have a circular dependency, and that's a strong sign that you ought to refactor. Such a refactoring would involve moving enough code from lib.cpp to lib.h that the circular dependency can be removed (one of the files would no longer #include the other).
Whatever implementation code is not suitable for use in a header obviously must not be included into the header. If any such code remains in the converted library then perhaps you keep that in lib.cpp, and move the rest into lib.h
Is it even possible to create an archived object file? What about shared object library (.so)
Templates cannot be compiled. They are templates for code that can be compiled: their instantiations. If your conversion leaves nothing else then no, you cannot usefully create an object file or a shared library. And you don't need to do, for this is a header-only library.
I have a project directory structure of:
Root
Source
Common
MyFolder
++ My 3 source files and header
When I am building my project it generates 3 to 4 shared libraries. Lib1 compiled using c++98 and others using c++11. Flags are added in CmakeList.txt which is at root.
I need my 3 source files to be compiled for Lib1 and for other Libs as as well. but here what happens is compiler is first compiling my source file for lib using c++11 and then it is trying to use same .o file for Lib1 as well. So for .o file which is generated using c++11 is throwing exception when same is used for c++98 compiled library.
So how do write this in CmakeList.txt such that compiler rather than trying to use same .o file will compile source file again for Lib1(c++98 compiled library)
Is there any flag I can specify so that it won't take precompiled .o file and will compile it again ?
Here flags are not being overridden for different shared libraries but actually same object file by make file is being used for different flags
This is sort of counter to how makefiles and cmake usually work.
Most users consider it really important that make performs an incremental build.
The usual way with makefiles is to do make clean which is supposed to remove any binaries and object files that were created.
However, sometimes I write cmake scripts that use globbing over the source directory to assemble the project. (That means, it says "just grab all *.cpp files in the /src folder and make an executable from them".) A makefile cannot check what files in a directory, so the make build will be broken after I add a new file, and make clean won't fix it -- the whole makefile will need to be regenerated by cmake.
Usually what I do is, I write a simple bash script, named rebuild.sh or something,
#!/bin/bash
rm -rf build
mkdir build
cd build
cmake ..
make -j3
./tests
And I put that in the root of my repository, and add /build to my .gitignore. I call that when I want to do a full rebuild -- it nukes the build directory, so its foolproof. When I want an incremental rebuild, I just type make again in the /build directory.
The rebuild.sh script can also serve a double purpose if you use travis-ci for continuous integration.
Most build system assume the compiled objects remain the same within the same pass. To avoid shooting your foot I would suggest telling the build system they were actually different objects, while still compiled from same source files.
I'm not familiar with cmake but this is how you do with make:
For example you have a a.cpp which you want to compile 2 times for different compiler options:
#include <stdio.h>
int main(int argc, char* argv[]) {
printf ("Hello %d\n", TOKEN);
return 0;
}
And the Makefile would looks like:
SRC := $(wildcard *.cpp)
OBJ_1 := $(patsubst %.cpp,%_1.o,$(SRC))
OBJ_2 := $(patsubst %.cpp,%_2.o,$(SRC))
all: pass1 pass2
pass1: $(OBJ_1)
gcc -o $# $(OBJ_1) -lstdc++
pass2: $(OBJ_2)
gcc -o $# $(OBJ_2) -lstdc++
%_1.o: %.cpp
gcc -DTOKEN=1 -c $< -o $#
%_2.o: %.cpp
gcc -DTOKEN=2 -c $< -o $#
clean:
rm -f $(OBJ_1) $(OBJ_2)
What I do here is generate two different list of object from the same source files, which you can even do the same for dependency(-MMD -MP flags).
The background about this question is: my project(C++ language) contains too many files, which including boost, thrift, zookeeper, etc. Now the compilation duration takes too long.
As you know, Visual Studio supports precompiled headers, so as GCC. Because I use automake to manager the Make procedure, so What I want to ask for is whether automake supports precompiled headers? How can I write automake files if so?
Waiting and thranks for your answers.
The whole thing is not about automake, but rather about writing makefiles.
Actually, you can try the .gch, but there're restrictons:
write one common header to include everything (like stdafx.h)
It should be the 1st include in all your sources
use same CFLAGS to compile all your sources
It's a new functionality. You'd want to detect it in your configure.ac
Write a rule in your Makefile.am, name it like stdafx.gch:. Make it empty, if gch is not supported:
stdafx.gch: stdafx.h
$(OPTIONAL_COMPILE_GCH)
.PHONY: $(OPTIONAL_STDAFX_GCH)
Make your _OBJECTS depend on stdafx.gch:
$(foo_SOURCES:.cpp=.$(OBJEXT)): stdafx.gch
# or (is it documented var?)
$(foo_OBJECTS)): stdafx.gch
Youc can use the documented CXXCOMPILE command to be sure that all CXXFLAGS are the same.
Detect in your configure.ac:
if ...; then
[OPTIONAL_COMPILE_GCH='$(CXXCOMPILE) -c -o $# $<']
[OPTIONAL_STDAFX_GCH=]
else
[OPTIONAL_COMPILE_GCH=]
[OPTIONAL_STDAFX_GCH='stdafx.gch']
fi
AC_SUBST(OPTIONAL_COMPILE_GCH)
AC_SUBST(OPTIONAL_STDAFX_GCH)
Update
You want your .gch to be recompiled, when any header file indirectly used by stdafx.h is modified.
Then you could add stdafx.cpp that does nothing, but includes stdagx.h and make .gch depend on stdafx.o. This way the dependency tracking would be managed by automake, but there's a problem:
stdafx.o itself could make use of .gch to compile faster, but if we add such dependency, it would be circular. It's not easy for me to find a solution.
Here's an example that uses status files to solve this: https://gist.github.com/basinilya/e00ea0055f74092b4790
Ideally, we would override the compilation command for stdafx.o, so it first creates .gch and then runs the standard automake compilation, but automake does not use $(CXXCOMPILE) as is. It creates a complex recipe from it (and it depends on automake version):
Update 2
Another solution is to use gcc dependency tracking directly:
stdafx.h.gch: stdafx.h
g++ -MT stdafx.h.gch -MD -MP -MF .deps/stdafx.Tpo -c -o stdafx.h.gch stdafx.h
mv .deps/stdafx.Tpo .deps/stdafx.Po
-include .deps/stdafx.Po
By default, if precompiled header is used, gcc -MD will not put the list of header files into the generated dependencies file, just .gch. There's an option to fix this : -fpch-deps (see bug), but perhaps the default behavior is not that bad: if only the .gch depends on the headers, make will have less work to do.
I am trying to compile my project which has the following structure
Project:
MakeFile
Executable
Source1
.cxx
.h
Source2
.cxx
.h
Build
*.o
And I'm having difficulty writting a Makefile to compile. I currently have commands like:
Src1 = $(wildcard $(SRCDIR1)/*.cxx)
Obj1 = $(patsubst $(SRCDIR1)/%.cxx, $(OBJDIR)/%.o, $(Src1))
But then I have difficulty making the compile rules for the object files a) Because I can no longer do:
$(Obj1): %.cxx
$(CXX) $(CFLAGS) -c $(#:.o=.cxx) -o $#
Because the '$#' command now includes the path of the build directory and b) because the prerequisites now include the build path and I should have a source path. I have read large bits of the make manual to try and find a solution but no luck.
Any help towards a solution appreciated!
Jack
From personal experience, after playing around a bit with "raw" Makefiles, I'd really recommend using some tool building the Makefiles for you, like automake or cmake.
You'll still have to specify all the source files manually - but at least I prefer that to manually fiddling around with the Makefiles.
One option I prefer is building an isomorphic directory structure in the build directory. That is, a source file ${src_dir}/project_x/main.cxx builds into ${build_dir}/project_x/main.o. This way you are protected from name clashes when there are source files with the same name in different source directories. The compiler rule would look something like:
${obj_dir}/%.o : ${src_dir}/%.cxx # % includes directory name, e.g. project_x/main
#-mkdir -p ${#D}
${CXX} -c -o $# ${CPPFLAGS} ${CXXFLAGS} $<
Notice how in the above it creates the target directory on the fly. This is a bit simplistic, in a real-world build system object files depend (using order-only dependency) on its directory, so that make automatically creates the destination directory if it does not exist instead of speculatively trying to create them for each target object file even if it already exists.
Is there a clean/portable way to descend recursively from a given directory, compiling all found .cpp files into a single output file? I'm not sure if makefiles are capable of this sort of thing, or if it's a job for some kind of build script, but I'd like to avoid maintaining various IDEs' project files along with my code.
There are different things that you can do here. I would suggest that you use a multiplatform build system, and follow the documentation for it. I have used CMake in the past, but I wouldn't know how to tell it to compile all files in a directory.
The advantage is that the user can use CMake to generate project files for most common IDEs, so it would allow VisualStudio users to generate VS solutions, MacOSX users to generate Xcode projects, Eclipse CDK projects in pretty much any environment, Makefiles...
There's the wildcard function which can be used to match a pattern like so:
CXX_FILES = $(wildcard src/*.cpp) # All .cpp files in the directory
This is not recursive, but will at least save you from having to manually specify the files in a certain directory. The rule for building them would look something like this:
CXX_FILES = $(wildcard src/*.cpp) # All .cpp files in the directory
OBJ_FILES = $(CXX_FILES:src/%.cpp=$(OBJ_DIR)/%.o) # Corresponding .o files
# Rules
all: $(OBJ_FILES)
g++ $(OBJ_FILES) -o output_filename
$(OBJ_DIR)/%.o: src/%.cpp
g++ -c $< -o $#
Oh, and to answer your question, this method is completely portable.