I'm new to C and wanted to use a library (MLT Multimedia Framework)
I've built it and it produced the following directories: include lib share
Inside lib there are .so .a .la files
Inside include there are .h files
Now, I'm instructed to do this:
#include <framework/mlt.h> which is inside include/mlt/framework/
Questions:
Why I do I need to place the header file that contains only function prototypes? Where are the real functions then? are they linked someway to the ones included in lib directory?
Where to place my own files and how to compile it?
How to learn more about the topics:
Dynamic/Static libraries
Building / making / installing
How to use any C library
If you don't have the function prototypes, how would the compiler know what functions exist in the library? Short answer is: It doesn't. Longer answer: The compiler doesn't care about library files, static (files ending in .a) or shared (files ending in .so), all it cares about is the current translation unit. It's up to the linker to handle resolving undefined references.
When you use libraries, you include the header files that contain the needed declarations (structures, classes, types, function prototypes) into the source file. The source file plus all included header files forms the translation unit that the compiler uses to generate code. If there are undefined references (for example a call to a function in the library) the compiler adds special information about that to the generated object file. The linker then looks through all object files, and if it finds an unresolved reference it tries to find it in the other object files and the provided libraries. If all definitions are resolved the linker generates the final executable, otherwise it will report the unresolved definitions as errors.
To answer your other questions:
Where to place my own files and how to compile it?
This is two questions, the answer to the first one (about placement of your files) is that it doesn't really matter. For small project with only a few source and header files, it's common to place all files in a common project directory.
The second question, about compiling, there are different ways to do it too. If there are only one or two source files you could use the compiler frontend (e.g. gcc) to compile and link and generate your executable all in one go:
$ gcc -Wall -g source1.c source2.c -o your_program_name
The above command takes two source files, compiles and links them into the program your_program_name.
If you need to use a library, there are one or two things that you need to add to the above command line:
You need to tell the linker to link with the library, this is done with e.g. the -l (lower case L) option:
$ gcc -Wall -g source1.c source2.c -o your_program_name -lthe_library
It's important to note that the_library is the base name of the library. If the library file is named libthe_library.so then only the_library part is needed, the linker will add the other parts automatically.
If the library is not in a standard location, then you need to tell the compiler and linker where the library file are. This is done with the -I (capital i) option to tell the preprocessor where the header files are, and the -L (capital l) where the linker files are.
Something like
$ gcc -Wall -g -Ilocation/of/headers source1.c source2.c -o your_program_name -Llocation/of/libraries -lthe_library
If you have more than a couple of source files, it's common to use so called makefiles that lists all source files, their dependencies, compiler and linker flags, and contain rules on how to build object files and link the final program. Such a makefile could look like
CFLAGS = -Wall -g
LDFLAGS = -g
SOURCES = source1.c source2.c
OBJECTS = $(SOURCES:.c=.o)
TARGET = your_program_name
.PHONY: all
all: $(TARGET)
$(TARGET): $(OBJECTS)
$(LD) $(LDFLAGS) $^ -o $#
%.o: %.c
$(CC) $(CFLAGS) $< -c -o $#
The above makefile should do just about the same as the previous command line. The big difference is that it's much easier to add more source files, add special rules for special files, and most importantly, the make program will handle dependencies so that if one source file haven't been modified since last build then it won't be compiled. The last bit will make big projects with many source files build much quicker when only one or a few source files has been modified.
How to learn more about the topics [...]
By going to your favorite search engine, and looking for those topics there. I also recommend e.g. Wikipedia.
Of course, if you use an Integrated Development Environment (a.k.a. an IDE) then you don't have to compile from the command line, or to make your own makefiles, the IDE will handle all that for you. It will also have dialogs for the project settings where you can enter include paths and library paths, and what libraries to link with.
Why I do I need to place the header file that contains only function prototypes?
So as to satisfy your compiler for declaration of those functions or declaration of classes. As C++ is static type checking language, they must know the type of objects which they will be using.
Where to place my own files and how to compile it?
You can place you code anywhere in you filesystem; only make sure to include .h files in includes path and lib while compiling. Usually you need to modify your path.
You can check about building on this link:
https://en.wikipedia.org/wiki/GNU_build_system
Check the README file that came with the code. It should tell you how to install it into the system properly. Usually there is an install build target which installs the resulting files into the proper directories.
The usual sequence of commands to build and install most products is:
$ ./configure
$ make
$ sudo make install
Related
I am having difficulties with re-writting my C++ library into template form; main problem concerns re-designing the Makefile.
In the previous state, when it was non-template I had:
a header file lib.h in which (protected by include guards) we have the declarations of classes and overloaded operators. This file does not include any other libraries at all.
an implementation file lib.cpp in which, on top of the file, I include many headers from the standard library (cmath, iostream, etc) as well as I include the header file for this custom library: #include "lib.h"
a Makefile with commands for build/install the library:
CC = g++
CFLAGS = -O2
SRC = lib.cpp
HDR = $(SRC:.cpp=.h)
OBJ = $(SRC:.cpp=.o)
LIB = $(OBJ:.o=.a)
.PHONY: all install clean uninstall
# =========================================================
# Build
# =========================================================
all: $(LIB)
# Create an object file
$(OBJ):
$(CC) $(CFLAGS) -c -o $# $(SRC)
# Create a static library file
$(LIB): $(OBJ)
ar rcs $# $<
# =========================================================
# Install
# =========================================================
install: ~/lib/lib/$(LIB) ~/lib/include/$(HDR)
# Create top-level directory for the libraries
~/lib:
mkdir -p $#;
# Create top-level directory for the static library files
~/lib/lib:
mkdir -p $#;
# Create top-level directory for the headers
~/lib/include:
mkdir -p $#;
# Copy the library file into the right directory
~/lib/lib/$(LIB): $(LIB) ~/lib/lib
cp $< $#
# Copy the header file into the right directory
~/lib/include/$(HDR): $(HDR) ~/lib/include
cp $< $#
following the installation my CI (GH Actions) would compile a small test program, which includes my library header (#include <lib.h>) with the following command:
g++ -O0 -Wall --std=c++14 test.cpp -I ~/lib/include/ ~/lib/lib/lib.a -o test
This setup worked fine.
Problems emerge now, when I want to re-write my class to be template.
Following the information in this post I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc). With that change I needed to adjust the compilation process and do not provide my implementation file in the command line (as it is already included in the header and I have to avoid re-definition errors). That works fine. The core of the problem is now in the Makefile, in the command to build an object file. As I try to compile the implementation file of my library, it includes the header and the header includes the implementation again... I red about the problem in this port and they suggest to remove the header inclusion from the implementation file. So I did that, I commented the #include "lib.h" and tried to run:
g++ -O2 -c -o lib.o lib.cpp
And I end up with a lot of error: use of undeclared identifier errors...
How can I build the library properly with make? My constraint is that the library stays in two separate files: header and implementation. What I want in the end is to be able to #include <lib.h> in my further programs.
Is it even possible to create an archived object file? What about shared object library (.so)
I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc).
I can't say I think much of the advice you followed there. Code that is intended to be used as or in a header should be named appropriately (.h, .hpp, or whatever convention your project follows), and code that is suitable for direct compilation pretty much never belongs in a header. Perhaps your conversion involves changing everything of the latter kind into to former kind, so maybe you want to rename lib.cpp to lib_impl.h, or something similar, and skip trying to compile it at all. Maybe.
Note well, however, that if you name and structure the implementation code as a header, then it needs its own include guards. Note also that it must not, then, contain any external, non-template functions. If it did, then no two separate translation units contributing to the same program could both include the header, as that would result in duplicate function definitions, include guards notwithstanding. And if there are no external, non-template functions (and no external object definitions) then it is pointless to try to compile to an object file, as there would be no accessible entry point to any function within.
How can I build the library properly with make?
This isn't really about make in particular. That just provides automation. The issue is with the structure of the library itself and your expectations.
My constraint is
that the library stays in two separate files: header and
implementation.
This constraint makes sense only if the implementation contains any instantiated templates for external objects or functions, or any external non-template functions or objects. These are the things that would contribute to a buildable object file that you could directly or indirectly link to applications. If converting to a template library means that there are no longer any such entities in your library then the constraint is arbitrary, as the converted result is a header-only library. Nevertheless, you can split up your headers any way you like, as long as each resulting one is structured as a header, with only contents appropriate for a header, and with its own inclusion guards.
On the other hand, if your converted implementation code does contain any of those things then they must not be #included into any header, as discussed above.
What I want in the end is to be able to #include <lib.h> in my further programs.
If the converted implementation code is suitable for use as or in a header, then you are already there, but it would be much better style to rename your lib.cpp as a header. If you want to be able to include that header directly into code other than the main library header then it needs its own inclusion guards -- that will take care of the duplicate declarations. Do note, however, that those errors arise from the fact that you have a circular dependency, and that's a strong sign that you ought to refactor. Such a refactoring would involve moving enough code from lib.cpp to lib.h that the circular dependency can be removed (one of the files would no longer #include the other).
Whatever implementation code is not suitable for use in a header obviously must not be included into the header. If any such code remains in the converted library then perhaps you keep that in lib.cpp, and move the rest into lib.h
Is it even possible to create an archived object file? What about shared object library (.so)
Templates cannot be compiled. They are templates for code that can be compiled: their instantiations. If your conversion leaves nothing else then no, you cannot usefully create an object file or a shared library. And you don't need to do, for this is a header-only library.
I have a conceptual question regarding makefile dependencies and this is because I see inconsistency online about this.
Let's say I have the following files:
main.cpp uses-> my_math.cpp and my_strings.cpp
my_math.cpp uses-> my_math.h
my_strings.cpp uses-> my_strings.h
If I have a makefile, with the general outlay of:
program: $(all_objs)
g++ $(all_objs) -o program
main.o: ...
.......
my_math.o: ...
.......
my_strings.o: ...
.......
I don't know what should go into each dependency. Like, math.o #includes my_math.h and my_strings.h, does that mean that main.cpp needs to recompile if I change my_math.h? But why? It uses it like a library, right? It doesn't need to recompile main.cpp or does it?
As in, should the result for main.o be:
1) main.o: main.cpp
gcc -c main.cpp
2) main.o: main.cpp my_strings.cpp my_strings.h my_math.cpp my_math.h
gcc -c main.cpp
3) main.o: main.cpp my_strings.cpp my_strings.h my_math.cpp my_math.h
gcc -c main.cpp my_strings.cpp my_math.cpp
I'm a bit lost on dependencies and how linking works.
Any help would be appreciated! Thank you!
The dependencies are everything whose change requires recompiling the source code. That includes not only your #include-d headers, but also the indirectly included system headers, and even (in principle) the compiler and build chain itself (when you upgrade your C++ compiler, you should recompile all your software). If some of your C++ code is generated from some source (e.g. by tools like GNU bison or Qt moc, or by your own script), both the sources and the generating tools are dependencies. Read also about package managers.
Practically speaking, the GCC compiler is able to output most make dependencies, notably with -M and related processor options. Read also about auto dependencies generation. See also this.
(in practice, you generally don't code in your Makefile some explicit dependency on the compiler itself; but you should not forget to make clean when the compiler has been upgraded)
Unless your main.cpp is including my_strings.cpp (which is not conventional and is very bad taste), your make rule won't have a dependency from my_strings.cpp to main.o. But probably your main.cpp is #include-ing (directly or indirectly) my_strings.h so main.o should depend not only on main.cpp but also on my_strings.h
As a rule of thumb, your object file my_strings.o depends on the source file my_strings.cpp and all the header files which are directly or indirectly #include-d in it. Your main program executable depends on all its object files and the libraries you are linking into it. Order of program arguments to g++ matters a lot.
It uses it like a library, right?
From what you are showing, you don't have any own libraries (but you probably use the standard C++ library, and perhaps some other system libraries). On Linux these are lib*.a files (static libraries) or lib*.so files (shared libraries). A library is an organized agglomeration of object code -and sometimes other resources.
I'm a bit lost on dependencies and how linking works.
Understand the difference between source code files, object files (they contain relocation information) and executables (on Linux, object files and executable files and shared libraries are using the ELF format). Read also about the role of compilers, linkers (the g++ program can run both) & build automation (for which you are using make).
Read Program Library HowTo and much more about translation units and linkers (& name mangling), notably Levine's book on Linkers & loaders.
See also this & that & this (examples about Makefile for C++ programs).
BTW, you should use g++ (not gcc) when compiling C++ code. There are significant differences (even if gcc is sometimes able to compile C++ or Fortran code, you'll mostly use gcc to compile C code). And (assuming you use specifically GNU make) your Makefile should mention $(CXX) (not g++). You need to understand the builtin rules of make (run once make -p to get them) and you'll better take advantage of them (e.g. use $(COMPILE.cc) or $(COMPILE.cpp) etc...). You certainly should pass -Wall -Wextra (to get all warnings, and even more), and -g (to get debugging information) to g++. Practically speaking, you should set your CXXFLAGS variable in your Makefile.
Take time to carefully read GNU make documentation and Invoking GCC.
Look into the Makefile-s of existing free software projects. For various reasons, some projects are generating their Makefile-s with tools like autoconf or cmake. But most simple projects don't need that generality, and you should be able to write your own Makefile for your C++ projects. Of course, take inspiration from existing code.
If you have
main.cpp uses-> my_math.cpp and my_strings.cpp
my_math.cpp uses-> my_math.h
my_strings.cpp uses-> my_strings.h
The purpose of Make is to maintain dependency between modules in two different ways, by building .o files and by linking .o files.
you can picture it as a dependency tree where main.o is the root
main.o
/ \
my_math.o my_strings.o
for each .o there is also a dependency tree with regard to source files e.g.
main.o my_math.o my_strings.o
/ \ / \ / \
main.cpp main.h my_math.cpp my_math.h my_strings.cpp my_strings.h
So when make builds, it sets up a dependency tree with main.o at its root and then tries to build all .o files needed for main. When all .o files have been built they are linked.
By following the dependency tree Make ensures that main will be linked/built when one of the dependent modules is changed.
However if you have used something like say a constant from one of the included headers #define MAXSTRING 32 you are no longer merely dependent on the .o file, you are then dependent on the header content so you need to make sure that main.o is built if the header is changed since linking is then not enough so you add the .h in the dependency
main.o
/ | \
my_math.o my_strings.o my_strings.h
Of course there are ways to make the header more robust to avoid that dependency but is another question.
Your cpp files do not depend on other cpp files in terms of compilation. Simple cpp file should only depend on h files.
In your question you say main.cpp depends on my_math.cpp and my_strings.cpp, but I think it is not true. I can guess you have #includes there and these are your dependecies.
Generally speaking dependencies for cpp files are all #included h files.
Usually there are no dependencies between cpp files. You just generate o files by compiling them. Then your final binary depends on all o files.
We link in a library(TAO) which is composed of many header files.
Every time I run the pre-processor command on a cpp file( g++ -MM $< $# ), these library files are automatically included in every .d file generated.
These are obviously not system files and almost never change as far as we're concerned, so I would like to eliminate them from my .d files.
Short of filtering out these header files using sed, is there any built in way to accomplish this?
You may instruct gcc to consider some path as system headers with -isystem; and g++ -MM ignore system-headers.
Is it possible to build a library that uses another, already compiled library?
I have some make files that are used to normally build an executable. When making the executable, I use -L ../include/lib1.a to include the original library.
Now, I am building a separate program that is calling the classes from the executable, which was never built into a library, just compiled to the executable with a link like
${CPP} -c ${INC} ${CFLAGS} MyFile.cpp ${OBJ} ${LIB2} -lm
Where LIB2 includes the reference to lib1.a
Now I want to access the class MyFile.cpp directly, and when I build it to its own library lib2.a, and try to call it from the new program, I get a bunch of errors that the classes it references are missing. This is in spite of the fact that when building the new program I am linking in both lib1.a and lib2.a
Should:
-L../include/lib1.a
not be:
-L../include -llib1
I.e. -L denotes the library search path and -l the archive name?
Yes, you should be able to do it. It would be helpful to see the exact errors you are getting.
If you compile your sources into a library and supply the libraries they use on the command line, the compiler is liable to put the other library(s) into your new one. If that happens, and then someone tries to build a program against your library and those others, they will get a whole mess of "doubly-defined symbol" errors.
When you build lib2.a it will not contain the objects files contained in lib1.a.
Your final executable has to link in both of them.
I am running Windows 7 with gcc/g++ under Cygwin. What would be the Makefile format (and extension, I think it's .mk?) for compiling a set of .cpp (C++ source) and .h (header) files into a static library (.dll). Say I have a variable set of files:
file1.cpp
file1.h
file2.cpp
file2.h
file3.cpp
file3.h
....
What would be the makefile format (and extension) for compiling these into a static library? (I'm very new to makefiles) What would be the fastest way to do this?
The extension would be none at all, and the file is called Makefile (or makefile) if you want GNU Make to find it automatically.
GNU Make, at least, lets you rely on certain automatic variables that alone give you control over much of the building process with C/C++ files as input. These variables include CC, CPP, CFLAGS, CPPFLAGS, CXX, CXXFLAGS, and LDFLAGS. These control the switches to the C/C++ preprocessor, compiler, and the linker (the program that in the end assembles your program) that make will use.
GNU Make also includes a lot of implicit rules designed to enable it automatically build programs from C/C++ source code, so you don't [always] have to write your own rules.
For instance, even without a makefile, if you try to run make foobar, GNU Make will attempt to first build foobar.o from foobar.c or foobar.cpp if it finds either, by invoking appropriate compiler, and then will attempt to build foobar by assembling (incl. linking) its parts from system libraries and foobar.o. In short, GNU Make knows how to build the foobar program even without a makefile being present -- thanks to implicit rules. You can see these rules by invoking make with the -p switch.
Some people like to rely on GNU Make's implicit rule database to have lean and short makefiles where only that specific to their project is specified, while some people may go as far as to disable the entire implicit rule database (using the -r switch) and have full control of the building process by specifying everything in their makefile(s). I won't comment on superiority of either strategy, rest assured both do work to some degree.
There are a lot of options you can set when building a dll, but here's a basic command that you could use if you were doing it from the command line:
gcc -shared -o mydll.dll file1.o file2.o file3.o
And here's a makefile (typically called Makefile) that will handle the whole build process:
# You will have to modify this line to list the actual files you use.
# You could set it to use all the "fileN" files that you have,
# but that's dangerous for a beginner.
FILES = file1 file2 file3
OBJECTS = $(addsuffix .o,$(FILES)) # This is "file1.o file2.o..."
# This is the rule it uses to assemble file1.o, file2.o... into mydll.dll
mydll.dll: $(OBJECTS)
gcc -shared $^ -o $# # The whitespace at the beginning of this line is a TAB.
# This is the rule it uses to compile fileN.cpp and fileN.h into fileN.o
$(OBJECTS): %.o : %.cpp %.h
g++ -c $< -o $# # Again, a TAB at the beginning.
Now to build mydll.dll, just type "make".
A couple of notes. If you just type "make" without specifying the makefile or the target (the thing to be built), Make will try to use the default makefile ("GNUMakefile", "makefile" or "Makefile") and the default target (the first one in the makefile, in this case mydll.dll).