I'm trying to use a shared library written in C++ from other programs. I have done the shared library and I have tested that it works.
The shared library mentioned above has dependencies to other libraries. I have avoided this problem compiling my shared library telling the linker where to find that dependencies, and deploying that shared library with all its dependencies into a folder.
The problem
Now my problem is the header files. I have a header file to deploy with my shared library, but this header file also includes 3rd-party headers. For example, I have my shared library A, with its header file AH. AH needs to include BH and CH header files to get their definitions, and this is what I want to avoid.
Then, I think the solution could be to use the extern keyword on the 3rd-party functions that I need. The problem with this is that I also need the definitions of some structs (that are declared in the 3rd-party header files mentioned above). So, how can I include this struct definitions using the extern keyword? Is there any better way to do what I want?
I have a header file to deploy with my shared library
Why? Headers are required only at compile time. You do not need to provide headers along with the library if all you want to do is run programs that use it.
But maybe you do want to provide for building additional programs that use your library.
but this header file also includes 3rd-party headers. For example, I have my shared library A, with its header file AH. AH needs to include BH and CH header files to get their definitions, and this is what I want to avoid.
You're not making much sense. If AH requires definitions (or maybe you really mean declarations) provided by BH and CH then you have only two options:
Ensure that BH and CH are available, or
Provide alternative, compatible versions of the definitions / declarations that AH requires.
You describe needing function declarations. Certainly if you have calls to B and C functions in AH, then you need full declarations of those functions. Function declarations are extern by default, so no worries there, but it is essential to have full prototypes. You could copy those from BH and CH if you wanted, though that's a bit questionable.
You also describe needing struct declarations. Type declarations have no linkage, so extern is irrelevant to them; extern would matter here only if you were talking about global variables. You do not need complete declarations of the structs if you use only opaque pointers to instances of those structs. That means never instantiating them directly; never dereferencing them, not even to invoke methods on them; and generally never doing anything that depends on their size or layout. If your usage -- including in your own header file AH -- meets those requirements, then instead of complete declarations of the structs in question you can provide your own, incomplete ones:
struct b_struct;
Of course, since you can't do anything directly with pointers to those, that only makes sense if the pointers to them are provided by functions from libraries B and C, and your own code does nothing with them apart from store them and pass them as function or method arguments. If you need them in any more substantial capacity, then you do need the full declarations. Like with the functions, you can copy those. Even more than with the functions, that's questionable.
More generally, you inquire
how can I include [...] definitions using the extern keyword?
It is essential to understand that extern is about linkage. It is primarily concerned with the run-time visibility of functions, methods, and variables. It relates to compile-time visibility only inasmuch as the compiler and linker implement linkage rules. Slapping the extern keyword on the declaration of a struct type does not magically make that type visible to code that does not contain an actual declaration of that type.
Related
This answer says we need header files for declarations of other functions placed somewhere else, for example in .lib or .dll or .cpp files.
My question is, why do we really need to use declarations in header files at all? Why can't we just use definitions straight away? For example, place a definition instead of a declaration and address to it directly. To a beginner, all this declaration stuff seems to be sort of a redundant work.
A few reasons come to mind:
C++ being an old language, it is sensitive to the ordering of code. Things must be declared before they can be used, and header files are a convenient and consistent way to do this. For example, this example would work in most modern languages, but not in C++:
int main() {
foo(); // error: 'foo' has not been declared
return 0;
}
void foo() { /* ... */ }
It separates the interface from the implementation. This means you can push out updates to a library's implementation, and as long as none of the interfaces have changed, clients do not need to re-compile their code. This also means that you can have multiple implementations corresponding to a single interface - for example, maybe you want to write a filesystem library, but the implementation of that library will have to look significantly different on Linux than it does on Windows. Both of these implementations can share the same header, making the implementation details "opaque" to users.
Bear in mind that all definitions are declarations in C++ (but not all declarations are definitions).
Non-inline functions are usually declared but not defined in a header to avoid breaking the one-definition rule (and linking problems) when that header is included in multiple sources.
Declarations (not definitions) of some class/struct types are needed in headers to break circular dependencies (which causes diagnosable errors and prevents compilation).
These types of concerns rarely arise in "toy" problems used in programming courses and ivory towers, but are critically important in real-world development. For example, in toy problems, a header file might only be included in a single source file - so the concerns of breaking the one-definition rule don't arise. But real-world projects typically include headers in multiple source files - so are affected by problems that arise when definitions are inappropriately placed into header files.
In short, the approaches used in small learning exercises (which cause students to wonder why there is any need to put declarations into headers rather than definitions) don't scale to real-world development.
Some types of definitions (templates, inline functions, and class/struct definitions) are routinely placed into headers. That doesn't mean that all definitions should be placed in headers.
I am writing a header-only template library in C++. I want to able to write some helper functions inside that header file that will not be visible from a cpp file that includes this header library.
Any tips on how to do this?
I know static keyword can be used in cpp files to limit visibility to that one translation unit. Is there something similar for header files?
There isn't really a way.
The convention is to use a namespace for definitions that are not meant to be public. Typical names for this namespace are detail, meaning implementation details, or internal meaning internal to your library.
And as mentioned in comments, C++20 modules changes this situation.
The easy answer is no.
Headers do not exist to the linker, so all functions in the headers are actually in the module that included them. Technically static (or anonymous namespace) functions in a header, are static to the module that included them. This might work, but you will end up with multiple functions, and bloated code-sizes.
Due to this you should always inline functions in header files, or use something that implies inline - like constexpr; If possible...
Function in headers usually rely on either being inline, or templated. A templated function is "weak", meaning the linker assumes that they are all the same, and just uses a random one, and discards the others.
I have a 10+ years old C library which -- I believe -- used to work just fine in the good old days, but when I tried to use it with a C++ source (containing the main function) the other day I ran into some difficulties.
Edit: to clarify, the C library compiles just fine with gcc, and it generates an object file old_c_library.o. This library was supposed to be used in a way so that the C header file old_c_library.h is #included in your main.c C source file. Then your main C source file should be compiled and linked together with old_c_library.o via gcc. Here I want to use a C++ source file main.cpp instead, and compile/link it with g++.
The following three problems occurred, during the compilation of the C++ source file:
one of the header files of the C library contains the C++ reserved word new (it is the name of an integer), which resulted in fatal error; and
one of the header files of the C library contains a calloc call (an explicit typecast is missing), which resulted in fatal error; and
various files of the C library contain code where comparison of signed and unsigned integers happen, which resulted in warnings.
Edit: I tried to use the #extern "C" { #include "obsolete_c_library.h" } "trick", as suggested in the comments, but that did not solve any of my problems.
I can sort out problem 1 by renaming all instances of the reserved words and replacing them by -- basically -- anything else. I can sort out problem 2 by typecasting the calloc call. I might try to sort out the warnings by ideas suggested here: How to disable GCC warnings for a few lines of code.
But I still wonder, is there a way to overcome these difficulties in an elegant, high-level way, without actually touching the original library?
Relevant:
Where is C not a subset of C++? and Do I cast the result of malloc? and How do I use extern to share variables between source files?.
Generally speaking, it is not safe to #include C header files into C++ sources if those headers were not built in anticipation of such usage. Under some circumstances it can be made to work, but you need to be prepared to either modify the headers or write your own declarations for the functions and global variables you want to access.
At minimum, if the C headers declare any functions and you are not recompiling those functions in C++ then you must ensure that the declarations are assigned C linkage in your C++ code. It's not uncommon for C headers to account for that automatically via conditional compilation directives, but if they do not then you can do it on the other side by wrapping the inclusion(s) in a C linkage block:
extern "C" {
#include "myclib.h"
}
If the C headers declare globals whose names conflict with C++ keywords, and which you do not need to reference, then you may be able to use the preprocessor to redefine them:
#define new extern_new
#include "myclib.h"
#undef new
That's not guaranteed to work, but it's worth a try. Do not forget to #undef such macros after including the C headers, as shown.
There may be other fun tricks you can play with macros to adapt specific headers to C++, but at some point it makes more sense to just copy / rewrite the needed declarations (and only those), either in your main C++ source or in your own C++ header. Note that doing so does not remove the need to declare C linkage -- that requirement comes from the library having been compiled by a C compiler rather than a C++ compiler.
You can write a duplicate of the c header with the only difference being lack of declaration of extern int new. Then use the newly created c++ friendly header instead of the old, incompatible one.
If you need to refer that variable from c++, then you need to do something more complex. You will need to write a new wrapper library in c, that exposes read and write functions a pointer to c++ that can be used to access the unfortunately named variable.
If some inline functions refer to the variable, then you can leave those out from the duplicate header and if you need to call them from c++, re-implement them in a c++ friendly manner. Just don't give the re-implementation same name within extern "C", because that would give you a conflict with the original inline function possibly used by some existing c code.
Do the same header duplication as described in 1. leaving out the problematic inline function and re-implementing if needed.
Warnings can be ignored or disabled as you already know. No need to modify the original headers. You could rewrite any {const,type}-unsafe inline functions with versions that don't generate warnings, but you have to consider whether your time is worth it.
The disadvantage of this approach is that you now have two vesions of some/all of the headers. New ones used by c++, and the old ones that may be used by some old code.
If you can give up the requirement of not touching the original library and don't need to work with an existing compiled binary, then an elegant solution would be to simply rename the problematic variables (1.). Make non-c++-compatible inline functions (2.) non-inline and move them into c source files. Fix unsafe code (3.) that generates warnings or if the warning is c++ specific, simply make the function non-inline.
I want to write a library that to use, you only need to include one header file. However, if you have multiple source files and include the header in both, you'll get multiple definition errors, because the library is both declared and defined in the header. I have seen header-only libraries, in Boost I think. How did they do that?
Declare your functions inline, and put them in a namespace so you don't collide:
namespace fancy_schmancy
{
inline void my_fn()
{
// magic happens
}
};
The main reason why Boost is largely header-only is because it's heavily template oriented. Templates generally get a pass from the one definition rule. In fact to effectively use templates, you must have the definition visible in any translation unit that uses the template.
Another way around the one definition rule (ODR) is to use inline functions. Actually, getting a free-pass from the ODR is what inline really does - the fact that it might inline the function is really more of an optional side-effect.
A final option (but probably not as good) is to make your functions static. This may lead to code bloat if the linker isn't able to figure out that all those function instances are really the same. But I mention it for completeness. Note that compilers will often inline static functions even if they aren't marked as inline.
Boost uses header-only libraries a lot because like the STL, it's mostly built using class and function templates, which are almost always header-only.
If you are not writing templates I would avoid including code in your header files - it's more trouble than it's worth. Make this a plain old static library.
There are many truly header-only Boost libraries, but they tend to be very simple (and/or only templates). The bigger libraries accomplish the same effect through some trickery: they have "automatic linking" (you'll see this term used here). They essentially have a bunch of preprocessor directives in the headers that figure out the appropriate lib file for your platform and use a #pragma to instruct the linker to link it in. So you don't have to explicitly link it, but it is still being linked.
When writing a header library (like Boost), can one define free-floating (non-method) functions without (1) bloating the generated binary and (2) incurring "unused" warnings?
When I define a function in a header that's included by multiple source files which in turn is linked into the same binary, the linker complains about redefinitions. One way around this is to make the functions static, but this reproduces the code in each translation unit (BTW, can linkers safely dereplicate these?). Furthermore, this triggers compiler warnings about the function being unused.
I was trying to look for an example of a free-floating function in Boost, but I couldn't find one. Is the trick to contain everything in a class (or template)?
If you really want to define the function (as opposed to declaring it), you'll need to use inline to prevent linker errors.
Otherwise, you can declare the function in the header file and provide its implementation separately in your source file.
You can use the inline keyword:
inline void wont_give_linker_errors(void)
{
// ...
}
Er... The answer to your question is simply don't. You just don't define functions in header files, unless they are inline.
'static' function can also be defined in headers, but it is only useful for very specific rare purposes. Using 'static' just to work around a multiple-definition problem is utter nonsense.
Again, header files are for non-defining function declarations. Why on Earth would you want to define functions there?
You said you are writing "header library". What's a "header library"? Please note, that Boost defines its "functions" in header files because their "functions" are not really functions, they are function templates. Function templates have to be defined in header files (well, almost). If that's wasn't the case, Boost wouldn't be doing something as strange as defining anything in header files.
Besides the already mentioned inline, with most compilers templates have to be defined in headers (and with all compilers it's allowed). Since boost is mostly templates, that explains why it is almost all headers.
People have suggested inline but that violates the very first part of your question i.e. it bloats the code as the full definition is inserted into the code at each call of the function. The answer to your overall question is therefore "No".
If you mark them as static then they are still defined in each source file as you rightly pointed out but only once and so that's a better option than inline if code size is the only issue. I don't know if linkers can, or are allowed to, spot the duplicates and merge them. I suspect not.
Edit:
Just to clear up any confusion as to whether I support the notion of using static and/or defining functions within headers files generally then rest assured I don't. This was simply meant as a technical response as to the differences between functions marked inline and static defined in header files. Nothing more.