I've been trying to make an attempt to make a Game Engine, and I started out by making the Mathmatical foundation ( Vector, Matrix and Point classes. )
I would like to make a static library file (.lib) which I can then use in the rest of my Game Engine where it's needed.
What I was wondering is the following. Since most functions in the library need to be as fast as possible. I'd like to see the functions for example a Vector class being inlined. Can the compiler automatically do this? Or is some kind of hint needed?
I know I can put all the source in the header files but I think that's not really an elegant solution since some functions might be too large to be inlined. ( Header files wouldn't kind of be messy for the poeple using them. )
I hope you guys could help me out.
Christian
Inlining of functions is only guaranteed to work with current compiler technology if the compileunit has visibility of the source code. Once you compile code into a library, it will not inline the function, no matter what the function does.
There is some "whole program optimization" schemes (available in at least MSVC and GCC, probably other compilers too) that produce a "object file" that is only part of the way to machine-code, so that some of the information about "what the source code wanted" is available to the final code-generation/linking stage to move code around, perform inlining and such, but this is only applicable when all source-files are available at the compile-time. Once something is made into final object code (which your static library is), it won't be making it into the "inliner" part.
Obviously, if you have classes, they will be in header files, so if you put functions suitable for inlining into the header file, the compiler will do the right thing.
Yes, you can do this with link time optimization. If you generate a "fat" library, it would include both the compiled source and also an intermediate form that can be used to inline functions at link time (essentially, at link time a final compilation step is done, where everything is visible).
The technique is somewhat fragile since you need to ensure you are using the same flags at compile and link time and there are a few other restrictions: it wouldn't be a good way to publicly distribute a library. If it's for a controlled internal use though, it may work well.
Related
Well, I'm getting in C++ universe and this doubt came over.
It's too boring to have 2 files for each meaning-unit I choose to include in my program.
I know I can have multiple classes in the same (pair of) archive(s), however I would like to clarify if really there's no way to write just one file, instead of a .h and a .cpp ones.
I found some other answers (like this, that and that other) there are actually pretty explicative, but a quite older too. So hopping the language have got some improvement I came to ask:
Is there some compilation option, any another alternative extension, or whatever, that allows me to write just one file?
I appreciate it!
Okay, you need to understand what is going on. This isn't Java.
The .h file is the definition of your class. It's just an include file that can be used other places so those other places understand your class. Now, you CAN actually do the constructor inline, like this:
public:
Foo() { ... your code here ... }
This is perfectly legal. But the problem with this is simple. Everywhere you hit this constructor, the compiler has to insert that code inline. This leads to lots of the same code everywhere you create a new Foo.
If you put the code in your .cpp file, and then compile it, you get a .o file. That .o file includes a single copy of your constructor, and that's what gets called everywhere you create a Foo.
So separating the definition from the code results in smaller programs. That's less important nowadays than it used to be.
This is the nature of C++, and you should accept it.
The .h is an include file, used in other places. The .cpp is the implementation. It's not really onerous, once you grow accustomed.
You have to understand that C++ is a compiled language. When you compile a library, for example, the library contains machine-specific code. If you want to write a program that uses that library, your program has to be able to see function and class definitions to properly link that library. On the other hand, it is absolutely possible to write your entire program in header files -- indeed, the term header-only library exists to describe libraries that have no pre-compiled machine code. That means the responsibility of compiling it falls to you. You'll likely have longer compile times, and because of this, very large libraries are almost exclusively pre-compiled into binaries that are platform-specific (in the absence of a set of binaries for you machine, you must compile from source and link against the result). In theory, one could rewrite the C++ spec in such a way that only one file was necessary, but then those files would need to be present within any project that incorporates that library. For very large libraries, this can be a pain -- why include the full source of some engine when you could include just the definitions necessary to link into the binaries? This provides the added advantage of obfuscating the algorithms and implementation details from the client program. C++ is not an interpreted programming language -- it's important to think about it from the compiler's perspective.
I have seen a couple of questions on how to detect unnecessary #include files in a C++ project. This question has often intrigued me, but I have never found a satisfactory answer.
If there are some header files included which, are not being used in a c++ project, is that an overhead? I understand that it means that before compilation the contents of all the header files would be copied into the included source files and that would result in a lot of unnecessary compilation.
How far does this kind of overhead spread to the compiled object files and binaries?
Aren't compilers able to do some optimizations to make sure that this
kind of overhead is not transferred to the resulting object files and
binaries ?
Considering the fact, that I probably know nothing about compiler optimization, I still want to ask this, in case there is an answer.
As a programmer who uses a wide variety of c++ libraries for his work,
what kind of programming practices should I follow to keep avoiding
such overheads ? Is making myself intimately familiar with each
library's working the only way out ?
It does not affect the performance of the binary or even the contents of the binary file, for almost all headers. Declarations generate no code at all, inline/static/anonymous-namespace definitions are optimized away if they aren't used, and no header should include externally visible definitions (that breaks if the header is included by more than one translation unit).
As #T.C. points out, the exception are internally visible static objects with nontrivial constructors. iostream does this, for example. The program must behave as if the constructor is called, and the compiler usually doesn't have enough information to optimize the constructor away.
It does, however, affect how long compilation takes and how many files will be recompiled when a header is changed. For large projects, this is enough incentive to care about unnecessary includes.
Besides the obviously longer compile times, there might be other issues. The most important one IMHO is dependencies to external libraries. You don't want your program to depend on more libraries then necessary.
You also then need to install those libraries in every system you want to the program to build on. This can become a nightmare, especially when the next programmer needs to install some database client library although the program never uses a database.
Also, especially library headers often tend to define macros. Sometimes those macros have very generic names which will break you code or which are incompatible with other library headers you might actually need.
Of course any #include is an overhead. The compiler needs to parse that file.
So avoid them. Use forward declarations where ever possible.
It will speed up compilation. See Scott Myers book on the subject
The simple answer is YES its an overhead as far as the compilation is concerned but for runtime it is merely going to create any difference. Reason being lets say you add #include <iostream> (just for example) and assume that you are not using any of its function then g++ 4.5.2 has some additional 18,560 lines of code to process(compilation). But as far as the runtime overhead is concerned I hardly think that it creates a performance issue.
You can also refer Are unused includes harmful in C/C++? where I really liked this point made by David Young
Any singletons declared as external in a header and defined in a
source file will be included in your program. This obviously increases
memory usage and possibly contributes to a performance overhead by
causing one to access their page file more often (not much of a
problem now, as singletons are usually small-to-medium in size and
because most people I know have 6+ GB of RAM).
I've working on a project at work where there's loads and loads of code in the header files. If I were using Visual Studio this wouldn't be an issue, as this has pre-compiled headers etc, but this is Linux GCC code.
Anyway, its starting to become a bit of an issue with compilation times. Of course the templates are going to have to remain in the headers etc, but most of this code could be extracted into implementation files and linked against as a static library. All of the projects uses these headers and get compiled each time, so it makes sense to create a static lib.
Implementations in the header files are in-lined, or is this only a hint, like the inline keyword? This code is VERY time critical and I'm concerned about moving the implementations out of the headers. Can I achieve the same thing if I use the inline keyword as opposed to having implementations in header files?
** UPDATE **
I know that inline is only a hint to the compiler. I'm not in control of everything in the project and I just want to move everything out of the headers into a library without affecting performance. Is this actually going to be a try it and see thing? I just want to keep performance exactly the same but enhance compile time.
The inline keyword is only a hint to the compiler that it may wish to inline that function. Its real purpose is to allow you to legally "violate" the one definition rule.
In order to inline a function its body has to be visible at the point of call, which typically means that if you move the function to an implentation file it may not be inlined anymore.
But keep in mind that most likely large functions in the header will not be inlined anyway. Also consider that in many cases inlined functions may actually be slower than called functions due to a variety of architecture-specific issues.
inline is a hint for optimization, but it is also used to work around ODR.
Consider using whole program optimization / link-time optimization instead. It allows you to have implementation in multiple files and basically everything has the same opportunity to be optimized (and inlined) as if it were in the same translation unit.
Your compile times become much quicker, but link times usually suffer, sometimes quite a bit. You don't need to enable it for debug builds though, so it can manifest a pretty immediate improvement to dev time.
If I were using Visual Studio this wouldn't be an issue, as this has pre-compiled headers etc
GCC has them too.
http://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html
The inline keyword does NOT mean that the function has to be implemented "in the line" where you define the function. As you know it is a hint for the compiler, to let it try compiling as if the few lines in the function were at the place where you call it. Thus avoiding the overhad of saving the adress to jump back, a vtable lookup etc.
Thinking it is faster called because it is in the header, is wishfull thinking (of the original guy who wrote the code).
Try it and move the implementation to the cpp file in a minimal example - main one object one inline function one call to it. Once implemented in the header and once in a cpp file. Then look at the assembly. No difference.
How do I find which parts of code are taking a long time to compile?
I am already using precompiled headers for all of my headers, and they definitely improve the compilation speed. Nevertheless, whenever I make a change to my C++ source file, compiling it takes a long time (this is CPU/memory-bound, not I/O-bound -- it's all cached). Furthermore, this is not related to the linking portion, just the compilation portion.
I've tried turning on /showIncludes, but of course, since I'm using precompiled headers, nothing is getting included after stdafx.h. So I know it's only the source code that takes a while to compile, but I don't know what part of it.
I've also tried doing a minimal build, but it doesn't help. Neither does /MP, because it's a single source file anyway.
I could try dissecting the source code and figuring out which part is a bottleneck by adding/removing it, but that's a pain and doesn't scale. Furthermore, it's hard to remove something and still let the code compile -- error messages, if any, come back almost immediately.
Is there a better way to figure out what's slowing down the compilation?
Or, if there isn't a way: are there any language constructs (e.g. templates?) that take a lot longer to compile?
What I have in my C++ source code:
Three (relatively large) ATL dialog classes (including the definitions/logic).
They could very well be the cause, but they are the core part of the program anyway, so obviously they need to be recompiled whenever I change them.
Random one-line (or similarly small) utility functions, e.g. a byte-array-to-hex converter
References to (inline) classes found inside my header files. (One of the header files is gigantic, but it uses templates only minimally, and of course it's precompiled. The other one is the TR1 regex -- it's huge, but it's barely used.)
Note:
I'm looking for techniques that I can apply more generally in figuring out the cause of these issues, not specific recommendations for my very particular situation. Hopefully that would be more useful to other people as well.
Two general ways to improve the compilation time :
instead of including headers in headers, use forward declare (include headers only in the source files)
minimize templated code (if you can avoid using templates)
Only these two rules will greatly improve your build time.
You can find more tricks in "Large-Scale C++ Software Design" by Lakos.
For visual studio (I am not sure if it is too old), take a look into this : How should I detect unnecessary #include files in a large C++ project?
Template code generally takes longer to compile.
You could investigate using "compiler firewalls", which reduce the frequency of a .cpp file having to build (they can reduce time to read included files as well because of the forward declarations).
You can also shift time spent doing code generation from the compiler to the linker by using Link-Time Code Generation and/or Whole Program Optimization, though generally you lose time in the long run.
I wish to send some components to my customers. The reasons I want to deliver source code are:
1) My class is templatized. Customer might use any template argument, so I can't pre-compile and send .o file.
2) The customer might use different compiler versions for gcc than mine. So I want him to do compilation at his end.
Now, I can't reveal my source code for obvious reasons. The max I can do is to reveal the .h file. Any ideas how I may achieve this. I am thinking about some hooks in gcc that supports decryption before compilation, etc. Is this possible?
In short, I want him to be able to compile this code without being able to peek inside.
Contract = good, obfuscation = ungood.
That said, you can always do a kind of PIMPL idiom to serve your customer with binaries and just templated wrappers in the header(s). The idea is then to use an "untyped" separately compiled implementation, where the templated wrapper just provides type safety for client code. That's how one often did things before compilers started to understand how to optimize templates, that is, to avoid machine-code level code bloat, but it only provides some measure of protection about trivial copy-and-paste theft, not any protection against someone willing to delve into the machine code.
But perhaps the effort is then greater than just reinventing your functionality?
Just adding some terminology to Alf's answer: The Thin template idiom is what you might look at. It basically simulates the functionality of a generic. Don't get confused by the wikipedia article which pops up in google, you don't have to use void*...
This, of course, does not guarantee binary compatibility. As usual with 'native' c++, you either compile the component for customers platform yourself and deploy the binary, or give them your code... The difference to the pure generic component code is that you can do the former at all.
use some c++ obfuscators may be help?: http://www.semdesigns.com/products/obfuscators/CppObfuscationExample.html or Magle It
First, if you're going to provide the source code, then you have to provide the source code. Sure, you could encrypt it, but even if GCC had a "decrypt before compile" option, it would need to decrypt the code, and if GCC can decrypt the code, so can your customer.
What you're asking is impossible. (If you find a way to do it, I believe the movie industry might have a multi-million contract for you. They currently have to resort to expensive custom hardware to prevent people from ripping content, and that only works to a limited degree)
As for your "obvious reasons" why you don't want to provide the source code, I don't see why they're obvious. What would happen if you provided the source code?
You have two options:
provide the source code in its entirety, or
compile everything that can be precompiled into a (static or dynamic) library, and provide your customer with that, plus the header files.
what about pimpls?
1) My class is templatized. Customer might use any template argument, so I can't pre-compile and send .o file.
2) The customer might use different compiler versions for gcc than mine. So I want him to do compilation at his end.
Now, I can't reveal my source code for obvious reasons. The max I can do is to reveal the .h file. Any ideas how I may achieve this. I am thinking about some hooks in gcc that supports decryption before compilation, etc. Is this possible?
In short, I want him to be able to compile this code without being able to peek inside.
Consideration 2) above encompasses A) ABI differences such that the same code compiled with different compiler versions/vendors on the same platform is incompatible, as well as B) the differences in system libraries, kernel versions etc. that the code might be dependent on. The only general solution is to compile on the specific platforms. Either you do it for all platforms, or you give them all the source code and they do it. That's not just the headers and template implementation, that's your out-of-line functions too. You might mitigate A) a little by building a wall of more interoperable extern "C" functions, but you're basically stuck when it comes to B).
So, can you decrypt during compilation? Only if you ship your own hacked GCC binaries to them, built for their specific system, which is probably more hassle than providing different builds of your own libraries (though it may address the template/header exposure issue).
Alternatively, you could employ source code obfuscation techniques. This is probably - practically - as good as it gets. I don't know what tools are out there, but it's an approach that people have pursued for decades (though I'm yet to hear anyone recommend it), so there's sure to be some mature tools.
Re templated code - other people have suggested a templated front end to a C-style generic implementation shipped as a precompiled object. That may or may not be practical (clearly risks performance degradation, and you have to capture the set of type-specific operations you want - e.g. by instantiating a type-specific class derived from an abstract operations base class) but anyway the precompiled object still runs afoul of B).
One other thought... clients might take your source code, but are unlikely to understand it as well as you. Even if they build more systems dependent on their version of it, in a way they're getting more locked in, and may have more need for your services in future. And, if you see they've not played fair, you charge them for it appropriately when the time comes.
It seems with gcc 4.5 comes the support for plugins. So you can provide your own .so which would be, for instance, called before compilation stage starts. So you can have all kinds of tricks(decryption of source file) in there, neatly hidden. This would also be portable solution as no change is made to g++ per se.
This is exactly what I was looking for. You can read more here:
http://www.codesynthesis.com/~boris/blog/2010/05/03/parsing-cxx-with-gcc-plugin-part-1/