I am building a Shared Object Project in ' IBM Rational Rhapsody 7.5 ' with qcc which in turn uses gcc .
A binary mismatch is observerd when the same code is built on two different machines having the exactly same build environment.
After some trial and error methods I observed that the mismatch is due to the pre-processsor directive used for conditional execution of code (Using #ifdef).When the pre-processor directive is removed and the project is built on different machines.The two binaries match exactly.
Is there is any reason for 'Why this mismatch occurs ?.
Is Pre-Processor directive adding machine dependent parameters in object file and so they are reflected in Executable, giving the mismatch ?
Please guide me.
The preprocessor directive #ifdef does conditional compilation. Whether or not this leads to binary incompatibility depends on what's between #ifdef and the corresponding #endif. For example, the following would obviously not have any effect at all:
#ifdef FOO
#endif
No matter whether FOO is defined, the compiled code looks exactly the same.
However the following will certainly cause an incompatibility:
#ifdef HAS_FOO_STRUCT
typedef foo foo_type;
#else
struct foo_type {};
#endif
void bar(foo_type);
Here, if HAS_FOO_STRUCT is defined, bar(foo_type) is actually bar(foo) and will be mangled as such. On the other hand, if HAS_FOO_STRUCT is not defined, foo_type is really a type of this name, thus bar(foo_type) will have that name mangled in. This there will be binary incompatibility.
Note that differences in defined macros can come from several different places. First, different compiler versions may define different macros (or define macros to different values). For example, there are macros identifying the type of system you are compiling on, and identifying the compiler version. Differences in macros can also come from system header files, which may have different #defines. Moreover, the build system may define macros indicating whether certain things (e.g. pre-installed libraries) were found or not found on the system.
Note that the binary incompatibility you see may well be intentional, in order to prevent running into a more subtle binary incompatibility which doesn't show up in compiling, but causes strange bugs under certain conditions.
Related
I need to use pycparser on preprocessed C code (the results produced by 'gcc -E'). However I am currently running into issue that I can't understand or solve.
I am using the provided samples year2.c and func_defs.py, which i modified to use a variety of preprocessors and fake libraries to no avail. Maybe some of you can look into this and see if you can reproduce/solve the issue. I will append all necessary code.
The errors were generated using year2.c (regular sample file) and year2.i ('gcc -E' output). There was no useable result for the latter while the former worked with both preprocessor/fakelib variants.
I have created a bitbucket repo with all relevant errors, the script used (albeit only its last variation) and the year2.c and year2.i files.
Error & Sample Repo
Thanks for your time.
The error you're getting is:
pycparser.plyparser.ParseError: /usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdarg.h:40:27: before: __gnuc_va_list
The line indicated as causing the error (stdarg.h:40):
typedef __builtin_va_list __gnuc_va_list;
In gcc, __builtin_va_list is, as its name indicates, built in to the compiler. Consequently, no declaration of that type is necessary (or allowed).
It's pretty common for C compilers to use a symbol-table-based technique to parse typenames, since there are a number of ambiguities in the grammar if you cannot distinguish a typename from another identifier. Such a parser will assume that an undeclared identifier is not a typename, and if __builtin_va_list is not a typename, that typedef is a syntax error.
So I suppose that the pyparser grammar you're using doesn't know about gcc builtin types (and why should it?).
Your fakelib seems to be including the same header file. That's not surprising since it is hard to fake stdarg.h; although technically a library header, it is part of the small set of headers which must be provided by the compiler even in a freestanding (no standard library) implementation: <float.h>, <iso646.h>, <limits.h>, <stdalign.h>,
<stdarg.h>, <stdbool.h>, <stddef.h>, <stdint.h>, and
<stdnoreturn.h> (C11 standard, clause 4, paragraph 6). These must be implemented by the compiler because there is no way an external library can know enough about the nature of the compiled code to properly define them.
Depending on what you require from the pyparsed output, you may be able to workaround this for pyparser by including a definition of __builtin_va_list, such as:
typedef struct __builtin_va_list { } __builtin_va_list;
__builtin_va_list is not the only builtin gcc datatype, although you may not run into the other ones. So you might have to iterate this solution a few times until you achieve whatever it is you are trying to achieve.
As #rici has explained the cause of the error. I'd focus more on how to solve it. I've taken my answer from pycparser author's blog -
http://eli.thegreenplace.net/2015/on-parsing-c-type-declarations-and-fake-headers
The idea is that pycparser needs to know what anyheader.h contains so it can properly parse the code. As actually parsing anyheader.h and all the other headers it transitively includes, could be very time consuming and perhaps not required for your task, fakeheaders can be used. A fake anyheader.h will only contain the parts of the original that are necessary for parsing - the #defines and the typedefs.
gcc -nostdinc -E -I/home/rg/pycparser-master/utils/fake_libc_include test.c > testPP.c
The above command preprocess test.c which contains <stdio.h> using fake headers provided with pycparser package. -nostdinc flag is used to block some pre-set system header directories that gcc automatically includes. Now, parsing the preprocessed file, using e.g. below code
import pycparser
pycparser.parse_file('testPP.c')
should work in the most of the cases. If it doesn't make sure you provide all the dependencies for preprocessing.
In case, for some headers fakes are not provided, you can fake error causing typedef using #defining e.g. to resolve an error caused by __builtin_va_list, you can try faking it as follows:
gcc -nostdinc -E -D'__builtin_va_list(x)=' -I/home/rg/pycparser-master/utils/fake_libc_include test.c > testPP.c
I've a C++ application that works in actual compilers (I compile it with eclipse).
Now, I need compile it on a very old compiler version (gcc/c++ v2.96) on a Redhat 7.3 with Kdevelop.
When I compile the app it gives the following error: swprintf undeclared.
wchar.h header it's included, but I saw this file in the RH7.3 OS and only declare this function if __USE_UNIX98 __USE_ISOC99 are declared.
How can I enable __USE_UNIX98?
GNU libc defines the features that should be enabled in all of its headers using a special system header <features.h>. If you define the appropriate macros, <features.h> will define __USE_UNIX98 for you.
The typical way to get all functions, regardless of what standard (if any) covers them, is by adding -D_GNU_SOURCE on the command-line. Getting only the functions covered by a specific standard requires defining the macro as specified in that standard using the value specified in that standard, such as -D_POSIX_C_SOURCE=200112L. The precise values that are supported on your particular implementation are probably easiest found by inspecting /usr/include/features.h manually.
From inspection of <features.h> defining _XOPEN_SOURCE to 500 or greater will cause __USE_UNIX98 to be defined
I have created several C++ libraries that currently are header-only. Both the interface and the implementation of my classes are written in the same .hpp file.
I've recently started thinking that this kind of design is not very good:
If the user wants to compile the library and link it dynamically, he/she can't.
Changing a single line of code requires full recompilation of existing projects that depend on the library.
I really enjoy the aspects of header-only libraries though: all functions get potentially inlined and they're very very easy to include in your projects - no need to compile/link anything, just a simple #include directive.
Is it possible to get the best of both worlds? I mean - allowing the user to choose how he/she wants to use the library. It would also speed up development, as I'd work on the library in "dynamically-linking mode" to avoid absurd compilation times, and release my finished products in "header-only mode" to maximize performance.
The first logical step is dividing interface and implementation in .hpp and .inl files.
I'm not sure how to go forward, though. I've seen many libraries prepend LIBRARY_API macros to their function/class declarations - maybe something similar would be needed to allow the user to choose?
All of my library functions are prefixed with the inline keyword, to avoid "multiple definition of..." errors. I assume the keyword would be replaced by a LIBRARY_INLINE macro in the .inl files? The macro would resolve to inline for "header-only mode", and to nothing for the "dynamically-linking mode".
Preliminary note: I am assuming a Windows environment, but this should be easily transferable to other environments.
Your library has to be prepared for four situations:
Used as header-only library
Used as static library
Used as dynamic library (functions are imported)
Built as dynamic library (functions are exported)
So let's make up four preprocessor defines for those cases: INLINE_LIBRARY, STATIC_LIBRARY, IMPORT_LIBRARY, and EXPORT_LIBRARY (it is just an example; you may want to use some sophisticated naming scheme).
The user has to define one of them, depending on what he/she wants.
Then you can write your headers like this:
// foo.hpp
#if defined(INLINE_LIBRARY)
#define LIBRARY_API inline
#elif defined(STATIC_LIBRARY)
#define LIBRARY_API
#elif defined(EXPORT_LIBRARY)
#define LIBRARY_API __declspec(dllexport)
#elif defined(IMPORT_LIBRARY)
#define LIBRARY_API __declspec(dllimport)
#endif
LIBRARY_API void foo();
#ifdef INLINE_LIBRARY
#include "foo.cpp"
#endif
Your implementation file looks just like usual:
// foo.cpp
#include "foo.hpp"
#include <iostream>
void foo()
{
std::cout << "foo";
}
If INLINE_LIBRARY is defined, the functions are declared inline and the implementation gets included like a .inl file.
If STATIC_LIBRARY is defined, the functions are declared without any specifier, and the user has to include the .cpp file into his/her build process.
If IMPORT_LIBRARY is defined, the functions are imported, and there isn't a need for any implementation.
If EXPORT_LIBRARY is defined, the functions are exported and the user has to compile those .cpp files.
Switching between static / import / export is a really common thing, but I'm not sure if adding header-only to the equation is a good thing. Normally, there are good reasons for defining something inline or not to do so.
Personally, I like to put everything into .cpp files unless it really has to be inlined (like templates) or it makes sense performance-wise (very small functions, usually one-liners). This reduces both compile time and - way more important - dependencies.
But if I choose to define something inline, I always put it in separate .inl files, just to keep the header files clean and easy to understand.
It is operating system and compiler specific. On Linux with a very recent GCC compiler (version 4.9) you might produce a static library using interprocedural linktime optimization.
This means that you build your library with g++ -O2 -flto both at compile and at library link time, and that you use your library with g++ -O2 -flto both at compile and link time of the invoking program.
This is to complement #Horstling's answer.
You can either create a static or a dynamic library. When you create statically-linked libraries, compiled code for all functions/objects will be saved to a file (with .lib extension in Windows). At main project (the project using the library) 's link time, these codes will be linked into your final executable together with the main project codes. So the final executable wouldn't have any runtime dependency.
Dynamically linked libraries will be merged into the main project at run time (and not link time). When you compile the library you get a .dll file (which contains actual compiled code) and a .lib file (which contains enough data for the compiler/runtime to find functions/objects in the .dll file). At link time, the executable will be configured to load the .dll and use compiled code from that .dll as needed. You will need to distribute the .dll file with your executable to be able to run it.
There is no need to choose between static or dynamic linking (or header-only) when designing your library, you create multiple project/makefiles, one to create a static .lib, another to create a .lib/.dll pair, and distribute both versions, for the user to choose between. (You'll need to use preprocessor macros like the ones #Horstling suggested).
You cannot put any templates in a pre-compiled library, unless you use a technique called Explicit Instantiation, which limits template parameters.
Also note that modern compiler/linkers usually do not respect the inline modifier. They may inline a function even if it's not designated as inline, or may dynamically call another that has inline modifier, as they see fit. (Regardless, I'll advise explicitly putting inline where applicable for maximum compatibility). So, there won't be any runtime performance penalty if you use a statically linked library instead of a header-only library (and enable compiler/linker optimizations, of course). As others have suggested, for really small functions that are sure to benefit from being called inline, it is best practice to put them in header files, so dynamically linked libraries will also not suffer any significant performance loss. (In any case, inlining functions will only affect performance for functions that are being called very often, inside loops that are going to be called thousands/millions of times).
Instead of putting inline functions in header files (with an #include "foo.cpp" in your header), you can change makefile/project settings and add foo.cpp to the list of source files to be compiled. This way, if you change any function implementation there will be no need to re-compile the whole project and only foo.cpp will be re-compiled. As I mentioned earlier, your small functions will still be inlined by the optimizing compiler, and you don't need to worry about that.
If you use/design a pre-compiled library, you should consider the case where the library is compiled with a different version of compiler to the main project. Each different compiler version (even different configurations, like Debug or Release) uses a different C runtime (things like memcpy, printf, fopen, ...) and C++ standard library runtime (things like std::vector<>, std::string, ...). These different library implementations may complicate linking, or even create runtime errors.
As a general rule, always avoid sharing compiler runtime objects (data structures that are not defined by standards, like FILE*) across libraries, because incompatible data structures will lead to runtime errors.
When linking your project, C/C++ runtime functions must be linked into your library .lib or .lib/.dll, or your executable .exe. C/C++ runtime itself can be linked as static or dynamic library (you can set this in makefile/project settings).
You will find that dynamically linking to C/C++ runtime in both the library and the main project (even when you compile the library itself as a static library) avoids most linking problems (with duplicate function implementations in multiple runtime versions). Of course you would need to distribute runtime DLLs for all used versions with your executable and library.
There are scenarios that statically linking to C/C++ runtime is needed, and the best approach in these cases would be to compile the library with the same compiler setting as the main project to avoid linking problems.
Rationale
Put as little as necessary in header files and as much as possible in library modules, because of the very reasons that you mentioned: compile-time dependency and long compilation time. The only good reasons for header-only modules are:
generic templates for user-defined template parameter;
very short convenience functions when inlining gives significant
performance.
In case 1, it is often possible to hide some functionality that does not depend on the user-defined type in a .cpp file.
Conclusion
If you stick to this rationale, then there is no choice: templated functionality that must allow user-defined types cannot be pre-compiled, but requires a header-only implementation. Other functionality should be hidden from the user in a library to avoid exposing them to the implementation details.
Rather than a dynamic library, you could have a precompiled static library and thin header file. In an interactive quick build, you get the benefit of not having to recompile the world if implementation details changes. But a fully optimized release build can do global optimization and still figure out it can inline functions. Basically, with "link-time code generation" the toolset does the trick you were thinking about.
I'm familiar with Microsoft's compiler, which I know for sure does this as of Visual Studio 2010 (if not earlier).
Templated code will necessarily be header-only: for instantiating this code, the type parameters must be known at compilation time. There is no way to embed template code in shared libraries. Only .NET and Java support JIT instantiation from byte-code.
Re: non-template code, for short one-liners I suggest keeping it header-only. Inline functions give the compiler a lot more opportunities to optimize the final code.
To avoid "insane compilation time", Microsoft Visual C++ has a "precompiled headers" feature. I do not think GCC has a similar feature.
Long functions should not be inlined in any case.
I had one project which had header-only bits, compiled library bits and some bits I could not decide where belonged. I ended up having .inc files, conditionally included in either .hpp or .cxx depending on #ifdef. Truth to be told, the project was always compiled in "max inline" mode, so after a while I got rid of the .inc files and simply moved the contents to .hpp files.
Is it possible to get the best of both worlds?
In terms; limitations arise because tools aren't smart enough. This answer gives the current best effort that is still portable enough to be used effectively.
I've recently started thinking that this kind of design is not very good.
It ought to be. Header-only libraries are ideal because they simplify deployment: makes the reusing mechanism of the language similar to almost all others', which is just the sane thing to do. But this is C++. Current C++ tools still rely on half-a-century-old linking models that remove important degrees of flexibility, such as choosing which entry points to import or export on an individual level without being forced to change the library's original source code. Also, C++ lacks a proper module system and still relies on glorified copy-paste operations to work (although this is just a side factor to the problem in question).
In fact, MSVC is a little better in this regard. It is the only major implementation trying to achieve some degree of modularity in C++ (by attempting e.g. C++ modules). And it is the only compiler that actually allows e.g. the following:
//// Module.c++
#pragma once
inline void Func() { /* ... */ }
//// Program1.c++
#include <Module.c++>
// Inlines or "vague" links Func(), whatever is better.
int main() { Func(); }
//// Program2.c++
// This forces Func() to be imported.
// The declaration must come *BEFORE* the definition.
__declspec(dllimport) __declspec(noinline) void Func();
#include <Module.c++>
int main() { Func(); }
//// Program3.c++
// This forces Func() to be exported.
__declspec(dllexport) __declspec(noinline) void Func();
#include <Module.c++>
Note that this can be used to selectively import and export individual symbols from the library, although still cumbersomely.
GCC also accepts this (but the order of the declarations must be changed) and Clang does not have any way to achieve the same effect without changing the library's source.
We appear to have developed a strange situation in our application. An ASSERT is being triggered which should only run if _DEBUG is defined, but it is being evaluated when the application is compiled in Release mode.
ASSERT is defined in a header file, and is being triggered from another header file, which is included into a source file.
On further inspection, the source file is indeed running in Release mode (_DEBUG is not defined, and NDEBUG is). However, the header files have _DEBUG defined, and not NDEBUG.
According to conventional wisdom, #including a header file is equal to cutting-and-pasting the lines of code into the source file. This would make the above behaviour impossible.
We are compiling a large, mixed language (Intel FORTRAN and C++) application in VS2010. This problem also occurs on our build server, though, so it doesn't seem to be just a VS2010 'feature'.
We have checked:
All projects are building in Release.
The affected cpp files do not have any unusual properties being set.
There are no files in our solution manually defining or undefining _DEBUG or NDEBUG.
We have established the above behaviour by including clauses such as:
bool is_debug = false;
#ifdef _DEBUG
is_debug = true
#endif
and breaking on the point immediately afterwards.
We are running out of things to test - about the only things that I can even hypothesise are:
Some standard library or external include is redefining _DEGUG and NDEBUG, or
Something has overridden the #include macro (is this possible?).
EDIT ----------------------------------------------------------
Thanks in part to the #error trick (below), we've found the immediate problem: In several of the projects the NDEBUG and _DEBUG are no longer defined. All of these project were meant to have inherited something from the macro $(PreprocessorDefinitions) - but this is not defined anywhere.
This still leaves some awkward questions:
The source file that was causing the above behaviour does have NDEBUG defined in its project settings, and yet the header files they include don't (although VS2010 does grey-out the correct #ifdef blocks).
If the PreprocessorDefinitions macro is inherited by all C++ projects (which it appears to be), then why isn't it defined anywhere?
My usual approach to problems like this is, to look where the symbol is defined or an #ifdef is used and then put `#error Some text´ in it. This way already the compilation process will break, instead of having to wait and run it. Then you can see what really is defined.
You could also add such an #ifdef - #error combination right where the assert occurs, then you can be absolutely sure what the compiler thinks should be valid.
From http://msdn.microsoft.com/en-us/library/9sb57dw4(v=vs.71).aspx:
The assert routine is available in both the release and debug versions of the C run-time libraries. Two other assertion macros, _ASSERT and _ASSERTE, are also available, but they only evaluate the expressions passed to them when the _DEBUG flag has been defined.
In other words: either use _ASSERT(...) or #define NDEBUG, so you don't get asserts in Release builds.
OK, the problem turns out to be because NDEBUG and _DEBUG are missing from the Properties->C/C++->Preprocessor->Preprocessor Definitions on several projects. Whether they were always missing, or whether they had originally been included via the $(PreprocessorDefinitions) macro is unclear.
Thanks to #Lamza, #Devolus and #Werner Henze - all of their input was useful, and the eventual problem was depressingly mundane.
I'm compiling code using gcc that comes from Visual C++ 2008. The code is using errno_t, but in some versions of gcc headers including <errno.h> doesn't define the type. How do I detect if the type is defined? Is there a define that signals that the type was defined? In the case it isn't defined I'd like to provide the typedef to let the code compile correctly on all platforms.
Microsoft's errno_t is redundant. errno is defined by the ISO C standard to be a modifiable lvalue of type int. If your code needs to store errno values, then you should put them into an int.
Do a global search and replace s/errno_t/int/ and you're done.
Edit: Also, you shouldn't be providing a typedef int errno_t in your code, because all names that end with _t are reserved.
You can't check for a typedef the way you can for a macro, so this is a bit on the tricky side. If you're using autoconf, this patch shows the minimum changes that you need to have autoconf check for the presence of errno_t and define it if it's missing (the typedef would be placed in a file that includes your generated config.h and is included by all files that need errno_t). If you're not using autoconf you need to come up with some way to do the same thing within your build system, or a very clever set of tests against compiler version macros.
This is typically the case where GNU autoconf comes to the rescue. Basically autoconf will generate a configure script that can detect various system-dependent features such as whether this type exists and how it is defined. You then include the generated C header file within your application.
If you know which versions of GCC are giving you trouble, you can test for them. You can check for versions of GCC using something like:
#if __GNUC__ == 3
...
#else
...
#endif