What is difference between sjlj vs dwarf vs seh? - c++

I can't find enough information to decide which compiler should I use to compile my project. There are several programs on different computers simulating a process. On Linux, I'm using GCC. Everything is great. I can optimize code, it compiles fast and uses not-so-much memory.
I do my own benchmark with MSVC and GCC compilers. Later one produces slightly faster binaries (for each subarchitecture). Though compile time is much more than MSVC.
So I decided to use MinGW. But can't find any explanation about exception handling methods and their implementations in MinGW. I can use different distributions for different operating systems and architectures.
Considerations:
Compile time and memory are not important for my usage. Only important thing is runtime optimization. I need my programs to be fast enough. A slow compiler is acceptable.
OS: Microsoft Windows XP / 7 / 8 / Linux
Architecture: Intel Core i7 / Core2 / and a very old i686 running XP :P

There's a short overview at MinGW-w64 Wiki:
Why doesn't mingw-w64 gcc support Dwarf-2 Exception Handling?
The Dwarf-2 EH implementation for Windows is not designed at all to
work under 64-bit Windows applications. In win32 mode, the exception
unwind handler cannot propagate through non-dw2 aware code, this means
that any exception going through any non-dw2 aware "foreign frames"
code will fail, including Windows system DLLs and DLLs built with
Visual Studio. Dwarf-2 unwinding code in gcc inspects the x86
unwinding assembly and is unable to proceed without other dwarf-2
unwind information.
The SetJump LongJump method of exception handling works for most
cases on both win32 and win64, except for general protection faults.
Structured exception handling support in gcc is being developed to
overcome the weaknesses of dw2 and sjlj. On win64, the
unwind-information are placed in xdata-section and there is the .pdata
(function descriptor table) instead of the stack. For win32, the chain
of handlers are on stack and need to be saved/restored by real
executed code.
GCC GNU about Exception Handling:
GCC supports two methods for exception handling (EH):
DWARF-2 (DW2) EH, which requires the use of DWARF-2 (or DWARF-3) debugging information. DW-2 EH can cause executables to be
slightly bloated because large call stack unwinding tables have to be
included in th executables.
A method based on setjmp/longjmp (SJLJ). SJLJ-based EH is much slower than DW2 EH (penalising even normal execution when no
exceptions are thrown), but can work across code that has not been
compiled with GCC or that does not have call-stack unwinding
information.
[...]
Structured Exception Handling (SEH)
Windows uses its own exception handling mechanism known as Structured Exception Handling (SEH). [...]
Unfortunately, GCC does not support SEH yet. [...]
See also:
Exception handling models of GCC
C++ Exception Handling for IA-64
EH newbies howto

SJLJ (setjmp/longjmp): – available for 32 bit and 64 bit – not “zero-cost”: even if an exception isn’t thrown, it incurs a minor
performance penalty (~15% in exception heavy code) – allows exceptions
to traverse through e.g. windows callbacks
DWARF (DW2, dwarf-2) – available for 32 bit only – no permanent runtime overhead – needs whole call stack to be dwarf-enabled, which
means exceptions cannot be thrown over e.g. Windows system DLLs.
SEH (zero overhead exception) – will be available for 64-bit GCC 4.8.
source: https://wiki.qt.io/MinGW-64-bit

Related

Is it safe to assume that any x86 compiled app would always run under x64 edition?

Is it safe to assume that any x86 compiled app would always run under x64 edition of same OS the app was compiled in?
As far as I know, For Windows OS the answer is "Yes". Windows x86 emulation layer is built for the same purpose. But, I just want to reconfirm this from experts here.
What about Unix, Linux? Are there any caveats?
No, for x86 code to run it need to be run in compatibility or legacy mode. If the OS doesn't support running processes in compatibility mode the program would most likely not being able to run.
Linux and IFAIK Windows currently supports compatibility mode and it looks like many more are supporting it too, more or less. My understanding is that NETBSD requires a special module to support this, so it's not necessarily without special care that it will be supported and it shows that it's quite possible that there exists OS'es where the possibility has been dropped completely.
Then in addition there's the possibility of breaking the backwards compatibility in the future, that's already happened on the CPU as virtual x86 mode is no longer available from long mode that is you can't run 16 bits program anymore under 64-bit Windows or Linux.
It could also happen on the OS side, that the developers could decide not to support compatibility mode anymore. Note that this may also have happened as it might be possible to support virtual x86 mode by first switching to legacy mode, but if possible no-one seem to have bothered doing it. Similarly neither Windows or Linux developers seems to have bothered implementing possibility to run kernel code in legacy mode in 64-bit kernel.
The preceding two sections shows that there are future or even present indications on this might not always be possible.
Besides as this is a C++ question you would have to ask yourself the question why you would want to make such a assumption? If well written your code should be able to be compiled for 64-bit mode - for you haven't relied on data types being of specific width, have you?
No. We have a whole bunch of Debian servers which miss the Multi-Arch i386 (32 bits) libraries. On Windows Server Core, WoW64 (the 32 bit subsystem) is optional. So for both Linux and Windows, there are known 64 bit systems that won't run x86 executables.

GCC runtime libraries vs Microsoft Visual C++ runtime redistributables

Could anyone shed some light on C++ library versioning and distribution of
GCC library (libgcc, libstdc++,..?)
Microsoft Visual C++ runtime libraries (6.0, 2005, 2008, 2010, 2012, 2013, 2015,....)
With my limited exposure to GCC programming, I have never seen C++ runtime libraries being distributed along with program. That is often the case with MS Windows programs.
Can a relatively old linux system run a newer C++14 program (which is compiled on a newer system and then copied over to old system)?
Do GCC programmers distribute runtime libraries along with programs? If no, Why do Windows programs distribute them? How GCC distributions ensure that a C++ program always works when installed?
What about the frameworks such as Qt, How does Qt handle versioning and distribution on Linux and Windows? Does Qt also distribute runtimes for different versions?
May be it has to do with platform, how Linux is designed vs How Windows is designed.
What is so fundamentally different in approaches GCC & MS Windows take?
GCC's runtime libraries, like GNU's C library, provide a stable binary interface (small footnote: GCC 5.1 kind of blew this up, due to new C++ features that had to be implemented).
Microsoft's libraries don't, and each little version difference may and probably will break the ABI (application binary interface).
Additionally, Microsoft compilers change their ABI with version increments as well, making it a bad idea to combine code built by different versions of their tools. Also here, GCC maintains a strict ABI, which makes object code perfectly compatible (if no ABI breaking codegen options are given of course).
This ABI consists of object size and layout, which a compiler uses when generating code. Therefore running code built against one version but using a different version at runtime may produce unexpected results because the memory layout and use is just different.
GNU/Linux is quite strong in this regard, and is generally able to maintain strong backwards compatibility.
As long as the compiled program was compiled against an older version of the library, it will run perfectly if loaded with a newer version that a user has installed.
The same story goes for Qt, which only breaks ABI between major version numbers (Qt 4 and Qt 5 cannot be loaded at runtime interchangeable).
There are some small exceptions, GCC 5's libstdc++ being a big problem there. I know of no big glibc ABI breakages though.
The new Microsoft Universal CRT attempts to solve this issue, by providing a stable C runtime interface and as the story would have us believe, provide a glibc style library ABI stability. This UCRT is available for Windows Vista and up, but applications need to be compiled specifically against this. The first version of VS that has this capability, is VS2015.

What are the differences between C/C++ bare-metal compilation and compilation for a specific OS (Linux)?

Suppose you have a cross-compilation tool-chain that produces binaries for the ARM architecture.
Your tool-chain is like this (running on a X86_64 machine with Linux):
arm-linux-gnueabi-gcc.exe : for cross-compilation for Linux, running on ARM.
arm-gcc.exe : for bare-metal cross-compilation targeting ARM.
... and the plethora of other tools for cross-compilation on ARM.
Points that I'm interested in are:
(E)ABI differences between binaries (if any)
limitations in case of bare-metal (like dynamic memory allocations, usage of static constructors in case of C++, threading models, etc)
binary-level differences between the 2 cases in terms of information specific to each of them (like debug info support, etc);
ABI differences is up to how you invoke the compiler, for example GCC has -mabi and that can be one of ‘apcs-gnu’, ‘atpcs’, ‘aapcs’, ‘aapcs-linux’ and ‘iwmmxt’.
On bare-metal limitations for various runtime features exists because someone hasn't provided them. Be them initializing zero allocated areas or providing C++ features. If you can supply them, they will work.
Binary level differences is also up to how you invoke compiler.
You can check GCC ARM options online.
I recently started a little project to use a Linux standard C library in a bare-metal environment. I've been describing it on my blog: http://ellcc.org/blog/?page_id=289
Basically what I've done is set up a way to handle Linux system calls so that by implementing simplified versions of certain system calls I can use functions from the standard library. For example, the current state for the ARM implements simplified versions of read(), readv(), write(), writev() and brk(). This allows me to use printf(), fgets(), and malloc() unchanged.
I'm my case, I use the same compiler for targeting Linux and bare-metal. Since it is clang/LLVM based, I can also use the same compiler to target other processors. I'm working on a bare-metal example for the Mips right now.
So I guess the answer is that there doesn't have to be any difference.

cc1plus: Virtual memory exhausted

I am trying to build a project in QNX Momentics IDE (4.6) using qcc in Linux. I fail to succeed the build process with the following error:
virtual memory exhausted: Cannot allocate memory
/opt/qnx641/host/linux/x86/usr/lib/gcc/i386-pc-nto-qnx6.4.0/4.3.3/cc1plus error 1
The project has a cpp file with more than 1.3 MLOC. This one is an autogenerated code from a large Matlab/SIMULINK simulation model so it is not easy to divide and conquer.
It is hard to understand if it is LOC limit of qcc compiler or due to a programming practice in the autogenerated code.
I would like to ask:
Is there any source file size limit for qcc?
What are the bad programming practices that cause this?
Any suggestions to fix virtual memory exhausted problem of cc1plus?
Q1: Is there any source file size limit for qcc?
A1: qcc = gcc. More accurately: qcc is a lightweight wrapper that calls gcc; all the real work is done by gcc. GNU software is, as a general philosophy, designed to not impose arbitrarily limit and I presume this is especially true for gcc. Even if there exist arbitrarily limits you are not hitting those because you are running out of system memory.
Random links:
preprocessor limits: http://gcc.gnu.org/onlinedocs/cpp/Implementation-limits.html
some gcc limits benchmarking: gcc module size limits
Q2: What are the bad programming practices that cause this?
A2: E.g., dumping all source code into a single file, as you demonstrated. I'd say this question is not relevant to your case because you already stated you don't have control over the generated code.
Q3: Any suggestions to fix virtual memory exhausted problem of cc1plus?
A3: a) put more ram into your host computer (may or may not help depending on how much you have and if your OS is 32 or 64 bit); b)increase your swap-space (same applies); c) if a/b does not help then upgrade your OS to 64 bit and try a/b again. Unfortunately, this 64-bit suggestion almost surely does not apply to the gcc version that QNX shipped with 6.4.1. Maybe not even to the latest one.
As a general suggestion, since qcc is using gcc I'd recommend that you have the same code build using the host's gcc (gcc that is shipped with your Linux). Once that works you may start looking for the differences, which likely boil down to 64-bit support.

Same program faster on Linux than Windows -- why?

The solution to this was found in the question Executable runs faster on Wine than Windows -- why? Glibc's floor() is probably implemented in terms of system libraries.
I have a very small C++ program (~100 lines) for a physics simulation. I have compiled it with gcc 4.6.1 on both Ubuntu Oneiric and Windows XP on the same computer. I used precisely the same command line options (same makefile).
Strangely, on Ubuntu, the program finishes much faster than on Windows (~7.5 s vs 13.5 s). At this point I thought it's a compiler difference (despite using the same version).
But even more strangely, if I run the Windows executable under wine, it's still faster than on Windows (I get 11 s "real" and 7.7 s "user" time -- and this includes wine startup.)
I'm confused. Surely if the same code is run on the same CPU, there shouldn't be a difference in the timing.
What can be the reason for this? What could I be doing wrong?
The program does minimal I/O (outputs a single line), and only uses a fixed-length vector from the STL (i.e. no system libraries should be involved). On Ubuntu I used the default gcc and on Windows the Nuwen distribution. I verified that the CPU usage is close to zero when doing the benchmarking (I closed most programs). On Linux I used time for timing. On Windows I used timethis.exe.
UPDATE
I did some more precise timings, comparing the running time for different inputs (run-time must be proportional to the input) of the gcc and msvc-compiled programs on Windows XP, Wine and Linux. All numbers are in seconds and are the minimum of at least 3 runs.
On Windows I used timethis.exe (wall time), on Linux and Wine I used time (CPU time). (timethis.exe is broken on Wine) I made sure no other programs were using the CPU and disabled the virus scanner.
The command line options to gcc were -march=pentium-m -Wall -O3 -fno-exceptions -fno-rtti (i.e. exceptions were disabled).
What we see from this data:
the difference is not due to process startup time, as run-times are proportional to the input
The difference between running on Wine and Windows exists only for the gcc-compiled program, not the msvc-compiled one: it can't be casued by other programs hogging the CPU on Windows or timethis.exe being broken.
You'd be surprised what system libraries are involved. Just do ldd on your app, and see which are used (ok, not that much, but certainly glibc).
In order to completely trust your findings about execution speed, you would need to run your app a couple of times sequentially and take the mean execution time. It might be that the OS loader is just slower (although 4s is a long loading time).
Other very possible reasons are:
Different malloc implementation
Exception handling, if used to the extreme might cause slowdown (Windows GCC, MinGW, might not be the optimal exception handling star of the show)
OS-dependent initialization: stuff that needs to be done at program startup on Windows, but not on Linux.
Most of these are easily benchmarkable ;-)
An update to your update: the only thing you can now do is profile. Stop guessing, and let a profiler tell you where time is being spent. Use gprof and the Visual Studio built-in profiler and compare time spent in different functions.
Do benchmarking in code. Also try to compile with visual studio. On windows if you have some application like Yahoo Messenger, that are installing hooks, they can very easy slow down your application loading times.
On windows you have: QueryPerformanceCounter
On Linux: clock_gettime
Apparently the difference is system related.
You might use strace to understand what system calls are done, eg
strace -o /tmp/yourprog.tr yourprog
and then look into /tmp/yourprog.tr
(If an equivalent of strace existed on Windows, try to use it)
Perhaps your program is allocating memory (using mmap system call), and perhaps the memory related system calls are faster on Linux (or even on Wine) than on Windows? Or some other syscalls give faster functionality on Linux that on Windows.
NB. I know nothing about Windows, since I'm using Unix systems since 1986 and Linux since 1993.