In a release build on Visual Studio 2008 (though this probably applies to any Microsoft compiler), I can set /Zi ("Produces a program database (PDB) that contains type information and symbolic debugging information for use with the debugger. The symbolic debugging information includes the names and types of variables, as well as functions and line numbers.") and also /Ox ("Full Optimization").
These two flags are valid together, but seem to be in conflict. One of the things the compiler might choose to do to fully optimize the code, is to rearrange code and change which code is in which line, but that screws up the information that the program database is compiling.
When I have both flags, which one "wins"? Does my code get fully optimized, and some line numbers are not available? Or does the optimizer get set to "less-than-full", so that it leaves line numbers alone?
Related
When I run my code (any code), in debug mode I have the option of "Disassembly".
I know that "Disassembly" is the creation of assembly code from machine code ('1' and '0' that stored in object files).
So, I can assume that when I compile my code in Visual Studio, there is no "Assembly/Assembler" stage between the C/C++ code to the machine code/object file, and if I want to see the Assembly code I need to ask Visual Studio to disassemble it from object file to assembly.
So my questions are:
am I right, is there no way to see asm other than disassembly?
(if 1 is no): why does Visual Studio's debugger show a "Disassembly" option, not "Assembly"?
Does Clang/GCC have an assembly stage, or does the C/C++ code compile to object code directly?
editor's note: the answer to 3 actually differs for GCC vs. most other mainstream C/C++ compilers like clang/LLVM and MSVC. Since that's kind of a separate question from how MSVC works, that part is a duplicate of these other SO Q&As:
Does a compiler always produce an assembly code?
What do C and Assembler actually compile to?
What is the need to generate ASM code in gcc, g++
When a debugger provides a disassembly option, it is generally examining memory of the running process, augmented by debugging information embedded in or associated with the executable file, which can include references that help the debugger identify associated source code.
Separately, compilers usually have an option to show you the assembly language that is either the assembly language they use internally (perhaps in temporary files passed to specific subprocesses, such as an assembler) or its equivalent (what it would generate if it were not generating object code directly). For GCC and Clang, the -S switch does this. For Visual Studio, /FA does this.
Background
There are several different debug flags you can use with the Visual Studio C++ compiler. They are:
(none)
Create no debugging information
Faster compilation times
/Z7
Produce full-symbolic debugging information in the .obj files using CodeView format
/Zi
Produce full-symbolic debugging information in a .pdb file for the target using Program Database format.
Enables support for minimal rebuilds (/Gm) which can reduce the time needed for recompilation.
/ZI
Produce debugging information like /Zi except with support for Edit-and-Continue
Issues
The /Gm flag is incompatible with the /MP flag for Multiple Process builds (Visual Studio 2005/2008)
If you want to enable minimal rebuilds, then the /Zi flag is necessary over the /Z7 flag.
If you are going to use the /MP flag, there is seemingly no difference between /Z7 and /Zi looking at MSDN. However, the SCons documentation states that you must use /Z7 to support parallel builds.
Questions
What are the implications of using /Zi vs /Z7 in a Visual Studio C++ project?
Are there other pros or cons for either of these options that I have missed?
Specifically, what is the benefit of a single Program Database format (PDB) file for the target vs multiple CodeView format (.obj) files for each source?
References
MDSN /Z7, /Zi, /ZI (Debug Information Format)
MSDN /MP (Build with Multiple Processes)
SCons Construction Variables CCPDBFLAG
Debug Info
Codeview is a much older debugging format that was introduced with Microsoft's old standalone debugger back in the "Microsoft C Compiler" days of the mid-1980s. It takes up more space on disk and it takes longer for the debugger to parse, and it's a major pain to process during linking. We generated it from our compiler back when I was working on the CodeWarrior for Windows in 1998-2000.
The one advantage is that Codeview is a documented format, and other tools can often process it when they couldn't deal with PDB-format debug databases. Also, if you're building multiple files at a time, there's no contention to write into the debug database for the project. However, for most uses these days, using the PDB format is a big win, both in build time and especially in debugger startup time.
One advantage of the old C7 format is that it's all-in-one, stored in the EXE, instead of a separate PDB and EXE. This means you can never have a mismatch. The VS dev tools will make sure that a PDB matches its EXE before it will use it, but it's definitely simpler to have a single EXE with everything you need.
This adds new problems of needing to be able to strip debug info when you release, and the giant EXE file, not to mention the ancient format and lack of support for other modern features like minrebuild, but it can still be helpful when you're trying to keep things as simple as possible. One file is easier than two.
Not that I ever use C7 format, I'm just putting this out there as a possible advantage, since you're asking.
Incidentally, this is how GCC does things on a couple platforms I'm using. DWARF2 format buried in the output ELF's. Unix people think they're so hilarious. :)
BTW the PDB format can be parsed using the DIA SDK.
/Z7 keeps the debug info in the .obj files in CodeView format and lets the linker extract them into a .pdb while /Zi consolidates it into a common .pdb file during compilation already by sync'ing with mspdbsrv.exe.
So /Z7 means more file IO, disc space being used and more work for the linker (unless /DEBUG:FASTLINK is used) as there is lots of duplicate debug info in these obj files. But it also means every compilation is independent and thus can actually still be faster than /Zi with enough parallelization.
By now they've improved the /Zi situation though by reducing the inter-process communication with mspdbsrv.exe: https://learn.microsoft.com/en-us/cpp/build/reference/zf
Another use-case of /Z7 is for "standalone" (though larger) static libraries that don't require shipping a separate .pdb if you want that. That also prevents the annoying issues arising from the awful default vcxxx.pdb name cl uses as long as you don't fix it with a proper https://learn.microsoft.com/en-us/cpp/build/reference/fd-program-database-file-name, which most people forget.
/ZI is like /Zi but adds additional data etc. to make the Edit and Continue feature work.
There is one more disadvantage for /Z7:
It's not compatible with incremental linking, which may alone be a reason to avoid it.
Link: http://msdn.microsoft.com/en-us/library/4khtbfyf%28v=vs.100%29.aspx
By the way: even though Microsoft says a full link (instead of an incremental) is performed when "An object that was compiled with the /Yu /Z7 option is changed.", it seems this is only true for static libraries build with /Z7, not for object files.
Another disadvantage of /Z7 is the big size of the object files. This has already been mentioned here, however this may escalate up to the point where the linker is unable to link the executable because it breaks the size limit of the linker or the PE format (it gives you linker error LNK1248). It seems Visual Studio or the PE format has a hard limit of 2GB (also on x64 machines). When building a debug version you may run into this limit. It appears this does not only affect the size of the final compiled executable, but also temporary data. Only Microsoft knows about the linker internals, but we ran into this problem here (though the executable was of course not 2gigs large, even in debug). The problem miraculously went away and never came back when we switched the project to /ZI.
I'm aware that mixing debug and release libraries that pass STL containers to each other causes big problems. But what exactly in 'debug' or 'release' causes this?
I have a QT project that gets built as 'release', but it has /DEBUG added to the compiler flags.
If I build another QT project under 'debug' (which also has the /DEBUG flag), are they compatible?
Or is there an optimization flag or some other flag that makes them incompatible?
Basically, is there a way I can look at the compilation line of the 2 libraries and see something that says "DON'T MIX THESE!"?
This is undefined behaviour, so it might work but more probably it will crash your app. It also depends on stl implementation, in debug version it might enable additional checks and allocate additional memory making data layout different in release modes. In the end this violates ODR rule (one definition rule) once again causing undefined behaviour.
I have a QT project that gets built as 'release', but it has /DEBUG added to the compiler flags. If I build another QT project under 'debug' (which also has the /DEBUG flag), are they compatible?
If your Release contains exactly the same compiler flags as Debug then I would say they are compatible.
Or is there an optimization flag or some other flag that makes them incompatible?
you should be able to see those flags, I dont think optimization flags should cause UB problems, its rather different macros used during compilation phase that cause ODR violation. Maybe there are optimization flags which changes alignment in structures...
Basically, is there a way I can look at the compilation line of the 2 libraries and see something that says "DON'T MIX THESE!"?
no idea here.
Why are you mixing differently build libraries in the first place? this is asking for trouble.
Not really. Concerning Visual C++'s STL implementation, there are data members of the containers corresponding to the iterator checking that is only compiled in if some preprocessor variables are set. These variables are default valued base on the NDEBUG preprocessor variable that is quasi standard for "no debug". But these could be also set directly from the command line, or from header files, or Visual Studio property pages, etc.
E.g.: all containers are derived from _Container_base, while that is a typedef depending on _ITERATOR_DEBUG_LEVEL. The different _Container_base implementations have different memory layout, and that is what causes incompatibilities between the debug and release version.
The /DEBUG compiler flag tells the compiler whether to generate debug information or not, and might also affect default values for optimization settings, but I am not sure about that, and of course that is compiler dependent.
Just like there could be an STL implementation that does not depend on any preprocessor directives, and in that case it would not matter how you compile it, debug or release, the memory layout would be identical, and so it could be passed around between modules compiled the different ways.
Firstly the /DEBUG flag doesn't actually create a 'debug' build. It just tells the linker to generate debugging information and create a .pdb file along with the resulting binary.
As to the difference between the debug and release MSVC++ runtimes the issue is to do with the fact that different runtimes can have different sizes for the same object. e.g. in debug extra information might be placed in iterators to ensure their validity at run time. If code has been compiled against the release version of these structures then corruption is likely. Another issue would be if an object if allocated from the heap of one runtime and is attempted to be freed on the heap of another runtime.
Are there any disadvantages, side effects, or other issues I should be aware of when using the "Multi-processor Compilation" option in Visual Studio for C++ projects? Or, to phrase the question another way, why is this option off by default in Visual Studio?
The documentation for /MP says:
Incompatible Options and Language Features
The /MP option is incompatible with some compiler options and language features. If you use an incompatible compiler option with the /MP option, the compiler issues warning D9030 and ignores the /MP option. If you use an incompatible language feature, the compiler issues error C2813then ends or continues depending on the current compiler warning level option.
Note:
Most options are incompatible because if they were permitted, the concurrently executing compilers would write their output at the same time to the console or to a particular file. As a result, the output would intermix and be garbled. In some cases, the combination of options would make the performance worse.
And it gives a table that lists compiler options and language features that are incompatible with /MP:
#import preprocessor directive (Converts the types in a type library into C++ classes, and then writes those classes to a header file)
/E, /EP (Copies preprocessor output to the standard output (stdout))
/Gm (Enables an incremental rebuild)
/showIncludes (Writes a list of include files to the standard error (stderr))
/Yc (Writes a precompiled header file)
Instead of disabling those other options by default (and enabling /MP by default), Visual Studio makes you manually disable/prevent these features and enable /MP.
From our experience the main issues found were:
browse information failing to build due to multiple projects calling bscmake at the same time (useless information nowadays so should be removed as a project setting)
linker failures due to dependency issues and build order issues, something you would not normally see when building normally
Batch builds do not take advantage of multi-processor compilation, at least this was certainly true for 2005-2008 VS editions
warnings generated about pre-compiled headers being incompatible, this occurs when you build stdafx and can be ignored but when doing a rebuild it generates this message
However, the above are configuration issues which you can resolve, otherwise it should be enabled as it will speed up builds.
Because multi-processor compilation isn't compatible with many other compilation options and also has higher usage of system resources. It should be up to the developer to decide whether or not it's worth for him. You can find the full documentation here: http://msdn.microsoft.com/en-us/library/bb385193.aspx
While using /MP will bring some benefit to the compilation speed, there is still some performance left on the table due to the way workload is scheduled: https://randomascii.wordpress.com/2014/03/22/make-vc-compiles-fast-through-parallel-compilation/
The compiler receives jobs in "batches" (a set of source files passed to compiler) and will only start the next batch when the prior one is finished. That means there are cycles left unused on other cores until the longest translation unit is compiled. There is no sharing of data between the compiler subprocesses.
To improve utilization on multiple cores even further I suggest switching to ninja. I've implemented it in a few projects and it was always a win, e.g. https://github.com/openblack/openblack/pull/68#issuecomment-529172980
My definition of powerful is ability to customize.
I'm familiar with gcc I wanted to try MSVC. So, I was searching for gcc equivalent options in msvc. I'm unable to find many of them.
controlling kind of output
Stop after the preprocessing stage; do not run the compiler proper.
gcc: -E
msvc: ???
Stop after the stage of compilation proper; do not assemble.
gcc: -S
msvc: ???
Compile or assemble the source files, but do not link.
gcc: -c
msvc:/c
Useful for debugging
Print (on standard error output) the commands executed to run the stages of compilation.
gcc: -v
msvc: ???
Store the usual “temporary” intermediate files permanently;
gcc: -save-temps
msvc: ???
Is there some kind of gcc <--> msvc compiler option mapping guide?
gcc Option Summary lists more options in each section than Compiler Options Listed by Category. There are hell lot of important and interesting things missing in msvc. Am I missing something or msvc is really less powerful than gcc.
MSVC is an IDE, gcc is just a compiler. CL (the MSVC compiler) can do most of the steps that you are describing from gcc's point of view. CL /? gives help.
E.g.
Pre-process to stdout:
CL /E
Compile without linking:
CL /c
Generate assembly (unlike gcc, though, this doesn't prevent compiling):
CL /Fa
CL is really just a compiler, if you want to see what commands the IDE generates for compiling and linking the easiest thing to look at the the command line section of the property pages for an item in the IDE. CL doesn't call a separate preprocessor or assembler, though, so there are no separate commands to see.
For -save-temps, the IDE performs separate compiling and linking so object files are preserved anyway. To preserve pre-processor output and assembler output you can enable the /P and /Fa through the IDE.
gcc and CL are different but I wouldn't say that the MSVC lacks "a hell lot" of things, certainly not the outputs that you are looking for.
For the equivalent of -E, cl.exe has /P (it doesn't "stop after preprocessing stage" but it outputs the preprocessor output to a file, which is largely the same thing).
For -S, it's a little murkier, since the "compilation" and "assembling" steps happen in multiple places depending on what other options you have specified (for example, if you have whole program optimization turned on, then machine code is not generated until the link stage).
For -v, Visual C++ is not the same as GCC. It executes all stages of compilation directly in cl.exe (and link.exe) so there are no "commands executed" to display. Similarly for -save-temps: because everything happens inside cl.exe and link.exe directly, the only "temporary" files are the .obj files that cl.exe produces and they're always saved anyway.
At the end of the day, though, GCC is an open source project. That means anybody with an itch to scratch can add whatever command-line options they like with relatively little resistance. For Visual C++, a commercial closed-source product, every option needs to have a business case, design meetings, test plans and so on. Every new feature starts with minus 100 points.
Both compilers have a plethora of options for modifying... everything. I suspect that any option not present in either is an option for something not worth doing in the first place. Most "normal" users don't find a use for most of those options anyway.
If you're looking purely at the number of available options as a measure of "power" or "flexibility" then you'll probably find gcc to be the winner, simply because gcc handles many platforms other than Windows and has specific options for many of those platforms that you obviously won't find in MSVC. gcc (well, the gcc toolchain) also compiles a whole lot of languages beyond C and C++; I recently used it for Objective-C, for example.
EDIT: I'm with Dean in questioning the validity of your question. Yes, MSVC (cl) has options for the equivalent of many of gcc's options, but no, the number of options doesn't really mean much.
In short: Unless you're doing something very special, you'll find MSVC easily "powerful enough" on the Windows platform that you will likely not be missing any gcc options.