We find a slight difference in how bcc32 and bcc32ide behave on a
file.
bcc32ide works and bcc32 crashes.
Would there be any detrimental side effects to switching to bcc32ide for our automated builds?
Also, what is the difference between these two compilers (other than one crashes and one doesn't)?
See Re: BCB 6 Compiler slowness and how to speed it up dramatically for an explanation.
Quote:
bcc32ide.exe is a bcc32.exe, but it uses the IDE compiler. For
example, some compiler assertions do not raise with the IDE compiler, but
with bcc32.exe. So replacing bcc32.exe in makefiles by bcc32ide.exe will
allow to compile projects that compile within the IDE, but not at console.
bcc32ide.exe also adds some additional parameters:
-automake Compile only modified files
-verbose Output current compiled line number during compilation
-pch=filename_pch.h Use advanced precompiled headers
-FIfilename Force include (multiple, usage like -Iincludedir)
And it supports multi-process shared access to the advanced PCH file
(required for mtbcc32.exe).
Related
I'm looking for examples of code that triggers non-determinism in GCC or Clang's compilation process.
One prominent example is the usage of the __DATE__ macro.
GCC and Clang have a plethora of compiler flags to control the outcome of non-deterministic actions within the compiler eg. -frandom-seed and -fno-guess-branch-probability
Are there any small examples that are affected by these flags?
To be more precise:
$ c++ main.cpp -o main && shasum main
aabbccddee
$ c++ main.cpp -o main && shasum main
eeddccbbaa
I'm looking for macro-free code examples where multiple runs of the compiler lead to different outputs, but can be fixed by e.g. -frandom-seed
EDIT:
related: from the gcc docs:
-fno-guess-branch-probability:
Sometimes gcc will opt to use a randomized model to guess branch probabilities,
when none are available from either profiling feedback (-fprofile-arcs)
or __builtin_expect.
This means that different runs of the compiler on the same program
may produce different object code.
The default is -fguess-branch-probability at levels -O, -O2, -O3, -Os.
While old, this question is interesting for reproducible builds.
As you've stated, there are multiple source of non-determinism while compiling some C/C++ source.
Non-determinism in preprocessor
The preprocessor usually implements some numerous super macro which are changing between runs. There's the obvious __DATE__ and __TIME__ but also the non obvious __cplusplus or __STD_C_VERSION__ or __GNUC_PATCHLEVEL__ which can changes when the OS updates.
There's also the __FILE__ that will contain the path of the building environment (different from machine to machine).
Please notice that for the former macro, GCC observes the environment variable SOURCE_DATE_EPOCH to overwrite the date and time macro. Other compilers might have some other behavior.
Non-determinism in the compiler
The compiler might have different optimization strategies based on non-deterministic approach. You've cited one in GCC, but other might exists.
For MSVC, you might be interested in the /BREPRO compiler flag.
You'll have to RTFM for your compiler to know more.
Non-determinism in the linker
On some architecture, the linked object and/or library will contain a timestamp. MacOS is one of them. So for the same set of .o files, you'll get a different resulting executable.
Also, if you use Link Time Optimization, many compiler will create different versions of the .o files named randomly. Again for GCC, you'll use -frandom-seed=31415 to "fix" this randomness, but YMMV.
Non-determinism in the build-process
Sometimes repositories contain additional operation that are performed outside of the compilation stage. Like generating header files based on some configuration flags (or other steps).
In that case, this per-project's specific operations might not be deterministic either.
For a good overview of the deterministic builds, please refer to this post
Several static analysis tools designed for C/C++ exist, but they are not particularly useful for testing CUDA sources.
Since clang version 6 is able to compile CUDA, I wanted to check what are my options with using clang-tidy, which does not seem to have option for switching architectures.
Is there a way to make it work? For example compile time switch for turning on CUDA parser, extension in form of custom check, or is it maybe planned feature?
One of the problem with the clang-based tools is that they are not parsing the files in exactly the same way as clang does.
The first problem is that unlike C/C++ compilation, CUDA compilation compiles the source multiple times. By default clang creates multiple compilation jobs when you give it a CUDA file and that trips many tools that expect only one compilation. In order to work that around you need to pass --cuda-host-only option to clang-tidy.
You may also need to pass --cuda-path=/path/to/your/CUDA/install/root so clang can find CUDA headers.
Another problem you may run into would be related to include paths. Clang-derived tools do not have the same default include paths that clang itself uses and that occasionally causes weird problems. At the very least clang-tidy needs to find __clang_cuda_runtime_wrapper.h which is installed along with clang. If you run clang-tidy your-file.c -- -v it will print clang's arguments and include search paths it uses. Compare that to what clang -x c /dev/null -fsyntax-only -vprints. You may need to give clang-tidy extra include paths to match those used by clang itself. Note that you should not explicitly add the path to the CUDA includes here. It will be added in the right place automatically by --cuda-path=....
Once you have it all in place, clang-tidy should work on CUDA files.
Something like this:
clang-tidy your-file.cu -- --cuda-host-only --cuda-path=... -isystem /clang/includes -isystem /extra/system/includes
Are there any disadvantages, side effects, or other issues I should be aware of when using the "Multi-processor Compilation" option in Visual Studio for C++ projects? Or, to phrase the question another way, why is this option off by default in Visual Studio?
The documentation for /MP says:
Incompatible Options and Language Features
The /MP option is incompatible with some compiler options and language features. If you use an incompatible compiler option with the /MP option, the compiler issues warning D9030 and ignores the /MP option. If you use an incompatible language feature, the compiler issues error C2813then ends or continues depending on the current compiler warning level option.
Note:
Most options are incompatible because if they were permitted, the concurrently executing compilers would write their output at the same time to the console or to a particular file. As a result, the output would intermix and be garbled. In some cases, the combination of options would make the performance worse.
And it gives a table that lists compiler options and language features that are incompatible with /MP:
#import preprocessor directive (Converts the types in a type library into C++ classes, and then writes those classes to a header file)
/E, /EP (Copies preprocessor output to the standard output (stdout))
/Gm (Enables an incremental rebuild)
/showIncludes (Writes a list of include files to the standard error (stderr))
/Yc (Writes a precompiled header file)
Instead of disabling those other options by default (and enabling /MP by default), Visual Studio makes you manually disable/prevent these features and enable /MP.
From our experience the main issues found were:
browse information failing to build due to multiple projects calling bscmake at the same time (useless information nowadays so should be removed as a project setting)
linker failures due to dependency issues and build order issues, something you would not normally see when building normally
Batch builds do not take advantage of multi-processor compilation, at least this was certainly true for 2005-2008 VS editions
warnings generated about pre-compiled headers being incompatible, this occurs when you build stdafx and can be ignored but when doing a rebuild it generates this message
However, the above are configuration issues which you can resolve, otherwise it should be enabled as it will speed up builds.
Because multi-processor compilation isn't compatible with many other compilation options and also has higher usage of system resources. It should be up to the developer to decide whether or not it's worth for him. You can find the full documentation here: http://msdn.microsoft.com/en-us/library/bb385193.aspx
While using /MP will bring some benefit to the compilation speed, there is still some performance left on the table due to the way workload is scheduled: https://randomascii.wordpress.com/2014/03/22/make-vc-compiles-fast-through-parallel-compilation/
The compiler receives jobs in "batches" (a set of source files passed to compiler) and will only start the next batch when the prior one is finished. That means there are cycles left unused on other cores until the longest translation unit is compiled. There is no sharing of data between the compiler subprocesses.
To improve utilization on multiple cores even further I suggest switching to ninja. I've implemented it in a few projects and it was always a win, e.g. https://github.com/openblack/openblack/pull/68#issuecomment-529172980
My definition of powerful is ability to customize.
I'm familiar with gcc I wanted to try MSVC. So, I was searching for gcc equivalent options in msvc. I'm unable to find many of them.
controlling kind of output
Stop after the preprocessing stage; do not run the compiler proper.
gcc: -E
msvc: ???
Stop after the stage of compilation proper; do not assemble.
gcc: -S
msvc: ???
Compile or assemble the source files, but do not link.
gcc: -c
msvc:/c
Useful for debugging
Print (on standard error output) the commands executed to run the stages of compilation.
gcc: -v
msvc: ???
Store the usual “temporary” intermediate files permanently;
gcc: -save-temps
msvc: ???
Is there some kind of gcc <--> msvc compiler option mapping guide?
gcc Option Summary lists more options in each section than Compiler Options Listed by Category. There are hell lot of important and interesting things missing in msvc. Am I missing something or msvc is really less powerful than gcc.
MSVC is an IDE, gcc is just a compiler. CL (the MSVC compiler) can do most of the steps that you are describing from gcc's point of view. CL /? gives help.
E.g.
Pre-process to stdout:
CL /E
Compile without linking:
CL /c
Generate assembly (unlike gcc, though, this doesn't prevent compiling):
CL /Fa
CL is really just a compiler, if you want to see what commands the IDE generates for compiling and linking the easiest thing to look at the the command line section of the property pages for an item in the IDE. CL doesn't call a separate preprocessor or assembler, though, so there are no separate commands to see.
For -save-temps, the IDE performs separate compiling and linking so object files are preserved anyway. To preserve pre-processor output and assembler output you can enable the /P and /Fa through the IDE.
gcc and CL are different but I wouldn't say that the MSVC lacks "a hell lot" of things, certainly not the outputs that you are looking for.
For the equivalent of -E, cl.exe has /P (it doesn't "stop after preprocessing stage" but it outputs the preprocessor output to a file, which is largely the same thing).
For -S, it's a little murkier, since the "compilation" and "assembling" steps happen in multiple places depending on what other options you have specified (for example, if you have whole program optimization turned on, then machine code is not generated until the link stage).
For -v, Visual C++ is not the same as GCC. It executes all stages of compilation directly in cl.exe (and link.exe) so there are no "commands executed" to display. Similarly for -save-temps: because everything happens inside cl.exe and link.exe directly, the only "temporary" files are the .obj files that cl.exe produces and they're always saved anyway.
At the end of the day, though, GCC is an open source project. That means anybody with an itch to scratch can add whatever command-line options they like with relatively little resistance. For Visual C++, a commercial closed-source product, every option needs to have a business case, design meetings, test plans and so on. Every new feature starts with minus 100 points.
Both compilers have a plethora of options for modifying... everything. I suspect that any option not present in either is an option for something not worth doing in the first place. Most "normal" users don't find a use for most of those options anyway.
If you're looking purely at the number of available options as a measure of "power" or "flexibility" then you'll probably find gcc to be the winner, simply because gcc handles many platforms other than Windows and has specific options for many of those platforms that you obviously won't find in MSVC. gcc (well, the gcc toolchain) also compiles a whole lot of languages beyond C and C++; I recently used it for Objective-C, for example.
EDIT: I'm with Dean in questioning the validity of your question. Yes, MSVC (cl) has options for the equivalent of many of gcc's options, but no, the number of options doesn't really mean much.
In short: Unless you're doing something very special, you'll find MSVC easily "powerful enough" on the Windows platform that you will likely not be missing any gcc options.
I have built an open-source application from the source code. Unfortunately, the original executable runs significantly faster. I tried to enable few compiler optimizations, but the result wasn't satisfactory enough. What else do I need to make in Visual Studio 2008 to increase the executable performance?
Thanks!
Basically try enabling everything under Optimisation in project settings, then ensure Link Time Code Generation is on, enable Function-level linking and full COMDAT folding (that only reduces the size of the EXE but could help with caching), and turn off security features such as by defining _SECURE_SCL=0. Remember some of these settings have other implications, especially the security ones.
Define _SECURE_SCL=0.
http://msdn.microsoft.com/en-us/library/aa985896(VS.80).aspx
Try to enable SSE instructions, when compiling. Also - you can try to compile using different compiler (GNU GCC).
+There might be enabled some debug defines, shich also can reduce speed.
+Check, that original .exe has same version as one you are trying to compile.
The open-source precompiled binary is most likely (whit out know which project you are working with) compiled with GNU GCC (Mingw on Windows). That might be the reason that it is faster. According to question: performance g++ vs. VC++ some things are considerably slower if you use VC++.