clang compiler and #ident preprocessor directive - c++

I've been working on modernizing our source management control and
compiler in my organization. We've been thus far using the Oracle
Developer Studio compilers on Solaris / Linux and the latest compiler
hasn't been supported on RHEL 8+. It also seems that it won't be
maintained in the future so I've started using clang. We've also in the
past been using RCS (ugh) and have been moving our C++ code into SVN
repositories in line with what we currently do for java projects.
Typically we use #ident tags to identify source information and comes in
handy, at least with RCS in looking for dirty headers etc. Checked out
files are less of an issue with SVN and compiling in our own
self-contained areas, but its still useful to have this information in
the object and eventual binary so we can determine differences between
releases easily.
It looks as if clang however, with the compiler options I've been using
doesn't use either #ident or the windows - #pragma comment(exestr,
"XXX"). At least nothing appears in the object as strings that I can see.
Is there any compiler options or pre-processor compiler options that I
can use with clang to retain this information?

That's a bit naughty, if clang accepts #ident but throws the string away.
This is actually a surprisingly intractable problem. Before the days of fiercely optimising compilers, it was sufficient to put, say:
static const char *id = "whatever";
at the top of each file, but no longer. If id isn't referenced anywhere, this now (a) generates a warning, and (b) has no effect. But you could try:
static const volatile char *id = "xxx";
^^^^^^^^
This compiles OK but I don't know whether the compiler puts the string in the object file; you'd have to check.
I don't suppose the prospect of editing every file in your codebase fills you with glee, but it looks like you have no choice if you want to get those strings in there.

In case anyone is interested. I ended up doing the following:
I added to the source a #pragma message
#ifdef __clang __
#pragma message ("VERSION File: $Id: $")
#else
#ident "VERSION File: $Id: $"
#endif
Note here, that SVN will populate the $Id keyword
#pragma message generates a warning in the compile output, so I created a bash script filter to redirect the output through. If it is a pragma message warning, it will put the version information into a THE_OBJECT_FILE.txt
#!/bin/sh
if [ -f $1.txt ]; then
\rm -rf $1.txt
touch $1.txt
fi
while read line
do
TEST=`echo $line | grep MAGENTA | grep VERSION`
if [ "$TEST" = "" ]; then
echo $TEST
else
TEST2=`echo $line | grep pragma-messages`
if [ "$TEST2" = "" ]; then
TEST3=`echo $line | awk -F'"' '{print $2}'`
echo $TEST3 >> $1.txt
fi
fi
done
After the object has been compiled and passed through the filter, use objcopy to embed the versioning into the comments section in the object.
#if [ -f "$#.txt" -a -f "$#" -a ! "$(FILTER_VERSION)" = "" ]; then
objcopy --add-section .version_information=$#.txt
--set-section-flags .version_information=noload,readonly $#; rm -f $#.txt;
fi

Related

How can I make the GCC error messages more readable (e.g., with color)? [duplicate]

For some long errors, the gcc output is dense and has lots of line-wrapping etc. Especially when errors are subtle, it can take me 10-30 seconds of squinting to parse it with my eyes.
I've taken to pasting this in an open code-editor window to get some basic syntax highlighting and enable reformatting with regex's.
Has anyone invented a more automated method?
I use this script, called colorize:
#!/bin/bash
while read x ; do echo $x ; done \
| sed -e "s/.*error:.*/\x1b[1;36m&\x1b[0m/" \
-e "s/.*warning:.*/\x1b[1;36m&\x1b[0m/" \
-e "s/^\(.*\)\(required from\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
-e "s/^\(.*\)\(In instantiation of\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
-e "s/^\(.*\)\(In member\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
| sed -e "s/error:/\x1b[1;31m&\x1b[1;36m/" \
-e "s/warning:/\x1b[1;35m&\x1b[1;36m/" \
-e "s/note:/\x1b[1;30m&\x1b[0m/"
Then I just call it like this(using make or whatever build system):
make |& colorize
And I get color output similar to clang.
I've found colorgcc to be invaluable. By introducing coloring, it becomes much easier to mentally parse the text of gcc error messages, especially when templates are involved.
If your errors are template related, take a look at STLfilt:
http://www.bdsoft.com/tools/stlfilt.html
gccfilter does coloring & simplification of messages.
http://www.mixtion.org/gccfilter/
If you use GCC 4.9, you can add -fdiagnostics-color=auto as an additonal compilation flag. At some later version, the color has been enabled by default.
check diagcc out, you can get something like this:
If your gcc ≥ 4.9, you can use argument -fdiagnostics-color=always.
To answer your question 4 years later, clang should be mentioned here.
Here's my current hack, which mostly inserts newlines and indentation in strategic locations along with a little extra annotation, but does nothing to address STL verbosity.
Note that as currently implemented, this script does not return an error if the compiler returned one, so doing something like this will not work properly: (make && ./runApplication). This could surely be remedied by someone with better bash-fu.
#!/bin/bash
# SUBSTITUTION RULES:
# Note: All substitution rules must end in a semi-colon, inside of the closing quote
subColonSpace='s/: /:\n /g;'
subSrc='s/^src/\nsrc/;'
subError='s/error:/error:\n\n======================================\nERROR:/;'
subWarning='s/ *error: *\n/ERROR: /;'
subWarning='s/ *warning: *\n/WARNING: /;'
subNote='s/note:/\n NOTE:/g;'
subOpenTic='s/‘/\n ‘/g;'
subOpenParen='s/(/(\n /g; s/(\n *)/()/g;'
subCommaSpace='s/, /,\n /g;'
# Note: The order of these may matter
sedExpr="$subColonSpace $subSrc $subError $subWarning $subNote $subOpenTic
$subOpenParen $subCommaSpace"
makelogFile=makelog.tmp
make "$#" 2>&1 | sed "$sedExpr" | tee $makelogFile
if you like Ruby there is GilCC! GilCC is very easy to install (just copy it to the bin folder) and easy to use (just type GilCC instead of "gcc" or "make") and it works with GCC version. Unlike Perl based scripts GilCC has statistics such as # of warnings and error and compile time. You don't have to mess with .bash files and it is cross platform as long as you can run Ruby on your machine. Since it has the power of Ruby; you can make GilCC do different things such as trigger test automation, unit test or program external hardware after a successful build.
Here is the link to the download page: http://www.onlysolutionssoftware.com/gilcc/

Include Git commit hash and/or branch name in C/C++ source

I would like to know how you can get a Git commit hash and/or other information into the contents of a C++ variable in the compiled binary without having it be part of the source that's tracked by Git.
I need to keep track of firmware release information in a compiled executable running on an embedded processor. Means to identify the particular release of the firmware binary such as meaningful filenames, MD5 checksums or even date/time stamps are not available in this closed environment (i.e., there is no filesystem).
One approach is to have the device's console output produce identifying text, such as 'Release 1.2.3', 'commit hash 1bc123...', or similar. The firmware release information is only of interest to maintenance personnel, so a trained operator could examine the console output. To implement this it could potentially involve manual editing of a version string, which is then compiled into the code and output to the console at program startup.
This is acceptable for major releases where a signoff workflow is used to double-check that the version information is correct. However, this being a manual process it is inherently unreliable. For example, what if the developer forgets to update the version information? - There is now a disconnect between the compiled code and its reported version string.
A workflow where the code is freshly compiled and downloaded each time the user wants to test the hardware is not practical in the situation in question, ie., it is quite onerous to update the firmware.
An automatic way of identifying the version of the code is thus required. In the situation in question, Git is used, and the developers regularly commit their work to feature branches. Displaying the Git commit hash, and perhaps also whether or not there are unstaged changes, would be a way of identifying the status of the source code used to compile the firmware.
The requirement is that I would like my application to have information available to it so that it is able to display:
"Git commit:[01abcdef...etc], branch: experimentalStuffDoNotRelease"
Thus, I would like to automatically include Git information, such as commit hash and branch, in the compiled C and/or C++ code.
The development environment has developers using both Windows and Linux, and uses Eclipse CDT with a relatively unsophisticated workflow of: check out; compile; download to the hardware.
I use a makefile, like so:
GIT_HASH=`git rev-parse HEAD`
COMPILE_TIME=`date -u +'%Y-%m-%d %H:%M:%S UTC'`
GIT_BRANCH=`git branch | grep "^\*" | sed 's/^..//'`
export VERSION_FLAGS=-DGIT_HASH="\"$(GIT_HASH)\"" -DCOMPILE_TIME="\"$(COMPILE_TIME)\"" -DGIT_BRANCH="\"$(GIT_BRANCH)\""
all:
g++ main.cpp $(VERSION_FLAGS)
When the makefile is run, the git hash and the time of compilation are both loaded into macros accessible within the source, like so:
#include <iostream>
int main(){
std::cerr<<"hash="<<GIT_HASH<<", time="<<COMPILE_TIME<<", branch="<<GIT_BRANCH<<std::endl;
}
Which gives output like:
hash=35f531bf1c959626e1b95f2d3e1a7d1e4c58e5ec, time=2017-05-18 04:17:25 UTC, branch=master
In Eclipse CDT, use a pre-build step to generate an include file containing the relevant information, and a conditional inclusion to check that the file was created:
Right-click the project
Select Properties
Expand the C/C++ Build
Select Settings In the Build Steps tab
Enter the following in the Command text box:
git log --pretty=format:'#define GIT_INFO_PRESENT%n static const char* GIT_INFO = "Version Information=[%H,%d]\r\n";' -n 1 > ../src/gitcommit.h
This will, upon build, create a file gitcommit.h that will be included in the source code. To customise it, adjust the string to your needs. (See https://git-scm.com/docs/pretty-formats)
As an example, I produce a debug output at the beginning of the main routine to inform the user of the commit and branch (not strictly needed knowing the commit, but certainly helpful):
Put this in the file, probably at the top
#if __has_include("gitcommit.h")
#include "gitcommit.h"
#else
static const char* GIT_INFO = "Git version information not present.\r\n";
#endif
To display the information somewhere in your code, do similar to this:
printf(GIT_INFO);
Note that I haven't, in this case, made the pre-build step a shell script or Windows/DOS .bat file, as I work often in Linux or Windows.
Note that this isn't tested in Windows.
In both cases, 'git' must be executable from the standard command line.
There is a dependency on provision of __has_include. This was intended to provide simplicity so that a default include file need not be provided.
Note that the gitcommit.h file's path should be discoverable by the compiler.
Usually as part of your build you run some command that generates something like that.
For example, git describe gives you something that you could use:
echo // auto generated version: > version.h
git describe > echo // auto generated version: > version.h
echo -e "#define VERSION " >> version.h
git describe >> version.h
For example x264 use this simple script to generate it:
if [ -d .git ] && command -v git >/dev/null 2>&1 ; then
localver="$(($(git rev-list HEAD | wc -l)))"
if [ "$localver" -gt 1 ] ; then
ver_diff="$(($(git rev-list origin/master..HEAD | wc -l)))"
ver="$((localver-ver_diff))"
echo "#define X264_REV $ver"
echo "#define X264_REV_DIFF $ver_diff"
if [ "$ver_diff" -ne 0 ] ; then
ver="$ver+$ver_diff"
fi
if git status | grep -q "modified:" ; then
ver="${ver}M"
fi
ver="$ver $(git rev-list -n 1 HEAD | cut -c 1-7)"
version=" r$ver"
fi
fi
echo "#define X264_VERSION \"$version\""
echo "#define X264_POINTVER \"0.$ver\""
This will generate something like:
#define X264_VERSION " r2708 86b7198"
#define X264_POINTVER "0.148.2708 86b7198"
Here's simple solution if you need git hash and local modifications flag only:
$ cat update-version-info.sh
#!/bin/sh
version=$(git describe --always --dirty --tags)
echo "#define GIT_VERSION \"$version\"" > git-version.h
$ cat 1.c
#include <stdlib.h>
#include <stdio.h>
#include "git-version.h"
int main() {
#ifdef GIT_VERSION
printf("%s\n", GIT_VERSION);
#endif
return 0;
}
$ ./1
ee4f307-dirty
Based on #Richard's answer and my fixes for qmake/Qt, here is the solution I use:
With Qt 5.14 the following lines in .pro file work for me:
GIT_HASH="\\\"$$system(git -C \""$$_PRO_FILE_PWD_"\" rev-parse --short HEAD)\\\""
GIT_BRANCH="\\\"$$system(git -C \""$$_PRO_FILE_PWD_"\" rev-parse --abbrev-ref HEAD)\\\""
BUILD_TIMESTAMP="\\\"$$system(date -u +\""%Y-%m-%dT%H:%M:%SUTC\"")\\\""
DEFINES += GIT_HASH=$$GIT_HASH GIT_BRANCH=$$GIT_BRANCH BUILD_TIMESTAMP=$$BUILD_TIMESTAMP
In your code you can check the revision like this:
int main(int argc, char *argv[])
{
QStringList args;
for (int i = 0; i < argc; i++)
args << QString(argv[i]);
if (args.contains("-v") || args.contains("--version")) {
std::cout << QString("branch: %1, version: %2, built_at: %3").arg(GIT_BRANCH).arg(GIT_HASH).arg(BUILD_TIMESTAMP).toUtf8().constData() << std::endl;
return 0;
}
// ...
}
A C file or a C++ file can be generated by some program (for example, some shell script on Linux, or some GNU awk script, or your C program running system or popen(3) running some git command).
You just configure your build automation tool (e.g. your Makefile if you use GNU make, or your build.ninja file if you use ninja) appropriately.
Both Bismon and RefPerSys are doing so and keep their git id inside the obtained executable (displaying it when invoked with --version). Note: both are projects I created.
PS a different question is how to configure your IDE to run a particular build automation tool. This is specific to your IDE. But Eclipse CDT FAQ offers an insight.

Improving g++ output

g++ sometimes produces pretty convoluted outputs. Specially when dealing with templates. Is there any tool that makes g++ output more readable? ... at least some color?
It may sound silly to ask this question here, but I've been unable to google my way out of this.
GCC as of 4.9 has colourized output ☺
Append the following to your bashrc/zshrc/whatever.
export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
If you're free to switch compiler, try clang++. It's error outputs are most of the time very clear and sometimes it will even tell you what to do to fix your error. You can find a comparison of gcc vs clang error output here: http://clang.llvm.org/diagnostics.html.
Otherwise, as noted by Matteo Italia, look to STLFilt.
For the color aspect, Edwin Buck's solution (colorgcc) is very good. You can create symlinks to it named gcc, g++, etc so that it always gets used whenever you run a compiler command (provided you setup the paths in the colorgcc script correctly). I find this especially convenient when compiling some large project from a Makefile. Helps tremendously to detect where exactly something went wrong (especially when you do parallel compilations with make -j).
There's colorgcc, a perl script which wraps the gcc (g++) output with color for easier readability.
As far as the "output" of gcc (g++) I am guessing you're not complaining about the compiled binaries :)
I improve it by using sed to color the output with this script:
#!/bin/bash
while read x ; do echo $x ; done \
| sed -e "s/.*error:.*/\x1b[1;36m&\x1b[0m/" \
-e "s/.*warning:.*/\x1b[1;36m&\x1b[0m/" \
-e "s/^\(.*\)\(required from\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
-e "s/^\(.*\)\(In instantiation of\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
-e "s/^\(.*\)\(In member\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
| sed -e "s/error:/\x1b[1;31m&\x1b[1;36m/" \
-e "s/warning:/\x1b[1;35m&\x1b[1;36m/" \
-e "s/note:/\x1b[1;30m&\x1b[0m/"
If you use gcc 4.9 or higher there are a couple built-in options:
-fmessage-length=n, which optimizes the output for lines of n characters length.
-fdiagnostics-color=always, which applies some nice colors. (Works fine on linux, not so much on msys)
For more options and exact usage heres the documentation:
https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Message-Formatting-Options.html
When dealing with the STL1, STLFilt comes to the rescue.
On a related note, I also heard that the clang compiler produces by itself much better error messages in general.
Nitpickers' corner
Actually, not the SGI STL, but the parts of it that have been adopted in the C++ standard and are part of the standard library.
I can't answer your question directly, but note that in regards to templates if you look at the top message (the error caused by the templated) and the bottom "instantiated from here" message, it usually becomes much more clear. Ignore all the middle messages as they are probably not going to help.
There is colorgcc.
there is my personal function:
colorgcc()
{
perl -wln -M'Term::ANSIColor' -e '
m/not found$/ and print "\e[1;30m$`\e[0m", "$&", "\e[0m"
or
m/found$/ and print "\e[1;30m$`\e[0;32m", "$&", "\e[0m"
or
m/yes$/ and print "\e[1;30m$`\e[0;32m", "$&", "\e[0m"
or
m/no$/ and print "\e[1;30m$`\e[0m", "$&", "\e[0m"
or
m/ Error |error:/i and print "\e[1;91m", "$_", "\e[0m"
or
m/ Warning |warning:/i and print "\e[0;33m", "$_", "\e[0m"
or
m/nsinstall / and print "\e[0;36m", "$_", "\e[0m"
or
m/Linking |link: |\.a\b/ and print "\e[1;36m", "$_", "\e[0m"
or
m/Building|gcc|g++|\bCC\b|\bcc\b/ and print "\e[1;30m", "$_", "\e[0m"
or
print; '
}
you can use GilCC which is a Ruby tool that will convert GCC output to color in real-time. It is free and you can customize it to your preference. It is not intrusive as it does not change your gcc setup and you don't have to edit any system files such as .bash. You have to install Ruby and be able to invoke Ruby programs from the command line. Right now you have two options Perl script or GilCC and if you work with Ruby you will like GilCC.
Then whenever you call "GillCC" it will call "make" behind the scenes with your parameters such as "clean", "debug" or "release". for example if normally you call: "make debug" you should call "GilCC debug".
Almost forgot to mention that GilCC has some statistics such as # of warnings and error as well as build time. These things are handy when you are trying to improve your build.
The link to the download page is here.

Debugging the C++ preprocessor

I'm trying to build Amaya. When the build failed with
error: expected unqualified-id before ‘(’ token
I ran g++ with only the preprocessor (replacing the -c option with -E) on the file that failed to compile to see what was going on. This produced an 80,000 line file, showing me that 'Blue' had been replaced by (2 << 8), which clearly explained the error. If I correct this, the file compiles fine. I guess I could live with that, but I would like to find out why this is happening.
Is there any way I can track how the preprocessor is replacing a specific string, in this case 'Blue'?
================= Update ===================
Well, I found the culprit:
> headers=`g++ [omited for brevity] -M \
../../thotlib/dialogue/AmayaClassicNotebook.cpp`
> for file in $headers ; do grep -s -H Blue $file | grep "(2 << 8)";done
/usr/local/include/gc.h:#define Blue (2 << 8)
So adding #undef Blue fixed the problem. So using this combination of -M and grep seems OK but sometimes C++ preprocessor definitions can be a real forest; I was curious whether there were some better way, some clever GNU tool maybe.
I find running
g++ ... -dD -E $file > $file.ii
to be very useful in untangling preprocessing problems. From man g++:
-dD Dump all macro definitions, at the end of preprocessing,
in addition to normal output.
If nobody comes up with anything better (based on the source line info in the preprocessed file), you could use the -M option to get a list of headers included by the source file, and search those for "Blue". I expect it's possible for some kind of obfuscation to mean that this doesn't find what you're looking for, but normally you'll turn up the definition somewhere.
What's wrong with the perennial
find /src -exec grep Blue {} /dev/null ';'
That usually works for me, at least as a first cut.

Is there any way to get readable gcc error and warning output at the command line?

For some long errors, the gcc output is dense and has lots of line-wrapping etc. Especially when errors are subtle, it can take me 10-30 seconds of squinting to parse it with my eyes.
I've taken to pasting this in an open code-editor window to get some basic syntax highlighting and enable reformatting with regex's.
Has anyone invented a more automated method?
I use this script, called colorize:
#!/bin/bash
while read x ; do echo $x ; done \
| sed -e "s/.*error:.*/\x1b[1;36m&\x1b[0m/" \
-e "s/.*warning:.*/\x1b[1;36m&\x1b[0m/" \
-e "s/^\(.*\)\(required from\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
-e "s/^\(.*\)\(In instantiation of\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
-e "s/^\(.*\)\(In member\)/\x1b[1;36m\1\x1b[0mnote: \2/" \
| sed -e "s/error:/\x1b[1;31m&\x1b[1;36m/" \
-e "s/warning:/\x1b[1;35m&\x1b[1;36m/" \
-e "s/note:/\x1b[1;30m&\x1b[0m/"
Then I just call it like this(using make or whatever build system):
make |& colorize
And I get color output similar to clang.
I've found colorgcc to be invaluable. By introducing coloring, it becomes much easier to mentally parse the text of gcc error messages, especially when templates are involved.
If your errors are template related, take a look at STLfilt:
http://www.bdsoft.com/tools/stlfilt.html
gccfilter does coloring & simplification of messages.
http://www.mixtion.org/gccfilter/
If you use GCC 4.9, you can add -fdiagnostics-color=auto as an additonal compilation flag. At some later version, the color has been enabled by default.
check diagcc out, you can get something like this:
If your gcc ≥ 4.9, you can use argument -fdiagnostics-color=always.
To answer your question 4 years later, clang should be mentioned here.
Here's my current hack, which mostly inserts newlines and indentation in strategic locations along with a little extra annotation, but does nothing to address STL verbosity.
Note that as currently implemented, this script does not return an error if the compiler returned one, so doing something like this will not work properly: (make && ./runApplication). This could surely be remedied by someone with better bash-fu.
#!/bin/bash
# SUBSTITUTION RULES:
# Note: All substitution rules must end in a semi-colon, inside of the closing quote
subColonSpace='s/: /:\n /g;'
subSrc='s/^src/\nsrc/;'
subError='s/error:/error:\n\n======================================\nERROR:/;'
subWarning='s/ *error: *\n/ERROR: /;'
subWarning='s/ *warning: *\n/WARNING: /;'
subNote='s/note:/\n NOTE:/g;'
subOpenTic='s/‘/\n ‘/g;'
subOpenParen='s/(/(\n /g; s/(\n *)/()/g;'
subCommaSpace='s/, /,\n /g;'
# Note: The order of these may matter
sedExpr="$subColonSpace $subSrc $subError $subWarning $subNote $subOpenTic
$subOpenParen $subCommaSpace"
makelogFile=makelog.tmp
make "$#" 2>&1 | sed "$sedExpr" | tee $makelogFile
if you like Ruby there is GilCC! GilCC is very easy to install (just copy it to the bin folder) and easy to use (just type GilCC instead of "gcc" or "make") and it works with GCC version. Unlike Perl based scripts GilCC has statistics such as # of warnings and error and compile time. You don't have to mess with .bash files and it is cross platform as long as you can run Ruby on your machine. Since it has the power of Ruby; you can make GilCC do different things such as trigger test automation, unit test or program external hardware after a successful build.
Here is the link to the download page: http://www.onlysolutionssoftware.com/gilcc/