Summary: C++ preprocessor output includes some lines that say <built-in>. I'm curious to know what these are for.
Details:
When I compile the following code in a file named test.cpp with clang++ -E (output from g++ is similar):
#include <iostream>
int main()
{
std::cout << "Hello World!" << std::endl;
return 0;
}
the first few lines of output are as follows:
# 1 "test.cpp"
# 1 "test.cpp" 1
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 156 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "test.cpp" 2
My question is what do the <built-in> statements mean.
A macro expands to "1", and in case of built-in, the macro is defined by default, e.g., __cplusplus, in case of command line, the macro is defined on the command-line, i.e., -DMACRO=1.
You can see a list of predefined macros:
cpp -dM foo.h
Related
An option is defined (value = 1 or 2) to chose between two instructions and I would like to use with an instruction which have a comma.
#define option 1
#if option == 1
#define my_instr(instr1, instr2) instr1
#else if option == 2
#define my_instr(instr1, instr2) instr2
#endif
It works but when there is a comma in the instruction, I have a problem.
For example :
program main
my_instr(print *,"opt 1", print * ,"opt 2")
end program main
does not compile (gftran -cpp) : Too much args. I am ok.
Thus, to escape the comma, parentheses are added : my_instr((print *,"opt 1"), (print * ,"opt 2"))
But it does not compile any more because of parentheses.
How can I solve that ?
Using the answer (https://stackoverflow.com/a/46311121/7462275), I found a " solution ".
#define option 2
#define unparen(...) __VA_ARGS__
#if option == 1
#define my_instr(instr1, instr2) unparen instr1
#elif option == 2
#define my_instr(instr1, instr2) unparen instr2
#endif
program main
my_instr((print *,"opt 1"), (print * ,"opt 2"))
end program main
But,
gfortran -cpp does not compile (problem with __VA_ARGS__). So, cpp -P is used before gfortran
__VA_ARGS__ : It is not standard to use __VA_ARGS__ without something before (cf comments of :Matthew D. Scholefield in the answer used :
Just something to note, this still doesn't conform to the C standard because of the use of a variadic macro.
and KamilCuk in this question)
Even instructions without comma need to be enclosed between parentheses
This will select the correct string.
#ifdef my_instr
#undef my_instr
#endif
#define my_instr(x) print *, x
#if option == 1
#define str "Opt 1"
#else if option == 2
#define str "Opt 2"
#endif
program foo
my_instr(str)
end program foo
% gfortran -E -Doption=2 a.F90 | cat -s
# 1 "a.F90"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "a.F90"
program foo
print *, "Opt 2"
end program foo
I was trying to check some single line macros which have 2 pre-processor directives.
#define REPLACE { \
#if EXT == 42 \
#warning "Got 42" \
#endif \
}
int main(void){
REPLACE;
return 0;
}
The pre-processor parses this fine yielding:
$g++ -E includetest.cpp
# 1 "includetest.cpp"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "includetest.cpp"
int main(void){
{ #if EXT == 42 #warning "Got 42" #endif };
return 0;
}
which of course is illegal code, since just macro substitution occurs and the ifdef lookalike macro isn't processed again even though it looks like one.
Now if I slightly alter the macro to look something like
#define REPLACE(a) { a + 2 ; \
#if EXT == 42 \
#warning "Got 42" \
#endif \
}
int main(void){
REPLACE(0);
return 0;
}
Which yields this pre-processor error:
$g++ -E includetest.cpp
# 1 "includetest.cpp"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "includetest.cpp"
includetest.cpp:1:18: error: '#' is not followed by a macro parameter
#define REPLACE(a) { a + 2 ; \
^
int main(void){
REPLACE(0);
return 0;
}
Why does this error come up ? Of course, this won't compile but I want to know why addition of a parameter ensues in a parsing error from pre-processor ?
People would say " You can't nest another directive in another", but in the first case too they are nested, why doesn't pre-processor error out then ? Or is that responsibility delegated to the compiler ?
EDIT: I am not trying to achieve any functionality per se, this is just an exercise (in futility?) to understand the pre-processor.
Only inside a function macro does a # have special meaning ([cpp.stringize]p1). The standard says that a # needs to be followed by a function argument, which is not happening in your second case (if, warning and endif are not parameters).
Your first case is valid (you can actually have directives inside the replacement list of object macros) exactly because there # doesn't have any special meaning.
(I've spent quite some time getting this to work, so I thought I'd document it - first, to put it formally as a question):
Is there a simple example of systemtap probing/tracing functions in a user-space application, preferably in C++? My system is Ubuntu 14.04:
$ uname -a
Linux mypc 4.2.0-42-generic #49~14.04.1-Ubuntu SMP Wed Jun 29 20:22:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ g++ --version
g++ (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4 ...
$ stap --version
Systemtap translator/driver (version 2.3/0.158, Debian version 2.3-1ubuntu1.4 (trusty))
OK, so this didn't turn out to be trivial - first of all, I somehow ended up with a (newer) kernel 4.2.0 on Ubuntu 14.04; and apparently the systemtap that comes with Ubuntu 14.04 is too old for that kernel (see below). That means that I had to build systemtap from source - this was my procedure:
cd /path/to/src
git clone git://sourceware.org/git/elfutils.git elfutils_git
git clone git://sourceware.org/git/systemtap.git systemtap_git
cd systemtap_git
./configure --with-elfutils=/path/to/src/elfutils_git --prefix=/path/to/src/systemtap_git/local --enable-docs=no
make
make install
# after this, there are `stap` executables in:
# /path/to/src/systemtap_git/stap
# /path/to/src/systemtap_git/local/bin/stap
This is the thing:
you shouldn't build elfutils separately, and then systemtap - you should instead pass the elfutils source directory to --with-elfutils of systemtap's configure, which will then configure and build elfutils as well.
you MUST do make install of systemtap, even if it is in a non-system/private (local) directory! - otherwise, some errors occur (unfortunately, didn't log them)
After building, stap reports version:
$ ./stap --version
Systemtap translator/driver (version 3.2/0.170, commit release-3.1-331-g0efba6fc74c8 + changes) ...
Ok, so I found a basic Fibonacci C++ example for analysis, which I slightly modified, and called /tmp/fibo.cpp:
// based on: http://www.cplusplus.com/articles/LT75fSEw/
#include <iostream>
using namespace std;
class Fibonacci{
public:
int a, b, c;
void generate(int);
void doFibonacciStep(int);
};
void Fibonacci::doFibonacciStep(int istep){
c = a + b;
cout << " istep: " << istep << " c: " << c << endl;
a = b;
b = c;
}
void Fibonacci::generate(int n){
a = 0; b = 1;
cout << " Start: a "<< a << " b " << b << endl;
for(int i=1; i<= n-2; i++){
doFibonacciStep(i);
}
}
int main()
{
cout << "Hello world! Fibonacci series" << endl;
cout << "Enter number of items you need in the series: ";
int n;
cin >> n;
Fibonacci fibonacci;
fibonacci.generate(n);
return 0;
}
First I tried compiling it like this:
cd /tmp
g++ -g fibo.cpp -o fibo.exe
Now, the first thing that we want to do, is to figure out which functions are available for probing in our executable; for that, we can use stap -L (note, here I'm still using the old, Ubuntu 14.04 system stap):
$ stap -L 'process("/tmp/fibo.exe").function("*").call'
process("/tmp/fibo.exe").function("_GLOBAL__sub_I__ZN9Fibonacci15doFibonacciStepEi").call
process("/tmp/fibo.exe").function("__static_initialization_and_destruction_0").call $__initialize_p:int $__priority:int
process("/tmp/fibo.exe").function("doFibonacciStep#/tmp/fibo.cpp:13").call $this:class Fibonacci* const $istep:int
process("/tmp/fibo.exe").function("generate#/tmp/fibo.cpp:20").call $this:class Fibonacci* const $n:int
process("/tmp/fibo.exe").function("main#/tmp/fibo.cpp:28").call
Nice - so I'd like to probe/trace the doFibonacciStep and its input argument, istep. So I try from the command line:
$ sudo stap -e 'probe process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep").call { printf("stap do step: %d\n", $istep) }' -c /tmp/fibo.exe
WARNING: "__tracepoint_sched_process_fork" [/tmp/stap51A5tV/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko] undefined!
WARNING: "__tracepoint_sys_exit" [/tmp/stap51A5tV/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko] undefined!
WARNING: "__tracepoint_sys_enter" [/tmp/stap51A5tV/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko] undefined!
WARNING: "__tracepoint_sched_process_exec" [/tmp/stap51A5tV/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko] undefined!
WARNING: "__tracepoint_sched_process_exit" [/tmp/stap51A5tV/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko] undefined!
ERROR: Couldn't insert module '/tmp/stap51A5tV/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko': Unknown symbol in module
WARNING: /usr/bin/staprun exited with status: 1
Pass 5: run failed. [man error::pass5]
Tip: /usr/share/doc/systemtap/README.Debian should help you get started.
$ sudo stap -e 'probe process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep").call { printf("stap do step: %d\n", $istep) }' -c /tmp/fibo.exe
ERROR: Couldn't insert module '/tmp/stapmo60OW/stap_ab5b824c79b38b5207910696c49c4e22_1760.ko': Unknown symbol in module
WARNING: /usr/bin/staprun exited with status: 1
Pass 5: run failed. [man error::pass5]
Ouch, errors like these - not good. The post "__tracepoint_sched_process_fork undefined" when run systemstap script explains that basically the stap version is too old for the kernel that I have - which required the building from source (above). So let's see now how the new stap -L works:
$ /path/to/src/systemtap_git/stap -L 'process("/tmp/fibo.exe").function("*").call'
process("/tmp/fibo.exe").function("_GLOBAL__sub_I__ZN9Fibonacci15doFibonacciStepEi#/tmp/fibo.cpp:37").call
process("/tmp/fibo.exe").function("__do_global_dtors_aux").call
process("/tmp/fibo.exe").function("__libc_csu_fini").call
process("/tmp/fibo.exe").function("__libc_csu_init").call
process("/tmp/fibo.exe").function("__static_initialization_and_destruction_0#/tmp/fibo.cpp:37").call $__initialize_p:int $__priority:int
process("/tmp/fibo.exe").function("_fini").call
process("/tmp/fibo.exe").function("_init").call
process("/tmp/fibo.exe").function("_start").call
process("/tmp/fibo.exe").function("deregister_tm_clones").call
process("/tmp/fibo.exe").function("doFibonacciStep#/tmp/fibo.cpp:13").call $this:class Fibonacci* const $istep:int
process("/tmp/fibo.exe").function("frame_dummy").call
process("/tmp/fibo.exe").function("generate#/tmp/fibo.cpp:20").call $this:class Fibonacci* const $n:int
process("/tmp/fibo.exe").function("main#/tmp/fibo.cpp:28").call
process("/tmp/fibo.exe").function("register_tm_clones").call
Nice, this is already a bit more verbose than the old version. Anyways, I'd like to probe the doFibonacciStep function, and its input argument, here $istep. So I write this on the command line:
$ sudo /path/to/src/systemtap_git/stap -e 'probe process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep").call { printf("stap do step: %d\n", $istep) }' -c /tmp/fibo.exe
semantic error: while processing probe process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep#/tmp/fibo.cpp:13").call from: process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep").call
semantic error: No cfa_ops supplied, but needed by DW_OP_call_frame_cfa: identifier '$istep' at <input>:1:107
source: probe process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep").call { printf("stap do step: %d\n", $istep) }
Pass 2: analysis failed. [man error::pass2]
Ouch - a nasty error, and doesn't really tell me anything - there are very few bug reports on this error (and mostly from 2010). So I was about to get stuck here, when for some reason, I remembered that the other day, I compiled some programs with -gdwarf-2 (for reasons I've forgotten by now); so I thought I'd try it - and whaddayaknow, it actually started working now:
$ g++ -gdwarf-2 fibo.cpp -o fibo.exe
$ sudo /path/to/src/systemtap_git/stap -e 'probe process("/tmp/fibo.exe").function("Fibonacci::doFibonacciStep").call { printf("stap do step: %d\n", $istep) }' -c /tmp/fibo.exe
Hello world! Fibonacci series
Enter number of items you need in the series: 5
Start: a 0 b 1
istep: 1 c: 1
istep: 2 c: 2
istep: 3 c: 3
stap do step: 1
stap do step: 2
stap do step: 3
Nice! Note that the stap prints are actually printed after the program has finished (that is, they are not interleaved with the actual program output where they occured).
Instead of specifying the probe points and behavior directly on the command line, we could write a script instead - so here is check-do-step.stp - here with some extra stuff:
#!/usr/bin/env stap
global stringone = "Testing String One"
global stringtwo = "Testing String Two"
probe begin {
printf("begin: %s\n", stringone)
#exit() # must have; else probe end runs only upon Ctrl-C if we only have `begin` and `end` probes!
}
probe process(
"/tmp/fibo.exe"
).function(
"Fibonacci::doFibonacciStep"
).call {
printf("stap do step: %d\n", $istep)
}
probe end {
newstr = "We Are " . stringtwo . " And We're Done" # string concat
printf("%s\n", newstr)
}
... and with this script, our call and results look like this:
$ sudo /path/to/src/systemtap_git/stap check-do-step.stp -c /tmp/fibo.exe
Hello world! Fibonacci series
Enter number of items you need in the series: begin: Testing String One
6
Start: a 0 b 1
istep: 1 c: 1
istep: 2 c: 2
istep: 3 c: 3
istep: 4 c: 5
stap do step: 1
stap do step: 2
stap do step: 3
stap do step: 4
We Are Testing String Two And We're Done
Notice again - the begin: Testing ... string does not hit at the very start as we'd otherwise expect, but only after the program already started generating output.
Well, I guess this is it - certainly good enough for me, for a simple example...
Minimal code of the bigger problem:
struct S { int i; };
typedef int (S::*Type);
Type foo (int) { return &S::i; }
#define FOO(X) *foo(X)
int main ()
{
S s;
s.*foo(0) = 0; // ok
s.FOO(0) = 0; // error <--- ??
}
If foo() method is replaced with FOO() macro to avoid '*', then it results in the error posted in title. When I checked the preprocessing using g++ -E option, then both the "ok" & "error" lines look same.
Why is this error with macro?
With clang 3.8 I got the next output for your program:
# 1 "test.cpp"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 325 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "test.cpp" 2
struct S { int i; };
typedef int (S::*Type);
Type foo (int) { return &S::i; }
int main ()
{
S s;
s.*foo(0) = 0;
s. *foo(0) = 0;
}
One can see the space in the line:
s. *foo(0) = 0;
This space is the reason for "expected the unqualified-id before..." error. The space itself should be a product of token spacing.
I do not know why g++ does not show the space. Possibly it's a compiler bug concerning representing the output of preprocessing.
Having this string s=";123;;123;;456;;124;;123;;567;" in R, which shows some Ids separated by ";", I want to find the repeated IDs, so in this case ";123;" is repeated. I used the following command in R:
gregexpr("(;[1-9]+;).*\1", s)
but it doesn't find the repeated patterns. Any idea what is wrong?
One example of a long string:
1760381;;1774536;;1774614;;1774617;;1774705;;1774723;;1775013;;1902321;;1928678;;2105486;;2105514;;2105544;;2105575;;2105585;;2279115;;2379236;;290927;;542280;;555749;;641540;;683822;;694934;;713228;;713248;;713249;;726949;;727204;;731434;;754522;;7693856;;100095;;1003838;;1045582;;1079057;;1108697;;1231229;;124087;;1249672;;1328126;;1412065;;1419930;;1441743;;1470580;;1476585;;1502106;;1556149;;1637775;;1643922;;1655644;;1755547;;1759001;;1760295;;1760296;;1760320;;1760326;;1760338;;1760348;;1760349;;1760350;;1760353;;1760375;;1760376;;1760377;;1760378;;1760388;;1760401;;1760402;;1760403;;1760410;;1760421;;1760425;;1760426;;1760642;;1760654;;1770463;;1774365;;1774366;;1774394;;1774449;;1774453;;1774454;;1774455;;1774456;;1774457;;1774458;;1774461;;1774462;;1774463;;1774464;;1774466;;1774469;;1774504;;1774505;;1774506;;1774519;;1774520;;1774525;;1774527;;1774529;;1774532;;1774533;;1774539;;1774542;;1774593;;1774595;;1774604;;1774610;;1774616;;1774617;;1774641;;1774660;;1774671;;1774674;;1774684;;1774687;;1774694;;1774704;;1774706;;1774713;;1774717;;1774722;;1774723;;1774726;;1774733;;1774745;;1774750;;1774753;;1774754;;1774766;;1774784;;1774786;;1774795;;1774799;;1774800;;1774803;;1774809;;1774813;;1774835;;1774849;;1774852;;1774853;;1774854;;1774857;;1774858;;1774861;;1774862;;1774867;;1774868;;1774869;;1774870;;1774877;;1774878;;1774880;;1774884;;1774885;;1774886;;1774902;;1774905;;1774934;;1774935;;1774937;;1774939;;1774946;;1774949;;1774950;;1774958;;1774959;;1774960;;1774961;;1774962;;1774964;;1774965;;1774966;;1774967;;1774969;;1774971;;1774972;;1774973;;1774975;;1774977;;1774978;;1774999;;1775000;;1775003;;1775005;;1775006;;1775009;;1775013;;1775014;;1775017;;1775024;;1775026;;1775033;;1775038;;1775040;;1775041;;1775044;;1775087;;1785544;;1811645;;1837210;;1864356;;1928674;;1928678;;1932882;;1954203;;2066856;;2076876;;2105349;;2105351;;2105458;;2105464;;2105476;;2105480;;2105482;;2105484;;2105489;;2105496;;2105500;;2105510;;2105514;;2105518;;2105532;;2105545;;2105550;;2172257;;2172762;;218438;;2228198;;2229827;;2247909;;2262250;;2263135;;2287260;;2335872;;2335873;;2335874;;2335877;;2338682;;2352560;;2420902;;263946;;265370;;303060;;330571;;338764;;387492;;387750;;388362;;431807;;436056;;436442;;444058;;458026;;491696;;504783;;513098;;529228;;539799;;549649;;559957;;562574;;563116;;576418;;582851;;592273;;599952;;614463;;626416;;645122;;652363;;665854;;668048;;682877;;683822;;688317;;709795;;710684;;723114;;724447;;724526;;725177;;731389;;731434;;876958;;879962;;947924;;987322;;987446;;61326;;1025952;;1095970;;1338018;;1349990;;1373122;;1419930;;1760310;;1760320;;1774705;;1774706;;1774708;;1774712;;1774952;;1774954;;1774963;;1774972;;1774977;;1775077;;1901075;;2022080;;2117779;;2143723;;441554;;450517;;549649;;1010402;;113311;;1148258;;1374348;;1419930;;1606449;;1606515;;1606608;;1606610;;1760320;;1760338;;1760618;;1760642;;1774504;;1774520;;1774595;;1774705;;1774909;;1774977;;1775011;;1775043;;179542;;1928678;;2105598;;2105721;;2188303;;2335873;;340762;;387759;;436442;;504783;;588336;;646185;;682877;;715644;;725080;;741661;;760924
m<-gregexpr("[0-9]+",s)
n<-regmatches(s,m)
[[1]]
[1] "123" "123" "456" "124" "123" "567"
data.frame(table(unlist(n)))
Var1 Freq
1 123 3
2 124 1
3 456 1
4 567 1
The code works for your long form string too: Here is the head and tail of the output:
head(data.frame(table(unlist(n))),10)
Var1 Freq
1 100095 1
2 1003838 1
3 1010402 1
4 1025952 1
5 1045582 1
6 1079057 1
7 1095970 1
8 1108697 1
9 113311 1
10 1148258 1
tail(data.frame(table(unlist(n))),10)
Var1 Freq
316 731434 2
317 741661 1
318 754522 1
319 760924 1
320 7693856 1
321 876958 1
322 879962 1
323 947924 1
324 987322 1
325 987446 1
1) In the examples the ids are all the same length so we assume that is a general feature. Try this pattern where (?=...) is a zero width lookahead expression (see ?regex)
pat <- ";([1-9]+);(?=.*\\1)"
gregexpr(pat, s, perl = TRUE)
or this:
library(gsubfn)
strapply(s, pat, perl = TRUE)[[1]]
## [1] "123" "123"
This lists each id one fewer times than its occurrence (zero times for ids not duplicated) in s so to list each duplicated id uniquely try unique(st) where st is the result of this last line of code above.
Note: In the second example in the question, i.e. the long string, there is no ; at the end of the string so the last id can never be matched by the expression unless we first paste a ; onto the end.
2) Instead of matching the contents we could match the delimiters instead:
strsplit(s, ";")[[1]])[-1]
If st is the result of this line of code then st is just a vector of all the ids so unique(st[duplicated[st]) uniquely lists each duplicated id and involves no regular expressions.