Why is token replacement done before macro expansion? - c++

Considering the following snippet:
#include <stdio.h>
#define MY_MACRO(\
arg) \
arg
#define MY_MACRO2(t1, t2) t1##t2
#define MY_ a
#define MACRO b
int main() {
printf("%d\n", MY_MACRO2(MY_,MACRO)(45));
return 0;
}
It turns out to compile and display 45, however, if MY_ and MACRO were expanded before substitution, this code should not compile.
The reason why I notice this is when I read in the C standard the following:
6.10.3.1 (but also in C++ standard)
After the arguments for the invocation of a function-like macro have been identified,argument substitution takes place.A parameter in the replacement list, unless preceded by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is replaced by the corresponding argument after all macros contained therein have been expanded. Before being substituted, each argument’s preprocessing tokens are completely macro replaced as if they formed the rest of the preprocessing file; no other preprocessing tokens are available
So if all macros contained in the arguments were expanded before replacement, why don't we end up with ab(45)?

To let constructions like X(X()) work. Note that while X() is expanded the X macro is disabled to avoid infinite recursions. Expanding arguments before expanding the macro let's one use X in the arguments.
A practical application of X(X()):
#define TEN(x) x x x x x x x x x x
#define HUNDRED(x) TEN(TEN(x))

Related

Replacing __LINE__ macro

I read about macros using Cppreference.
__LINE__ : expands to the source file line number, an integer constant, can be changed by the #line directive
I made c++ program to test __LINE__ macro.
#include <iostream>
using namespace std;
#line 10
#define L __LINE__
int main()
{
#line 20
int i = L;
cout<<i<<endl;
return 0;
}
Output :
20
Why output of the above code is 20? Why does not 10?
If you want to print 10 then change L into something that is not a macro:
constexpr int L = __LINE__;
Otherwise the macro L would be substituted on the line int i = L; and become:
int i = __LINE__;
Where it will have to be substituted again for the line number, and read the last #line directive.
Recall that macros perform token substitution. When you #define L __LINE__ it only specifies what tokens should L be substituted for when it appears in the source. It does not substitute anything at the point of L's own definition.
C++ - [cpp.replace]/9 or C - [6.10.3 Macro replacement]/9
A preprocessing directive of the form
# define identifier replacement-list new-line
defines an object-like macro that causes each subsequent instance of
the macro name to be replaced by the replacement list of preprocessing
tokens that constitute the remainder of the directive. The replacement
list is then rescanned for more macro names as specified below.
#define y 42
#define x y
This makes x defined as a sequence of preprocessing tokens that contains one token y. Not the token 42.
cout << x;
This will expad x to y and then y to 42.
#undef y
#define y "oops"
x is still defined as y.
cout << x;
You guess what happens. __LINE__ isn't special in this regard.
Your #line preprocessor directive changes the value to 20:
#line 20
The macro is expanded where used (not where defined) which is in your main() function, after the preprocessor directive which changes the value to 20.

trying to understand syntax [duplicate]

What does this line mean? Especially, what does ## mean?
#define ANALYZE(variable, flag) ((Something.##variable) & (flag))
Edit:
A little bit confused still. What will the result be without ##?
A little bit confused still. What will the result be without ##?
Usually you won't notice any difference. But there is a difference. Suppose that Something is of type:
struct X { int x; };
X Something;
And look at:
int X::*p = &X::x;
ANALYZE(x, flag)
ANALYZE(*p, flag)
Without token concatenation operator ##, it expands to:
#define ANALYZE(variable, flag) ((Something.variable) & (flag))
((Something. x) & (flag))
((Something. *p) & (flag)) // . and * are not concatenated to one token. syntax error!
With token concatenation it expands to:
#define ANALYZE(variable, flag) ((Something.##variable) & (flag))
((Something.x) & (flag))
((Something.*p) & (flag)) // .* is a newly generated token, now it works!
It's important to remember that the preprocessor operates on preprocessor tokens, not on text. So if you want to concatenate two tokens, you must explicitly say it.
## is called token concatenation, used to concatenate two tokens in a macro invocation.
See this:
Macro Concatenation with the ## Operator
One very important part is that this token concatenation follows some very special rules:
e.g. IBM doc:
Concatenation takes place before any
macros in arguments are expanded.
If the result of a concatenation is a
valid macro name, it is available for
further replacement even if it
appears in a context in which it
would not normally be available.
If more than one ## operator and/or #
operator appears in the replacement
list of a macro definition, the order
of evaluation of the operators is not
defined.
Examples are also very self explaining
#define ArgArg(x, y) x##y
#define ArgText(x) x##TEXT
#define TextArg(x) TEXT##x
#define TextText TEXT##text
#define Jitter 1
#define bug 2
#define Jitterbug 3
With output:
ArgArg(lady, bug) "ladybug"
ArgText(con) "conTEXT"
TextArg(book) "TEXTbook"
TextText "TEXTtext"
ArgArg(Jitter, bug) 3
Source is the IBM documentation. May vary with other compilers.
To your line:
It concatenates the variable attribute to the "Something." and adresses a variable which is logically anded which gives as result if Something.variable has a flag set.
So an example to my last comment and your question(compileable with g++):
// this one fails with a compiler error
// #define ANALYZE1(variable, flag) ((Something.##variable) & (flag))
// this one will address Something.a (struct)
#define ANALYZE2(variable, flag) ((Something.variable) & (flag))
// this one will be Somethinga (global)
#define ANALYZE3(variable, flag) ((Something##variable) & (flag))
#include <iostream>
using namespace std;
struct something{
int a;
};
int Somethinga = 0;
int main()
{
something Something;
Something.a = 1;
if (ANALYZE2(a,1))
cout << "Something.a is 1" << endl;
if (!ANALYZE3(a,1))
cout << "Somethinga is 0" << endl;
return 1;
};
This is not an answer to your question, just a CW post with some tips to help you explore the preprocessor yourself.
The preprocessing step is actually performed prior to any actual code being compiled. In other words, when the compiler starts building your code, no #define statements or anything like that is left.
A good way to understand what the preprocessor does to your code is to get hold of the preprocessed output and look at it.
This is how to do it for Windows:
Create a simple file called test.cpp and put it in a folder, say c:\temp.
Mine looks like this:
#define dog_suffix( variable_name ) variable_name##dog
int main()
{
int dog_suffix( my_int ) = 0;
char dog_suffix( my_char ) = 'a';
return 0;
}
Not very useful, but simple. Open the Visual studio command prompt, navigate to the folder and run the following commandline:
c:\temp>cl test.cpp /P
So, it's the compiler your running (cl.exe), with your file, and the /P option tells the compiler to store the preprocessed output to a file.
Now in the folder next to test.cpp you'll find test.i, which for me looks like this:
#line 1 "test.cpp"
int main()
{
int my_intdog = 0;
char my_chardog = 'a';
return 0;
}
As you can see, no #define left, only the code it expanded into.
According to Wikipedia
Token concatenation, also called token pasting, is one of the most subtle — and easy to abuse — features of the C macro preprocessor. Two arguments can be 'glued' together using ## preprocessor operator; this allows two tokens to be concatenated in the preprocessed code. This can be used to construct elaborate macros which act like a crude version of C++ templates.
Check Token Concatenation
lets consider a different example:
consider
#define MYMACRO(x,y) x##y
without the ##, clearly the preprocessor cant see x and y as separate tokens, can it?
In your example,
#define ANALYZE(variable, flag) ((Something.##variable) & (flag))
## is simply not needed as you are not making any new identifier. In fact, compiler issues "error: pasting "." and "variable" does not give a valid preprocessing token"

Using boost preprocessor for token comparison

I found a page that it is explained how identifier-token comparison can be implemented using c preprocessor directives. This could be done by some macros like this:
#define COMPARE_foo(x) x
#define COMPARE_bar(x) x
#define PRIMITIVE_COMPARE(x, y) IS_PAREN \
( \
COMPARE_ ## x ( COMPARE_ ## y) (()) \
)
PRIMITIVE_COMPARE(foo, bar) // expands to 0
PRIMITIVE_COMPARE(bar, bar) // expands to 1
Which IS_PAREN checks that it is completely expanded or not(which occurs just when two macros are different because of painting blue).
Now I am looking for a similar command in Boost Preprocessor. I want to have a list of accepted types of a macro and if the macro called with one of this type it expands to what it must otherwise it does not anything. My pseudo code is something like this:
#define ACCEPTED_TYPE (float)(int)(string)
#define Macro(x) // If one of accepted type do a otherwise do nothing
If boost preprocessor has not the exact solution what is your suggestion to make implementation easier.

Repeated Class Instantiation without using Identifier

I want to repeatedly instantiate a class, within module scope, without providing a unique name. Something like this.
MyClass name##__LINE__(); // doesn't work because __LINE__ won't stringify
MyClass name##__LINE__(); // duplicate identifier error - two name__LINE__ variables
Is there some way to do this, either creating a unique name or using some anonymous context, such as an initializer or struct?
See this answer
#define CONCATENATE_DETAIL(x, y) x##y
#define CONCATENATE(x, y) CONCATENATE_DETAIL(x, y)
#define MAKE_UNIQUE(x) CONCATENATE(x, __LINE__)
MyClass MAKE_UNIQUE(name);
MyClass MAKE_UNIQUE(name);
...
Or just make an array:
MyClass arr[N];
Why these macros work
C11 standard, 6.10.3.1 Argument substitution:
After the arguments for the invocation of a function-like macro have been identified,
argument substitution takes place. A parameter in the replacement list, unless preceded
by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is
replaced by the corresponding argument after all macros contained therein have been
expanded. Before being substituted, each argument’s preprocessing tokens are
completely macro replaced as if they formed the rest of the preprocessing file; no other
preprocessing tokens are available.
Corresponding paragraph in C++ standard (16.3.1 Argument substitution) is exact copy of C standard's.
You need to double nest the concatenation operator
struct A{};
#define JOIN(X, Y) JOIN_DETAIL(X, Y)
#define JOIN_DETAIL(X, Y) JOIN_DETAIL2(X, Y)
#define JOIN_DETAIL2(X, Y) X##Y
int main() {
A JOIN(a, __LINE__);
A JOIN(a, __LINE__);
}

How to compose valid token depending on pre-defined macros (C++ preprocessor)?

Suppose some tokens FOO, BAR, and DUD (and possibly more) are #defined or not. I would like to have a macro EXTEND(name) that generates a valid extended name, e.g.
#define FOO
#undef BAR
#define DUD
EXTEND(object)
expands to
object_foo_dud
Is it possible to write macro EXTEND with less than O(2n) lines if there are n macro tokens (like FOO, BAR, and DUD)? I think it should be possible with O(n) lines, but how?
I've tried this:
#ifdef FOO
# define ExtFOO(name) name ## _foo
#else
# define ExtFOO(name) name
#endif
#ifdef BAR
# define ExtBAR(name) ExtFOO(name) ## _bar
#else
# define ExtBAR(name) ExtFOO(name)
#endif
#ifdef DUD
# define ExtDUD(name) ExtBAR(name) ## _dud
#else
# define ExtDUD(name) ExtBAR(name)
#endif
#define EXTEND(name) ExtDUD(name)
but
test.cc:26:5: error: pasting formed ')_dud', an invalid preprocessing token
EXTEND(object)
^
The ## operator concatenates two preprocessing tokens and must yield a single valid token. For example, from section 6.10.3.3 of the C99 spec:
For both object-like and function-like macro invocations, before the replacement list is
reexamined for more macro names to replace, each instance of a ## preprocessing token
in the replacement list (not from an argument) is deleted and the preceding preprocessing
token is concatenated with the following preprocessing token. Placemarker preprocessing tokens are handled specially: concatenation of two placemarkers results in a single placemarker preprocessing token, and concatenation of a placemarker with a non-placemarker preprocessing token results in the non-placemarker preprocessing token. If the result is not a valid preprocessing token, the behavior is undefined. The resulting token is available for further macro replacement. The order of evaluation of ## operators is unspecified.
So the expansion ExtBAR(name) ## _dud is invalid since it would yield ExtBAR(object)_dud.
I'd go with the following approach:
#ifdef FOO
# define ValFOO _foo
#else
# define ValFOO
#endif
#ifdef BAR
# define ValBAR _bar
#else
# define ValBAR
#endif
#ifdef DUD
# define ValDUD _dud
#else
# define ValDUD
#endif
#define CONCAT(a, b, c, d) a ## b ## c ## d
#define XCONCAT(a, b, c, d) CONCAT(a, b, c, d)
#define EXTEND(name) XCONCAT(name, ValFOO, ValBAR, ValDUD)
The intermediate XCONCAT step is needed because macro arguments are not expanded if they're concatenated using ##.