C++ Macro Indirection - c++

Question

Could someone please explain why this works:
#include <iostream>
using namespace std;
#define cat(a,b) cat_1(a,b)
#define cat_1(a,b) a ## b
int main()
{
cat(c,cat(o,cat(u,t))) << "Hello world!";
return 0;
}
but the same code with one less level of macro indirection does not:
#include <iostream>
using namespace std;
#define cat(a,b) a ## b
int main()
{
cat(c,cat(o,cat(u,t))) << "Hello world!";
return 0;
}
I've looked at this:
http://www.boost.org/doc/libs/1_55_0/libs/preprocessor/doc/
but while it illustrates the problem I still don't understand how this solves it. When I run the preprocessor on this (g++ -E):
#include <iostream>
using namespace std;
#define cat(a,b) cat_1(a,b)
int main()
{
cat(c,cat(o,cat(u,t))) << "Hello world!";
return 0;
}
it expands the line to:
cat_1(c,cat_1(o,cat_1(u,t))) << "Hello world!"
so it looks like the problem should still be there since it maps directly to the line with just 'cat'

Answer 1

## operator is applied before the macro substitution is re-scanned for more macro invocations. The outer invocation first expands to ccat(o,cat(u,t)), then to ccat(o, ut) and stops there.
Extra indirection allows re-scanning to work before token pasting.

Answer 2

Short answer: the presence of an ## operator in a macro definition stops the normal cascading substitution of macro parameters near it.
Take a subset of the first (working) version:
#define cat(a,b) cat_1(a,b)
#define cat_1(a,b) a ## b
cat(o,cat(u,t))
The last line is tokenised:
cat ( o , cat ( u , t ) )
The cat and ( tokens indicate the start of a call to the cat() macro, so the preprocessor starts replacing it with the definition of cat():
cat_1(
at this point it has to replace "a" with the argument passed in (ie. "o"), so it continues:
cat_1(o,
now it has to replace the parameter "b" with the argument passed in (ie. "cat(u,t))"), but this argument is itself a macro call, so that gets expanded before substitution to "cat_1(u,t)" and then to "u ## t" and finally to "ut", so, getting back to the top level, we end up with:
cat_1(o,ut)
which is the re-scanned, turning into:
o ## ut
and finally to
out
as expected.
In the non-working case, the rule about non-expansion near the ## comes into play:
#define cat(a,b) a ## b
cat(o,cat(u,t))
This time when the preprocessor starts replacing the outer cat() call, it immediately encounters the parameter "a" and has to replace it with the passed argument "o", which is fine, following by the ##:
o ##
Now it gets to "b" which it must replace with the argument "cat(u,t)". However, unlike the working example above, this time, the argument isn't recursively expanded because, according to the C Standard, parameters immediately preceding or following a ## operator must not be recursively expanded. So, it just leaves the "cat(u,t)" as it got it and ends up with:
o ## cat(u,t)
which is then collapsed into
ocat(u,t)
which is where the preprocessor stops, since it doesn't know about "ocat".
The ## (and #) preprocessor operators stopping recursive parameter expansion is set out in section 6.10.3.1 of the C Standard.