CWG3224 [cpp.replace.general] p14: Clarification of directive-like preprocessing tokens in macro arguments

Full name of submitter (unless configured in github; will be published with the issue): Yin Xinyu

Reference (section label): [cpp.replace.general]

Issue description:

[cpp.replace.general]/14 says:

If there are sequences of preprocessing tokens within the list of arguments that would otherwise act as preprocessing directives, the program is ill-formed.

The intention is possibly avoiding multi-line macro invocations that cross text lines and preprocessing directives ([cpp.pre]). Consider:

// ill-formed
#define FOO(...)
FOO(                // text-line
#include <iostream> // pp-directive
)                   // text-line

Although it's impossible to perform a multi-line macro invocation starting at a preprocessing directive, the current wording would accept the following

// well-formed
#define FOO(...)
FOO(                // text-line
#bluh ) bluh        // pp-directive

since the conditionally-supported directive #bluh ) bluh is not within the list of arguments consisting of merely #bluh.

Implementations have no consensus on how to deal with this case (godbolt). By the way they have not even diagnosed the first example as ill-formed.

This can result in an even weirder situation, although results are all ill-formed. Consider:

#define FOO(...)
#define LPAREN (
#define NIL
FOO(                // text-line
#if LPAREN X ) NIL  // pp-directive
#endif              // pp-directive

and

#define FOO(...)
#ifdef X
FOO(                // text-line
#else ))            // pp-directive
#endif              // pp-directive

It's hard to tell what would happen, where X may be in different branches.

To avoid these complex and anti-intuitive circumstances, we'd better edit the wording of [cpp.replace.general]/14 to forbid these cases.

If an application such as stringization is needed, the following is well-formed.

#define STR(x) STR2(x)
#define STR2(x) #x
#define HASH #
#define NIL
STR(
    HASH include <iostream>
)
STR(
    NIL #include <iostream>
)

The arguments in STR2 is safe, considering [cpp.rescan]/4

The resulting completely macro-replaced preprocessing token sequence is not processed as a preprocessing directive even if it resembles one, but all pragma unary operator expressions within it are then processed as specified in [cpp.pragma.op] below.

Suggested resolution:

Change [cpp.replace.general]/14:

If there are sequences of preprocessing tokens within the list of arguments that , along with all preprocessing tokens immediately following them, would otherwise act as preprocessing directives, the program is ill-formed.

What's the behavior of existing implementations for these examples?

In any case, I don't think those two rules conflict, but they are complementary. [cpp.replace.general] p14 is talking about the argument list (before any replacement happens). [cpp.rescan] p4 is talking about the result of the replacement and the rescanning. At that stage, preprocessing directives are no longer recognized (and the token sequence survives until phase 6, where they are ill-formed, because # is not a valid phase 7 token).

In any case, I don't think those two rules conflict, but they are complementary. [cpp.replace.general] p14 is talking about the argument list (before any replacement happens). [cpp.rescan] p4 is talking about the result of the replacement and the rescanning. At that stage, preprocessing directives are no longer recognized (and the token sequence survives until phase 6, where they are ill-formed, because # is not a valid phase 7 token).

That's exactly my viewpoint. However even though the first ECHO example would be ill-formed afterward, the second STR example works.

What's the behavior of existing implementations for these examples?

See godbolt. It seems that their preprocessors remain to respect the old specification ([cpp.replace.general]/2 used to be a UB) so they accept both STR(#py comment) and STR(PY comment).

Edit: adding C++ version flag (-std=c++26 etc.) doesn't result in any change.

the old specification ([cpp.replace.general]/2 used to be a UB)

Ah, we changed this in "P2843R3 Preprocessing is never undefined". My guess is implementations haven't caught up to this change. Feel free to post bug reports for

STR(#py comment)  // ill-formed

the old specification ([cpp.replace.general]/2 used to be a UB)

Ah, we changed this in "P2843R3 Preprocessing is never undefined". My guess is implementations haven't caught up to this change. Feel free to post bug reports for
STR(#py comment)  // ill-formed

That's why I post this issue: is STR(PY comment) well-formed after P2843R2 instead of being an edge case of a UB? In the past, whether STR(PY comment) is well-formed depends on a UB so it doesn't matter at all.

P2843 wanted to remove UB from the preprocessor.

Given the current state of the wording in the standard, is there any reason to believe that STR(PY comment) would be ill-formed?

P2843 wanted to remove UB from the preprocessor.

Given the current state of the wording in the standard, is there any reason to believe that STR(PY comment) would be ill-formed?

I think STR(PY comment) is well-formed according to the current standard. I wonder if this case needs an informative note or example, since it used to associate with a UB and now the UB is removed but this case becomes well-formed.

Why was it UB? [cpp.replace.general]/14 used to say "UB" (instead of ill-formed), but we agreed this rule is not triggered by your example.

I found another ambiguity here.

If there are sequences of preprocessing tokens within the list of arguments that would otherwise act as preprocessing directives, the program is ill-formed.

Does it mean that there shall be no # within the list of arguments since a single # suffices to form a null directive?

Or we should consult [cpp.pre]/2 and whitespaces matter?

A preprocessing directive consists of a sequence of preprocessing tokens that satisfies the following constraints: At the start of translation phase 4, the first preprocessing token in the sequence, referred to as a directive-introducing token , begins with the first character in the source file (optionally after whitespace containing no new-line characters) or follows whitespace containing at least one new-line character, and is

a # preprocessing token, or

...
The last preprocessing token in the sequence is the first preprocessing token within the sequence that is immediately followed by whitespace containing a new-line character.

Consider:

STR(
#
) // ill-formed
STR(#) // ill-formed?
STR(#
) // ill-formed?
STR(
#) // ill-formed?

STR(bluh
#
bluh) // ill-formed?
STR(bluh#) // ill-formed?
STR(bluh#if) // ill-formed?  (triggered by #, not by incomplete #if)
STR(bluh#
bluh) // ill-formed?
STR(bluh
#if) // ill-formed?

Should I open a new issue? Or may I edit the current issue to update this problem?

Why was it UB? [cpp.replace.general]/14 used to say "UB" (instead of ill-formed), but we agreed this rule is not triggered by your example.

Yes if we adopt my interpretation, then STR(PY comment) doesn't belong to the UB.

I found another ambiguity here.

If there are sequences of preprocessing tokens within the list of arguments that would otherwise act as preprocessing directives, the program is ill-formed.

Does it mean that there shall be no # within the list of arguments since a single # suffices to form a null directive?

Or we should consult [cpp.pre]/2 and whitespaces matter?

Yes, the "otherwise" means if macro expansion did not take place I believe. So only the examples that have the # on the start of a line are invalid.

Yes, the "otherwise" means if macro expansion did not take place I believe. So only the examples that have the # on the start of a line are invalid.

How about the ending condition?

The last preprocessing token in the sequence is the first preprocessing token within the sequence that is immediately followed by whitespace containing a new-line character.

Do you mean that only

STR(bluh
#
bluh)

is ill-formed? Or

STR(bluh
#if)

is also ill-formed? Note that here #if is imcomplete and cannot act as a preprocessing directive otherwise.

Yes, the "otherwise" means if macro expansion did not take place I believe. So only the examples that have the # on the start of a line are invalid.

How about the ending condition?

The last preprocessing token in the sequence is the first preprocessing token within the sequence that is immediately followed by whitespace containing a new-line character.

Do you mean that only

STR(bluh
#
bluh)
is ill-formed? Or

STR(bluh
#if)
is also ill-formed? Note that here #if is imcomplete and cannot act as a preprocessing directive otherwise.

With your previous example:

STR(
#
) // ill-formed
STR(#) // valid
STR(#
) // valid
STR(
#) // ill-formed

STR(bluh
#
bluh) // ill-formed
STR(bluh#) // valid
STR(bluh#if) // valid
STR(bluh#
bluh) // valid
STR(bluh
#if) // ill-formed

The #if) should still be considered a preprocessor directive, just an invalid one. Note that it cannot be treated as a normal text line "A sequence of preprocessing tokens is only a text-line if it does not begin with a directive-introducing token." [cpp.pre]/3. Since it is not a valid if-group, it appears that the whole file is simply not a valid preprocessing-file like the example after the quoted text.

As for that ending condition, I do not understand the point you are making. "sequences of preprocessing tokens within the list of arguments" does not imply that it has to be a whole argument, if that is what you were thinking of.

As for that ending condition, I do not understand the point you are making. "sequences of preprocessing tokens within the list of arguments" does not imply that it has to be a whole argument, if that is what you were thinking of.

Well [cpp.pre]/2 actually specifies two constraints on the "sequence of preprocessing tokens" that to be considered as a preprocessing directive:

The first pp token shall be at the start of a line (optionally after some whitespace containing no new-line).
The last pp token shall be immediately followed by some whitespace containing a new-line.

In my opinion, the second constraint is somewhat more important since otherwise there are two possible "sequences of preprocessing tokens" in

STR(
#bluh)

that would otherwise act as preprocessing directives (one is the null directive # and another is the conditionally-supported directive #bluh). Afterall, there is no constraint on when to stop if the second constraint is omiited.

My intention to use imcomplete #if is to distinguish whether the former case (# itself with immediately following pp tokens) is ill-formed. If the if-section causes complexity, one can use

STR(
#line) // is the # itself forms a sequence of pp tokens that would otherwise act as pp-directive?

instead to prohibit the latter case (#line needs more pp tokens) as a valid preprocessing directive otherwise. Don't you agree that "act as" may need the pp-directive to be valid? If we adopt this interpretation of "act as pp-directive", then # itself would make it ill-formed even though #line would not. However this divergence may result in no difference: nevertheless it would be ill-formed because of the appearance of# (my opinion) or because of the invalid but sufficient appearance of #line (your opinion).

As for that ending condition, I do not understand the point you are making. "sequences of preprocessing tokens within the list of arguments" does not imply that it has to be a whole argument, if that is what you were thinking of.

Well [cpp.pre]/2 actually specifies two constraints on the "sequence of preprocessing tokens" that to be considered as a preprocessing directive:

The first pp token shall be at the start of a line (optionally after some whitespace containing no new-line).

The last pp token shall be immediately followed by some whitespace containing a new-line.

In my opinion, the second constraint is somewhat more important since otherwise there are two possible "sequences of preprocessing tokens" in

STR(
#bluh)
that would otherwise act as preprocessing directives (one is the null directive # and another is the conditionally-supported directive #bluh). Afterall, there is no constraint on when to stop if the second constraint is omiited.

Does it matter which directive it is considering? Though I think it should be considering the whole #bluh), perhaps the wording should be adjusted to not imply that the whole directive needs to be in the arguments.

My intention to use imcomplete #if is to distinguish whether the former case (# itself with immediately following pp tokens) is ill-formed. If the if-section causes complexity, one can use

STR(
#line) // is the # itself forms a sequence of pp tokens that would otherwise act as pp-directive?
instead to prohibit the latter case (#line needs more pp tokens) as a valid preprocessing directive otherwise. Don't you agree that "act as" may need the pp-directive to be valid? If we adopt this interpretation of "act as pp-directive", then # itself would make it ill-formed even though #line would not. However this divergence may result in no difference: nevertheless it would be ill-formed because of the appearance of# (my opinion) or because of the invalid but sufficient appearance of #line (your opinion).

I do not think there is any reason to outright forbid # tokens entirely in preprocessing macros, that was surely never the intent. The intent as far as I understand is that if a macro invocation starts in a text line, it shall not cross over into lines other than text lines. For example:

#define FOO(...)
FOO(
    #include<iostream>
)

This is not valid, but FOO(#) has no issues with it being interpreted as a preprocessing directive so it does not make sense to forbid it. For macro invocations that start in lines other than text lines, there is also a constraint that they must be entirely contained within that line.

So, we've established that the original examples are broken because the # isn't at the start of a line (modulo whitespace).

Do you still want an example for this:

STR(
#
) // ill-formed

(and yes, the point here is the # symbol appearing before macro expansion)?

I do not think there is any reason to outright forbid # tokens entirely in preprocessing macros, that was surely never the intent. The intent as far as I understand is that if a macro invocation starts in a text line, it shall not cross over into lines other than text lines. For example:

#define FOO(...)
FOO(
#include<iostream>
)
This is not valid, but FOO(#) has no issues with it being interpreted as a preprocessing directive so it does not make sense to forbid it. For macro invocations that start in lines other than text lines, there is also a constraint that they must be entirely contained within that line.

I admit at the very beginning that this example (with both heading and trailing new-line) is ill-formed without doubt.

The detail #bluh) of your response reminds me that maybe you interpret [cpp.replace.general]/14

The sequence of preprocessing tokens bounded by the outside-most matching parentheses forms the list of arguments for the function-like macro...

as including the bounding parentheses. If so, then I can understand why you cannot tell the difference between the example showing above and the following

#define FOO(...)
FOO(
    #include<iostream>)

since I interpret it as excluding the bounding parentheses. For this example

either we include the parentheses (your opinion), realize #include <iostream>) is an invalid pp-directive, and its subsequences #include <iostream> or # do not satisfy the ending constarint,
or we exclude the parentheses (my opinion), then there is no sequence of pp-tokens that satisfies the ending constraint because of the ) followed immediately.

Thus whether an invalid pp-directive serves as "otherwise act as a pp-directive" does matter; it would determine the well-formedness in the former case.

Thus whether the ending constaint needs to be satisfied does matter; it would determine the well-formedness in the latter case.

It's worth saying that even if we include the parentheses, consider:

#define FOO(...)
FOO(
    #include<iostream>) FOO()

Then #include<iostream>) violates the ending constraint and perhaps is no longer to be a pp-directive.

However, I think the correct interpretation based on the context is exlcuding parentheses, since [cpp.replace.general]/14 continues as

...The individual arguments within the list are separated by comma preprocessing tokens, but comma preprocessing tokens between matching inner parentheses do not separate arguments.

If we including parentheses, then it seems that the individual arguments of MACRO(a,b) would be (a and b), which is absurd (see also CWG 2003).

Does it matter which directive it is considering? Though I think it should be considering the whole #bluh), perhaps the wording should be adjusted to not imply that the whole directive needs to be in the arguments.

Yes, the key point is whether the whole directive needs to be in the list of arguments. Since we'd better exlcude the bounding parentheses, if we moreover require the sequence of pp-tokens to satisfy the two constraints, then

MACRO(#) // well-formed
MACRO(#
) // well-formed
MACRO(
#) // well-formed
MACRO(
#
) // ill-formed

which may be a satisfying resolution.

So, we've established that the original examples are broken because the # isn't at the start of a line (modulo whitespace).

Do you still want an example for this:
STR(
#
) // ill-formed
(and yes, the point here is the # symbol appearing before macro expansion)?

A detailed example such as

MACRO(#) // well-formed
MACRO(#
) // well-formed
MACRO(
#) // well-formed
MACRO(
#
) // ill-formed
#define HASH #
MACRO(
HASH
) // well-formed

might be helpful.

MACRO(
#) // well-formed

#) is a conditionally-supported-directive.

MACRO(
#) // well-formed

#) is a conditionally-supported-directive.

As I said above, we'd better exclude the bounding parentheses, so

If there are sequences of preprocessing tokens within the list of arguments that would otherwise act as preprocessing directives, the program is ill-formed.

means that #) is NOT within the list of arguments which consists of merely a single #.

#define FOO(...)
FOO(
#include)
since I interpret it as excluding the bounding parentheses. For this example

either we include the parentheses (your opinion), realize #include <iostream>) is an invalid pp-directive, and its subsequences #include <iostream> or # do not satisfy the ending constarint,

or we exclude the parentheses (my opinion), then there is no sequence of pp-tokens that satisfies the ending constraint because of the ) followed immediately.

It is not really about specifically the closing ), I think the whole line should be interpreted as part of the directive. Consider the following:

#define LPAREN (
#define FOO(...)
#ifdef X
FOO(
#elif LPAREN 1)
42
#endif

Does this create an elif-group if and only if X is not defined? Or does it need to examine what FOO is even when X is not defined, therefore not actually ignoring the skipped tokens?

#) is a conditionally-supported-directive.

As I said above, we'd better exclude the bounding parentheses, so

Grammatically it forms a conditionally-supported-directive, it cannot be a text-line and it cannot be another form of directive. I do not see how it can be understood as anything else. The grammar of preprocessing files does not depend upon macro expansion.

Grammatically it forms a conditionally-supported-directive, it cannot be a text-line and it cannot be another form of directive. I do not see how it can be understood as anything else. The grammar of preprocessing files does not depend upon macro expansion.

I think I get your point. So you opinion is

MACRO(
#)

is well-formed, since it would not be parsed as a multi-line macro invocation (which can only happen within text lines) so it has nothing to do with [cpp.replace.general]/14?

But if so, [cpp.replace.general]/14 seems to be redundant since there is actually no ill-formed case. Consider:

MACRO(#) // well-formed macro invocation
MACRO(#
) // well-formed macro invocation
MACRO(
#) // well-formed conditionally-supported directive; no macro invocation here
MACRO(
#
) // well-formed null directive; no macro invocation here
MACRO(
#include <iostream>) // ill-formed #include directive; no macro invocation here

Is there any ill-formed case for which [cpp.replace.general]/14 is needed?

Furthermore, if this analysis does work, this UB was also never triggered before?

Grammatically it forms a conditionally-supported-directive, it cannot be a text-line and it cannot be another form of directive. I do not see how it can be understood as anything else. The grammar of preprocessing files does not depend upon macro expansion.

I think I get your point. So you opinion is

MACRO(
#)
is well-formed, since it would not be parsed as a multi-line macro invocation (which can only happen within text lines) so it has nothing to do with [cpp.replace.general]/14?

The semantics of #) are entirely up to the implementation, so I would say it is conditionally well-formed (that is how I understand conditionally supported) and can do absolutely anything. Programs intending to be strictly conforming cannot use it, even in skipped conditional inclusions (at least that is how Clang justifies supporting #elifdef in all language modes, when it would be a conditionally-supported-directive aka non-directive in earlier versions).

But if so, [cpp.replace.general]/14 seems to be redundant since there is actually no ill-formed case. Consider:

MACRO(#) // well-formed macro invocation
MACRO(#
) // well-formed macro invocation
MACRO(
#) // well-formed conditionally-supported directive; no macro invocation here
MACRO(
#
) // well-formed null directive; no macro invocation here

This is not valid:

#define F()
F(

It would not make sense that adding a preprocessing directive then a closing parenthesis after would suppress macro expansion.

This is not valid:

#define F()
F(
It would not make sense that adding a preprocessing directive then a closing parenthesis after would suppress macro expansion.

You are absolutely right. Now consider:

MACRO(#) // well-formed macro invocation
MACRO(#
) // well-formed macro invocation
MACRO(
#) // ill-formed?
MACRO(
#
) // well-formed null directive; ill-formed macro invocation

The remaining problem is that whether the following example is well-formed or not

#define FOO(...)
FOO(
#line )

depends on which directive goes first. If #line goes first then #line ) is invalid hence ill-formed. If #define FOO(...) goes first then FOO( leads a list of arguments consisting of merely #line (excluding parentheses). But #line itself does not form a preprocessing directive to trigger [cpp.replace.general]/14, while #line ) does but is not within the list of arguments.

Perhaps we should consider your suggestion on changing the wording of [cpp.replace.general]/14.

However, the if-sections coincidently work well. Consider:

#define LPAREN (
#define FOO(...)
#ifdef X
FOO(
#elif LPAREN 1)
(42
#endif
)

To parse a preprocessing-file first, one needs to know when to relax syntax rules. Therefore evaluation of #ifdef X is sequenced before macro invocations associated with #define FOO(...). When the first #ifdef X is met, we should see these directives above to determine whether the controling expression evaluates to zero. This split into two cases:

if X is not defined, then the if group is skipped and we try to recognize a strict #elif directive. In this case the FOO( is skipped and #elif LPAREN 1) is expanded to be valid #elif ( 1). This branch works well and only (42) survives;
if X is defined, then the if group is valid and the following #elif could have relaxed syntax. In this case the FOO( is not skipped and thus demands for a list of arguments. The following #elif directive ending with ) is skipped but #endif isn't ([cpp.cond]/13). So the result is FOO(\n#endif\n), which is ill-formed without doubt.

(If there is no ) after #endif and X is defined then FOO( has no terminating ), which is clearly ill-formed.)

Also consider:

#define LPAREN (
#define FOO(...)
FOO(
#if LPAREN X)
)
#else
)
#endif

The file is processed to contain either FOO(\n#if(1)\n) (if X nonzero) or FOO(\n#else\n) (if X zero) both ill-formed.

#define FOO(...)
FOO(
#line )

depends on which directive goes first. If #line goes first then #line ) is invalid hence ill-formed. If #define FOO(...) goes first then FOO( leads a list of arguments consisting of merely #line (excluding parentheses). But #line itself does not form a preprocessing directive to trigger [cpp.replace.general]/14, while #line ) does but is not within the list of arguments.

Perhaps we should consider your suggestion on changing the wording of [cpp.replace.general]/14.

If it is necessary to clearly forbid this, then I think it would be worth adding wording that the entire directive does not need to be contained within the arguments for this condition to occur. Though because this wording is from C (aside from the change to make it ill-formed instead of undefined) it would make sense to file an issue to WG14 too.

If it is necessary to clearly forbid this, then I think it would be worth adding wording that the entire directive does not need to be contained within the arguments for this condition to occur. Though because this wording is from C (aside from the change to make it ill-formed instead of undefined) it would make sense to file an issue to WG14 too.

I think there is a similar situation in [cpp.rescan]/1:

After all parameters in the replacement list have been substituted and # and ## processing has taken place, all placemarker preprocessing tokens are removed. Then the resulting preprocessing token sequence is rescanned, along with all subsequent preprocessing tokens of the source file, for more macro names to replace.

Suggested resolution:
Change [cpp.replace.general]/14:

If there are sequences of preprocessing tokens within the list of arguments that along with all preprocessing tokens immediately followed would otherwise act as preprocessing directives, the program is ill-formed.

I have editted the original issue to reveal the discussion.

CWG3224

Navigation Menu

Repository navigation

CWG3224 [cpp.replace.general] p14: Clarification of directive-like preprocessing tokens in macro arguments #975

Issue description:

Suggested resolution:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

CWG3224 [cpp.replace.general] p14: Clarification of directive-like preprocessing tokens in macro arguments #975

Description

Issue description:

Suggested resolution:

Activity

jensmaurer commented on Aug 14, 2026

jensmaurer commented on Aug 14, 2026

BlowingWind314 commented on Aug 15, 2026

jensmaurer commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

jensmaurer commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

jensmaurer commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

Halalaluyafail3 commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

Halalaluyafail3 commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

Halalaluyafail3 commented on Aug 15, 2026

jensmaurer commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

Halalaluyafail3 commented on Aug 15, 2026

BlowingWind314 commented on Aug 15, 2026

Halalaluyafail3 commented on Aug 15, 2026

BlowingWind314 commented on Aug 16, 2026

Halalaluyafail3 commented on Aug 16, 2026

BlowingWind314 commented on Aug 16, 2026

Halalaluyafail3 commented on Aug 16, 2026

BlowingWind314 commented on Aug 16, 2026

BlowingWind314 commented on Aug 16, 2026

jensmaurer commented on Aug 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions