CWG3078 [cpp.include][cpp.cond][lex.digraph] Discrepancy between domains of #include _pp-tokens_ and _header-name-tokens_ #770

Full name of submitter (unless configured in github; will be published with the issue): Hubert Tong

Reference (section label): [cpp.include], [cpp.cond], [lex.digraph]

Link to reflector thread (if any): N/A

Issue description:

The resolution of CWG 3015 (https://cplusplus.github.io/CWG/issues/3015.html) has further clarified that

#define X >
#include <<X

performs the same inclusion as

#include <<>

There is implementation divergence (https://godbolt.org/z/TvceKr39G): Clang "accepts". GCC, EDG, and both MSVC preprocessor implementations reject.

Additionally, there are two issues with respect to language-specification consistency.

Firstly, the following is rejected by the header-name-tokens grammar in [cpp.cond] (because <% is a digraph) although the token sequence is specified to succeed when used as the pp-tokens for #include:

#define X >
#if __has_include(<%X)
#endif

Secondly, the footnote from https://wg21.link/lex.digraph#2 seems to be too broad in its statement of interchangeability between alternative and primary tokens.

Note that header-name-tokens does not prevent IFNDR cases such as a sequence consisting of the tokens <, >>, and >.

Suggested resolution:

With reference to a non-IFNDR case such as <, <:, >; strike the footnote from https://wg21.link/lex.digraph#2:

Thus the “stringized” values [cpp.stringize] of [ and <: will be different, maintaining the source spelling, but the tokens can otherwise be freely interchanged.

Modify in https://wg21.link/cpp.include#7:

The preprocessing tokens after include in the directive are processed just as in normal text (i.e., each identifier currently defined as a macro name is replaced by its replacement list of preprocessing tokens). The resulting sequence of preprocessing tokens shall be of the form
 header-name-tokens
Then, an An attempt is then made to form a header-name preprocessing token ([lex.header]) from the whitespace and the characters of the spellings of the resulting sequence of preprocessing tokens header-name-tokens; the treatment of whitespace is implementation-defined.