"\|" is an undefined sequence. It looks like an escape sequence, but there is no such defined escape sequence in string literals. Some awk string parsers (as the quoted error message makes clear) will discard the backslash and keep the pipe symbol, which is not special in a string. Other awks will keep both characters. However, an implementation that aborts with a compilation error (to my knowledge, none do) is not violating any standard.

The same is true when any character that is not part of a defined escape sequence (\\n, t, \\, etc) follows a backslash.

This is also an issue with sed escape sequence handling. If someone actually wrote an implementation that strictly refused undefined escape sequences, most non-trivial scripts posted in these forums would fail (which one could argue would be preferable to unknowingly harboring unreliable behavior).

The problem has nothing to do with the string literals and that their value is later processed by the regular expression parser. The problem is a misplaced semicolon forming an empty if-statement.

You can move or drop the semicolon, or just use a bare pattern. With your string literals as I quoted them, the following will work fine.

Code:

BEGIN { X=...; Y=...; Z=X...Y }
$0 !~ Z

Regards,
Alister