From c0c84d41e7c368b1fb47e26f4bebf6f501febd35 Mon Sep 17 00:00:00 2001 From: Peter Willendrup Date: Wed, 11 Nov 2026 09:35:17 +0100 Subject: [PATCH] Lexer: keep last line of a C code file lacking a final newline Such a line matched no rule and was echoed to stdout instead of reaching the generated C, e.g. losing a closing #endif or }. Co-Authored-By: Claude Fable 5.2 --- .../ADR_20261111_UNTERMINATED_LAST_LINE.md | 27 +++++++++++++++++++ mccode/src/instrument.l | 9 +++++++ 2 files changed, 36 insertions(+) create mode 100644 docs/GRAMMAR/ADR-records/ADR_20261111_UNTERMINATED_LAST_LINE.md diff --git a/docs/GRAMMAR/ADR-records/ADR_20261111_UNTERMINATED_LAST_LINE.md b/docs/GRAMMAR/ADR-records/ADR_20261111_UNTERMINATED_LAST_LINE.md new file mode 100644 index 0000000000..35f154baf5 --- /dev/null +++ b/docs/GRAMMAR/ADR-records/ADR_20261111_UNTERMINATED_LAST_LINE.md @@ -0,0 +1,27 @@ +# Keep the last line of a C code file that lacks a final newline + +## Status + +proposed + +## Context + +Inside `%{ ... %}` blocks, and in files embedded there with `%include`, the lexer (`instrument.l`, state ``) only knew newline-terminated lines. A last line without a newline matched no rule, so flex echoed it to stdout and it was missing from the generated C. + +Seen with the ESS BIFROST model, where `rfc-lib.h` and `rfc-lib.c` (mcstas-radial-filter-collimator) end in `#endif // RFC_LIB_H` and `}` without a newline. McStas printed `#endif // RFC_LIB_H}` and the compiler reported `function definition is not allowed here` for everything that followed. + +mccode-antlr accepts such files, so they are easily produced and go unnoticed. + +## Decision + +Add one `` rule matching a line without a newline. It can only be the longest match on the last line of a file. The rule pushes the line back with a newline appended, so it is rescanned by the existing rules (`%}`, `%include`, plain code line). + +## Consequences + +Library and code files no longer need to end with a newline. No new syntax, and nothing changes for files that do end with one. + +## Behaviour + +- An unterminated last line is kept, and gets a newline in the generated C. +- Generated C was compared before/after for the 369 McStas example instruments: 365 identical, 2 fail to generate both before and after. 2 differ by one restored line (`Supermirror_thin_substrate`, `Test_lib_supermirror_lib`): the closing comment of `supermirror-lib.h`, which also lacks a final newline and was silently dropped until now. +- The lexer is shared with McXtrace; only McStas was built and tested (macOS arm64). diff --git a/mccode/src/instrument.l b/mccode/src/instrument.l index 87784a3bce..06dafdad95 100644 --- a/mccode/src/instrument.l +++ b/mccode/src/instrument.l @@ -249,6 +249,15 @@ METADATA return TOK_METADATA; yylval.string = str_dup(yytext); return TOK_CODE_LINE; } + /* last line of a file, lacking its newline (only then can this rule be + the longest match): rescan it with a newline appended */ + [^\n]+ { + char *line = str_dup(yytext); + int i = yyleng; + unput('\n'); + while (i > 0) unput(line[--i]); + str_free(line); + } } /* end ccode */ /* C character literals, e.g. 'a' or '\n', kept verbatim as a single token in