Add support for BEGINFILE and ENDFILE rules#515
Open
bertysentry wants to merge 2 commits into
Open
Conversation
Implement gawk's BEGINFILE/ENDFILE special patterns, the nextfile statement, and the ERRNO and ARGIND special variables: - Parser: BEGINFILE/ENDFILE/nextfile keywords, per-file input-loop tuple layout (NEXT_FILE / CONSUME_FILE_INPUT scaffolding emitted only when these features are used, so existing programs keep the flat layout), and gawk-compatible compile-time restrictions (next rejected in BEGINFILE/ENDFILE, nextfile rejected in BEGIN/END/ENDFILE). - Runtime: per-file stepping in StreamInputSource with FILENAME, FNR=0, cleared $0, ARGIND, and ERRNO on open failure; open failures become non-fatal only when a BEGINFILE rule skips the file with nextfile; nextfile unwinds user-defined function calls like exit does; non-redirected getline is fatal inside BEGINFILE/ENDFILE. - New opcodes appended at the end of the Opcode enum to preserve precompiled tuple (-L) compatibility. - BEGINFILE/ENDFILE are not special in POSIX mode (--posix), as in gawk. - Lexer: a slash inside a regexp bracket expression (/[/]/) no longer terminates the literal, which beginfile1.awk requires; this also makes the vendored gawk regexpbrack test pass. - FNR now counts records read from standard input (POSIX: FNR is the record number in the current input file); custom InputSource implementations keep the documented isFromFilenameList() contract. - Tests: BeginFileEndFileTest (25 cases), re-enabled the POSIX nextfile conformance test, new stdin-FNR conformance tests, and wired the vendored beginfile1 fixture plus six beginfile2 sub-tests into GawkExtensionIT (beginfile1 matches beginfile1.ok byte for byte). - Docs: README, cli.md, cli-reference.md, extensions.md, java.md, java-input.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 697ebee809
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
gawk resets ERRNO to the empty string whenever the next main input (file operand or stdin) is selected successfully, not only on the BEGINFILE per-file path. Mirror that in the ordinary input flow and pin the behavior with unit tests. Addresses review feedback on #515. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #457.
What changed
Jawk now parses and executes gawk's
BEGINFILE/ENDFILEspecial patterns, together with thenextfilestatement and theERRNO/ARGINDspecial variables.AwkParser): newBEGINFILE/ENDFILE/nextfilekeywords and rule ASTs. When any of these features is used, the main input loop is compiled with the per-file tuple layout from the issue (NEXT_FILEbeforebegin_file,CONSUME_FILE_INPUT → end_fileat each file's EOF, ENDFILE rules, thenNEXT_FILE → no_more_inputfalling into the END blocks). Programs that don't use these features keep the exact previous flat layout, so there is zero overhead for existing scripts.AVM,JRT,StreamInputSource): per-file stepping with gawk-compatibleFILENAME,FNR = 0, cleared$0,ARGIND, andERRNOon open failure (No such file or directory/Is a directory, synthesized deterministically cross-platform). A file that cannot be opened is non-fatal only if a BEGINFILE rule skips it withnextfile; otherwise the usual gawk-style fatal error fires.nextfileworks in ordinary rules and BEGINFILE — including from user-defined functions (it unwinds the runtime/operand stacks the same wayexitdoes) — and runs ENDFILE for successfully opened files while skipping it for failed ones, matching the vendored.okfixtures.exitinside the hooks jumps to the END blocks.nextis rejected inside BEGINFILE/ENDFILE andnextfileinside BEGIN/END/ENDFILE (compile time when direct, runtime when reached through a function); non-redirectedgetlineis a fatal error inside both hooks, redirected forms are allowed.NEXT_FILE,CONSUME_FILE_INPUT,EXEC_NEXTFILE,SET_ENDFILE_ADDRESS,SET_NEXTFILE_ADDRESS,PUSH/ASSIGN_ERRNO,PUSH/ASSIGN_ARGIND— appended at the end of theOpcodeenum becausefromIdresolves by ordinal, preserving precompiled tuple (-L) compatibility.BEGINFILEandENDFILEare not special under--posix/setPosix(true)and lex as ordinary identifiers.nextfilestays available in both modes (POSIX.1-2024 standardizes it)./inside a regexp bracket expression (/.*[/]/, used bybeginfile1.awk) no longer terminates the literal. This also flipped the vendored gawkregexpbrackIT from error to pass.printf 'a\nb\n' | jawk '{print FNR}'now prints1 2instead of0 0. CustomInputSourceimplementations keep the documentedisFromFilenameList()contract.Tests
BeginFileEndFileTest(25 cases): execution order across files, empty files, stdin, ERRNO +nextfileskip, fatal open failures,nextfilefrom functions and without hooks, all restriction diagnostics, ARGIND,$0clearing,exitin hooks, POSIX-mode non-specialness, bracket-slash regexp.nextfileconformance test; added stdin-FNR conformance tests and anInputSourceTestguard for the custom-source FNR contract.GawkExtensionIT#test_beginfile1now runs the vendored fixture for real and matchesbeginfile1.okbyte for byte (including the ERRNO lines for.and a missing file). Sixbeginfile2sub-tests (2, 5, 9a, 9b, 12, 15) extract their programs and expected transcripts from the vendoredbeginfile2.in/.okand pass. The remaining sub-tests rely on non-redirectedgetlinein BEGIN/END driving the BEGINFILE/ENDFILE hooks mid-statement (or compare non-zero gawk CLI transcripts) — that reentrancy isn't reproducible in the tuple VM; the umbrella skip documents it.getfile/readfile2need gawk's dynamic extension API (@load,get_file()), which Jawk doesn't provide; their skip reasons now say so.Verification
mvn verify site: BUILD SUCCESS; checkstyle / PMD / SpotBugs at 0 findings; 543 unit tests pass.regexpbrackturning green; zero regressions across the 939 pre-existing cases, including after the FNR change.Notes for reviewers
test_beginfile2skip): non-redirectedgetlinein BEGIN/END reads ahead in the main input without firing BEGINFILE/ENDFILE; the loop then adopts the already-open file instead of reopening it.$0and the fields are cleared in BEGINFILE.🤖 Generated with Claude Code