Implement gawk array sorting and type introspection builtins#504
Merged
bertysentry merged 20 commits intoJul 15, 2026
Merged
Conversation
Add GawkExtension, enabled by default, providing asort(), asorti(), typeof(), isarray(), mkbool(), and gensub(), with runtime support for PROCINFO["sorted_in"] traversal order, IGNORECASE-aware sorting, and gawk's typed/untyped value distinctions. Extension SPI additions: @JawkBeforeStart startup hook, @JawkRawValue and @JawkRegexp parameter annotations, method-level @JawkAssocArray positions, and a ForInKeyOrder hook so extensions control for-in traversal without gawk-specific code in AVM/JRT/AwkParser. Also fix regexp literal expression semantics (/re/ evaluates as $0 ~ /re/ per POSIX), accept extra user-function arguments with a gawk-style runtime warning (evaluated then discarded), route runtime warnings to stderr instead of the output sink, expose honest SYMTAB/FUNCTAB content, and keep extension state per-engine. Closes #454 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4a8e20523b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
typeof()/isarray() on a missing array element now bring that element into existence (observable via `in`, delete, for-in), matching gawk. Achieved by reading array-element raw values through the normal element dereference, which already returns the untyped marker while creating the element; the special PEEK_ARRAY_ELEMENT_RAW opcode and JRT.peekAwkValue are removed as no longer needed. An explicit empty extension list (new Awk(emptyList) or empty varargs) is now honored as "no extensions" instead of silently installing the default set, giving Java hosts a way to reclaim names such as gensub or typeof. The no-argument constructors still install the default set. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tics Extension SPI: - Add @JawkOptional for trailing optional parameters (null when omitted); asort/asorti/typeof/gensub now declare real signatures instead of varargs. @JawkAssocArray is parameter-level only again: the method-level indexed form is gone (indexes were unfriendly), and a pure-runtime Map check cannot work because a fresh dest variable must be compiled as an array reference. - Make StrNum public: it is the strnum type class and extensions need it. Diagnostics: - Replace the per-tuple runtimeWarning fields (CallFunctionTuple, ExtensionTuple) with a single WARNING opcode the parser plants before the instruction it describes. - Move the gensub third-argument warning out of the parser into GawkExtension itself, where it now checks the actual runtime value (variable selectors warn too, like gawk); AVM exposes the source description and current call line for extension diagnostics, and extension call sites are stamped with the call-site line. Core cleanups: - Rename AwkSettings.setAllowArraysOfArrays to setPosix (logic reversed): the flag now gates all POSIX compile-time behavior, including typed regexp literals. - Rename JRT.getAwkValue to getAssocArrayValue. - Single source of truth for special variable names: AVM exposes getSpecialVariableNames() backed by the same map getVariable() uses; GawkExtension and execMatch consume it. - HashAssocArray/SortedAssocArray normalize null values to the untyped marker so callers never special-case null. - VariableManager.getVariable is a plain abstract method (implemented in the JMH benchmark); sourceBasename uses File.getName; warnings use String.format; comparator helpers documented, including why they cannot reuse JRT.compare2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a92bcbfc10
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ExtensionRegistry now stores factories instead of instances: resolve() and listExtensions() return a fresh instance for factory-registered extensions, so the stateful built-ins (GawkExtension, StdinExtension) are never shared across engines by programmatic callers. A new register(String, Supplier) overload lets third parties register factories for their own stateful extensions; register(String, instance) keeps its sharing semantics. The CLI's GawkExtension clone is obsolete. SYMTAB now observes initialized runtime globals: the SET_NUM_GLOBALS handler consumes the runtime-managed preamble tuples (ENVIRON, ARGC, ARGV offsets) before running the beforeStart hooks, so the gawk extension's SYMTAB snapshot sees their real values instead of untyped placeholders. The tuple stream stays the source of truth, which keeps sandboxed programs (which omit the ARGV tuple on purpose) intact. ARGC/ARGV also join AVM's special-variable map, answered through their lazy synthetic accessors, so SYMTAB["ARGV"][0] works in embedded runs where ARGV is never materialized. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6143c74242
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Add PEEK_DEREFERENCE and EXTENSION to the opcodes requiring the eval global frame: an AwkExpression containing a raw-value extension call such as typeof(x) had its SET_NUM_GLOBALS initializer optimized away, crashing on the null globals array. Extension calls also read globals (e.g. IGNORECASE) and run beforeStart hooks, so they keep the frame too. gensub() now honors IGNORECASE: when it is nonzero, the pattern is recompiled with CASE_INSENSITIVE, matching gawk's rule that IGNORECASE affects all regexp operations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ENVIRON and ARGV preamble tuples are now emitted only when the script references them, or when it references SYMTAB (whose snapshot exposes them); unreferenced ones are answered by the synthetic accessors, so plain scripts no longer pay for building the environment map. ARGC stays always materialized: it is a single cheap assignment and its slot must remain authoritative for ARGC=n command-line operand assignments to affect input traversal. ARGV population derives its count from the argument list so it no longer depends on the ARGC slot. IGNORECASE is now a JRT-managed special variable like FS or RS: reads and writes compile to dedicated PUSH_IGNORECASE/ASSIGN_IGNORECASE opcodes, the JRT precomputes the boolean on assignment, and every regexp/sorting consumer tests jrt.isIgnoreCase() instead of coercing the value per match. -v/API assignments flow through the same setter. Wiring IGNORECASE surfaced a general parser hole: ++/-- on JRT-managed specials compiled to slot-based INC/DEC opcodes that never touched the JRT (gawk's aasort fixture loops on IGNORECASE++ and span forever). Pre/post increment and decrement of specials now compile to a read/adjust/assign sequence through the special-variable opcodes, with correct prefix/postfix value semantics. The duplicated special-variable push/assign switches are consolidated into single parser-level methods. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SYMTAB and FUNCTAB move from GawkExtension into the interpreter, where they belong: the parser emits UPDATE_SYMTAB/UPDATE_FUNCTAB preamble tuples only when the script references them outside POSIX mode, and the AVM populates them from its own metadata (script globals, JRT-managed specials, host-supplied variables, function names, extension keywords). This fixes gawk parity for the reported case: -v variables without a compiled slot now appear in SYMTAB, and command-line name=value operand assignments update the array live between input files. The extension beforeStart hooks now run from a dedicated BEFORE_START_HOOKS opcode emitted by the parser at the end of the preamble, replacing the AVM's nested preamble-consumption loop, and execute at most once per AVM: reused interpreters (repeated executions, expression evaluations) no longer reinitialize their extensions. The special-variables accessor table is now a static Map<String, Function<AVM, Object>>, so constructing an AVM allocates nothing for it. GawkExtension loses all SYMTAB/FUNCTAB code (its beforeStart hook only registers the for-in ordering), which also retires the shouldMaterialize/getVariable lookup concern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AVM.getVariable(String) is back to a plain string switch (a compiled jump table, no map allocation and no lookup indirection); the special variable names live in a static Set that a unit test keeps in sync with the switch. @unsorted now performs no sorting at all: asort()/asorti() skip Collections.sort entirely (the for-in hook already returned keySet() directly), and the constant-comparator branch is gone. JRT.toAssignedScalar is renamed untypedToBlank with documentation of where the untyped marker comes from (missing-array-element reads) and why assignments must strip it (x = a[missing] must leave x an assigned blank scalar, as in gawk). The pointless call on a freshly boxed Integer in populateArgc is removed; the two real call sites remain, costing one monomorphic instanceof on the assignment paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 19b1689ece
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
IGNORECASE now follows AWK truthiness (nonzero or non-null, strnum
aware) instead of numeric coercion, so IGNORECASE = "yes" activates it,
matching gawk.
Runtime name=value assignments (command-line operands processed between
input files) now reach the JRT for its managed specials: IGNORECASE=1
or FS=: as an operand takes effect for subsequent files instead of
being written to a global slot or dropped. The duplicated special-
variable dispatch chains in JRT are consolidated into a single
applySpecialVariable(name, value) used by initial assignment, override
application, and the new runtime path. ARGC is excluded (its setter
delegates back to the VariableManager and its slot stays
authoritative).
IGNORECASE now applies to precompiled regexp constants at runtime:
pattern rules (/re/ { ... }), match expressions, sub()/gsub(), and
gensub() all consult a per-JRT cache of case-insensitive pattern twins,
compiled once per pattern, so the matching hot path stays allocation
free. Two gawk IGNORECASE fixtures (ignrcase, ignrcas4) now pass.
Unknown @-prefixed sort modes are fatal, as in gawk, instead of
silently applying the default ordering; names of user-defined
comparison functions (unsupported) fall back to the default ordering
with a one-time warning. symtabOffset is reset with the other
runtime-managed offsets so a reused AVM cannot write operand
assignments into a stale array.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RegexRuntimeSupport is gone: sub()/gsub() replacement now lives in JRT as replaceFirst/replaceAll instance methods that consult IGNORECASE internally (no flags parameter to thread around), with the reused result buffer moving along and exposed through getReplaceResult(). prepareReplacement() becomes a JRT static used by gensub() as well. The AVM's replaceFirst/replaceAll wrapper indirection is deleted; the sub/gsub opcode handlers call the JRT directly. regexpFlags() also moves to JRT next to isIgnoreCase(): dynamic-regexp call sites compile with jrt.regexpFlags(), while boolean consumers (sorting, pattern-twin cache) keep isIgnoreCase(). setFilelistVariable() now delegates to assignVariable() after parsing, so the specials-routing/slot/SYMTAB logic exists in exactly one place; the operand value is already an input scalar, which assignVariable's normalization passes through unchanged. The prepareReplacement unit test moves to io.jawk.jrt and gains backreference-mode coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ated The AWK match operator and the match() builtin move into JRT next to the IGNORECASE state and the pattern-twin cache: jrt.matches(text, regexp) handles both precompiled regexp constants and dynamic expressions, and jrt.matchPosition(s, ere) locates the match while updating RSTART and RLENGTH. The AVM's MATCHES case and execMatch are now thin delegations, and the interpreter no longer references java.util.regex.Matcher at all. The three inline copies of the "special name, but not ENVIRON/ARGV" expression in the parser now call isJrtManagedSpecialName, whose documentation explains the distinction: ENVIRON and ARGV are special names backed by plain global slots, while SYMTAB and FUNCTAB are not special names at all (ordinary globals populated by the runtime), so they need no exclusion. The caseAwarePattern documentation now records why the cache exists: the JDK's Pattern.compile performs no caching of its own, so every record matched against a regexp constant would otherwise recompile it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e3fb6eea33
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
gensub() no longer aborts when the replacement references a capture group the regexp does not define: prepareReplacement gains a group-count-aware variant, and \N beyond the pattern's groups expands to the empty string, as in gawk. The boolean variant keeps its semantics for sub()/gsub(). split() and FS field splitting honor IGNORECASE. Tokenizer selection is consolidated into JRT.splitTokenizer(input, separator), which replaces the three pasted copies in execSplit, JRT.split, and the field-parsing path. A regexp-literal separator now stays a precompiled Pattern all the way to the tokenizer (served by the case-aware twin cache instead of being stringified), a single-letter separator goes through the regexp path when IGNORECASE is set (letters are regex-safe, so POSIX literal single-character semantics are preserved otherwise), and multi-character separators compile with the IGNORECASE flags. RegexTokenizer gains Pattern and flags constructors. The gawk splitwht fixture now passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review once more! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2fc8a65c29
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
All four findings verified against gawk 5.0.0 and fixed to match:
- @val_str_* compares every scalar as text, with no numbers-before-
strings type ranking (the number 2 sorts after the string "10").
- @val_num_* coerces every scalar to a number, non-numeric strings
counting as 0, and breaks numeric ties with a string comparison;
subarrays still sort last.
- An empty sort mode ("") behaves like an omitted one instead of
crashing on charAt(0); handled in effectiveSortMode so both the
asort()/asorti() and for-in paths get it, which also let
comparator() drop its redundant null/empty guard.
- asorti() defaults to @ind_str_asc, gawk's actual default: indexes
are strings, so no type ranking applies and "10" sorts before "2".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 400eb21350
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- The SYMTAB snapshot no longer contains the SYMTAB and FUNCTAB
globals themselves; gawk keeps the meta tables out of the symbol
table ("SYMTAB" in SYMTAB is 0).
- @ind_num_* now breaks numeric ties with a string comparison, like
@val_num_*: non-numeric indexes all coerce to 0, so without the
tiebreak their order leaked the backing map's insertion order.
Verified against gawk 5.0.0 ({"b","a","0","B"} sorts 0 B a b, and
_desc fully reverses). The shared comparator is renamed
compareNumericThenText.
- @JawkBeforeStart docs now state the deliberate contract: hooks run
once per interpreter instance, not once per execution, because
extension initialization may be heavy and the AVM/JRT pair is stable
for the engine's lifetime. The example no longer suggests seeding
per-run globals there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review once more. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a47c61c850
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- gensub's occurrence selector now goes through AWK numeric conversion (JRT.toDouble) and truncates, matching gawk 5.0.0: " 2" selects the 2nd occurrence, "1e1" the 10th, and the "treated as 1" warning fires exactly when the converted value is below 1 (so non-numeric strings, 0, and negatives all warn; valid numeric strings no longer do). This replaces the integer-prefix JRT.toLong parse and the separate isParseableNumber warning check. - Only the exact "@unsorted" mode skips sorting; "@unsortedx" and "@unsorted_asc" are undefined comparison modes and fatal, as in gawk, for asort()/asorti() and PROCINFO["sorted_in"] alike. - VariableManager.getVariable(String) is now a default method returning null: it was added to a pre-existing public interface, and a default keeps third-party VariableManager implementations source and binary compatible. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d61f909777
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- SYMTAB and FUNCTAB are marked array-typed when their global ID is created (non-POSIX only), so scalar misuse like "SYMTAB = 1" now fails at parse time with the same array/scalar guard as any other array, instead of the less precise runtime error. gawk is fatal on such use too. In --posix mode both names stay ordinary variables. - IGNORECASE now applies to string relational operators and index(), as in gawk: compare2 gains a case-aware variant fed with jrt.isIgnoreCase() by the comparison opcodes (numeric/strnum comparisons are untouched), and index() delegates to a case-aware JRT.index(). Constant folding of comparisons is restricted to numeric operands, because a folded string comparison would bake in the case-sensitive result before runtime IGNORECASE can act. Verified byte-identical with gawk 5.0.0, including strnum cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e3c62ea597
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The SYMTAB snapshot and AVM.getVariable's slotless fallbacks read baseInitialVariables, so variables supplied per run through AwkRunBuilder.variable(...) / execute(..., variableOverrides) were invisible when the script gave them no compiled slot. All three sites now read executionInitialVariables, which prepareExecutionInputs already maintains as the base map merged with the current run's overrides (and aliases the base map when there are none, so no-override executions are untouched). A host-supplied PROCINFO with "sorted_in" now drives for-in ordering even when the script never references PROCINFO; tests cover that and SYMTAB visibility. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex Please review once more. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 65b60c5e7b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- mkbool() now applies ordinary AWK truthiness via jrt.toBoolean, the same conversion IGNORECASE uses: a non-empty string like "abc" (or the string constant "0") is true, while a strnum judged numerically zero stays false. Numeric coercion via toDouble wrongly returned false for truthy non-numeric strings. - asorti() writes the source indices as string values, as gawk does: the internal key object may be a Long for numeric-looking indexes, which leaked "number" out of typeof(d[i]) where gawk 5.0.0 says "string". Note: for (k in a) also types numeric-looking keys as "number" where gawk says "string"; that lives in the interpreter's iteration path, not the extension, and is left for a separate discussion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@codex please review again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1ab7186b3d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
No code change: the reported null-frame scenario is not reachable. setNumGlobals is the sole populator of both the globals array and the by-name offset map, so when the eval optimizer drops SET_NUM_GLOBALS (only for expressions with no compiled globals) no name can resolve to an index into the missing frame. Hooks run once, on the first execution of a fresh AVM, so no stale layout from a prior run can exist either; every by-name path a hook can take (getVariable, assignVariable) falls back to JRT specials or the initial-variables maps. Treating BEFORE_START_HOOKS as frame-requiring would instead disable the optimization entirely, since the opcode is emitted in every eval preamble. A regression test pins this: an extension whose beforeStart reads and assigns variables by name runs against a frame-optimized expression stream and completes safely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- GawkExtension.compareStrings folds case with compareToIgnoreCase instead of two toLowerCase(Locale.ROOT) allocations per comparison inside O(n log n) sorts, and now applies the same folding rule as JRT.compare2 on string relational operators. - JRT.replaceFirst/replaceAll share one private replace(orig, repl, ere, global) worker; their bodies differed by a single token. - GawkExtension members are grouped by design order: constants and fields, the before-start hook, the six @JawkFunction builtins, the diagnostics helper, for-in and sort-mode plumbing, sort machinery, comparators, typeof helpers. Pure block moves, no content changes. - The sort entry and key lists are pre-sized to the array size. Remaining observations from the final review are tracked in #511. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bertysentry
deleted the
454-implement-gawk-array-sorting-and-type-introspection-builtins
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #454
What changed
GawkExtension, enabled by default (innew Awk()and the CLI when no-lis given), implementing:asort(source [, dest [, how]])/asorti(...)— gawk array sorting with the predefined comparison modes (@ind_str_asc,@val_num_desc,@val_type_asc,@unsorted, …), honoringIGNORECASEtypeof(x)— gawk type categories (number,string,strnum,array,regexp,number|bool,unassigned,untyped)isarray(x),mkbool(x), andgensub(re, repl, how [, target])(gawk semantics: anyg/G-prefixed selector is global,\Nbackreferences)PROCINFO["sorted_in"]-controlledfor (i in a)traversal, re-read at each loop so mid-execution changes applySYMTAB(script globals + Jawk special variables, snapshot) andFUNCTAB(user-defined functions + extension keywords) for scripts that reference themExtension SPI — gawk-specific behavior lives entirely in the extension; the core gained neutral hooks:
@JawkBeforeStart— setup method run after globals are allocated, before execution@JawkRawValue— parameter receives the runtime value uncoerced (howtypeof/isarraysee untyped state); backed by newPEEK_DEREFERENCE/PEEK_ARRAY_ELEMENT_RAWopcodes that observe without autovivifying@JawkRegexp— parameter keeps regexp literals as precompiled patterns@JawkAssocArray({...})for array positions consumed through varargsForInKeyOrder— extension-providedfor-intraversal order; the default path is unchanged (singlekeySet()copy, null-check only)Interpreter fixes riding along:
$0 ~ /re/per POSIX (x = /re/yields 0/1)@/re/) accepted; rejected under--posixerrorStream(...)captures it in the Java API)AVM.getVariable(name)falls back to-v/API-supplied initial variables, so e.g.-v IGNORECASE=1is honored even when the script never references itsub()/gsub()replacement escaping is shared withgensubviaRegexRuntimeSupportReviewer notes
GawkExtensionperAwk; the singletonINSTANCEpattern was removed becauseAbstractExtensionholds per-enginevm/jrtbindings.ExtensionFunctionistransientand recomputed from the resolvedMethodon deserialization, so.tuplesfiles from other versions stay loadable.VariableManager.getVariable(String)is adefaultmethod to keep third-party implementors source- and binary-compatible.GawkExtensionIT.test_asortsymtabgoes from pass to failure. Its.okfixture only encodes the entry counts of the gawk build's own symbol table (30 globals / 42 builtins); the previous implementation faked those counts with placeholder entries, which corrupted any script actually readingSYMTAB/FUNCTAB. The honest content can't match gawk's internal table sizes.index.md.vm(gawk departures list),extensions.md,extensions-writing.md,cli.md,cli-reference.md, andjava.md— including thatgensub/typeof/asort/… become reserved names by default.Verification
mvn verify: 479 unit tests, 0 failures; checkstyle, PMD, SpotBugs clean;mvn compile -P benchmarkpassesmainbaseline: 29 cases error→pass (incl. all 13 representative cases from Implement gawk array sorting and type-introspection builtins #454), 10 error→failure (now run, output differs — follow-up candidates), 1 pass→failure (asortsymtab, intentional, see above), zero other regressions across POSIX/BWK/gawk suites🤖 Generated with Claude Code