C backend: a segment with more than 247 live one-word values fails to compile #1069

Activity

1. nicolas-abril

   added

   bugSomething isn't workingSomething isn't working

   on Sep 25, 2026
2. aldeni commented on Sep 26, 2026

   

   aldeni

   on Sep 26, 2026

   Contributor

   More actions

   Reproduced on 2.0.29 (clang 22.1.8, Linux x86-64), and I tried to see what the check is protecting today, by lifting it in a scratch copy and building three shapes of the same size, 300 one-word values live across a call:

   shape	interpreter / JS	C, check lifted, FID_T as u8	C, check lifted, FID_T as uint16_t
   the issue's continuation (Mk{(s ++ "0"), …})	x299	x299 at 1 and 4 threads	x299
   a fork with 300 values live across it	46851	46851 at 1 thread; frontier drained without a result at 4	46851 at 1 and 4
   a closure capturing 300 values	44857	35362, silently, at 1 and 4 threads	44857

   So on this clang the wrong answers all come from one place: FID_T[][3] is u8, and an arity of 301 reads back as 45. Clo~apply loads 44 captured words and the rest of the bank is whatever it was; task_node sizes the join's node from the same truncated count. The plain continuation only survives because a program without a fork never leaves the frame path, where the pop count is a literal. With the table at 16 bits and nothing else changed, all three build and answer correctly at 1 and 4 threads, with a 303-register bank, preserve_none and musttail included; clang 22 places it. I could not try clang 19-21 here, so the crash you saw stands as the other half of the limit.

   The bank is one for the whole program (WL_SIG), so one wide join widens every segment's signature. I measured what that costs before assuming: four of bench/runtime built as they are and again with the bank forced to 303 words in the emitted C, medians of 3 interleaved runs:

   bench	threads	bank as emitted	bank of 303 words	change
   queens	1	13.58 s	13.53 s	−0.4%
   queens	4	4.07 s	3.94 s	−3.0%
   symreg	1	5.34 s	5.32 s	−0.3%
   symreg	4	1.65 s	1.55 s	−6.4%
   lexer	1	5.20 s	5.06 s	−2.6%
   lexer	4	1.55 s	1.45 s	−6.4%
   hashmap	1	5.86 s	5.96 s	+1.7%
   hashmap	4	1.73 s	1.83 s	+5.5%

   Within noise, and on both sides of it: the words nobody writes ride the stack in place through every musttail, so a wide bank moves nothing extra. (The 4-thread rows spread more; the machine was not alone.) The binary grows by 40 bytes.

   So on the host, with this clang, what stands between the program and a correct build is the u8 table, and the register bank itself costs nothing to widen. What still argues for the spill you describe is what I could not measure here: clang 19-21, where you saw the backend crash, and the device lane, where the bank is a lane's registers and 303 words would be a real limit. Which of the two you want, a wider FID_T with WIDE raised for host segments, or the spill for every lane, seems like your call.

   I'd be glad to implement either, and I have the three reproducers above as tests (each seeded from IO.now() so nothing folds, each built to hold at 1 and 4 threads; they are generated files of 300 fields, so I left them out of this comment and can attach them or put them in the PR). If you'd rather keep this one in-house, they are yours to take as they are.
3. paymog commented on Sep 27, 2026

   

   paymog

   on Sep 27, 2026

   Last edited by paymog

   More actions

   Another shape that hits this, from ordinary code: four known Word(64n) values live in one function. Bend 2.0.29, macOS 26 arm64 (Apple clang 17).

   import Base
   
   def f(n: Nat, a: Word(64n), b: Word(64n), c: Word(64n), d: Word(64n)) -> Word(64n):
     match n:
       case 0n:
         a
       case 1n+p:
         f(p, b, c, d, a)
   
   def ones() -> Word(64n):
     Word.not(64n, Word.zero(64n))
   
   def main() -> IO(Unit):
     IO.print(Bool.show(Cmp.is_eq(Word.cmp(64n, f(3n, ones(), ones(), ones(), ones()), ones()))))
   

   bend arity.bend prints True. bend arity.bend -o a prints Error: an arity over 247. With three Word(64n) parameters it builds. With Word.zero(64n) arguments instead of ones() it also builds.

   It shows up in a U64 type over Word(64n): a loop such as rem(n, +a, +c, +m, +x, acc) over five U64 values does not build natively, so the loop has to be split.
4. ajoslin commented on Sep 27, 2026

   

   ajoslin

   on Sep 27, 2026

   Last edited by ajoslin

   More actions

   We also hit this in an application on 2.0.29 (macOS arm64), through flattened nested records rather than wide source constructors. A diagnostic in a scratch compiler copy printed these continuation widths:

   Function	Words
   finish_staged	310
   close_trade	310
   adopt	249 and 250

   These functions take only a few source arguments. The close paths built an updated State record before computing commands and projections, so both the original and the updated record were live across later calls.

   Workaround: compute the projections first, then build the updated record in a helper. With that change, the unmodified compiler builds the program, and native output matches the interpreter on the cases we checked. Adding pass-through helpers alone did not change the widths. The workaround changes evaluation order only; it doesn't fix the limit, which also shows up in @paymog's four-Word(64n) example above.
5. VictorTaelin

   added

   laterAccepted; parked until we return to itAccepted; parked until we return to it

   on Sep 27, 2026
6. paymog

   added a commit that references this issue on Sep 28, 2026

   fix(http): build natively again by keeping cookie times out of Cav (#242

   ...

   7bd95b0
7. o1lo01ol1o commented on Oct 3, 2026

   

   o1lo01ol1o

   on Oct 3, 2026

   More actions

   Posted by Claude (an AI assistant) on behalf of @o1lo01ol1o, after some cursory experiments.

   A direction not yet raised here: give each segment its exact width instead of one program-wide bank with a cap. From the experiments below, that isn't expressible in C, and LLVM IR looks like the only way to get it.

   Motivating example

   The limit applies to the type of a value that is live across a call, not to how much data it holds. Any record of one-word fields hits it once it is live across one call:

   type State is Data:
     S{f0: U32, f1: U32, ..., f259: U32}
   
   def bump(+n: U32) -> U32:
     U32.add(n, 1)
   
   def rebuild(+k: U32, s: State) -> State:
     match s:
       case S{+f0, ..., +f259}: S{U32.add(f0, k), ..., U32.add(f259, k)}
   
   # s is live across the call to bump.
   def step(s: State, +n: U32) -> State:
     rebuild(bump(n), s)
   

   The interpreter answers (42 for main = first(step(S{0, ..., 259}, 41))). The C build refuses with an arity over 255; on 2.0.25 the limit is 255, and #1068 later moved it to 247. In our case the record is the state of a composed state machine. Composing machines pairs their states, so the width adds up across components while every component stays small. Today the options are to box the state by hand, which is acceptable for us only because every step waits on IO, or to reorder evaluation as @ajoslin did. A compute-bound program pays an allocation and an indirection per step for a limit that comes from the calling convention.

   Why a cap is needed at all

   Every segment shares WL_SIG because the host jumps between segments with musttail, and C's musttail requires identical prototypes. I checked this with clang 21.1.8 (arm64) under the default convention, preserve_none and swiftasynccall. All three reject different parameter counts:

   __attribute__((noinline, preserve_none)) u64 g(u64 a, u64 b, u64 c);
   __attribute__((noinline, preserve_none)) u64 f(u64 a) { __attribute__((musttail)) return g(a, a, a); }
   // error: cannot perform a tail call to function 'g' because its signature is incompatible
   //        with the calling function (expected 1 but has 3)

   The spill proposed above keeps the shared bank and moves the excess through memory. That works, but it keeps a cap (whose value is a tuning choice per lane), a second code path for the segments past it, and a program-wide signature set by its widest segment.

   LLVM IR can express exact widths

   LLVM's tailcc convention has the callee pop its own arguments, and it guarantees tail calls between functions of different signatures. With the same clang 21.1.8 (arm64):

   • a 2-argument and a 15-argument tailcc function calling each other 10^8 times (15 is past arm64's 8 argument registers, so some arguments go on the stack): correct result at -O0 and -O2, no stack growth;
   • a 2-argument and a 300-argument tailcc function calling each other 10^7 times: correct result at -O0 and -O2 (checked against an independent computation).

   define tailcc i64 @narrow(i64 %n, i64 %acc) {
     %z = icmp eq i64 %n, 0
     br i1 %z, label %out, label %go
   out:
     ret i64 %acc
   go:
     %m = sub i64 %n, 1
     %r = tail call tailcc i64 @wide(i64 %m, i64 %acc, i64 1, i64 2, ..., i64 13)
     ret i64 %r
   }
   define tailcc i64 @wide(i64 %n, i64 %acc, i64 %a, ..., i64 %m) {
     ; ...
     %r = tail call tailcc i64 @narrow(i64 %n, i64 %acc2)
     ret i64 %r
   }

   So the host lane could emit each segment at its exact width, with no cap, no shared bank and no spill path. A segment's cost would follow its own width. The parts that would need design rather than a constant:

   • Unknown calls (Clo~apply, wl_tab), where the callee's width is known only at run time. The usual answer is GHC's eval/apply ("Making a fast curry", Marlow and Peyton Jones). Known calls jump to the exact-width entry. Unknown calls go through a uniform entry per segment that unpacks from the closure node, which already holds the captures.
   • Suspend and resume (WL_SAVE/WL_TAKE, joins, IO parking) would save exactly the suspended segment's width, which the emitter already records.
   • Tables and buffers (FID_ARITY_T, Term fs[256]) widen to match, which also closes the u8 truncation class @aldeni found.
   • The device lane is one switch per kernel with per-lane registers. It has no per-segment signatures, so it would still need a register budget and a spill past it. The cap only goes away on the host.

   Caveats

   • This is a move from emitting C to emitting LLVM IR, at least for the segment and call layer, which is a large change.
   • I could not reproduce the clang crashes reported here and in ARM64 native emission triggers clang backend crashes; isolated LLVM14 IR reproducer and Apple clang21 report #1233 with synthetic code. A shared 300-word preserve_none/musttail bank in C also built and ran correctly on clang 21.1.8 (arm64). So I have no evidence either way that exact widths avoid those crashes, which seem to depend on the emitted code, not only on the width.
   • I haven't measured tailcc's cost against the current shared bank. Given @aldeni's measurement that a wide bank costs about nothing on clang 22 (x86-64), the case for this is structural (no cap, no truncation class, no program-wide coupling to the widest segment), not performance.
8. rofrol commented on Oct 15, 2026

   

   rofrol

   on Oct 15, 2026

   More actions

   Posted by Claude (an AI assistant) on behalf of @rofrol.

   Another ordinary shape that hits this: a list literal whose elements are calls. Each element's result stays live in the continuation that builds the next one, so the widest continuation carries one word per element, and the C build is refused at exactly 248 elements. The checker, the interpreter and JS accept it.

   n=248
   { echo 'import Base'
     echo 'def f(n: U32) -> String:'
     echo '  U32.show(n)'
     printf 'def xs() -> List<&2, String>:\n  ['
     i=1; while [ $i -le $n ]; do printf 'f(%d)' $i; [ $i -lt $n ] && printf ', '; i=$((i+1)); done
     echo ']'
     echo 'def main() -> IO(Unit):'
     echo '  IO.print(String.join(xs(), " "))'
   } > list.bend
   bend list.bend -o list

   elements	C build
   247	builds, prints 247 numbers
   248	Error: an arity over 247

   Same result on 2.0.37 and on main at d2aeefc (macOS 27 arm64, Apple clang 21). A diagnostic in a scratch copy of comp.ts named the segments FID_XS_K258, FID_XS_K259, ... with 248, 249, ... parameters.

   We hit it in a real program: a registry of 25 package definitions, [("name", G.Full{Pkg.pkg()}), ...], where each element is a record built by calls, so the continuations grow by 18 words per element (253, 271, ... 379 parameters) and the build fails well below 248 elements. JS (-o gen.js) works and runs on Bun or Node. A diagnostic naming the source function (here xs, or registry) instead of only the limit would have saved the search.
9. ScottMoore0

   added 2 commits that reference this issue on Oct 18, 2026

   word64: a W64 is two native U32 halves

   ...

   1155157

   caps: a CapSet is a native U32 plus a 9-bit word (builds natively)

   ...

   9f2a88e
10. dearlordylord commented on Oct 29, 2026

    

    dearlordylord

    on Oct 29, 2026

    Last edited by dearlordylord

    Contributor

    More actions

    Posted by Codex Sol

    Additional production reproducer for #1069, observed while upgrading from Bend 2.0.38 to 2.0.39.

    The same source emits C successfully with the official 2.0.38 Linux ARM64 compiler, but the official 2.0.39 Linux ARM64 compiler exits with:

    Error: an arity over 247
    

    Reproduction using public source, with no npm dependencies required:

    git clone https://github.com/dearlordylord/hapsland.git
    cd hapsland
    git checkout c32f148cd10ca3f859efceab2a54f1b914543498
    /path/to/bend-2.0.38/bin/bend prototypes/canonical-defense/DefenseMain.bend -o /tmp/game38.c
    /path/to/bend-2.0.39/bin/bend prototypes/canonical-defense/DefenseMain.bend -o /tmp/game39.c

    Observed results:

    • 2.0.38: exits 0 and emits C.
    • 2.0.39: exits 1 with the error above, before invoking clang.

    A diagnostic-only build of the 2.0.39 compiler, with logging added at the arity guard in bend2/comp.ts, identifies one oversized segment:

    NativeRun.command_selected$k1063: 254 parameters
    limit: 247
    oversized constructor IDs: none
    

    The original Bend definition has seven parameters; the oversized continuation is generated internally. This may be another useful case for #1788. No compiler patches or alternate runtime settings were used for the official-release comparison.