Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,8 @@ deno task binding:audit # comment↔token binding audit: does forma
deno task authoring:audit # authoring-independence over Svelte boundary whitespace: every render-equivalent authoring of one document (hug ↔ space ↔ newline at a tag's content boundary; space ↔ newline between siblings) must reach ONE tsv fixed point (pure Rust, no sidecar; gated in `deno task check`) — exits 1 on any non-idempotency, site-level or a base-non-idempotent FILE; see Debug Tooling
deno task fuzz:audit # seeded mutational fuzzer over tests/fixtures (fixed --seed 0 --iterations 5000; pure Rust, no sidecar; gated in `deno task check`) — asserts no-panic + idempotency + structural-reparse, on every seed file AS AUTHORED and then on mutated input; see Debug Tooling
deno task comments:audit # print-once comment ledger: every comment a document PARSES must be EMITTED exactly once (pure Rust, no sidecar; gated in `deno task check`) — reports DROPPED (silent content loss) and DOUBLE-PRINTED; the structural guard on the detached comment model, tsv's `ensureAllCommentsPrinted`; see Debug Tooling
deno task gaps:audit # gap-injection audit: inject a comment into EVERY gap (five payloads, one per ownership path) and re-run the ledger — the discovery arm `comments:audit` can't be, since it only formats each file AS AUTHORED and no fixture covers most positions (eight such drops were found BY HAND, all green on every gate). Pure Rust, no sidecar; gated in `deno task check` as a RATCHET over a generated shape snapshot (`gap_audit_known.txt`): every line is a known bug and the file shrinking is the goal, so a shape not on the list, one on it that no longer fires, or any PANIC, FAILS. ~17 s. Full reference: ./docs/gap_audit.md
deno task gaps:audit:update # regenerate that snapshot after fixing a shape (or when a new fixture merely REACHES a pre-existing one); refuses a narrowed run
deno task idempotency:sweep # F1 (idempotency) sweep over the real-code corpus (the `perf` view — sibling dev repos + upstream framework source). NOT in `deno task check`: machine-dependent corpus, minutes not seconds. Run at conformance cadence or after a printer change; see Debug Tooling
```

Expand Down Expand Up @@ -882,6 +884,30 @@ cargo run -p tsv_debug --features comment_check comment_audit ~/dev/zzz/src #
# source), so they are outside the model by construction.
```

**Gap-Injection Audit (dropped-comment discovery):**

```bash
# gap_audit - inject a comment into EVERY gap and re-run the print-once ledger. The
# DISCOVERY arm `comments:audit` can't be: the ledger only ever sees a document AS
# AUTHORED, so a gap no fixture puts a comment in is one it never checks (eight such
# drops were found BY HAND, all green on every gate). Pure Rust, no sidecar.
cargo run --profile corpus -p tsv_debug --features comment_check gap_audit # tests/fixtures
cargo run --profile corpus -p tsv_debug --features comment_check gap_audit ~/dev/zzz/src
# Also: --json, --jobs N, --limit N, --payload <one>, --all-bytes, --update.
# Build with `--profile corpus` (release + panic=unwind): plain `--release` is
# panic=abort, so a formatter panic kills the process instead of being caught + reported.
#
# GATED as a RATCHET, not a green gate: `gap_audit_known.txt` is a machine-generated
# snapshot of the ~717 shapes tests/fixtures produces, every line a KNOWN BUG, the file
# shrinking is the goal. A shape not on the list, one on it that no longer fires, or any
# PANIC, FAILS. `--limit`/`--payload`/`--all-bytes`/a path narrow a run, so they skip the
# ratchet and refuse `--update`. ~17 s.
```

Full reference — flags, the ratchet, reading a finding, triage + re-pin workflow,
scope: ./docs/gap_audit.md. Design rationale (why byte offsets and not tokens, why the
ledger is the oracle, why five payloads) lives in the `gap_audit` module docs.

**Build-Fanout Audit (exponential-rebuild regression guard):**

```bash
Expand Down
1,937 changes: 1,937 additions & 0 deletions crates/tsv_debug/src/cli/commands/gap_audit.rs

Large diffs are not rendered by default.

713 changes: 713 additions & 0 deletions crates/tsv_debug/src/cli/commands/gap_audit_known.txt

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions crates/tsv_debug/src/cli/commands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ pub mod fixtures_update_parsed;
pub mod fixtures_validate;
pub mod format_prettier;
pub mod fuzz;
#[cfg(feature = "comment_check")]
pub mod gap_audit;
pub mod json_profile;
pub mod lex_diff;
pub mod line_width;
Expand Down
8 changes: 8 additions & 0 deletions crates/tsv_debug/src/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ pub mod commands;
use argh::FromArgs;
#[cfg(feature = "comment_check")]
use commands::comment_audit::CommentAuditCommand;
#[cfg(feature = "comment_check")]
use commands::gap_audit::GapAuditCommand;
#[cfg(feature = "swallow_check")]
use commands::swallow_audit::SwallowAuditCommand;
use commands::{
Expand Down Expand Up @@ -71,6 +73,10 @@ pub enum Subcommand {
// `comments:audit` deno task passes it.
#[cfg(feature = "comment_check")]
CommentAudit(CommentAuditCommand),
// Same `comment_check` gate as `CommentAudit` — it drives the same ledger; the
// `gaps:audit` deno task passes the feature.
#[cfg(feature = "comment_check")]
GapAudit(GapAuditCommand),
Compare(CompareCommand),
ConformanceAudit(ConformanceAuditCommand),
AstDiff(AstDiffCommand),
Expand Down Expand Up @@ -117,6 +123,8 @@ impl TopLevel {
Subcommand::BuildFanoutAudit(c) => c.run(),
#[cfg(feature = "comment_check")]
Subcommand::CommentAudit(c) => c.run(),
#[cfg(feature = "comment_check")]
Subcommand::GapAudit(c) => c.run(),
Subcommand::Compare(c) => c.run(),
Subcommand::ConformanceAudit(c) => c.run(),
Subcommand::AstDiff(c) => c.run(),
Expand Down
128 changes: 103 additions & 25 deletions crates/tsv_lang/src/doc/arena.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,21 @@ use super::types::{
TEXT_WIDTH_NOT_COMPUTED,
};

/// Whether a line-flattening walk may delete a **hard** line.
///
/// The two answers are two different operations wearing one walk — see
/// [`DocArena::remove_lines`] (prettier's `removeLines`, [`Self::Keep`]) versus
/// [`DocArena::flatten_all_lines`] (atomize, [`Self::Drop`]). Naming the axis is the point:
/// dropping a hard line deletes a newline the content required, so it is only ever sound
/// where the caller has proved none is required.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum HardLines {
/// Leave hard/literal lines (and `MultilineText`) alone — prettier's `!doc.hard` gate.
Keep,
/// Delete them, fusing whatever sat on either side.
Drop,
}

/// Index into `DocArena.nodes`.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct DocId(u32);
Expand Down Expand Up @@ -1537,14 +1552,61 @@ impl DocArena {
}
}

/// Remove all line breaks from a doc, forcing it to stay on a single line.
/// Statically flatten a doc's **soft and normal** lines.
/// Creates new nodes; old nodes remain in the arena (they're just unused).
///
/// **A hard line is deliberately left alone** — prettier's `removeLinesFn`
/// (`src/document/utilities/index.js`) gates on `!doc.hard` and says why: *"Hard lines
/// should still output because there's too great of a chance of breaking existing
/// assumptions otherwise."* Removing one doesn't relayout the doc, it **deletes a
/// newline the content required**, and the content on either side fuses. A multi-line
/// block comment in a flattened arrow signature is the case that bit us: `/* a⏎b */`
/// came out `/* ab */`, silently gluing `a` to `b` (fixture
/// `typescript/expressions/calls/arrow_array_return_multiline_comment`).
///
/// [`DocNode::MultilineText`] is left alone for the same reason: its `\n`s *are* hard
/// lines — the render arm emits each as a context-indented hardline — merely
/// pre-joined into one pooled body. Flattening it is removing hard lines by another
/// name.
///
/// So this cannot promise a single line, and never could: a caller flattening a doc
/// that contains a hard line gets a shorter doc, not a one-line one. That is prettier's
/// contract too — `expandLastArg` flattens the signature so *breakable* params can't
/// break, not to overrule content that must break. A caller that genuinely needs
/// one line no matter what wants [`Self::flatten_all_lines`] — a different question,
/// and now a different name.
pub fn remove_lines(&self, id: DocId) -> DocId {
self.remove_lines_impl(id, HardLines::Keep)
}

/// Flatten **every** line — hard ones included — onto a single line.
///
/// Not prettier's `removeLines` (that is [`Self::remove_lines`], which keeps hard
/// lines). This is the stronger, un-prettier-like *atomize*: force a doc onto one line
/// even past the print width. Prettier reaches the same place by re-rendering the doc
/// at `printWidth: Infinity` and substituting the resulting string
/// (`template-literal.js`); this achieves it as a doc transform.
///
/// **Only sound where the content provably has no required newline.** Deleting a hard
/// line does not relayout anything — it deletes a newline the content demanded, fusing
/// whatever sat on either side (`/* a⏎b */` → `/* ab */`, gluing `a` to `b`). Its one
/// caller, the template-interpolation atomizer, first routes any interpolation
/// containing a comment or a source newline down a *different* branch, so nothing that
/// must break can reach here.
///
/// The two used to be one function that kept prettier's name while quietly doing this,
/// which is how a multi-line comment in a flattened arrow signature got its newline
/// deleted. Two questions, two names.
pub fn flatten_all_lines(&self, id: DocId) -> DocId {
self.remove_lines_impl(id, HardLines::Drop)
}

fn remove_lines_impl(&self, id: DocId, hard: HardLines) -> DocId {
// Extract node info while borrowing, then release borrow before allocating.
// This pattern avoids RefCell conflicts since alloc() needs borrow_mut().
enum Info {
Keep, // Return id unchanged
MultilineText(String),
FlattenedMultilineText(String),
Line(LineKind),
Indent(DocId),
Dedent(DocId),
Expand All @@ -1568,13 +1630,15 @@ impl DocArena {
let nodes = self.nodes.borrow();
match &nodes[id.index()] {
DocNode::Text(_) | DocNode::LineSuffixBoundary => Info::Keep,
// Flatten: drop the internal hardlines (→ empty) and join the
// lines with no separator — identical to remove_lines over the
// per-line `concat([text, hardline, text, …])` this replaces.
DocNode::MultilineText { span, .. } => {
let pool = self.text_pool.borrow();
Info::MultilineText(span.slice(&pool).replace('\n', ""))
}
// `MultilineText`'s `\n`s are hard lines pre-joined into one body, so it
// follows `hard` for the same reason a `Line(Hard)` does — see the fn docs.
DocNode::MultilineText { span, .. } => match hard {
HardLines::Keep => Info::Keep,
HardLines::Drop => {
let pool = self.text_pool.borrow();
Info::FlattenedMultilineText(span.slice(&pool).replace('\n', ""))
}
},
DocNode::Line(kind) => Info::Line(*kind),
DocNode::Indent(inner) => Info::Indent(*inner),
DocNode::Dedent(inner) => Info::Dedent(*inner),
Expand Down Expand Up @@ -1608,21 +1672,29 @@ impl DocArena {

match info {
Info::Keep => id,
Info::MultilineText(flat) => self.text_pooled(&flat),
Info::FlattenedMultilineText(flat) => self.text_pooled(&flat),
Info::Line(kind) => match kind {
LineKind::Normal => self.text(" "),
LineKind::Soft | LineKind::Hard | LineKind::Literal => self.empty(),
LineKind::Soft => self.empty(),
// Prettier's `!doc.hard` gate: a hard line passes through untouched, because
// removing one deletes a required newline rather than relayouting anything.
// Only `flatten_all_lines`, whose content provably has no required newline,
// drops them.
LineKind::Hard | LineKind::Literal => match hard {
HardLines::Keep => id,
HardLines::Drop => self.empty(),
},
},
Info::Indent(inner) => {
let new_inner = self.remove_lines(inner);
let new_inner = self.remove_lines_impl(inner, hard);
self.indent(new_inner)
}
Info::Dedent(inner) => {
let new_inner = self.remove_lines(inner);
let new_inner = self.remove_lines_impl(inner, hard);
self.dedent(new_inner)
}
Info::Align(n, contents) => {
let new_contents = self.remove_lines(contents);
let new_contents = self.remove_lines_impl(contents, hard);
self.align(n, new_contents)
}
Info::Group {
Expand All @@ -1631,7 +1703,7 @@ impl DocArena {
id: group_id,
should_break,
} => {
let flat_contents = self.remove_lines(contents);
let flat_contents = self.remove_lines_impl(contents, hard);
if should_break {
self.alloc(DocNode::Group {
contents: flat_contents,
Expand All @@ -1647,8 +1719,10 @@ impl DocArena {
let children = self.children.borrow();
DocBuf::from_slice(expanded_states.resolve(&children))
};
let new_kids: DocBuf =
kids.into_iter().map(|kid| self.remove_lines(kid)).collect();
let new_kids: DocBuf = kids
.into_iter()
.map(|kid| self.remove_lines_impl(kid, hard))
.collect();
self.alloc_children(&new_kids)
};
self.alloc(DocNode::Group {
Expand All @@ -1659,25 +1733,29 @@ impl DocArena {
})
}
}
Info::IfBreakFlat(flat_doc) => self.remove_lines(flat_doc),
Info::IndentIfBreakContents(contents) => self.remove_lines(contents),
Info::IfBreakFlat(flat_doc) => self.remove_lines_impl(flat_doc, hard),
Info::IndentIfBreakContents(contents) => self.remove_lines_impl(contents, hard),
Info::Concat(kids) => {
let flattened: DocBuf =
kids.into_iter().map(|kid| self.remove_lines(kid)).collect();
let flattened: DocBuf = kids
.into_iter()
.map(|kid| self.remove_lines_impl(kid, hard))
.collect();
self.concat(&flattened)
}
Info::Fill(kids) => {
// Fill becomes regular concat when flattened
let flattened: DocBuf =
kids.into_iter().map(|kid| self.remove_lines(kid)).collect();
let flattened: DocBuf = kids
.into_iter()
.map(|kid| self.remove_lines_impl(kid, hard))
.collect();
self.concat(&flattened)
}
Info::WithContext(doc, context) => {
let new_doc = self.remove_lines(doc);
let new_doc = self.remove_lines_impl(doc, hard);
self.with_context(new_doc, context)
}
Info::LineSuffix(inner) => {
let new_inner = self.remove_lines(inner);
let new_inner = self.remove_lines_impl(inner, hard);
self.line_suffix(new_inner)
}
Info::BreakParent => self.empty(),
Expand Down
45 changes: 42 additions & 3 deletions crates/tsv_lang/src/doc/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -485,11 +485,50 @@ mod arena_tests {

#[test]
fn test_arena_multiline_text_remove_lines() {
// Flattening drops the internal hardlines (→ empty) and joins the lines
// with no separator — matches `remove_lines` over the per-line concat.
// `remove_lines` must NOT touch a `MultilineText`: its `\n`s are hard lines, and
// prettier's `removeLinesFn` gates on `!doc.hard` precisely so content that must
// break still breaks.
let a = DocArena::new();
let flat = a.remove_lines(a.multiline_text("/*a\n b\n c*/"));
assert_eq!(render_default(&a, flat), "/*a b c*/");
assert_eq!(render_default(&a, flat), "/*a\n b\n c*/");
}

/// The glue this guards against, in the shape that shows it.
///
/// The old behavior joined the lines with no separator, and the case above CANNOT see
/// that: every line of `/*a\n b\n c*/` already starts with a space, so dropping the
/// newlines still rendered `/*a b c*/` — which reads fine. Only a body whose lines
/// would FUSE reveals it, so pin one.
#[test]
fn test_arena_multiline_text_remove_lines_does_not_glue_words() {
let a = DocArena::new();
let flat = a.remove_lines(a.multiline_text("/*text1\ntext2*/"));
assert_eq!(
render_default(&a, flat),
"/*text1\ntext2*/",
"flattening must not fuse `text1` and `text2` into `text1text2`"
);
}

/// A hard line survives; a soft/normal one does not. The whole contract in one case.
#[test]
fn test_arena_remove_lines_keeps_hard_drops_soft_and_normal() {
let a = DocArena::new();
// Normal → space, soft → nothing: the flattening `remove_lines` really does do.
let soft_and_normal = a.concat(&[
a.text("a"),
a.line(),
a.text("b"),
a.softline(),
a.text("c"),
]);
assert_eq!(render_default(&a, a.remove_lines(soft_and_normal)), "a bc");

// Hard and literal → untouched, because removing one deletes a required newline.
let hard = a.concat(&[a.text("a"), a.hardline(), a.text("b")]);
assert_eq!(render_default(&a, a.remove_lines(hard)), "a\nb");
let literal = a.concat(&[a.text("a"), a.literalline(), a.text("b")]);
assert_eq!(render_default(&a, a.remove_lines(literal)), "a\nb");
}

#[test]
Expand Down
22 changes: 21 additions & 1 deletion crates/tsv_svelte/src/printer/nodes/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -630,7 +630,27 @@ impl<'a> Printer<'a> {
.map(|c| self.build_trailing_js_comment_doc(c))
.collect();

self.concat_with_surrounding_comments(leading_docs, expr_doc, trailing_docs)
let body = self.concat_with_surrounding_comments(leading_docs, expr_doc, trailing_docs);

// A leading LINE comment breaks the head (`build_leading_js_comment_doc` ends it
// with a hardline), so what follows starts a genuine continuation line and takes the
// continuation indent — the same shape a width-wrapped head gets:
//
// {#if a && {#if // c1
// b \tcond
// } }
//
// The rule is keyed on *that* the head broke, not on why; without the indent the
// condition sits at base with the `}` dangling below it, closing nothing.
//
// A leading BLOCK comment is deliberately excluded, multi-line or not. It ends with
// a space, never a hardline: a single-line one doesn't break the head at all, and a
// multi-line one's newlines live *inside* its verbatim source span, which renders
// with no context indent by design (the interior stays as authored), so its
// continuation is the comment's own line and there is nothing to indent.
let breaks_head =
comments_to_emit_in_range(self.comments, span_start, expr_start).any(|c| !c.is_block);
if breaks_head { d.indent(body) } else { body }
}

/// Build a block head's expression doc, deriving both the comment-scan start offset
Expand Down
Loading