Skip to content

schemachange: refuse a new column the copy cannot fill at the shadow build - #274

Open
Kiran01bm wants to merge 1 commit into
kiran01bm/cs-pd9-identity-column-refusalfrom
kiran01bm/cs-pd10-unfillable-new-column
Open

Kiran01bm wants to merge 1 commit into
kiran01bm/cs-pd9-identity-column-refusalfrom
kiran01bm/cs-pd10-unfillable-new-column

Conversation

@Kiran01bm

Copy link
Copy Markdown
Collaborator

The shadow build now refuses, as shadow-statement-new-column, a gated statement that adds a column the copy cannot fill or that leaves the shadow without the source's primary key, instead of committing a shadow whose first chunk fails.

Why

The copy list names only the source columns the shadow kept as the same column; every other shadow column is a new column that takes its default on every copied row. The re-add check covers a column dropped and added back under its own name, but a column added beside the other changes, or the key dropped or replaced under a new name, went unchecked: the build committed a shadow, wrote the checkpoint row, and the first chunk failed — with the server's not_null_violation, which the run treats as a data fault and abandons with code: data-fault, for a statement no data could have satisfied. From the CLI the shape is one multi-operation statement away: ADD COLUMN … NOT NULL alone is routed native and rejected by the server, but beside an ALTER COLUMN TYPE it rides into the copy-and-swap statement and reaches the build.

What

  • refuseUnfillableNewColumns runs right after the re-add check, inside the build's transaction: it refuses when the primary key is not among the kept columns, and when any other shadow column the copy leaves out refuses NULL (its own NOT NULL or a domain's) with no default, generation expression, or identity to fill it. Nothing is committed and no checkpoint row is written.
  • New cause shadow-statement-new-column (capability-boundary, ST-11); the message names the column and the remedy.
  • Docs: refusal-classes, limitations (new row; the re-add row's "still builds and stops at the first chunk" list shrinks to the CHECK case), invariants ST-11, capabilities reason note, README.

Before / after

The same statement on orders(id bigserial PK, qty integer NOT NULL, note text, sku text) with rows in it: ALTER TABLE orders ALTER COLUMN qty TYPE bigint, ADD COLUMN region text NOT NULL. What changed is where it stops; the retype-only statement beside it, and a new column with a default, build and run exactly as before.

Before                                              After
──────                                              ─────
planner: retype → copy-and-swap;                    planner: same route, same ride-along
         ADD COLUMN NOT NULL rides along

BuildShadow                                         BuildShadow
  CREATE TABLE orders_shadow (LIKE orders)            CREATE TABLE orders_shadow (LIKE orders)
  run the statement on the empty shadow               run the statement on the empty shadow
  read both relations' column positions               read both relations' column positions
  copy list = [id qty note sku]  (region left out)    copy list = [id qty note sku]  (region left out)
  readd check: region was not dropped → passes        readd check: region was not dropped → passes
  (no check on other left-out columns)                new-column check: region refuses NULL,
                                                        no default, not generated, no identity
  COMMIT  ─▶ shadow exists, checkpoint row written     ROLLBACK ─▶ no shadow, no fence, no row
                                                      refused: shadow-statement-new-column
first chunk                                             "adds column region, which refuses NULL
  INSERT INTO orders_shadow (id, qty, note, sku) …       and has no default … give the new column
  ─▶ 23502 not_null_violation on region                  a default, or add it in a later schema
  ─▶ classed as a data fault: shadow dropped,            change the server applies in place"
     slot/fence released, checkpoint row deleted
  verdict: failed, code: data-fault                   verdict: refused, capability-boundary
  (the operator is told to fix their data)            (the operator is told what to change)

The other shape, DROP COLUMN id, ADD COLUMN new_id bigint PRIMARY KEY, ALTER COLUMN qty TYPE bigint, is refused by the same check: it too built and committed a shadow before, one the copy would have to drive by id, which the shadow no longer has.

🤖 Generated with Amp (Claude Opus 4.6)

…build

A column added beside the gated changes, or the key dropped or replaced
under a new name, built and committed a shadow that failed at the first
chunk and was reported as a data fault. Refuse it in the build's
transaction instead, as shadow-statement-new-column.
@Kiran01bm
Kiran01bm marked this pull request as ready for review October 28, 2026 21:42
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-28T21:42:27.076846Z
We couldn’t complete this review. Comment @codex review to try again.
51f5765 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Kiran01bm
Kiran01bm requested a review from morgo as a code owner October 28, 2026 21:42
@JashLal

JashLal commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

🤖 Reviewed the full PR diff at 51f5765407e06ba0355f827430564be184e9da06. No concrete blocking defect found in the changes inspected.

The new guard checks that the source key remains among the kept columns before examining omitted shadow columns. Reusing requiresAValueTheCopyCannotGive keeps the new-column decision aligned with the re-add guard, and the call in buildShadow preserves the ordering of re-add, new-column, and new-sequence refusals. The new cause is registered as ST-11 / capability-boundary in code, classification tests, and documentation. The Runner regression asserts the important failure outcome: no shadow, write fence, or checkpoint row, with the source still writable.

One non-blocking coverage suggestion: pkg/schemachange/new_column_integration_test.go:75 (TestBuildShadowAdmitsANewColumnTheServerFills) currently checks successful construction and CopyColumns(). Consider a Runner case with actual source rows and ADD COLUMN priority integer NOT NULL DEFAULT 0, then assert the final row values after copy and cutover. That would directly cover the documented “builds and runs” behavior alongside the early-refusal regression; the present test does not exercise filling the omitted column during the copy.

Validation limits: all 16 reported CI checks are successful, including the PostgreSQL 14–18 jobs. I did not run a local build, tests, or database reproduction. The exact-head surrounding-source API request was unavailable in this execution environment, so I could not independently inspect the unchanged implementations of sameColumns, requiresAValueTheCopyCannotGive, or transaction cleanup. This review covers the complete diff and PR context, with those dependencies unverified.

@JashLal JashLal left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approving on Jash's behalf after the full-diff review at 51f5765. No concrete blocker found; the optional end-to-end default-fill test and exact-head surrounding-source verification limits are recorded in the review. No local build or tests were run. Generated by Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants