Skip to content

Keep stored external payloads readable after a namespace storage policy is disabled or moved #546

Closed

Description

@rmcdaniel

Delivered source

Native Server PR #1224 merged at e363a472dd9183a66d31e1554824eaceb595253b; its tree exactly matches qualified head 1c4fecd1c2dc6923ccd6767d9bdb12285bd1fc14. All 63 pre-merge checks, merge-queue checks and the four actual-main workflows passed. Original object bindings now preserve retained reads and cleanup after disabling or moving the write policy. SQLite, PostgreSQL, MySQL and MariaDB coverage includes upgrade/takeover and interrupted migrations. Local and remote topic branches are absent and the worktree is clean. This is accepted native source; no released-artifact qualification is claimed. The retired PHP line is unchanged.

Problem

An admin can disable or move a namespace's external payload storage policy with one API call. Every payload already offloaded then becomes unreadable through the Server, even though each object's full location is recorded:

Workers can't fetch inputs for retained runs, and history reads and queries fail. This is source-verified, not reproduced.

Acceptance criteria

  • Disabling a policy stops new offloads only. Stored payloads stay readable, and retention and namespace deletion can still remove them.
  • After a driver, bucket or prefix change, earlier objects stay readable and deletable, or the change is refused with a clear reason while stored objects depend on it, with a documented migration path.
  • An unresolvable stored object returns a specific reason, not "temporarily unavailable".
  • Feature tests offload a payload, disable the policy and change its prefix, then fetch the payload and prune its expired run.

Activity

  1. added
    bugSomething isn't working
    kind:defectA public product behavior is incorrect
    priority:P2Normal-priority product work
    on Oct 20, 2026
  2. rmcdaniel commented on Nov 4, 2026

    @rmcdaniel
    MemberAuthor

    Server open-item pass, 2026-11-04.

    Owner: Server for native storage-policy/read authority, General for shared compatibility. Disabling new offloads must not silently reinterpret or discard the location/authority of retained payloads. Current-policy routing and a retained object's original storage binding need separate treatment.

    Next source proposal chooses immutable per-object policy-version resolution or a typed refusal to move a still-referenced provider, with a documented migration path. Cover disable, prefix/bucket/driver change, read/delete/retention, missing credentials/provider and cross-namespace rejection. General: choose that compatibility boundary before a new policy producer is written. No policy, credential, provider object or retention state changed.

  3. rmcdaniel commented on Nov 11, 2026

    @rmcdaniel
    MemberAuthor

    Triage 2026-11-11: reproduces on native: stored references are read through the current policy, so disabling it returns 503 (external_resolution.rs, external_transport.rs). Next: resolve already-stored references regardless of enabled (ready). The move/prefix half needs a decision; asked General. Owner: server.

  4. rmcdaniel commented on Nov 11, 2026

    @rmcdaniel
    MemberAuthor

    Decision (General, 2026-11-11): already-stored references keep their original location and stay readable; policy changes (disable, move, prefix) apply to new writes only. Unblocked; next: server implements. Owner: server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions