Skip to content

feat(sync): define recovery semantics for incomplete chunk history聽#904

Description

@dnlrsls

馃搵 Pre-flight Checks

  • I have searched existing issues and this is not a duplicate
  • I understand this issue needs status:approved before a PR can be opened

馃攳 Problem Description

Local chunk sync has no single recovery contract for incomplete history. A manifest can reference chunks that are unavailable on the current machine, while existing chunks can contain partial session, prompt, observation, relation, or tombstone history.

The current behavior mixes several decisions: when missing history should be treated as unknown, when local SQLite should re-export idempotently, when import should defer a dependency, when corruption must fail loudly, and how long histories should be scanned without unbounded work. PR #780 experimented with several of these policies while solving #615, but #894 delivered only the scoped observation-watermark fix. The remaining lifecycle policy needs its own design before implementation.

馃挕 Proposed Solution

Define one explicit recovery contract for local and cloud sync:

  • distinguish unavailable history from corrupt readable history;
  • keep local SQLite as the source of truth and never silently drop data;
  • specify when to re-export idempotently, rebuild derived history, defer dependencies, or stop;
  • align dependency ordering and terminal lifecycle states across local and cloud import;
  • define bounded behavior for full-history scans;
  • cover missing, corrupt, partial, tombstoned, and dependency-stalled histories with deterministic push and pull tests.

Implementation should be split into focused work units only after the contract is approved.

馃摝 Affected Area

Sync (multi-instance)

馃攧 Alternatives Considered

Continuing the global timestamp-only watermark can silently miss identities. Porting the accumulated PR #780 logic would mix multiple roots and unresolved policies. A destructive full reset would discard useful delivery state and violate local-first expectations.

馃搸 Additional Context

PR #780 was closed after PR #894 resolved issue #615. Related but non-equivalent issues include #595 (explicit full re-sync after project migration), #649 (orphan relation import stalls), and #849 (dead-row retention). This tracker owns only the incomplete-history recovery contract.

Activity

  1. jemanuelp commented on Sep 17, 2026

    @jemanuelp
    Sponsor

    PR #1226 surfaced a concrete recovery case within this issue's scope: a manifest can retain a chunk ID after ReadChunk returns ErrChunkNotFound. Incremental export then omits historical observations and prompts, while a new content hash does not replace the stale manifest entry.

    Please update this issue with the Bug Report form's required reproduction, expected and actual behavior, environment, version, and agent fields so it can be the conforming tracker for this case. Source: #1226 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions