feat(memory): add compact Atomic Memory directory

[wutongyuonce]

Which issue or RFC does this PR close?

Refs #1656. Follows the Atomic Memory/shared Artifact model in #1809/#1859 and the explicit migration boundary in #1771.

Rationale for this change

Browsing Atomic Memory identities should not load complete content or history. This adds a compact current-head directory with exact Artifact references. A content, lifecycle, merge/restoration or effective tag change invalidates continuation; callers restart from page one.

What changes are included in this PR?

• Add POST /v1/atomic-memory/directory/query, query_atomic_memory_directory in the HTTP SDK/MCP, and the Python scoped query. OpenAPI owns the generated contracts; Codex session Scope binding includes the operation. Regenerate Desktop wire/API contracts and the distributed DSH bundle; include the additive tool in Skill mock catalogs and negative controls.
• Read compact shared heads with state/kind/tag selection before keyset limits, signed query/subject/policy/generation cursors, and response budgets. Directory requests expand no body, Source or history. Formal owner readiness still covers the whole Scope; SQLite may scan it, so this is not a constant database-work claim.
• Add no pagination tables. SQLite writer locks/increments reject noninteger stored generation values, preventing publication with a damaged counter. Extend Scope generation and head kind through frozen resources, and maintain both through shared publication, governance, tags and merge/restoration transactions. Retain caller-owned transactions and defer audit writes. All directory-enabled lock paths acquire the Scope boundary before heads/content.
• Register the optional persistent SQLite directory target in the existing server db-migrate entry point and single revision table. Append frozen p0004/p0005 resources, current-head backfill/validation, shared batch commits and original-backup recovery. The default four-table acceptance target remains p0003; released revision/env bytes are unchanged.
• Keep ordinary startup from adding or backfilling directory fields. Unfinished recorded maintenance prevents initializer writes. A structurally valid managed directory retains writer maintenance while damaged data disables querying. Topic-only child runtimes do not scan unrelated Atomic bodies.
• Update bilingual workflow documentation, the optional MCP tool declaration, and migration/application/transport regressions.

Are there any user-facing changes?

The additive directory returns compact exact references with default limit 50 / maximum 100. Continuation must restart after changes. Full-record APIs retain their current contracts.

Production enablement in this PR is restricted to the registered default current SQLite business layout, ordinary persistent files, no embedding extension, and trusted-local or qualified same-database Builtin/Casbin Scope-read authorization. Stop all writers, run server db-migrate plan/apply/verify --target atomic-memory-directory --env-file deployment.env, then restart the matching program. The target uses the database-adjacent default evidence directory. Migration does not invent owners.

Unmanaged targets and unsupported access configurations fail explicitly. Unregistered historical/archive layouts, vector configurations and seekDB/OceanBase directory execution are not supported by this target. Existing initializers in other domains remain unchanged; this PR does not complete the entire migration RFC rollout. Shared migration output remains readiness_scope=registered_bundle, server_ready=false. Restore under stopped writes and rotate the existing cursor signing secret before resuming.

How was this change tested?

• Final-head .venv/bin/python -m pytest -n 2 --doctest-modules tests -q: 5,290 passed, 429 skipped. Skips/deselections do not establish external backend, native-host or real-model acceptance.
• Final-head directory storage/application/migration core on Python 3.11, 3.12 and 3.14: 45 passed on each version, including the integer-storage guard regression. Current directory/migration/Handoff/API focused tests: 137 passed. Python 3.13 is covered by the full run; full four-version tox was not run.
• Formal API generation and Python/JS contract tests: 52 passed. All 11 pre-commit hooks and full ty check passed.
• Final-head hosted DSH package, native personal service and Windows Desktop installer/smoke checks passed. Skill guidance: 28 local tests, checksum-verified skill-up v0.12.0 schema validation and dry-run passed.
• make docs-test: lint/test/static build passed, with 921 pages and internal links verified. sdist/wheel builds passed; packaged code and frozen resources were checked against the source.
• Installed final wheel in an isolated environment and ran the unified CLI against a fresh SQLite target. A real listening Server with enforced authentication passed HTTP SDK first-page, network MCP continuation/read-only tool metadata, exact revision reads, write-invalidated cursor and missing-token 401 checks.
• Actual Server-created old business databases preserve exact history and Server identity across migration; 201 records exercise multiple committed batches. Interrupted and bad later batches retain the original recovery point, resume safely, and do not manufacture changes on repeated success. Post-completion data damage fails validation while managed writers retain generation maintenance.
• Eight concurrent same-Scope writes produced eight records and generation 8. The keyset plan uses the existing composite primary-key index. A native SQLite snapshot/restore plus signing-key rotation rejects pre-restoration cursors.

Final-head GitHub CI: 35 checks passed, 0 failed, 0 pending, including Python 3.11–3.14 unit/e2e, hosted quality, Skill guidance, ZCode, Windows Desktop, DSH, native services, documentation and existing backend acceptance/clean-migration checks. CLA status also passed. The separate optional Desktop installed-package job was skipped and is not counted as passed; the Windows job built the installer and ran its installed-package smoke.

Validation limit: Local make check stops at uv lock --locked: the local uv installation reports a pyproject.toml/uv.lock mismatch. The same failure was reproduced in clean checkouts of upstream b5bcfe24 and b9edaa5b; this contribution changes no dependency manifest or lockfile. All hooks/type checks were run separately, and final-head hosted quality passed. Current head 83213498 contains upstream ee50b36d, including the upstream ZCode pin fix. Existing backend regressions do not establish seekDB/OceanBase directory support. The PR has no merge conflicts; maintainer review and design acceptance are still required.

AI usage statement

OpenAI Codex assisted implementation, local source/contract review and validation. Local tests, real network/package evidence and unavailable external-backend checks are reported separately; no independent or hosted review pass is claimed.

[wutongyuonce]

@Teingi CI update: the previous dsh-package failure was caused by a stale checked-in bundle. This is fixed in 07375594, and the full DSH job now passes, including the real runtime checks. The conflicts with current master are also resolved.

23 checks are now passing, including Python 3.11–3.14 and OceanBase acceptance. The only remaining failure is Acceptance (sqlite). Its uploaded exception evidence shows that the acceptance, LoCoMo, and fail-fast batches all stopped during agent dependency installation, before scenario execution:

Failed to fetch: https://pypi.org/simple/inquirer-textual/
HTTP status server error (503 Service Unavailable)

The download failed after three retries. This remaining failure is an external dependency-download error, not a pagination assertion failure; it does not establish a SQLite acceptance pass yet.

Could a maintainer please rerun the failed SQLite acceptance job? My attempt to rerun it was rejected with HTTP 403 due to repository permissions.

[Zxf-xufeng]

Minimize additional tables and justify their responsibilities.
This PR introduces pc_memory_entry_directory, pc_memory_tag_generations, and pc_memory_query_index_schema. Please evaluate whether extending existing tables can reduce this footprint. For example, the tag generation could potentially be stored on the owning Artifact head, while migration state should be reviewed alongside the unified migration design.
A separate directory projection may still be justified for efficient historical pagination. Please document why existing tables cannot support it through appropriate extensions. Any alternative must preserve revision-pinned traversal, deactivation/reactivation history, compaction behavior, and bounded reads. Moving an unbounded history into a JSON field would require performance evidence before being considered equivalent.

[Zxf-xufeng]

Align schema changes and historical backfills with PR #1771.
#1771 proposes a unified database migration process. This PR currently adds startup table creation and marker writes, plus a separate memory-query-migrate command that independently creates tables, backfills data, and publishes readiness. Its standalone verify also updates the database.
Please separate schema migration, projection backfill, and readiness verification according to that proposed process:

• Deliver schema changes through versioned migrations, with historical projection reconstruction as an explicit, idempotent data task.
• Keep ordinary startup and inspection commands read-only for existing databases; publish projection readiness during explicit migration execution.
• Use the unified execution boundary for maintenance, locking across batches, backup choices, and final verification.
• Distinguish schema readiness from projection readiness. Legacy APIs may remain available during an optional backfill only after their required schema has been migrated and validated.
• Cover interrupted/resumed backfills and verification failures, including a previously complete projection that subsequently fails verification.
  docs(rfc): define unified versioned database migrations #1771 is still a proposal, so this does not necessarily require waiting for the framework to ship. If this PR lands before framework enablement, please document its baseline adoption path. If it remains unmerged when the framework is enabled, it should follow the unified migration process rather than retain an independent migration authority.

[PsiACE]

@wutongyuonce Thanks for the implementation, tests, and scale validation in this PR. We are reviewing unified database migrations (#1771) and the Memory redesign (#1809, dependent on #1803). These designs would affect the directory projection, revision cursors, and migration approach used here.

We therefore plan to close this PR for now and keep the pagination requirement in #1656 open until those designs are settled. We'd welcome your continued involvement, and the work you've done provides a useful reference for the next implementation.

[hidb4ai]

please update & resolve conflicts

[hidb4ai]

please update & resolve conflicts

[hidb4ai]

please update & resolve conflicts