diff --git a/docs/en/docs/operate/atomic-memory-migration.md b/docs/en/docs/operate/atomic-memory-migration.md index 463b3b0590..0f07738e15 100644 --- a/docs/en/docs/operate/atomic-memory-migration.md +++ b/docs/en/docs/operate/atomic-memory-migration.md @@ -59,7 +59,9 @@ Validation requires versions continuous from 1, consistent predecessor IDs, cont manifest references. The current manifest pointer, or the last pointer before compaction, must identify that entry's chain tail. Unexplained absence, missing changes, duplicate identities, skipped versions and lagging heads block conversion. Legacy active, inactive and compacted entries become active, forgotten and -retired respectively. Family state versions and common head governance summaries agree. +retired respectively. Lifecycle is authoritative in `pc_artifact_heads`: these states use `lifecycle_state` +values `active`, `deprecated` and `retired`, respectively, and newly imported heads have no `merged_into_id`. +Atomic Memory reads the head's `governance_generation` as its `state_version`. A formal Owner on an exact legacy entry resource becomes the new memory's Owner. Without enforced access, an entry without an Owner migrates without one. Several Owners on one entry or an invalid Owner block @@ -177,9 +179,9 @@ Cursors, CAS generations, high-water marks, pending/flush requests, accepted tas unchanged. Family `memory` and binding `memory-source-window` remain scheduling aliases. Old leases are invalidated and their fences advance before maintenance completes. Consumed Sources are not extracted again. -Each entry's history, head, state, Owner, tags, grant conversion and current projection commit together. +Each entry's history, head, Owner, tags, grant conversion and current projection commit together. Embedding preparation occurs outside the transaction. Repeat apply with the same configuration after an -interruption. Existing targets must match the exact imported history; differing data or orphan content/state +interruption. Existing targets must match the exact imported history; differing data or orphan content is rejected without overwriting authority. Subsequent revisions or lifecycle changes never cause the task to reset a target's head, state or tags. @@ -207,7 +209,7 @@ powercontext server atomic-memory-rebuild-projection --env-file .env --maintenan ``` This command checks that legacy objects, columns and foreign keys have been removed from public storage and -that Atomic heads, content and states are valid. It rebuilds only active current rows and removes nonactive +that Atomic heads, content revisions and head lifecycle data are valid. It rebuilds only active current rows and removes nonactive or orphan current rows. Heads, content revisions, lifecycle states, retained history and Source/processing progress keep their exact identities and values. Rebuilding reads neither the archive nor legacy entry tables. The offline migration `verify` command checks whether legacy history was imported completely; projection @@ -254,4 +256,4 @@ A current table created during development of this branch (with copied read gran to no release and is rejected at initialization. Before migration completes, apply drops and recreates that derived table and its report asks for a projection rebuild; after migration, stop writers and run the projection rebuild command above. Either way only that derived table is rebuilt from retained authority; -Artifact history, Family state and grant records are unchanged. +Artifact history, head lifecycle data and grant records are unchanged. diff --git a/docs/en/docs/workflows/atomic-memory.md b/docs/en/docs/workflows/atomic-memory.md index 7f73ce004b..8f8e998b68 100644 --- a/docs/en/docs/workflows/atomic-memory.md +++ b/docs/en/docs/workflows/atomic-memory.md @@ -136,10 +136,14 @@ Restore directly through `POST /v1/atomic-memory/restorations`: {"scope_id": "S", "target": {"artifact_id": "B"}} ``` -Restoring forgotten content makes it active. Restoring an already active memory without a revision returns unchanged. +Restoring forgotten content makes it active. Restoring an already active memory without a selected revision returns +`changed: false` and appends no revision. Restoring a merged input undoes the later merges that froze it. For A+B→C followed by C+D→E, restoring B makes A, B and D active and retires C and E. Ordinary downstream Artifacts and Source cursors are not rolled back. -Supplying `target.revision` saves that historical content as a new revision of the target. +Every memory actually restored or retired appends a revision under its existing Artifact identity, numbered as its current +revision plus one. Restored inputs use their frozen content; supplying `target.revision` uses that historical content for the +target's new revision. Existing revisions and the original merge relationships remain intact, and exact historical reads +continue to return the original content. Current reads and search follow the restored revisions. Inspect the impact first through `POST /v1/atomic-memory/restoration-previews`: @@ -161,9 +165,20 @@ After inspecting it, send the original token to restorations with the same princ ``` Success returns `changed`, exact resulting `restored` references, `retired` and `undo_merge_results`. +When `changed: true`, both `restored` and `retired` reference the newly appended revisions of the objects actually changed. +A no-op returns `changed: false` and retains the current primary Artifact reference in `restored`, without appending a revision. To undo the merge that created C, use `operation: "undo_merge"` and `target: {"artifact_id": "C"}`. That operation cannot also select a content revision. +The public Python runtime can read the exact historical group result using the operation's primary new revision: + +```python +outcome = await runtime.artifact_merge.for_scope("S", "atomic-memory").restoration_outcome("B", revision=2) +``` + +Replace `2` with the primary revision produced by that operation. The method returns `None` if that revision has no stored +restoration outcome and requires read permission for every affected Artifact. + | Error | HTTP status | Action | | --- | --- | --- | | `invalid_preview` | 422 | Check the token, principal, Scope, operation, target and shared signing configuration | @@ -173,7 +188,7 @@ That operation cannot also select a content revision. | `atomic_memory_changed` | 409 | Read the changed content/state before deciding to retry | | `invalid_memory_relation` | 409 | Investigate inconsistent stored relationships | -Restore and merge have no `idempotency_key` or durable receipt for replaying an earlier response. A direct restoration +Restore and merge have no `idempotency_key` and do not automatically replay an earlier response. A direct restoration interprets the relationships current on each invocation; repeating it after another merge can undo that newer merge. Replaying a successful request with a token can return `preview_stale`. If the connection fails during commit, inspect current states before deciding to preview again. The Python SDK does not blindly retry writes with an unknown outcome. diff --git a/docs/zh/docs/operate/atomic-memory-migration.md b/docs/zh/docs/operate/atomic-memory-migration.md index 2b6dc8f8dd..4c87132f26 100644 --- a/docs/zh/docs/operate/atomic-memory-migration.md +++ b/docs/zh/docs/operate/atomic-memory-migration.md @@ -53,7 +53,9 @@ revision 不会产生该条记忆的新 revision。 当前 manifest 或 compact 前最后一次出现的指针必须指向本 entry 的版本链末端。 缺失变化记录、无法解释的缺席、重复身份、跳过版本或落后的 head 都会阻断；迁移不会选取最大版本 来掩盖不一致。旧 active、inactive、compact 分别成为 active、forgotten、retired。 -状态版本计数与公共 head 的治理摘要同步。 +生命周期以 `pc_artifact_heads` 为权威：这三种状态分别使用 `lifecycle_state` 值 `active`、`deprecated`、 +`retired`，新导入的 head 不设置 `merged_into_id`。Atomic Memory 的 `state_version` 读取 head 的 +`governance_generation`。 旧 entry 的精确资源身份上有正式 Owner 时，它迁为新记忆的 Owner；未启用鉴权的部署中没有 Owner 的 entry 迁移后同样没有 Owner。同一 entry 有多个 Owner 或 Owner 不合法时会阻断，集合 Owner 不会分配给 @@ -156,9 +158,9 @@ Source Cursor、CAS generation、高水位、pending/flush 请求、已接受任 `memory` Family 与 `memory-source-window` binding 继续作为调度兼容身份。 迁移完成前失效旧 Lease 并推进 fence，已消费的 Source 不重新抽取。 -每条记忆的历史、head、状态、Owner、标签、授权转换与当前投影在同一个事务提交。 +每条记忆的历史、head、Owner、标签、授权转换与当前投影在同一个事务提交。 向量准备在事务外完成。中途退出后，可以使用相同配置重复 apply；已提交对象先核验精确导入历史。 -与确定性身份对应的内容不同、存在孤立状态或历史时明确报错，不覆盖目标数据。 +与确定性身份对应的内容不同或存在孤立正文历史时明确报错，不覆盖目标数据。 已经导入的对象后续产生新 revision 或生命周期变化时，迁移不回退其 head、状态或标签。 如果已转换的授权仍保留旧资源摘要，plan 和 verify 会返回 `ready: false`，并在 @@ -179,7 +181,7 @@ API、宿主、Worker、自动重启和输入写入，然后使用目标部署 powercontext server atomic-memory-rebuild-projection --env-file .env --maintenance-confirmed ``` -命令检查公共表中的旧对象、旧列和外键已清理，且 Atomic 的 head、正文和状态有效。它只重建 active 的 +命令检查公共表中的旧对象、旧列和外键已清理，且 Atomic 的 head、正文 revision 和 head 生命周期数据有效。它只重建 active 的 current 行，清除非 active 或孤立的 current 行；head、正文 revision、生命周期状态、保留历史及 Source/处理进度保持原身份和原值。重建不读取归档或旧 entry 表；旧历史是否已完整导入，由停服迁移的 `verify` 负责核验，投影重建不能补做缺失的历史导入。 @@ -219,4 +221,4 @@ plan、apply 和 verify 读取保留的全部旧历史和 Source/任务快照， 本分支开发过程中创建的 current 表（含读取授权副本或 Owner 列）不属于任何发布版本，初始化时会被 明确拒绝。迁移尚未完成时，apply 会删除并重建这张派生表，结果报告提示随后执行投影重建；迁移已完成 时，停服后直接执行上述投影重建命令。两种方式都只重建这张派生表，并从正式记录恢复活跃记忆， -不改变 Artifact 历史、Family 状态和授权记录。 +不改变 Artifact 历史、head 的生命周期数据和授权记录。 diff --git a/docs/zh/docs/workflows/atomic-memory.md b/docs/zh/docs/workflows/atomic-memory.md index 7af625e967..e6ec5f50c2 100644 --- a/docs/zh/docs/workflows/atomic-memory.md +++ b/docs/zh/docs/workflows/atomic-memory.md @@ -131,9 +131,11 @@ Atomic Memory 将每条事实、偏好或决策保存为独立的 `atomic-memory {"scope_id": "S", "target": {"artifact_id": "B"}} ``` -恢复 forgotten 使其重新在役；恢复已经 active 且未指定 revision 的记忆返回未变化。 +恢复 forgotten 使其重新在役；恢复已经 active 且未指定 revision 的记忆返回 `changed: false`，不追加版本。 恢复 merged 会撤销冻结该目标的后续合并。例如 A+B→C、C+D→E，恢复 B 会让 A、B、D 在役，C、E 退役。 -普通下游 Artifact 和 Source cursor 不随之回滚。指定 `target.revision` 时，将所选历史正文保存为目标的新 revision。 +普通下游 Artifact 和 Source cursor 不随之回滚。每个实际恢复或退役的记忆均在原 Artifact 身份下追加新 revision， +版本号为其当前 revision 加一。恢复输入使用被冻结时的正文；指定 `target.revision` 时，目标的新版本使用所选历史正文。 +旧版本和原始合并关系保留，精确历史读取仍返回原内容；当前读取和检索跟随恢复后的新版本。 先查看影响范围可调用 `POST /v1/atomic-memory/restoration-previews`： @@ -155,9 +157,19 @@ operation 和 target 向 restorations 提交原 token： ``` 成功响应返回 `changed`、恢复后的精确 `restored` 引用、`retired` 和 `undo_merge_results`。 +`changed: true` 时，`restored` 和 `retired` 均引用实际变化对象所追加的新版本。 +no-op 返回 `changed: false`，`restored` 保留当前主制品的引用，不追加版本。 若要撤销创建结果 C 的合并，改用 `operation: "undo_merge"`、`target: {"artifact_id": "C"}`； 该操作不能同时指定内容 revision。 +公共 Python runtime 可以按本次操作的主新版本查询精确的整组历史结果： + +```python +outcome = await runtime.artifact_merge.for_scope("S", "atomic-memory").restoration_outcome("B", revision=2) +``` + +将 `2` 替换为本次操作产生的主版本。该版本没有保存恢复结果时返回 `None`；读取需要所有受影响制品的读取权限。 + | 错误 | HTTP 状态 | 处理 | | --- | --- | --- | | `invalid_preview` | 422 | 检查 token、主体、Scope、operation、target 及共享签名配置 | @@ -167,7 +179,7 @@ operation 和 target 向 restorations 提交原 token： | `atomic_memory_changed` | 409 | 当前内容或状态已变，重新读取后决定是否重试 | | `invalid_memory_relation` | 409 | 关系数据不一致，需要排查存储数据 | -恢复与合并没有 `idempotency_key`，不返回某次旧请求的持久回执。不带 token 的恢复按每次调用时的当前关系解释； +恢复与合并没有 `idempotency_key`，不会自动重放某次旧请求的响应。不带 token 的恢复按每次调用时的当前关系解释； 目标后来又被合并，重复请求可能撤销新的合并。携带 token 的成功请求重发也可能返回 `preview_stale`。 连接在提交时中断，应先读取各对象状态确认结果，再决定是否重新预览。Python SDK 不盲目重试结果未知的写入。 签名密钥、有效期及直接恢复重试预算见[配置](../operate/configuration.md#atomic-memory)。 diff --git a/src/powercontext/builtin/artifacts/atomic_memory/restoration.py b/src/powercontext/builtin/artifacts/atomic_memory/restoration.py index bcde25d51d..f53222c80f 100644 --- a/src/powercontext/builtin/artifacts/atomic_memory/restoration.py +++ b/src/powercontext/builtin/artifacts/atomic_memory/restoration.py @@ -12,285 +12,21 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Whole-group restoration planning and independently typed signed previews.""" - -from __future__ import annotations - -import base64 -import binascii -import hmac -import json -from collections.abc import Awaitable, Callable, Mapping -from datetime import UTC, datetime, timedelta -from typing import Literal, cast - -from pydantic import BaseModel, ConfigDict, StrictInt, ValidationError - -from powercontext.artifacts import ArtifactRef -from powercontext.builtin.artifacts.atomic_memory.errors import ( - AtomicMemoryPreviewExpiredError, - AtomicMemoryRelationError, - InvalidAtomicMemoryPreviewError, - InvalidAtomicMemoryStateError, -) -from powercontext.builtin.artifacts.atomic_memory.models import ( - AtomicMemory, - AtomicMemoryDraft, - AtomicMemoryPlan, - AtomicMemoryRead, - AtomicMemoryRecord, - AtomicMemoryRestoreItem, - AtomicMemoryRestoreOperation, - AtomicMemoryStateValue, - AtomicMemoryWrite, -) - - -def merge_inputs(result: AtomicMemory) -> tuple[ArtifactRef, ...]: - """Resolve only selected exact inputs; ordinary lineage is not a merge relation.""" - - creation = result.content.creation - if result.revision != 1 or creation is None: - raise AtomicMemoryRelationError("the result has no valid merge creation revision") # noqa: TRY003 - selected: list[ArtifactRef] = [] - for artifact_id in creation.input_artifact_ids: - refs = tuple( - ref - for ref in result.lineage.artifacts - if ref.family == AtomicMemory.family and ref.artifact_id == artifact_id +"""Atomic preview wire compatibility over the shared endpoint signer.""" + +from powercontext.builtin.artifacts.atomic_memory.models import AtomicMemoryRead +from powercontext.builtin.artifacts.atomic_memory.service import ATOMIC_MERGE_ERRORS +from powercontext.builtin.artifacts.merge_restoration import ArtifactMergePreviewSigner + + +class AtomicMemoryPreviewSigner(ArtifactMergePreviewSigner): + def __init__(self, *, keys, active_key_id, ttl_seconds=300, clock=None): + super().__init__( + keys=keys, + active_key_id=active_key_id, + ttl_seconds=ttl_seconds, + clock=clock, + endpoint_type=AtomicMemoryRead, + preview_format="powercontext.atomic-memory.restoration-preview.v1", + errors=ATOMIC_MERGE_ERRORS, ) - if len(refs) != 1 or artifact_id == result.artifact_id: - raise AtomicMemoryRelationError("merge creation must select exactly one distinct exact input reference") # noqa: TRY003 - selected.append(refs[0]) - return tuple(selected) - - -async def calculate_restoration( - *, - scope_id: str, - subject: str, - target: AtomicMemoryRecord, - operation: AtomicMemoryRestoreOperation, - selected: AtomicMemory | None, - load_current: Callable[[str], Awaitable[AtomicMemoryRecord]], - load_first: Callable[[str], Awaitable[AtomicMemory]], -) -> AtomicMemoryPlan: - """Follow the current merge path, then interpret it from its endpoint inward.""" - - if target.state.state is AtomicMemoryStateValue.RETIRED: - raise InvalidAtomicMemoryStateError("retired memory identities cannot be restored") # noqa: TRY003 - if operation == "undo_merge" and selected is not None: - raise InvalidAtomicMemoryStateError("undo_merge does not accept a content revision") # noqa: TRY003 - records = {target.artifact.artifact_id: target} - result_ids: list[str] = [] - cursor = target - while cursor.state.state is AtomicMemoryStateValue.MERGED: - next_id = cursor.state.merged_into_id - if next_id is None or next_id in records: - raise AtomicMemoryRelationError("the current merge path is missing or cyclic") # noqa: TRY003 - cursor = await load_current(next_id) - if cursor.state.state is AtomicMemoryStateValue.RETIRED: - raise AtomicMemoryRelationError("a live merge path cannot end in a retired result") # noqa: TRY003 - records[next_id] = cursor - result_ids.append(next_id) - endpoint = cursor.as_read() - undo_ids = list(reversed(result_ids)) - if operation == "undo_merge": - undo_ids.append(target.artifact.artifact_id) - final = await _expand_undo(undo_ids, records, load_current, load_first) - target_id = target.artifact.artifact_id - if operation == "restore": - final[target_id] = AtomicMemoryStateValue.ACTIVE - writes, restored = _restoration_writes(final, records, target_id, selected) - return AtomicMemoryPlan( - scope_id=scope_id, - subject=subject, - reads=tuple(records[identity].as_read() for identity in sorted(records)), - writes=tuple(writes), - primary_artifact_id=target_id, - operation=operation, - endpoint=endpoint, - target_revision=None if selected is None else selected.revision, - undo_merge_results=tuple(undo_ids), - restore=tuple(restored), - ) - - -async def _expand_undo( - undo_ids: list[str], - records: dict[str, AtomicMemoryRecord], - load_current: Callable[[str], Awaitable[AtomicMemoryRecord]], - load_first: Callable[[str], Awaitable[AtomicMemory]], -) -> dict[str, AtomicMemoryStateValue]: - """Verify frozen input relations and compute the final state of the affected group.""" - - final: dict[str, AtomicMemoryStateValue] = {} - for result_id in undo_ids: - first = await load_first(result_id) - refs = merge_inputs(first) - selected_ids = {ref.artifact_id for ref in refs} - if any( - record.state.merged_into_id == result_id and identity not in selected_ids - for identity, record in records.items() - ): - raise AtomicMemoryRelationError("the merge path input is absent from the result creation selector") # noqa: TRY003 - final[result_id] = AtomicMemoryStateValue.RETIRED - for ref in refs: - current = records.get(ref.artifact_id) - if current is None: - current = await load_current(ref.artifact_id) - records[ref.artifact_id] = current - if ( - current.state.state is not AtomicMemoryStateValue.MERGED - or current.state.merged_into_id != result_id - or current.ref != ref - ): - raise AtomicMemoryRelationError( # noqa: TRY003 - "merge inputs do not match their current destination and frozen version" - ) - final[ref.artifact_id] = AtomicMemoryStateValue.ACTIVE - return final - - -def _restoration_writes( - final: dict[str, AtomicMemoryStateValue], - records: dict[str, AtomicMemoryRecord], - target_id: str, - selected: AtomicMemory | None, -) -> tuple[list[AtomicMemoryWrite], list[AtomicMemoryRestoreItem]]: - """Build only final changes; intermediate results never need projections.""" - - writes: list[AtomicMemoryWrite] = [] - restored: list[AtomicMemoryRestoreItem] = [] - for artifact_id, state in sorted(final.items()): - current = records[artifact_id] - draft = None - if artifact_id == target_id and selected is not None: - # Historical lineage_only Sources stay bound to their historical - # target; a precise historical Artifact reference retains evidence. - refs = tuple( - dict.fromkeys((ref.family, ref.artifact_id, ref.revision) for ref in (current.ref, selected.as_ref())) - ) - draft = AtomicMemoryDraft( - content=selected.content.without_creation(), - artifacts=tuple( - ArtifactRef(family=family, artifact_id=identity, revision=revision) - for family, identity, revision in refs - ), - ) - if state is AtomicMemoryStateValue.ACTIVE: - restored.append( - AtomicMemoryRestoreItem( - artifact_id=artifact_id, - source_revision=selected.revision - if draft is not None and selected is not None - else current.artifact.revision, - creates_revision=draft is not None, - ) - ) - if state != current.state.state or draft is not None: - writes.append(AtomicMemoryWrite(artifact_id=artifact_id, state=state, draft=draft, current=current)) - return writes, restored - - -class _PreviewClaims(BaseModel): - model_config = ConfigDict(extra="forbid", frozen=True) - format: Literal["powercontext.atomic-memory.restoration-preview.v1"] - key_id: str - scope_id: str - subject: str - operation: AtomicMemoryRestoreOperation - artifact_id: str - revision: StrictInt | None - endpoint: AtomicMemoryRead - expires_at: StrictInt - - -class AtomicMemoryPreviewSigner: - """Sign previews with explicit shared keys; no per-process random fallback.""" - - def __init__( - self, - *, - keys: Mapping[str, bytes], - active_key_id: str, - ttl_seconds: int = 300, - clock: Callable[[], datetime] | None = None, - ) -> None: - if active_key_id not in keys or any(not key_id or len(secret) < 32 for key_id, secret in keys.items()): - raise ValueError("preview keys need a configured active key and at least 32 secret bytes") # noqa: TRY003 - if isinstance(ttl_seconds, bool) or ttl_seconds < 1: - raise ValueError("preview TTL must be positive") # noqa: TRY003 - self._keys = dict(keys) - self._active_key_id = active_key_id - self._ttl = timedelta(seconds=ttl_seconds) - self._clock = clock or (lambda: datetime.now(UTC)) - - def encode(self, plan: AtomicMemoryPlan) -> tuple[str, datetime]: - if plan.endpoint is None or plan.operation not in {"restore", "undo_merge"}: - raise InvalidAtomicMemoryPreviewError("only restoration plans can be previewed") # noqa: TRY003 - expires_at = self._now() + self._ttl - claims = _PreviewClaims( - format="powercontext.atomic-memory.restoration-preview.v1", - key_id=self._active_key_id, - scope_id=plan.scope_id, - subject=plan.subject, - operation=cast(AtomicMemoryRestoreOperation, plan.operation), - artifact_id=plan.primary_artifact_id, - revision=plan.target_revision, - endpoint=plan.endpoint, - expires_at=int(expires_at.timestamp()), - ) - payload = json.dumps( - claims.model_dump(mode="json"), sort_keys=True, separators=(",", ":"), ensure_ascii=False - ).encode() - signature = hmac.digest(self._keys[self._active_key_id], payload, "sha256") - return f"{_encode(payload)}.{_encode(signature)}", datetime.fromtimestamp(claims.expires_at, UTC) - - def validate( - self, - token: str, - *, - scope_id: str, - subject: str, - operation: AtomicMemoryRestoreOperation, - artifact_id: str, - revision: int | None, - ) -> AtomicMemoryRead: - if len(token) > 16_384: - raise InvalidAtomicMemoryPreviewError("the restoration preview token is invalid") # noqa: TRY003 - try: - payload_part, signature_part = token.split(".") - payload, signature = _decode(payload_part), _decode(signature_part) - claims = _PreviewClaims.model_validate_json(payload, strict=True) - key = self._keys[claims.key_id] - except (ValueError, UnicodeError, binascii.Error, ValidationError, KeyError) as error: - raise InvalidAtomicMemoryPreviewError("the restoration preview token is invalid") from error # noqa: TRY003 - if not hmac.compare_digest(signature, hmac.digest(key, payload, "sha256")): - raise InvalidAtomicMemoryPreviewError("the restoration preview signature is invalid") # noqa: TRY003 - if (claims.scope_id, claims.subject, claims.operation, claims.artifact_id, claims.revision) != ( - scope_id, - subject, - operation, - artifact_id, - revision, - ): - raise InvalidAtomicMemoryPreviewError("the restoration preview does not match this request") # noqa: TRY003 - if int(self._now().timestamp()) >= claims.expires_at: - raise AtomicMemoryPreviewExpiredError("the restoration preview has expired") # noqa: TRY003 - return claims.endpoint - - def _now(self) -> datetime: - now = self._clock() - if now.tzinfo is None: - raise ValueError("preview clock must return an aware timestamp") # noqa: TRY003 - return now.astimezone(UTC) - - -def _encode(value: bytes) -> str: - return base64.urlsafe_b64encode(value).rstrip(b"=").decode("ascii") - - -def _decode(value: str) -> bytes: - raw = value.encode("ascii") - return base64.b64decode(raw + b"=" * (-len(raw) % 4), altchars=b"-_", validate=True) diff --git a/src/powercontext/builtin/artifacts/atomic_memory/service.py b/src/powercontext/builtin/artifacts/atomic_memory/service.py index a59e40c271..0947035cd5 100644 --- a/src/powercontext/builtin/artifacts/atomic_memory/service.py +++ b/src/powercontext/builtin/artifacts/atomic_memory/service.py @@ -12,19 +12,19 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Atomic Memory inspection, connection-free preparation and transactional publication.""" +"""Atomic content, projection and public-model adapters for shared Artifact merge.""" from __future__ import annotations -from collections.abc import Awaitable, Callable, Sequence -from dataclasses import replace -from typing import Any, Literal, Protocol, cast +from collections.abc import Sequence +from typing import TYPE_CHECKING, Any, Protocol, cast from sqlalchemy.ext.asyncio import AsyncConnection from powercontext.artifacts import ArtifactLineage, ArtifactRef from powercontext.builtin.artifacts.atomic_memory.errors import ( AtomicMemoryConflictError, + AtomicMemoryPreviewExpiredError, AtomicMemoryPreviewStaleError, AtomicMemoryRelationError, InvalidAtomicMemoryPreviewError, @@ -40,92 +40,275 @@ AtomicMemoryRead, AtomicMemoryRecord, AtomicMemoryRestorationPreview, + AtomicMemoryRestoreItem, AtomicMemoryRestoreOperation, + AtomicMemoryState, AtomicMemoryStateValue, AtomicMemoryWrite, PreparedAtomicMemory, ) -from powercontext.builtin.artifacts.atomic_memory.restoration import ( - AtomicMemoryPreviewSigner, - calculate_restoration, - merge_inputs, -) from powercontext.builtin.artifacts.memory.canonical import canonical_error_code, normalize_text from powercontext.builtin.artifacts.memory.errors import InvalidMemoryCandidateError +from powercontext.builtin.artifacts.merge import ArtifactMergeSecurity, ArtifactMergeService, MergeTags +from powercontext.builtin.artifacts.merge_models import ( + ArtifactMergeErrors, + ArtifactMergePlan, + ArtifactMergeRead, + ArtifactMergeRecord, + ArtifactMergeRestoreItem, + ArtifactMergeWrite, + PreparedArtifactMerge, +) +from powercontext.builtin.persistence.artifact_governance import ArtifactGovernance, ArtifactLifecycleState from powercontext.builtin.persistence.artifacts import ArtifactRepository from powercontext.builtin.persistence.atomic_memory import AtomicMemoryStateRepository from powercontext.builtin.persistence.atomic_memory_index import PreparedAtomicMemoryProjection -from powercontext.builtin.persistence.errors import RepositoryNotFoundError -from powercontext.errors import RevisionConflictError from powercontext.sources import SourceRef +if TYPE_CHECKING: + from powercontext.builtin.artifacts.atomic_memory.restoration import AtomicMemoryPreviewSigner -class AtomicMemoryAuthorization(Protocol): - """Shared access decisions applied inside the caller's business transaction.""" - def subject(self, context: Any) -> str: ... - - async def authorize( +class AtomicMemoryProjections(Protocol): + async def prepare(self, content: AtomicMemoryContent) -> PreparedAtomicMemoryProjection: ... + def validate_prepared(self, prepared: PreparedAtomicMemoryProjection) -> None: ... + async def publish( self, connection: AsyncConnection, scope_id: str, - context: Any, - action: Literal["read", "write", "create"], - ref: ArtifactRef | None = None, + record: AtomicMemoryRecord, + prepared: PreparedAtomicMemoryProjection, ) -> None: ... + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str) -> None: ... - async def authorize_sources( - self, connection: AsyncConnection, scope_id: str, context: Any, sources: tuple[SourceRef, ...] - ) -> None: ... - async def establish_owner( - self, connection: AsyncConnection, scope_id: str, artifact_id: str, context: Any - ) -> None: ... +ATOMIC_MERGE_ERRORS = ArtifactMergeErrors( + conflict=AtomicMemoryConflictError, + relation=AtomicMemoryRelationError, + invalid_state=InvalidAtomicMemoryStateError, + invalid_preview=InvalidAtomicMemoryPreviewError, + preview_expired=AtomicMemoryPreviewExpiredError, + preview_stale=AtomicMemoryPreviewStaleError, +) +AtomicMemorySecurity = ArtifactMergeSecurity + + +def _lifecycle(state: AtomicMemoryStateValue) -> ArtifactLifecycleState: + return { + AtomicMemoryStateValue.ACTIVE: ArtifactLifecycleState.ACTIVE, + AtomicMemoryStateValue.FORGOTTEN: ArtifactLifecycleState.DEPRECATED, + AtomicMemoryStateValue.MERGED: ArtifactLifecycleState.DEPRECATED, + AtomicMemoryStateValue.RETIRED: ArtifactLifecycleState.RETIRED, + }[state] + + +def _atomic_state(state: ArtifactGovernance) -> AtomicMemoryState: + if state.replacement_artifact_id is not None: + raise AtomicMemoryRelationError("Atomic Memory does not use generic governance replacements") # noqa: TRY003 + value = { + ArtifactLifecycleState.ACTIVE: AtomicMemoryStateValue.ACTIVE, + ArtifactLifecycleState.DEPRECATED: AtomicMemoryStateValue.FORGOTTEN + if state.merged_into_id is None + else AtomicMemoryStateValue.MERGED, + ArtifactLifecycleState.RETIRED: AtomicMemoryStateValue.RETIRED, + }[state.lifecycle_state] + return AtomicMemoryState( + state=value, state_version=state.governance_generation, merged_into_id=state.merged_into_id + ) + + +def _common_read(read: AtomicMemoryRead) -> ArtifactMergeRead: + return ArtifactMergeRead( + ref=read.ref, + lifecycle_state=_lifecycle(read.state), + state_version=read.state_version, + merged_into_id=read.merged_into_id, + ) + + +def _atomic_read(read: ArtifactMergeRead) -> AtomicMemoryRead: + return AtomicMemoryRead( + ref=read.ref, + **_atomic_state( + ArtifactGovernance( + artifact=read.ref, + lifecycle_state=read.lifecycle_state, + governance_generation=read.state_version, + merged_into_id=read.merged_into_id, + replacement_artifact_id=read.replacement_artifact_id, + ) + ).model_dump(), + ) + + +def _common_record(record: AtomicMemoryRecord) -> ArtifactMergeRecord: + return ArtifactMergeRecord( + artifact=record.artifact, + state=ArtifactGovernance( + artifact=record.ref, + lifecycle_state=_lifecycle(record.state.state), + governance_generation=record.state.state_version, + merged_into_id=record.state.merged_into_id, + ), + ) + + +def _atomic_record(record: ArtifactMergeRecord) -> AtomicMemoryRecord: + if not isinstance(record.artifact, AtomicMemory): + raise AtomicMemoryRelationError("the stored artifact is not an Atomic Memory") # noqa: TRY003 + return AtomicMemoryRecord(artifact=record.artifact, state=_atomic_state(record.state)) + + +def _common_plan(plan: AtomicMemoryPlan) -> ArtifactMergePlan: + return ArtifactMergePlan( + scope_id=plan.scope_id, + subject=plan.subject, + reads=tuple(_common_read(read) for read in plan.reads), + writes=tuple( + ArtifactMergeWrite( + artifact_id=write.artifact_id, + state=_lifecycle(write.state), + draft=write.draft, + current=None if write.current is None else _common_record(write.current), + merged_into_id=write.merged_into_id, + merge_input_ids=write.merge_input_ids, + ) + for write in plan.writes + ), + primary_artifact_id=plan.primary_artifact_id, + operation=plan.operation, + endpoint=None if plan.endpoint is None else _common_read(plan.endpoint), + target_revision=plan.target_revision, + undo_merge_results=plan.undo_merge_results, + restore=tuple(ArtifactMergeRestoreItem(**item.model_dump()) for item in plan.restore), + preview_token=plan.preview_token, + ) + + +def _atomic_plan(plan: ArtifactMergePlan) -> AtomicMemoryPlan: + return AtomicMemoryPlan( + scope_id=plan.scope_id, + subject=plan.subject, + reads=tuple(_atomic_read(read) for read in plan.reads), + writes=tuple( + AtomicMemoryWrite( + artifact_id=write.artifact_id, + state=AtomicMemoryStateValue.MERGED + if write.merged_into_id is not None + else { + ArtifactLifecycleState.ACTIVE: AtomicMemoryStateValue.ACTIVE, + ArtifactLifecycleState.DEPRECATED: AtomicMemoryStateValue.FORGOTTEN, + ArtifactLifecycleState.RETIRED: AtomicMemoryStateValue.RETIRED, + }[write.state], + draft=cast(AtomicMemoryDraft | None, write.draft), + current=None if write.current is None else _atomic_record(write.current), + merged_into_id=write.merged_into_id, + merge_input_ids=write.merge_input_ids, + ) + for write in plan.writes + ), + primary_artifact_id=plan.primary_artifact_id, + operation=plan.operation, + endpoint=None if plan.endpoint is None else _atomic_read(plan.endpoint), + target_revision=plan.target_revision, + undo_merge_results=plan.undo_merge_results, + restore=tuple(AtomicMemoryRestoreItem(**item.model_dump()) for item in plan.restore), + preview_token=plan.preview_token, + ) + + +class AtomicMemoryMergeAdapter: + family = AtomicMemory.family + + def __init__(self, projections: AtomicMemoryProjections) -> None: + self.projections = projections + def draft( + self, + content: Any, + lineage: ArtifactLineage, + *, + merge_inputs: tuple[ArtifactRef, ...] = (), + historical: bool = False, + ) -> AtomicMemoryDraft: + content = AtomicMemoryContent.model_validate(content) + if historical: + content = content.without_creation() + elif content.creation is not None: + raise AtomicMemoryRelationError("creation metadata is constructed only by the merge service") # noqa: TRY003 + if lineage.publication_source is not None: + raise AtomicMemoryRelationError("Atomic Memory accepts direct Sources and exact in-Scope Artifacts") # noqa: TRY003 + if historical: + return AtomicMemoryDraft(content=content, sources=lineage.sources, artifacts=lineage.artifacts) + try: + text = normalize_text(content.text) + except (TypeError, ValueError) as error: + raise InvalidMemoryCandidateError( + "canonical", str(error), canonical_code=canonical_error_code(error) + ) from error + content = content.model_copy(update={"text": text}) + if merge_inputs: + content = content.model_copy( + update={ + "creation": AtomicMemoryCreation(input_artifact_ids=tuple(ref.artifact_id for ref in merge_inputs)) + } + ) + return AtomicMemoryDraft(content=content, sources=lineage.sources, artifacts=lineage.artifacts) -class AtomicMemoryProjections(Protocol): - async def prepare(self, content: AtomicMemoryContent) -> PreparedAtomicMemoryProjection: ... + async def prepare(self, content: AtomicMemoryContent) -> PreparedAtomicMemoryProjection: + return await self.projections.prepare(content) - def validate_prepared(self, prepared: PreparedAtomicMemoryProjection) -> None: ... + def validate_prepared(self, prepared: PreparedAtomicMemoryProjection) -> None: + self.projections.validate_prepared(prepared) async def publish( self, connection: AsyncConnection, scope_id: str, - record: AtomicMemoryRecord, + record: ArtifactMergeRecord, prepared: PreparedAtomicMemoryProjection, - ) -> None: ... + execution_context: Any, + ) -> None: + await self.projections.publish(connection, scope_id, _atomic_record(record), prepared) - async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str) -> None: ... + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str) -> None: + await self.projections.remove(connection, scope_id, artifact_id) -MergeTags = Callable[[AsyncConnection, str, str, tuple[str, ...]], Awaitable[None]] +class _AtomicPreviewAdapter: + def __init__(self, signer): + self.signer = signer + def encode(self, plan): + return self.signer.encode(_atomic_plan(plan)) + + def validate(self, *args, **kwargs): + return _common_read(self.signer.validate(*args, **kwargs)) -class AtomicMemoryService: - """Own Family transitions while leaving transaction lifetime with the caller. - Inspect under a read transaction, close it, and prepare vectors without a - connection. Commit receives the caller's write transaction; it never - commits or retries a transaction whose outcome it does not own. - """ +class AtomicMemoryService: + """Keep Atomic public contracts while delegating all transitions to the common service.""" def __init__( self, *, artifacts: ArtifactRepository, states: AtomicMemoryStateRepository, - security: AtomicMemoryAuthorization, + security: AtomicMemorySecurity, projections: AtomicMemoryProjections, merge_tags: MergeTags, preview_signer: AtomicMemoryPreviewSigner | None = None, ) -> None: - self.artifacts = artifacts - self.states = states - self.security = security - self.projections = projections - self.merge_tags = merge_tags - self.preview_signer = preview_signer + self.artifacts, self.states, self.security = artifacts, states, security + self.projections, self.merge_tags, self.preview_signer = projections, merge_tags, preview_signer + self.shared = ArtifactMergeService( + artifacts=artifacts, + adapter=AtomicMemoryMergeAdapter(projections), + security=security, + merge_tags=merge_tags, + errors=ATOMIC_MERGE_ERRORS, + preview_signer=None if preview_signer is None else _AtomicPreviewAdapter(preview_signer), + ) async def get( self, @@ -137,18 +320,9 @@ async def get( revision: int | None = None, for_update: bool = False, ) -> AtomicMemoryRecord: - identity = ArtifactRef( - family=AtomicMemory.family, artifact_id=artifact_id, revision=1 if revision is None else revision + return _atomic_record( + await self.shared.get(connection, scope_id, artifact_id, context, revision=revision, for_update=for_update) ) - await self.security.authorize(connection, scope_id, context, "read", identity) - if revision is None: - stored = await self.artifacts.latest( - connection, scope_id, AtomicMemory.family, artifact_id, for_update=for_update - ) - else: - stored = await self.artifacts.get(connection, scope_id, identity, for_update=for_update) - state = await self.states.get(connection, scope_id, artifact_id, for_update=for_update) - return AtomicMemoryRecord(artifact=self._atomic(stored), state=state) async def inspect_change( self, @@ -163,40 +337,18 @@ async def inspect_change( lineage: ArtifactLineage | None = None, reads: Sequence[AtomicMemoryRead] = (), ) -> AtomicMemoryPlan: - self._ordinary_content(content) - draft = self._draft(content, lineage) - current = None - if expected_revision is None: - target = ArtifactRef(family=AtomicMemory.family, artifact_id=artifact_id, revision=1) - await self.security.authorize(connection, scope_id, context, "create", target) - try: - stored = await self.artifacts.latest(connection, scope_id, AtomicMemory.family, artifact_id) - except RepositoryNotFoundError: - pass - else: - raise RevisionConflictError(draft, stored) - state = AtomicMemoryStateValue.ACTIVE - else: - current = await self.get(connection, scope_id, artifact_id, context) - await self.security.authorize(connection, scope_id, context, "write", current.ref) - self._expected(current, expected_revision, expected_state_version, draft) - if current.state.state not in {AtomicMemoryStateValue.ACTIVE, AtomicMemoryStateValue.FORGOTTEN}: - raise InvalidAtomicMemoryStateError("merged and retired memories cannot be edited") # noqa: TRY003 - target = ArtifactRef( - family=AtomicMemory.family, artifact_id=artifact_id, revision=current.artifact.revision + 1 + return _atomic_plan( + await self.shared.inspect_change( + connection, + scope_id, + artifact_id, + content, + context, + expected_revision=expected_revision, + expected_state_version=expected_state_version, + lineage=lineage, + reads=tuple(_common_read(read) for read in reads), ) - state = current.state.state - draft = draft.model_copy(update={"artifacts": _unique_refs((*draft.artifacts, current.ref))}) - await self._validate_draft(connection, scope_id, target, draft, context) - dependencies = (*reads, *((current.as_read(),) if current is not None else ())) - await self._check_dependencies(connection, scope_id, dependencies, context) - return AtomicMemoryPlan( - scope_id=scope_id, - subject=self.security.subject(context), - reads=_unique_reads(dependencies), - writes=(AtomicMemoryWrite(artifact_id=artifact_id, state=state, draft=draft, current=current),), - primary_artifact_id=artifact_id, - operation="change", ) async def inspect_merge( @@ -211,60 +363,17 @@ async def inspect_merge( lineage: ArtifactLineage | None = None, reads: Sequence[AtomicMemoryRead] = (), ) -> AtomicMemoryPlan: - self._ordinary_content(content) - selected = tuple(inputs) - ids = tuple(read.ref.artifact_id for read in selected) - if len(ids) < 2 or len(set(ids)) != len(ids) or artifact_id in ids: - raise AtomicMemoryRelationError("merge needs at least two distinct inputs and a new result identity") # noqa: TRY003 - target = ArtifactRef(family=AtomicMemory.family, artifact_id=artifact_id, revision=1) - await self.security.authorize(connection, scope_id, context, "create", target) - try: - stored = await self.artifacts.latest(connection, scope_id, AtomicMemory.family, artifact_id) - except RepositoryNotFoundError: - pass - else: - raise RevisionConflictError(content, stored) - current_inputs: list[AtomicMemoryRecord] = [] - for expected in selected: - current = await self.get(connection, scope_id, expected.ref.artifact_id, context) - await self.security.authorize(connection, scope_id, context, "write", current.ref) - if current.as_read() != expected: - raise AtomicMemoryConflictError("merge inputs changed before preparation") # noqa: TRY003 - if current.state.state is not AtomicMemoryStateValue.ACTIVE: - raise InvalidAtomicMemoryStateError("only active memories can be merge inputs") # noqa: TRY003 - current_inputs.append(current) - draft = self._draft( - content.model_copy(update={"creation": AtomicMemoryCreation(input_artifact_ids=ids)}), lineage - ) - # An explicitly supplied lineage must already contain every selected - # exact reference. Internal callers may omit lineage and use inputs. - if lineage is None: - draft = draft.model_copy(update={"artifacts": tuple(read.ref for read in selected)}) - self._require_merge_draft(artifact_id, draft, selected) - await self._validate_draft(connection, scope_id, target, draft, context) - dependencies = _unique_reads((*reads, *selected)) - await self._check_dependencies(connection, scope_id, dependencies, context) - writes = ( - AtomicMemoryWrite( - artifact_id=artifact_id, state=AtomicMemoryStateValue.ACTIVE, draft=draft, merge_input_ids=ids - ), - *( - AtomicMemoryWrite( - artifact_id=current.artifact.artifact_id, - state=AtomicMemoryStateValue.MERGED, - merged_into_id=artifact_id, - current=current, - ) - for current in current_inputs - ), - ) - return AtomicMemoryPlan( - scope_id=scope_id, - subject=self.security.subject(context), - reads=dependencies, - writes=writes, - primary_artifact_id=artifact_id, - operation="merge", + return _atomic_plan( + await self.shared.inspect_merge( + connection, + scope_id, + artifact_id, + tuple(_common_read(read) for read in inputs), + content, + context, + lineage=lineage, + reads=tuple(_common_read(read) for read in reads), + ) ) async def inspect_forget( @@ -277,23 +386,15 @@ async def inspect_forget( expected_revision: int, expected_state_version: int, ) -> AtomicMemoryPlan: - current = await self.get(connection, scope_id, artifact_id, context) - await self.security.authorize(connection, scope_id, context, "write", current.ref) - self._expected(current, expected_revision, expected_state_version) - if current.state.state not in {AtomicMemoryStateValue.ACTIVE, AtomicMemoryStateValue.FORGOTTEN}: - raise InvalidAtomicMemoryStateError("only active or forgotten memories can be forgotten") # noqa: TRY003 - writes = ( - () - if current.state.state is AtomicMemoryStateValue.FORGOTTEN - else (AtomicMemoryWrite(artifact_id=artifact_id, state=AtomicMemoryStateValue.FORGOTTEN, current=current),) - ) - return AtomicMemoryPlan( - scope_id=scope_id, - subject=self.security.subject(context), - reads=(current.as_read(),), - writes=writes, - primary_artifact_id=artifact_id, - operation="forget", + return _atomic_plan( + await self.shared.inspect_forget( + connection, + scope_id, + artifact_id, + context, + expected_revision=expected_revision, + expected_state_version=expected_state_version, + ) ) async def inspect_restore( @@ -307,100 +408,41 @@ async def inspect_restore( revision: int | None = None, preview_token: str | None = None, ) -> AtomicMemoryPlan: - if operation not in {"restore", "undo_merge"}: - raise InvalidAtomicMemoryStateError("unknown restoration operation") # noqa: TRY003 - if operation == "undo_merge" and revision is not None: - raise InvalidAtomicMemoryStateError("undo_merge does not accept a content revision") # noqa: TRY003 - target = await self.get(connection, scope_id, artifact_id, context) - token_endpoint = ( - None - if preview_token is None - else self._signer().validate( - preview_token, - scope_id=scope_id, - subject=self.security.subject(context), + return _atomic_plan( + await self.shared.inspect_restore( + connection, + scope_id, + artifact_id, + context, operation=operation, - artifact_id=artifact_id, revision=revision, + preview_token=preview_token, ) ) - async def load_current(identity: str) -> AtomicMemoryRecord: - record = await self.get(connection, scope_id, identity, context) - await self.security.authorize(connection, scope_id, context, "write", record.ref) - return record - - async def load_first(identity: str) -> AtomicMemory: - ref = ArtifactRef(family=AtomicMemory.family, artifact_id=identity, revision=1) - await self.security.authorize(connection, scope_id, context, "read", ref) - return self._atomic(await self.artifacts.get(connection, scope_id, ref)) - - await self.security.authorize(connection, scope_id, context, "write", target.ref) - selected = None - if revision is not None: - selected = (await self.get(connection, scope_id, artifact_id, context, revision=revision)).artifact - plan = await calculate_restoration( - scope_id=scope_id, - subject=self.security.subject(context), - target=target, - operation=operation, - selected=selected, - load_current=load_current, - load_first=load_first, - ) - if token_endpoint is not None and plan.endpoint != token_endpoint: - raise AtomicMemoryPreviewStaleError("the restoration preview endpoint changed") # noqa: TRY003 - return replace(plan, preview_token=preview_token) - def restoration_preview(self, plan: AtomicMemoryPlan) -> AtomicMemoryRestorationPreview: - token, expires_at = self._signer().encode(plan) - if plan.endpoint is None: - raise InvalidAtomicMemoryPreviewError("the restoration plan has no endpoint") # noqa: TRY003 - retire_ids = {write.artifact_id for write in plan.writes if write.state is AtomicMemoryStateValue.RETIRED} + preview = self.shared.restoration_preview(_common_plan(plan)) return AtomicMemoryRestorationPreview( - preview_token=token, - expires_at=expires_at, - endpoint=plan.endpoint, - restore=plan.restore, - retire=tuple(read.ref for read in plan.reads if read.ref.artifact_id in retire_ids), - undo_merge_results=plan.undo_merge_results, + preview_token=preview.preview_token, + expires_at=preview.expires_at, + endpoint=_atomic_read(preview.endpoint), + restore=tuple(AtomicMemoryRestoreItem(**item.model_dump()) for item in preview.restore), + retire=preview.retire, + undo_merge_results=preview.undo_merge_results, ) async def prepare_change(self, plan: AtomicMemoryPlan) -> PreparedAtomicMemory: - """Perform external preparation only, with no database connection.""" - - projections: list[tuple[str, PreparedAtomicMemoryProjection]] = [] - for write in plan.writes: - if write.state is not AtomicMemoryStateValue.ACTIVE: - continue - content = ( - write.draft.content - if write.draft is not None - else write.current.artifact.content - if write.current is not None - else None - ) - if content is None: - raise AtomicMemoryRelationError("active publication has no content") # noqa: TRY003 - projections.append((write.artifact_id, await self.projections.prepare(content))) - return PreparedAtomicMemory(plan=plan, projections=tuple(projections)) + prepared = await self.shared.prepare_change(_common_plan(plan)) + return PreparedAtomicMemory(plan=plan, projections=prepared.projections) - async def prepare_merge(self, plan: AtomicMemoryPlan) -> PreparedAtomicMemory: - return await self.prepare_change(plan) - - async def prepare_forget(self, plan: AtomicMemoryPlan) -> PreparedAtomicMemory: - return await self.prepare_change(plan) - - async def prepare_restore(self, plan: AtomicMemoryPlan) -> PreparedAtomicMemory: - return await self.prepare_change(plan) + prepare_merge = prepare_change + prepare_forget = prepare_change + prepare_restore = prepare_change async def validate_read_set( self, connection: AsyncConnection, scope_id: str, reads: Sequence[AtomicMemoryRead], context: Any ) -> None: - """Allow no-op processing plans to protect their actual decision dependencies.""" - - await self.artifacts.lock_heads(connection, scope_id, tuple(read.ref for read in reads)) - await self._check_dependencies(connection, scope_id, reads, context, for_update=True) + await self.shared.validate_read_set(connection, scope_id, tuple(_common_read(read) for read in reads), context) async def commit( self, @@ -426,303 +468,23 @@ async def commit_window( read_set: Sequence[AtomicMemoryRead] = (), direct_source: SourceRef | None = None, ) -> tuple[AtomicMemoryMutationResult, ...]: - """Validate all window dependencies before publishing any of its final decisions.""" - - plans = tuple(item.plan for item in prepared) - subject = self.security.subject(context) - self._require_window(plans, scope_id, subject) - for item in prepared: - for _, projection in item.projections: - self.projections.validate_prepared(projection) - reads = _unique_reads((*read_set, *(read for plan in plans for read in plan.reads))) - await self.artifacts.lock_heads(connection, scope_id, tuple(read.ref for read in reads)) - for plan in plans: - await self._validate_preview(connection, plan, context) - current = await self._check_dependencies(connection, scope_id, reads, context, for_update=True) - for plan in plans: - if plan.operation in {"restore", "undo_merge"}: - verified = await self.inspect_restore( - connection, - scope_id, - plan.primary_artifact_id, - context, - operation=cast(AtomicMemoryRestoreOperation, plan.operation), - revision=plan.target_revision, - preview_token=plan.preview_token, - ) - if verified != plan: - raise AtomicMemoryConflictError("restoration relations changed; prepare the whole group again") # noqa: TRY003 - results: list[AtomicMemoryMutationResult] = [] - for item in prepared: - results.append(await self._commit_locked(connection, item, current, context, direct_source)) - return tuple(results) - - @staticmethod - def _require_window(plans: Sequence[AtomicMemoryPlan], scope_id: str, subject: str) -> None: - identities: set[str] = set() - for plan in plans: - if plan.scope_id != scope_id or plan.subject != subject: - raise InvalidAtomicMemoryStateError("window scope or identity differs from preparation") # noqa: TRY003 - AtomicMemoryService._require_plan(plan) - for write in plan.writes: - if write.artifact_id in identities: - raise AtomicMemoryRelationError("window decisions must coordinate each memory into one final write") # noqa: TRY003 - identities.add(write.artifact_id) - - async def _validate_preview(self, connection: AsyncConnection, plan: AtomicMemoryPlan, context: Any) -> None: - if plan.preview_token is None: - return - endpoint = self._signer().validate( - plan.preview_token, - scope_id=plan.scope_id, - subject=plan.subject, - operation=cast(AtomicMemoryRestoreOperation, plan.operation), - artifact_id=plan.primary_artifact_id, - revision=plan.target_revision, - ) - current = await self.get(connection, plan.scope_id, endpoint.ref.artifact_id, context, for_update=True) - if current.as_read() != endpoint: - raise AtomicMemoryPreviewStaleError("the restoration preview endpoint changed") # noqa: TRY003 - - async def _commit_locked( - self, - connection: AsyncConnection, - prepared: PreparedAtomicMemory, - current: dict[str, AtomicMemoryRecord], - context: Any, - direct_source: SourceRef | None, - ) -> AtomicMemoryMutationResult: - """Apply a plan against the already verified initial window state.""" - - plan = prepared.plan - projections = dict(prepared.projections) - records: dict[str, AtomicMemoryRecord] = {} - for write in plan.writes: - record = await self._commit_write( - connection, - plan.scope_id, - write, - current.get(write.artifact_id), - projections.get(write.artifact_id), - context, - direct_source, - ) - records[write.artifact_id] = record - if plan.primary_artifact_id not in records: - records[plan.primary_artifact_id] = current[plan.primary_artifact_id] - return AtomicMemoryMutationResult( - changed=bool(plan.writes), - records=tuple(records[identity] for identity in sorted(records)), - primary_artifact_id=plan.primary_artifact_id, - retired=tuple( - record.ref for record in records.values() if record.state.state is AtomicMemoryStateValue.RETIRED - ), - undo_merge_results=plan.undo_merge_results, - ) - - async def _commit_write( - self, - connection: AsyncConnection, - scope_id: str, - write: AtomicMemoryWrite, - before: AtomicMemoryRecord | None, - projection: PreparedAtomicMemoryProjection | None, - context: Any, - direct_source: SourceRef | None, - ) -> AtomicMemoryRecord: - """Publish one identity after the whole plan's locks and checks are complete.""" - - draft = write.draft - if draft is not None: - target = ArtifactRef( - family=AtomicMemory.family, - artifact_id=write.artifact_id, - revision=1 if before is None else before.artifact.revision + 1, - ) - if direct_source is not None: - sources = {(source.source_type, source.source_id): source for source in (*draft.sources, direct_source)} - draft = draft.model_copy(update={"sources": tuple(sources.values())}) - await self._validate_draft(connection, scope_id, target, draft, context, direct_source=direct_source) - if before is None: - await self.security.authorize(connection, scope_id, context, "create", target) - artifact = self._atomic(await self.artifacts.create(connection, scope_id, write.artifact_id, draft)) - state = await self.states.create(connection, scope_id, write.artifact_id) - await self.security.establish_owner(connection, scope_id, write.artifact_id, context) - if write.merge_input_ids: - await self.merge_tags(connection, scope_id, write.artifact_id, write.merge_input_ids) - else: - artifact = self._atomic(await self.artifacts.revise(connection, scope_id, before.artifact, draft)) - state = before.state - else: - if before is None: - raise AtomicMemoryRelationError("state transitions require an existing memory") # noqa: TRY003 - artifact, state = before.artifact, before.state - state = await self.states.transition( + results = await self.shared.commit_window( connection, scope_id, - write.artifact_id, - state, - write.state, - write.merged_into_id, - ) - record = AtomicMemoryRecord(artifact=artifact, state=state) - if state.state is AtomicMemoryStateValue.ACTIVE: - if projection is None: - raise AtomicMemoryRelationError("active publication requires a prepared projection") # noqa: TRY003 - await self.projections.publish(connection, scope_id, record, projection) - else: - await self.projections.remove(connection, scope_id, write.artifact_id) - return record - - async def _check_dependencies( - self, - connection: AsyncConnection, - scope_id: str, - reads: Sequence[AtomicMemoryRead], - context: Any, - *, - for_update: bool = False, - ) -> dict[str, AtomicMemoryRecord]: - records: dict[str, AtomicMemoryRecord] = {} - for expected in _unique_reads(reads): - current = await self.get(connection, scope_id, expected.ref.artifact_id, context, for_update=for_update) - await self.security.authorize(connection, scope_id, context, "write", current.ref) - if current.as_read() != expected: - raise AtomicMemoryConflictError("memory content, state or merge relation changed; prepare again") # noqa: TRY003 - if for_update: - await self.states.require_summary(connection, scope_id, expected.ref.artifact_id, current.state) - records[expected.ref.artifact_id] = current - return records - - async def _validate_draft( - self, - connection: AsyncConnection, - scope_id: str, - target: ArtifactRef, - draft: AtomicMemoryDraft, - context: Any, - *, - direct_source: SourceRef | None = None, - ) -> None: - # The host creates a lineage_only Source bound to this exact new target. - # It is not caller-supplied evidence and need not grant Scope body read. - ordinary_sources = tuple(source for source in draft.sources if source != direct_source) - await self.security.authorize_sources(connection, scope_id, context, ordinary_sources) - await self.artifacts.validate_lineage_sources(connection, scope_id, target, draft.sources) - for ref in draft.artifacts: - await self.security.authorize(connection, scope_id, context, "read", ref) - await self.artifacts.get(connection, scope_id, ref) - - @staticmethod - def _draft(content: AtomicMemoryContent, lineage: ArtifactLineage | None) -> AtomicMemoryDraft: - evidence = lineage or ArtifactLineage() - if evidence.publication_source is not None: - raise AtomicMemoryRelationError("Atomic Memory accepts direct Sources and exact in-Scope Artifacts") # noqa: TRY003 - try: - text = normalize_text(content.text) - except (TypeError, ValueError) as error: - raise InvalidMemoryCandidateError( - "canonical", str(error), canonical_code=canonical_error_code(error) - ) from error - normalized_content = content.model_copy(update={"text": text}) - return AtomicMemoryDraft(content=normalized_content, sources=evidence.sources, artifacts=evidence.artifacts) - - @staticmethod - def _ordinary_content(content: AtomicMemoryContent) -> None: - if content.creation is not None: - raise AtomicMemoryRelationError("creation metadata is constructed only by the merge service") # noqa: TRY003 - - @staticmethod - def _expected(current: AtomicMemoryRecord, revision: int, state_version: int | None, requested: Any = None) -> None: - if current.artifact.revision != revision: - raise RevisionConflictError(requested, current.artifact) - if state_version is not None and current.state.state_version != state_version: - raise AtomicMemoryConflictError("the expected memory state version is stale") # noqa: TRY003 - - @staticmethod - def _require_merge_draft(artifact_id: str, draft: AtomicMemoryDraft, inputs: Sequence[AtomicMemoryRead]) -> None: - result = AtomicMemory( - artifact_id=artifact_id, - revision=1, - content=draft.content, - lineage=ArtifactLineage(sources=draft.sources, artifacts=draft.artifacts), + tuple( + PreparedArtifactMerge(plan=_common_plan(item.plan), projections=item.projections) for item in prepared + ), + context, + read_set=tuple(_common_read(read) for read in read_set), + direct_source=direct_source, ) - selected = merge_inputs(result) - if selected != tuple(read.ref for read in inputs): - raise AtomicMemoryRelationError("merge selectors must identify the supplied exact active input versions") # noqa: TRY003 - - @staticmethod - def _require_plan(plan: AtomicMemoryPlan) -> None: - ids = tuple(write.artifact_id for write in plan.writes) - if len(ids) != len(set(ids)): - raise AtomicMemoryRelationError("one plan cannot write an identity twice") # noqa: TRY003 - reads = {read.ref.artifact_id: read for read in _unique_reads(plan.reads)} - for write in plan.writes: - if write.current is not None and write.current.state.state is AtomicMemoryStateValue.RETIRED: - raise InvalidAtomicMemoryStateError("retired memory identities cannot be modified") # noqa: TRY003 - if write.current is not None and reads.get(write.artifact_id) != write.current.as_read(): - raise AtomicMemoryRelationError("every existing write requires its exact current read dependency") # noqa: TRY003 - if write.current is None and (write.draft is None or write.state is not AtomicMemoryStateValue.ACTIVE): - raise AtomicMemoryRelationError("new memories require active content") # noqa: TRY003 - if write.draft is None: - continue - if write.merge_input_ids: - AtomicMemoryService._require_merge_plan(plan, write, reads) - else: - AtomicMemoryService._ordinary_content(write.draft.content) - if ( - write.current is not None - and plan.operation == "change" - and write.current.state.state - not in {AtomicMemoryStateValue.ACTIVE, AtomicMemoryStateValue.FORGOTTEN} - ): - raise InvalidAtomicMemoryStateError("merged and retired memories cannot be edited") # noqa: TRY003 - - @staticmethod - def _require_merge_plan( - plan: AtomicMemoryPlan, write: AtomicMemoryWrite, reads: dict[str, AtomicMemoryRead] - ) -> None: - if plan.operation != "merge" or write.current is not None or write.draft is None: - raise AtomicMemoryRelationError("merge creation belongs only to a new result") # noqa: TRY003 - if any(identity not in reads for identity in write.merge_input_ids): - raise AtomicMemoryRelationError("merge inputs require exact read dependencies") # noqa: TRY003 - inputs = tuple(reads[identity] for identity in write.merge_input_ids) - AtomicMemoryService._require_merge_draft(write.artifact_id, write.draft, inputs) - writes = {item.artifact_id: item for item in plan.writes} - for expected in inputs: - matching = writes.get(expected.ref.artifact_id) - if ( - expected.state is not AtomicMemoryStateValue.ACTIVE - or matching is None - or matching.state is not AtomicMemoryStateValue.MERGED - or matching.merged_into_id != write.artifact_id - ): - raise AtomicMemoryRelationError("all active merge inputs must freeze into the new result") # noqa: TRY003 - - @staticmethod - def _atomic(value: Any) -> AtomicMemory: - if not isinstance(value, AtomicMemory): - raise AtomicMemoryRelationError("the stored artifact is not an Atomic Memory") # noqa: TRY003 - return value - - def _signer(self) -> AtomicMemoryPreviewSigner: - if self.preview_signer is None: - raise InvalidAtomicMemoryPreviewError( # noqa: TRY003 - "restoration previews need an explicitly configured shared signing key" + return tuple( + AtomicMemoryMutationResult( + changed=result.changed, + records=tuple(_atomic_record(record) for record in result.records), + primary_artifact_id=result.primary_artifact_id, + retired=result.retired, + undo_merge_results=result.undo_merge_results, ) - return self.preview_signer - - -def _unique_refs(refs: Sequence[ArtifactRef]) -> tuple[ArtifactRef, ...]: - unique = {(ref.family, ref.artifact_id, ref.revision): ref for ref in refs} - return tuple(unique.values()) - - -def _unique_reads(reads: Sequence[AtomicMemoryRead]) -> tuple[AtomicMemoryRead, ...]: - unique: dict[str, AtomicMemoryRead] = {} - for read in reads: - previous = unique.get(read.ref.artifact_id) - if previous is not None and previous != read: - raise AtomicMemoryConflictError("a plan cannot mix versions of the same memory") # noqa: TRY003 - unique[read.ref.artifact_id] = read - return tuple(unique[identity] for identity in sorted(unique)) + for result in results + ) diff --git a/src/powercontext/builtin/artifacts/experience/merge.py b/src/powercontext/builtin/artifacts/experience/merge.py new file mode 100644 index 0000000000..09f596009f --- /dev/null +++ b/src/powercontext/builtin/artifacts/experience/merge.py @@ -0,0 +1,78 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Experience content and current-index adapter for the shared merge service.""" + +from __future__ import annotations + +from typing import Any, cast + +from sqlalchemy.ext.asyncio import AsyncConnection + +from powercontext.artifacts import ArtifactLineage, ArtifactRef +from powercontext.builtin.artifacts.experience.models import Experience, ExperienceContent, ExperienceDraft +from powercontext.builtin.artifacts.experience.search import experience_searchable_text +from powercontext.builtin.artifacts.merge_models import ArtifactMergeRecord, ArtifactMergeRelationError +from powercontext.builtin.persistence.experience_index import ExperienceIndex + + +class ExperienceMergeAdapter: + """Accept direct Sources and exact in-Scope Artifacts as explicit merge evidence. + + Legacy Memory citations and publication lineage remain supported by their + existing writers and exact reads; this adapter rejects them on new writes. + Stored historical content is restored through precise Artifact references. + """ + + family = Experience.family + + def __init__(self, index: ExperienceIndex) -> None: + self.index = index + + def draft( + self, + content: Any, + lineage: ArtifactLineage, + *, + merge_inputs: tuple[ArtifactRef, ...] = (), + historical: bool = False, + ) -> ExperienceDraft: + if lineage.publication_source is not None: + raise ArtifactMergeRelationError("Experience accepts direct Sources and exact in-Scope Artifacts") # noqa: TRY003 + return ExperienceDraft( + content=ExperienceContent.model_validate(content), sources=lineage.sources, artifacts=lineage.artifacts + ) + + async def prepare(self, content: Any) -> str: + return experience_searchable_text(ExperienceContent.model_validate(content)) + + def validate_prepared(self, prepared: Any) -> None: + if not isinstance(prepared, str) or not prepared: + raise ArtifactMergeRelationError("Experience publication requires prepared searchable text") # noqa: TRY003 + + async def publish( + self, + connection: AsyncConnection, + scope_id: str, + record: ArtifactMergeRecord, + prepared: Any, + execution_context: Any, + ) -> None: + experience = cast(Experience, record.artifact) + if type(experience) is not Experience or prepared != experience_searchable_text(experience.content): + raise ArtifactMergeRelationError("Experience projection does not match its exact content") # noqa: TRY003 + await self.index.replace(connection, scope_id, experience) + + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str) -> None: + await self.index.remove(connection, scope_id, artifact_id) diff --git a/src/powercontext/builtin/artifacts/merge.py b/src/powercontext/builtin/artifacts/merge.py new file mode 100644 index 0000000000..5fdf8cc13a --- /dev/null +++ b/src/powercontext/builtin/artifacts/merge.py @@ -0,0 +1,874 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Shared Artifact merge, whole-group restoration and transactional publication.""" + +from __future__ import annotations + +from collections.abc import Awaitable, Callable, Sequence +from dataclasses import replace +from typing import Any, Literal, Protocol, cast +from uuid import uuid4 + +from pydantic import BaseModel +from sqlalchemy.ext.asyncio import AsyncConnection + +from powercontext.artifacts import ArtifactDraft, ArtifactLineage, ArtifactRef +from powercontext.builtin.artifacts.merge_models import ( + ArtifactMergeErrors, + ArtifactMergeMutationResult, + ArtifactMergePlan, + ArtifactMergeRead, + ArtifactMergeRecord, + ArtifactMergeRestorationPreview, + ArtifactMergeRestoreOperation, + ArtifactMergeWrite, + PreparedArtifactMerge, +) +from powercontext.builtin.artifacts.merge_restoration import ArtifactMergePreviewSigner, calculate_restoration +from powercontext.builtin.persistence.artifact_governance import ArtifactGovernanceRepository, ArtifactLifecycleState +from powercontext.builtin.persistence.artifacts import ArtifactRepository +from powercontext.builtin.persistence.errors import RepositoryNotFoundError +from powercontext.builtin.persistence.sources import SourceRepository +from powercontext.builtin.sources.content import ( + ArtifactRestorationOutcome, + ArtifactRestorationWrite, + ContentSource, + ContentSourceInternal, + ContentSourceTarget, +) +from powercontext.errors import RevisionConflictError +from powercontext.sources import SourceMaterialization, SourceRef + + +class ArtifactMergeSecurity(Protocol): + """Explicit policy shared by HTTP and background callers; none is implicit.""" + + def subject(self, context: Any) -> str: ... + + async def authorize( + self, + connection: AsyncConnection, + scope_id: str, + context: Any, + action: Literal["read", "write", "create"], + ref: ArtifactRef | None = None, + ) -> None: ... + + async def authorize_sources( + self, connection: AsyncConnection, scope_id: str, context: Any, sources: tuple[SourceRef, ...] + ) -> None: ... + + async def establish_owner( + self, connection: AsyncConnection, scope_id: str, artifact_id: str, context: Any + ) -> None: ... + + +class ArtifactMergeAdapter(Protocol): + family: str + + def draft( + self, + content: Any, + lineage: ArtifactLineage, + *, + merge_inputs: tuple[ArtifactRef, ...] = (), + historical: bool = False, + ) -> ArtifactDraft[Any]: ... + async def prepare(self, content: Any) -> Any: ... + def validate_prepared(self, prepared: Any) -> None: ... + async def publish( + self, + connection: AsyncConnection, + scope_id: str, + record: ArtifactMergeRecord, + prepared: Any, + execution_context: Any, + ) -> None: ... + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str) -> None: ... + + +MergeTags = Callable[[AsyncConnection, str, str, tuple[str, ...]], Awaitable[None]] + + +class ArtifactMergeService: + """Own Family transitions while leaving transaction lifetime with the caller. + + Inspect under a read transaction, close it, and prepare vectors without a + connection. Commit receives the caller's write transaction; it never + commits or retries a transaction whose outcome it does not own. + """ + + def __init__( + self, + *, + artifacts: ArtifactRepository, + adapter: ArtifactMergeAdapter, + security: ArtifactMergeSecurity, + merge_tags: MergeTags, + preview_signer: Any = None, + errors: ArtifactMergeErrors | None = None, + sources: SourceRepository | None = None, + ) -> None: + self.artifacts = artifacts + self.states = ArtifactGovernanceRepository() + self.adapter = adapter + self.family = adapter.family + self.errors = errors or ArtifactMergeErrors() + self.security = security + self.projections = adapter + self.merge_tags = merge_tags + self.preview_signer = preview_signer + self.sources = artifacts.source_repository if sources is None else sources + + async def get( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + context: Any, + *, + revision: int | None = None, + for_update: bool = False, + ) -> ArtifactMergeRecord: + identity = ArtifactRef( + family=self.family, artifact_id=artifact_id, revision=1 if revision is None else revision + ) + await self.security.authorize(connection, scope_id, context, "read", identity) + if revision is None: + stored = await self.artifacts.latest(connection, scope_id, self.family, artifact_id, for_update=for_update) + else: + stored = await self.artifacts.get(connection, scope_id, identity, for_update=for_update) + state = await self.states.get(connection, scope_id, self.family, artifact_id, for_update=for_update) + return ArtifactMergeRecord(artifact=stored, state=state) + + async def inspect_change( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + content: BaseModel, + context: Any, + *, + expected_revision: int | None = None, + expected_state_version: int | None = None, + lineage: ArtifactLineage | None = None, + reads: Sequence[ArtifactMergeRead] = (), + ) -> ArtifactMergePlan: + draft = self.adapter.draft(content, lineage or ArtifactLineage()) + current = None + if expected_revision is None: + target = ArtifactRef(family=self.family, artifact_id=artifact_id, revision=1) + await self.security.authorize(connection, scope_id, context, "create", target) + try: + stored = await self.artifacts.latest(connection, scope_id, self.family, artifact_id) + except RepositoryNotFoundError: + pass + else: + raise RevisionConflictError(draft, stored) + state = ArtifactLifecycleState.ACTIVE + else: + current = await self.get(connection, scope_id, artifact_id, context) + await self.security.authorize(connection, scope_id, context, "write", current.ref) + self._expected(current, expected_revision, expected_state_version, draft) + if current.state.merged_into_id is not None or current.state.lifecycle_state not in { + ArtifactLifecycleState.ACTIVE, + ArtifactLifecycleState.DEPRECATED, + }: + raise self.errors.invalid_state("merged and retired Artifacts cannot be edited") # noqa: TRY003 + target = ArtifactRef(family=self.family, artifact_id=artifact_id, revision=current.artifact.revision + 1) + state = current.state.lifecycle_state + draft = draft.model_copy(update={"artifacts": _unique_refs((*draft.artifacts, current.ref))}) + await self._validate_draft(connection, scope_id, target, draft, context) + dependencies = (*reads, *((current.as_read(),) if current is not None else ())) + await self._check_dependencies(connection, scope_id, dependencies, context) + return ArtifactMergePlan( + scope_id=scope_id, + subject=self.security.subject(context), + reads=self._unique_reads(dependencies), + writes=( + ArtifactMergeWrite( + artifact_id=artifact_id, + state=state, + draft=draft, + current=current, + replacement_artifact_id=None if current is None else current.state.replacement_artifact_id, + ), + ), + primary_artifact_id=artifact_id, + operation="change", + ) + + async def restoration_outcome( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + revision: int, + context: Any, + ) -> ArtifactRestorationOutcome | None: + """Read a primary revision's saved outcome after authorizing every affected identity.""" + + record = await self.get(connection, scope_id, artifact_id, context, revision=revision) + if self.sources is None: + return None + for source_ref in record.artifact.lineage.sources: + source = (await self.sources.get(connection, scope_id, source_ref)).value + if not isinstance(source, ContentSource) or source.internal is None: + continue + internal = source.internal + if ( + internal.target + != ContentSourceTarget( + scope_id=scope_id, family=self.family, artifact_id=artifact_id, revision=revision + ) + or internal.restoration_outcome is None + ): + continue + outcome = internal.restoration_outcome + refs = _unique_refs(( + outcome.target, + *outcome.undo_merge_results, + *( + ref + for write in outcome.writes + for ref in (write.before_ref, write.content_from_ref, write.after_ref) + ), + )) + for ref in refs: + await self.security.authorize(connection, scope_id, context, "read", ref) + return outcome + return None + + async def inspect_merge( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + inputs: Sequence[ArtifactMergeRead], + content: BaseModel, + context: Any, + *, + lineage: ArtifactLineage | None = None, + reads: Sequence[ArtifactMergeRead] = (), + ) -> ArtifactMergePlan: + selected = tuple(inputs) + ids = tuple(read.ref.artifact_id for read in selected) + if ( + len(ids) < 2 + or len(set(ids)) != len(ids) + or artifact_id in ids + or any(read.ref.family != self.family for read in selected) + ): + raise self.errors.relation("merge needs at least two distinct inputs and a new result identity") # noqa: TRY003 + target = ArtifactRef(family=self.family, artifact_id=artifact_id, revision=1) + await self.security.authorize(connection, scope_id, context, "create", target) + try: + stored = await self.artifacts.latest(connection, scope_id, self.family, artifact_id) + except RepositoryNotFoundError: + pass + else: + raise RevisionConflictError(content, stored) + current_inputs: list[ArtifactMergeRecord] = [] + for expected in selected: + current = await self.get(connection, scope_id, expected.ref.artifact_id, context) + await self.security.authorize(connection, scope_id, context, "write", current.ref) + if current.as_read() != expected: + raise self.errors.conflict("merge inputs changed before preparation") # noqa: TRY003 + if ( + current.state.lifecycle_state is not ArtifactLifecycleState.ACTIVE + or current.state.merged_into_id is not None + ): + raise self.errors.invalid_state("only active Artifacts can be merge inputs") # noqa: TRY003 + current_inputs.append(current) + draft = self.adapter.draft( + content, lineage or ArtifactLineage(), merge_inputs=tuple(read.ref for read in selected) + ) + # An explicitly supplied lineage must already contain every selected + # exact reference. Internal callers may omit lineage and use inputs. + if lineage is None: + draft = draft.model_copy(update={"artifacts": tuple(read.ref for read in selected)}) + self._require_merge_draft(artifact_id, draft, selected) + await self._validate_draft(connection, scope_id, target, draft, context) + dependencies = self._unique_reads((*reads, *selected)) + await self._check_dependencies(connection, scope_id, dependencies, context) + writes = ( + ArtifactMergeWrite( + artifact_id=artifact_id, state=ArtifactLifecycleState.ACTIVE, draft=draft, merge_input_ids=ids + ), + *( + ArtifactMergeWrite( + artifact_id=current.artifact.artifact_id, + state=ArtifactLifecycleState.DEPRECATED, + merged_into_id=artifact_id, + current=current, + ) + for current in current_inputs + ), + ) + return ArtifactMergePlan( + scope_id=scope_id, + subject=self.security.subject(context), + reads=dependencies, + writes=writes, + primary_artifact_id=artifact_id, + operation="merge", + ) + + async def inspect_forget( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + context: Any, + *, + expected_revision: int, + expected_state_version: int, + ) -> ArtifactMergePlan: + current = await self.get(connection, scope_id, artifact_id, context) + await self.security.authorize(connection, scope_id, context, "write", current.ref) + self._expected(current, expected_revision, expected_state_version) + if current.state.merged_into_id is not None or current.state.lifecycle_state not in { + ArtifactLifecycleState.ACTIVE, + ArtifactLifecycleState.DEPRECATED, + }: + raise self.errors.invalid_state("only active or deprecated Artifacts can be deprecated") # noqa: TRY003 + writes = ( + () + if current.state.lifecycle_state is ArtifactLifecycleState.DEPRECATED + else ( + ArtifactMergeWrite(artifact_id=artifact_id, state=ArtifactLifecycleState.DEPRECATED, current=current), + ) + ) + return ArtifactMergePlan( + scope_id=scope_id, + subject=self.security.subject(context), + reads=(current.as_read(),), + writes=writes, + primary_artifact_id=artifact_id, + operation="forget", + ) + + async def inspect_restore( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + context: Any, + *, + operation: ArtifactMergeRestoreOperation = "restore", + revision: int | None = None, + preview_token: str | None = None, + ) -> ArtifactMergePlan: + if operation not in {"restore", "undo_merge"}: + raise self.errors.invalid_state("unknown restoration operation") # noqa: TRY003 + if operation == "undo_merge" and revision is not None: + raise self.errors.invalid_state("undo_merge does not accept a content revision") # noqa: TRY003 + target = await self.get(connection, scope_id, artifact_id, context) + token_endpoint = ( + None + if preview_token is None + else self._signer().validate( + preview_token, + scope_id=scope_id, + subject=self.security.subject(context), + operation=operation, + artifact_id=artifact_id, + revision=revision, + ) + ) + + async def load_current(identity: str) -> ArtifactMergeRecord: + record = await self.get(connection, scope_id, identity, context) + await self.security.authorize(connection, scope_id, context, "write", record.ref) + return record + + async def load_inputs(identity: str) -> tuple[ArtifactRef, ...]: + ref = ArtifactRef(family=self.family, artifact_id=identity, revision=1) + await self.security.authorize(connection, scope_id, context, "read", ref) + return await self.artifacts.merge_inputs(connection, scope_id, ref) + + await self.security.authorize(connection, scope_id, context, "write", target.ref) + selected = None + if revision is not None: + selected = (await self.get(connection, scope_id, artifact_id, context, revision=revision)).artifact + plan = await calculate_restoration( + scope_id=scope_id, + subject=self.security.subject(context), + target=target, + operation=operation, + selected=selected, + load_current=load_current, + load_inputs=load_inputs, + adapter=self.adapter, + errors=self.errors, + ) + if token_endpoint is not None and plan.endpoint != token_endpoint: + raise self.errors.preview_stale("the restoration preview endpoint changed") # noqa: TRY003 + return replace(plan, preview_token=preview_token) + + def restoration_preview(self, plan: ArtifactMergePlan) -> ArtifactMergeRestorationPreview: + token, expires_at = self._signer().encode(plan) + if plan.endpoint is None: + raise self.errors.invalid_preview("the restoration plan has no endpoint") # noqa: TRY003 + retire_ids = {write.artifact_id for write in plan.writes if write.state is ArtifactLifecycleState.RETIRED} + return ArtifactMergeRestorationPreview( + preview_token=token, + expires_at=expires_at, + endpoint=plan.endpoint, + restore=plan.restore, + retire=tuple(read.ref for read in plan.reads if read.ref.artifact_id in retire_ids), + undo_merge_results=plan.undo_merge_results, + ) + + async def prepare_change(self, plan: ArtifactMergePlan) -> PreparedArtifactMerge: + """Perform external preparation only, with no database connection.""" + + projections: list[tuple[str, Any]] = [] + for write in plan.writes: + if write.state is not ArtifactLifecycleState.ACTIVE: + continue + content = ( + write.draft.content + if write.draft is not None + else write.current.artifact.content + if write.current is not None + else None + ) + if content is None: + raise self.errors.relation("active publication has no content") # noqa: TRY003 + projections.append((write.artifact_id, await self.projections.prepare(content))) + return PreparedArtifactMerge(plan=plan, projections=tuple(projections)) + + async def prepare_merge(self, plan: ArtifactMergePlan) -> PreparedArtifactMerge: + return await self.prepare_change(plan) + + async def prepare_forget(self, plan: ArtifactMergePlan) -> PreparedArtifactMerge: + return await self.prepare_change(plan) + + async def prepare_restore(self, plan: ArtifactMergePlan) -> PreparedArtifactMerge: + return await self.prepare_change(plan) + + async def validate_read_set( + self, connection: AsyncConnection, scope_id: str, reads: Sequence[ArtifactMergeRead], context: Any + ) -> None: + """Allow no-op processing plans to protect their actual decision dependencies.""" + + await self.artifacts.lock_heads(connection, scope_id, tuple(read.ref for read in reads)) + await self._check_dependencies(connection, scope_id, reads, context, for_update=True) + + async def commit( + self, + connection: AsyncConnection, + prepared: PreparedArtifactMerge, + context: Any, + *, + direct_source: SourceRef | None = None, + ) -> ArtifactMergeMutationResult: + return ( + await self.commit_window( + connection, prepared.plan.scope_id, (prepared,), context, direct_source=direct_source + ) + )[0] + + async def commit_window( + self, + connection: AsyncConnection, + scope_id: str, + prepared: Sequence[PreparedArtifactMerge], + context: Any, + *, + read_set: Sequence[ArtifactMergeRead] = (), + direct_source: SourceRef | None = None, + ) -> tuple[ArtifactMergeMutationResult, ...]: + """Validate all window dependencies before publishing any of its final decisions.""" + + plans = tuple(item.plan for item in prepared) + subject = self.security.subject(context) + self._require_window(plans, scope_id, subject) + for item in prepared: + for _, projection in item.projections: + self.projections.validate_prepared(projection) + reads = self._unique_reads((*read_set, *(read for plan in plans for read in plan.reads))) + await self.artifacts.lock_heads(connection, scope_id, tuple(read.ref for read in reads)) + for plan in plans: + await self._validate_preview(connection, plan, context) + current = await self._check_dependencies(connection, scope_id, reads, context, for_update=True) + for plan in plans: + if plan.operation in {"restore", "undo_merge"}: + verified = await self.inspect_restore( + connection, + scope_id, + plan.primary_artifact_id, + context, + operation=cast(ArtifactMergeRestoreOperation, plan.operation), + revision=plan.target_revision, + preview_token=plan.preview_token, + ) + if verified != plan: + raise self.errors.conflict("restoration relations changed; prepare the whole group again") # noqa: TRY003 + results: list[ArtifactMergeMutationResult] = [] + for item in prepared: + results.append(await self._commit_locked(connection, item, current, context, direct_source)) + return tuple(results) + + def _require_window(self, plans: Sequence[ArtifactMergePlan], scope_id: str, subject: str) -> None: + identities: set[str] = set() + for plan in plans: + if plan.scope_id != scope_id or plan.subject != subject: + raise self.errors.invalid_state("window scope or identity differs from preparation") # noqa: TRY003 + self._require_plan(plan) + for write in plan.writes: + if write.artifact_id in identities: + raise self.errors.relation("window decisions must coordinate each Artifact into one final write") # noqa: TRY003 + identities.add(write.artifact_id) + + async def _validate_preview(self, connection: AsyncConnection, plan: ArtifactMergePlan, context: Any) -> None: + if plan.preview_token is None: + return + endpoint = self._signer().validate( + plan.preview_token, + scope_id=plan.scope_id, + subject=plan.subject, + operation=cast(ArtifactMergeRestoreOperation, plan.operation), + artifact_id=plan.primary_artifact_id, + revision=plan.target_revision, + ) + current = await self.get(connection, plan.scope_id, endpoint.ref.artifact_id, context, for_update=True) + if current.as_read() != endpoint: + raise self.errors.preview_stale("the restoration preview endpoint changed") # noqa: TRY003 + + async def _commit_locked( + self, + connection: AsyncConnection, + prepared: PreparedArtifactMerge, + current: dict[str, ArtifactMergeRecord], + context: Any, + direct_source: SourceRef | None, + ) -> ArtifactMergeMutationResult: + """Apply a plan against the already verified initial window state.""" + + plan = prepared.plan + projections = dict(prepared.projections) + records: dict[str, ArtifactMergeRecord] = {} + restoration = await self._record_restoration(connection, plan, current) + for write in plan.writes: + record = await self._commit_write( + connection, + plan.scope_id, + write, + current.get(write.artifact_id), + projections.get(write.artifact_id), + context, + direct_source, + tuple( + (read.ref, read.state_version) + for read in plan.reads + if read.ref.artifact_id in write.merge_input_ids + ), + restoration_source=None + if restoration is None or write.artifact_id != plan.primary_artifact_id + else restoration[1], + ) + records[write.artifact_id] = record + if plan.primary_artifact_id not in records: + records[plan.primary_artifact_id] = current[plan.primary_artifact_id] + if restoration is not None and any( + records[write.after_ref.artifact_id].ref != write.after_ref + or records[write.after_ref.artifact_id].state.lifecycle_state.value != write.lifecycle_state + for write in restoration[0].writes + ): + raise self.errors.relation("restoration outcome differs from committed revisions") # noqa: TRY003 + return ArtifactMergeMutationResult( + changed=bool(plan.writes), + records=tuple(records[identity] for identity in sorted(records)), + primary_artifact_id=plan.primary_artifact_id, + retired=tuple( + record.ref + for record in records.values() + if record.state.lifecycle_state is ArtifactLifecycleState.RETIRED + ), + undo_merge_results=plan.undo_merge_results, + ) + + async def _record_restoration( + self, + connection: AsyncConnection, + plan: ArtifactMergePlan, + current: dict[str, ArtifactMergeRecord], + ) -> tuple[ArtifactRestorationOutcome, SourceRef] | None: + """Persist one outcome inside the already locked group transaction.""" + + if plan.operation not in {"restore", "undo_merge"} or not plan.writes: + return None + if self.sources is None: + raise self.errors.invalid_state("restoration requires a configured Source repository") # noqa: TRY003 + writes: list[ArtifactRestorationWrite] = [] + for write in plan.writes: + before = current[write.artifact_id].ref + writes.append( + ArtifactRestorationWrite( + before_ref=before, + content_from_ref=before + if write.artifact_id != plan.primary_artifact_id or plan.target_revision is None + else before.model_copy(update={"revision": plan.target_revision}), + after_ref=before.model_copy(update={"revision": before.revision + 1}), + lifecycle_state=cast(Literal["active", "retired"], write.state.value), + ) + ) + outcome = ArtifactRestorationOutcome( + operation=cast(ArtifactMergeRestoreOperation, plan.operation), + target=current[plan.primary_artifact_id].ref, + writes=tuple(writes), + undo_merge_results=tuple(current[identity].ref for identity in plan.undo_merge_results), + ) + primary = next(write.after_ref for write in writes if write.after_ref.artifact_id == plan.primary_artifact_id) + source = await self.sources.add( + connection, + plan.scope_id, + ContentSource( + name=f"artifact-restoration_{uuid4().hex}", + materialization=SourceMaterialization.CAPTURED, + content="Artifact restoration completed.", + internal=ContentSourceInternal( + role="lineage_only", + operation="artifact_restore" if plan.operation == "restore" else "artifact_undo_merge", + target=ContentSourceTarget(scope_id=plan.scope_id, **primary.model_dump()), + restoration_outcome=outcome, + ), + ), + ) + return outcome, source.ref + + async def _commit_write( + self, + connection: AsyncConnection, + scope_id: str, + write: ArtifactMergeWrite, + before: ArtifactMergeRecord | None, + projection: Any | None, + context: Any, + direct_source: SourceRef | None, + merge_inputs: tuple[tuple[ArtifactRef, int], ...] = (), + *, + restoration_source: SourceRef | None = None, + ) -> ArtifactMergeRecord: + """Publish one identity after the whole plan's locks and checks are complete.""" + + draft = write.draft + if draft is not None: + target = ArtifactRef( + family=self.family, + artifact_id=write.artifact_id, + revision=1 if before is None else before.artifact.revision + 1, + ) + system_sources = tuple(source for source in (direct_source, restoration_source) if source is not None) + if system_sources: + sources = { + (source.source_type, source.source_id): source for source in (*draft.sources, *system_sources) + } + draft = draft.model_copy(update={"sources": tuple(sources.values())}) + await self._validate_draft(connection, scope_id, target, draft, context, system_sources=system_sources) + if before is None: + await self.security.authorize(connection, scope_id, context, "create", target) + if write.merge_input_ids: + artifact = await self.artifacts.create_merge_result( + connection, scope_id, write.artifact_id, draft, merge_inputs + ) + else: + artifact = await self.artifacts.create(connection, scope_id, write.artifact_id, draft) + state = await self.states.get(connection, scope_id, self.family, write.artifact_id, for_update=True) + await self.security.establish_owner(connection, scope_id, write.artifact_id, context) + if write.merge_input_ids: + await self.merge_tags(connection, scope_id, write.artifact_id, write.merge_input_ids) + else: + state = before.state + if state.merged_into_id is not None: + artifact = await self.artifacts.revise_for_restoration( + connection, scope_id, before.artifact, draft, state + ) + else: + artifact = await self.artifacts.revise(connection, scope_id, before.artifact, draft) + else: + if before is None: + raise self.errors.relation("state transitions require an existing Artifact") # noqa: TRY003 + artifact, state = before.artifact, before.state + state = await self.states.transition_merge( + connection, + scope_id, + self.family, + write.artifact_id, + state, + write.state, + write.merged_into_id, + write.replacement_artifact_id, + ) + record = ArtifactMergeRecord(artifact=artifact, state=state) + if state.lifecycle_state is ArtifactLifecycleState.ACTIVE: + if projection is None: + raise self.errors.relation("active publication requires a prepared projection") # noqa: TRY003 + await self.projections.publish(connection, scope_id, record, projection, context) + else: + await self.projections.remove(connection, scope_id, write.artifact_id) + return record + + async def _check_dependencies( + self, + connection: AsyncConnection, + scope_id: str, + reads: Sequence[ArtifactMergeRead], + context: Any, + *, + for_update: bool = False, + ) -> dict[str, ArtifactMergeRecord]: + records: dict[str, ArtifactMergeRecord] = {} + for expected in self._unique_reads(reads): + current = await self.get(connection, scope_id, expected.ref.artifact_id, context, for_update=for_update) + await self.security.authorize(connection, scope_id, context, "write", current.ref) + if current.as_read() != expected: + raise self.errors.conflict("Artifact content, state or merge relation changed; prepare again") # noqa: TRY003 + if for_update: + await self.states.require_summary( + connection, scope_id, self.family, expected.ref.artifact_id, current.state + ) + records[expected.ref.artifact_id] = current + return records + + async def _validate_draft( + self, + connection: AsyncConnection, + scope_id: str, + target: ArtifactRef, + draft: ArtifactDraft[Any], + context: Any, + *, + system_sources: tuple[SourceRef, ...] = (), + ) -> None: + # The host creates a lineage_only Source bound to this exact new target. + # It is not caller-supplied evidence and need not grant Scope body read. + ordinary_sources = tuple(source for source in draft.sources if source not in system_sources) + await self.security.authorize_sources(connection, scope_id, context, ordinary_sources) + await self.artifacts.validate_lineage_sources(connection, scope_id, target, draft.sources) + for ref in draft.artifacts: + await self.security.authorize(connection, scope_id, context, "read", ref) + await self.artifacts.get(connection, scope_id, ref) + + def _expected( + self, current: ArtifactMergeRecord, revision: int, state_version: int | None, requested: Any = None + ) -> None: + if current.artifact.revision != revision: + raise RevisionConflictError(requested, current.artifact) + if state_version is not None and current.state.governance_generation != state_version: + raise self.errors.conflict("the expected Artifact state version is stale") # noqa: TRY003 + + def _require_merge_draft( + self, artifact_id: str, draft: ArtifactDraft[Any], inputs: Sequence[ArtifactMergeRead] + ) -> None: + refs = tuple(read.ref for read in inputs) + if ( + len(refs) < 2 + or len({ref.artifact_id for ref in refs}) != len(refs) + or artifact_id in {ref.artifact_id for ref in refs} + ): + raise self.errors.relation("merge requires distinct inputs and a new result identity") # noqa: TRY003 + if draft.family != self.family or any(ref.family != self.family for ref in refs): + raise self.errors.relation("merge inputs and result must have the same Family") # noqa: TRY003 + selected = {(ref.family, ref.artifact_id, ref.revision) for ref in refs} + lineage = {(ref.family, ref.artifact_id, ref.revision) for ref in draft.artifacts} + if ( + any(sum(item == ref for item in draft.artifacts) != 1 for ref in refs) + or not selected.issubset(lineage) + or any( + ref.family == self.family + and ref.artifact_id in {item.artifact_id for item in refs} + and (ref.family, ref.artifact_id, ref.revision) not in selected + for ref in draft.artifacts + ) + ): + raise self.errors.relation("merge inputs require their supplied exact active versions in lineage") # noqa: TRY003 + + def _require_plan(self, plan: ArtifactMergePlan) -> None: + ids = tuple(write.artifact_id for write in plan.writes) + if len(ids) != len(set(ids)): + raise self.errors.relation("one plan cannot write an identity twice") # noqa: TRY003 + reads = {read.ref.artifact_id: read for read in self._unique_reads(plan.reads)} + for write in plan.writes: + if write.current is not None and write.current.state.lifecycle_state is ArtifactLifecycleState.RETIRED: + raise self.errors.invalid_state("retired Artifact identities cannot be modified") # noqa: TRY003 + if write.current is not None and reads.get(write.artifact_id) != write.current.as_read(): + raise self.errors.relation("every existing write requires its exact current read dependency") # noqa: TRY003 + if write.current is None and (write.draft is None or write.state is not ArtifactLifecycleState.ACTIVE): + raise self.errors.relation("new Artifacts require active content") # noqa: TRY003 + if write.draft is None: + continue + if write.merge_input_ids: + self._require_merge_plan(plan, write, reads) + else: + self.adapter.draft( + write.draft.content, + ArtifactLineage(sources=write.draft.sources, artifacts=write.draft.artifacts), + historical=plan.operation in {"restore", "undo_merge"}, + ) + if ( + write.current is not None + and plan.operation == "change" + and ( + write.current.state.merged_into_id is not None + or write.current.state.lifecycle_state + not in {ArtifactLifecycleState.ACTIVE, ArtifactLifecycleState.DEPRECATED} + ) + ): + raise self.errors.invalid_state("merged and retired Artifacts cannot be edited") # noqa: TRY003 + + def _require_merge_plan( + self, plan: ArtifactMergePlan, write: ArtifactMergeWrite, reads: dict[str, ArtifactMergeRead] + ) -> None: + if plan.operation != "merge" or write.current is not None or write.draft is None: + raise self.errors.relation("merge creation belongs only to a new result") # noqa: TRY003 + if any(identity not in reads for identity in write.merge_input_ids): + raise self.errors.relation("merge inputs require exact read dependencies") # noqa: TRY003 + inputs = tuple(reads[identity] for identity in write.merge_input_ids) + self._require_merge_draft(write.artifact_id, write.draft, inputs) + writes = {item.artifact_id: item for item in plan.writes} + for expected in inputs: + matching = writes.get(expected.ref.artifact_id) + if ( + expected.lifecycle_state is not ArtifactLifecycleState.ACTIVE + or matching is None + or matching.state is not ArtifactLifecycleState.DEPRECATED + or matching.merged_into_id != write.artifact_id + ): + raise self.errors.relation("all active merge inputs must freeze into the new result") # noqa: TRY003 + + def _signer(self) -> ArtifactMergePreviewSigner: + if self.preview_signer is None: + raise self.errors.invalid_preview( # noqa: TRY003 + "restoration previews need an explicitly configured shared signing key" + ) + return self.preview_signer + + def _unique_reads(self, reads: Sequence[ArtifactMergeRead]) -> tuple[ArtifactMergeRead, ...]: + unique: dict[str, ArtifactMergeRead] = {} + for read in reads: + if read.ref.family != self.family: + raise self.errors.relation("read dependencies must belong to the service Family") # noqa: TRY003 + previous = unique.get(read.ref.artifact_id) + if previous is not None and previous != read: + raise self.errors.conflict("a plan cannot mix versions of the same Artifact") # noqa: TRY003 + unique[read.ref.artifact_id] = read + return tuple(unique[identity] for identity in sorted(unique)) + + +def _unique_refs(refs: Sequence[ArtifactRef]) -> tuple[ArtifactRef, ...]: + return tuple({(ref.family, ref.artifact_id, ref.revision): ref for ref in refs}.values()) diff --git a/src/powercontext/builtin/artifacts/merge_models.py b/src/powercontext/builtin/artifacts/merge_models.py new file mode 100644 index 0000000000..5a1f8855c0 --- /dev/null +++ b/src/powercontext/builtin/artifacts/merge_models.py @@ -0,0 +1,160 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""In-memory plans for shared Artifact merge and whole-group restoration.""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import datetime +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, StrictInt + +from powercontext.artifacts import Artifact, ArtifactDraft, ArtifactRef +from powercontext.builtin.persistence.artifact_governance import ArtifactGovernance, ArtifactLifecycleState +from powercontext.errors import PowerContextError + + +class ArtifactMergeConflictError(PowerContextError): + code = "artifact_changed" + + +class ArtifactMergeRelationError(PowerContextError): + code = "invalid_merge_relation" + + +class InvalidArtifactMergeStateError(PowerContextError, ValueError): + code = "invalid_artifact_state" + + +class InvalidArtifactMergePreviewError(PowerContextError, ValueError): + code = "invalid_preview" + + +class ArtifactMergePreviewExpiredError(PowerContextError): + code = "preview_expired" + + +class ArtifactMergePreviewStaleError(ArtifactMergeConflictError): + code = "preview_stale" + + +@dataclass(frozen=True) +class ArtifactMergeErrors: + conflict: type[Exception] = ArtifactMergeConflictError + relation: type[Exception] = ArtifactMergeRelationError + invalid_state: type[Exception] = InvalidArtifactMergeStateError + invalid_preview: type[Exception] = InvalidArtifactMergePreviewError + preview_expired: type[Exception] = ArtifactMergePreviewExpiredError + preview_stale: type[Exception] = ArtifactMergePreviewStaleError + + +class ArtifactMergeRead(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + ref: ArtifactRef + lifecycle_state: ArtifactLifecycleState + state_version: StrictInt = Field(ge=0) + merged_into_id: str | None = None + replacement_artifact_id: str | None = None + + +@dataclass(frozen=True) +class ArtifactMergeRecord: + artifact: Artifact[Any] + state: ArtifactGovernance + + def as_read(self) -> ArtifactMergeRead: + return ArtifactMergeRead( + ref=self.ref, + lifecycle_state=self.state.lifecycle_state, + state_version=self.state.governance_generation, + merged_into_id=self.state.merged_into_id, + replacement_artifact_id=self.state.replacement_artifact_id, + ) + + @property + def ref(self) -> ArtifactRef: + return self.artifact.as_ref() + + +class ArtifactMergeRestoreItem(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + artifact_id: str + source_revision: StrictInt = Field(ge=1) + creates_revision: bool + + +class ArtifactMergeRestorationPreview(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + preview_token: str + expires_at: datetime + endpoint: ArtifactMergeRead + restore: tuple[ArtifactMergeRestoreItem, ...] + retire: tuple[ArtifactRef, ...] + undo_merge_results: tuple[str, ...] + + +@dataclass(frozen=True) +class ArtifactMergeWrite: + artifact_id: str + state: ArtifactLifecycleState + draft: ArtifactDraft[Any] | None = None + merged_into_id: str | None = None + current: ArtifactMergeRecord | None = None + merge_input_ids: tuple[str, ...] = () + replacement_artifact_id: str | None = None + + +@dataclass(frozen=True) +class ArtifactMergePlan: + scope_id: str + subject: str + reads: tuple[ArtifactMergeRead, ...] + writes: tuple[ArtifactMergeWrite, ...] + primary_artifact_id: str + operation: Literal["change", "merge", "forget", "restore", "undo_merge"] + endpoint: ArtifactMergeRead | None = None + target_revision: int | None = None + undo_merge_results: tuple[str, ...] = () + restore: tuple[ArtifactMergeRestoreItem, ...] = () + preview_token: str | None = None + + +@dataclass(frozen=True) +class PreparedArtifactMerge: + plan: ArtifactMergePlan + projections: tuple[tuple[str, Any], ...] + + +@dataclass(frozen=True) +class ArtifactMergeMutationResult: + changed: bool + records: tuple[ArtifactMergeRecord, ...] + primary_artifact_id: str + retired: tuple[ArtifactRef, ...] = () + undo_merge_results: tuple[str, ...] = () + + @property + def primary(self) -> ArtifactMergeRecord: + return next(record for record in self.records if record.ref.artifact_id == self.primary_artifact_id) + + @property + def restored(self) -> tuple[ArtifactRef, ...]: + return tuple( + record.ref for record in self.records if record.state.lifecycle_state is ArtifactLifecycleState.ACTIVE + ) + + +ArtifactMergeRestoreOperation = Literal["restore", "undo_merge"] diff --git a/src/powercontext/builtin/artifacts/merge_restoration.py b/src/powercontext/builtin/artifacts/merge_restoration.py new file mode 100644 index 0000000000..8a9b0178ff --- /dev/null +++ b/src/powercontext/builtin/artifacts/merge_restoration.py @@ -0,0 +1,299 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Whole-group traversal and optional signed endpoint previews.""" + +from __future__ import annotations + +import base64 +import binascii +import hmac +import json +from collections.abc import Awaitable, Callable, Mapping +from datetime import UTC, datetime, timedelta +from typing import Any, cast + +from pydantic import BaseModel, ConfigDict, StrictInt, ValidationError + +from powercontext.artifacts import Artifact, ArtifactLineage, ArtifactRef +from powercontext.builtin.artifacts.merge_models import ( + ArtifactMergeErrors, + ArtifactMergePlan, + ArtifactMergeRead, + ArtifactMergeRecord, + ArtifactMergeRestoreItem, + ArtifactMergeRestoreOperation, + ArtifactMergeWrite, +) +from powercontext.builtin.persistence.artifact_governance import ArtifactLifecycleState + + +async def calculate_restoration( + *, + scope_id: str, + subject: str, + target: ArtifactMergeRecord, + operation: ArtifactMergeRestoreOperation, + selected: Artifact[Any] | None, + load_current: Callable[[str], Awaitable[ArtifactMergeRecord]], + load_inputs: Callable[[str], Awaitable[tuple[ArtifactRef, ...]]], + adapter: Any, + errors: ArtifactMergeErrors, +) -> ArtifactMergePlan: + """Follow the current merge path, then interpret it from its endpoint inward.""" + + if target.state.lifecycle_state is ArtifactLifecycleState.RETIRED: + raise errors.invalid_state("retired Artifact identities cannot be restored") # noqa: TRY003 + if operation == "undo_merge" and selected is not None: + raise errors.invalid_state("undo_merge does not accept a content revision") # noqa: TRY003 + records = {target.artifact.artifact_id: target} + result_ids: list[str] = [] + cursor = target + while cursor.state.merged_into_id is not None: + next_id = cursor.state.merged_into_id + if next_id is None or next_id in records: + raise errors.relation("the current merge path is missing or cyclic") # noqa: TRY003 + cursor = await load_current(next_id) + if cursor.state.lifecycle_state is ArtifactLifecycleState.RETIRED: + raise errors.relation("a live merge path cannot end in a retired result") # noqa: TRY003 + records[next_id] = cursor + result_ids.append(next_id) + endpoint = cursor.as_read() + undo_ids = list(reversed(result_ids)) + if operation == "undo_merge": + undo_ids.append(target.artifact.artifact_id) + final = await _expand_undo(undo_ids, records, load_current, load_inputs, target.ref.family, errors) + target_id = target.artifact.artifact_id + if operation == "restore": + final[target_id] = ArtifactLifecycleState.ACTIVE + writes, restored = _restoration_writes(final, records, target_id, selected, adapter) + return ArtifactMergePlan( + scope_id=scope_id, + subject=subject, + reads=tuple(records[identity].as_read() for identity in sorted(records)), + writes=tuple(writes), + primary_artifact_id=target_id, + operation=operation, + endpoint=endpoint, + target_revision=None if selected is None else selected.revision, + undo_merge_results=tuple(undo_ids), + restore=tuple(restored), + ) + + +async def _expand_undo( + undo_ids: list[str], + records: dict[str, ArtifactMergeRecord], + load_current: Callable[[str], Awaitable[ArtifactMergeRecord]], + load_inputs: Callable[[str], Awaitable[tuple[ArtifactRef, ...]]], + family: str, + errors: ArtifactMergeErrors, +) -> dict[str, ArtifactLifecycleState]: + """Verify frozen input relations and compute the final state of the affected group.""" + + final: dict[str, ArtifactLifecycleState] = {} + for result_id in undo_ids: + refs = await load_inputs(result_id) + if ( + len(refs) < 2 + or len({ref.artifact_id for ref in refs}) != len(refs) + or any(ref.family != family or ref.artifact_id == result_id for ref in refs) + ): + raise errors.relation("the result has no valid marked merge inputs") # noqa: TRY003 + selected_ids = {ref.artifact_id for ref in refs} + if any( + record.state.merged_into_id == result_id and identity not in selected_ids + for identity, record in records.items() + ): + raise errors.relation("the merge path input is absent from the result marked inputs") # noqa: TRY003 + final[result_id] = ArtifactLifecycleState.RETIRED + for ref in refs: + current = records.get(ref.artifact_id) + if current is None: + current = await load_current(ref.artifact_id) + records[ref.artifact_id] = current + if ( + current.state.lifecycle_state is not ArtifactLifecycleState.DEPRECATED + or current.state.merged_into_id != result_id + or current.ref != ref + ): + raise errors.relation( # noqa: TRY003 + "merge inputs do not match their current destination and frozen version" + ) + final[ref.artifact_id] = ArtifactLifecycleState.ACTIVE + return final + + +def _restoration_writes( + final: dict[str, ArtifactLifecycleState], + records: dict[str, ArtifactMergeRecord], + target_id: str, + selected: Artifact[Any] | None, + adapter: Any, +) -> tuple[list[ArtifactMergeWrite], list[ArtifactMergeRestoreItem]]: + """Build only final changes; intermediate results never need projections.""" + + writes: list[ArtifactMergeWrite] = [] + restored: list[ArtifactMergeRestoreItem] = [] + for artifact_id, state in sorted(final.items()): + current = records[artifact_id] + source = selected if artifact_id == target_id and selected is not None else current.artifact + changed = ( + state != current.state.lifecycle_state + or current.state.merged_into_id is not None + or current.state.replacement_artifact_id is not None + or (artifact_id == target_id and selected is not None) + ) + draft = None + if changed: + # Historical lineage_only Sources stay bound to their historical + # target; a precise historical Artifact reference retains evidence. + refs = tuple( + dict.fromkeys((ref.family, ref.artifact_id, ref.revision) for ref in (current.ref, source.as_ref())) + ) + draft = adapter.draft( + source.content, + ArtifactLineage( + artifacts=tuple( + ArtifactRef(family=family, artifact_id=identity, revision=revision) + for family, identity, revision in refs + ) + ), + historical=True, + ) + if state is ArtifactLifecycleState.ACTIVE: + restored.append( + ArtifactMergeRestoreItem( + artifact_id=artifact_id, + source_revision=source.revision, + creates_revision=draft is not None, + ) + ) + if changed: + writes.append(ArtifactMergeWrite(artifact_id=artifact_id, state=state, draft=draft, current=current)) + return writes, restored + + +class _PreviewClaims(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + format: str + key_id: str + scope_id: str + subject: str + operation: ArtifactMergeRestoreOperation + artifact_id: str + revision: StrictInt | None + endpoint: dict[str, Any] + expires_at: StrictInt + + +class ArtifactMergePreviewSigner: + """Sign previews with explicit shared keys; no per-process random fallback.""" + + def __init__( + self, + *, + keys: Mapping[str, bytes], + active_key_id: str, + ttl_seconds: int = 300, + clock: Callable[[], datetime] | None = None, + endpoint_type: type[BaseModel] = ArtifactMergeRead, + preview_format: str = "powercontext.artifact.restoration-preview.v1", + errors: ArtifactMergeErrors | None = None, + ) -> None: + if active_key_id not in keys or any(not key_id or len(secret) < 32 for key_id, secret in keys.items()): + raise ValueError("preview keys need a configured active key and at least 32 secret bytes") # noqa: TRY003 + if isinstance(ttl_seconds, bool) or ttl_seconds < 1: + raise ValueError("preview TTL must be positive") # noqa: TRY003 + self._endpoint_type = endpoint_type + self._format = preview_format + self._errors = errors or ArtifactMergeErrors() + self._keys = dict(keys) + self._active_key_id = active_key_id + self._ttl = timedelta(seconds=ttl_seconds) + self._clock = clock or (lambda: datetime.now(UTC)) + + def encode(self, plan: Any) -> tuple[str, datetime]: + if plan.endpoint is None or plan.operation not in {"restore", "undo_merge"}: + raise self._errors.invalid_preview("only restoration plans can be previewed") # noqa: TRY003 + expires_at = self._now() + self._ttl + claims = _PreviewClaims( + format=self._format, + key_id=self._active_key_id, + scope_id=plan.scope_id, + subject=plan.subject, + operation=cast(ArtifactMergeRestoreOperation, plan.operation), + artifact_id=plan.primary_artifact_id, + revision=plan.target_revision, + endpoint=plan.endpoint.model_dump(mode="json"), + expires_at=int(expires_at.timestamp()), + ) + payload = json.dumps( + claims.model_dump(mode="json"), sort_keys=True, separators=(",", ":"), ensure_ascii=False + ).encode() + signature = hmac.digest(self._keys[self._active_key_id], payload, "sha256") + return f"{_encode(payload)}.{_encode(signature)}", datetime.fromtimestamp(claims.expires_at, UTC) + + def validate( + self, + token: str, + *, + scope_id: str, + subject: str, + operation: ArtifactMergeRestoreOperation, + artifact_id: str, + revision: int | None, + ) -> Any: + if len(token) > 16_384: + raise self._errors.invalid_preview("the restoration preview token is invalid") # noqa: TRY003 + try: + payload_part, signature_part = token.split(".") + payload, signature = _decode(payload_part), _decode(signature_part) + claims = _PreviewClaims.model_validate_json(payload, strict=True) + key = self._keys[claims.key_id] + endpoint = self._endpoint_type.model_validate_json( + json.dumps(claims.endpoint, ensure_ascii=False), strict=True + ) + except (ValueError, UnicodeError, binascii.Error, ValidationError, KeyError) as error: + raise self._errors.invalid_preview("the restoration preview token is invalid") from error # noqa: TRY003 + if claims.format != self._format: + raise self._errors.invalid_preview("the restoration preview format is invalid") # noqa: TRY003 + if not hmac.compare_digest(signature, hmac.digest(key, payload, "sha256")): + raise self._errors.invalid_preview("the restoration preview signature is invalid") # noqa: TRY003 + if (claims.scope_id, claims.subject, claims.operation, claims.artifact_id, claims.revision) != ( + scope_id, + subject, + operation, + artifact_id, + revision, + ): + raise self._errors.invalid_preview("the restoration preview does not match this request") # noqa: TRY003 + if int(self._now().timestamp()) >= claims.expires_at: + raise self._errors.preview_expired("the restoration preview has expired") # noqa: TRY003 + return endpoint + + def _now(self) -> datetime: + now = self._clock() + if now.tzinfo is None: + raise ValueError("preview clock must return an aware timestamp") # noqa: TRY003 + return now.astimezone(UTC) + + +def _encode(value: bytes) -> str: + return base64.urlsafe_b64encode(value).rstrip(b"=").decode("ascii") + + +def _decode(value: str) -> bytes: + raw = value.encode("ascii") + return base64.b64decode(raw + b"=" * (-len(raw) % 4), altchars=b"-_", validate=True) diff --git a/src/powercontext/builtin/persistence/artifact_governance.py b/src/powercontext/builtin/persistence/artifact_governance.py index 6a3a403d59..63659b9627 100644 --- a/src/powercontext/builtin/persistence/artifact_governance.py +++ b/src/powercontext/builtin/persistence/artifact_governance.py @@ -48,12 +48,19 @@ class ArtifactGovernance(BaseModel): artifact: ArtifactRef lifecycle_state: ArtifactLifecycleState replacement_artifact_id: str | None = None + merged_into_id: str | None = None governance_generation: int = Field(ge=0) @model_validator(mode="after") def validate_replacement(self) -> ArtifactGovernance: if self.replacement_artifact_id is not None and self.lifecycle_state is not ArtifactLifecycleState.DEPRECATED: raise ValueError("only a deprecated Artifact may name a replacement") # noqa: TRY003 + if self.merged_into_id is not None and ( + self.lifecycle_state is not ArtifactLifecycleState.DEPRECATED + or self.replacement_artifact_id is not None + or self.merged_into_id == self.artifact.artifact_id + ): + raise ValueError("a frozen input requires a distinct merge result and no ordinary replacement") # noqa: TRY003 return self @@ -67,24 +74,81 @@ async def get( family: str, artifact_id: str, /, + *, + for_update: bool = False, ) -> ArtifactGovernance: - row = ( - ( - await connection.execute( - select(ARTIFACT_HEADS_TABLE).where( - ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, - ARTIFACT_HEADS_TABLE.c.family == family, - ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, - ) - ) - ) - .mappings() - .one_or_none() + statement = select(ARTIFACT_HEADS_TABLE).where( + ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, + ARTIFACT_HEADS_TABLE.c.family == family, + ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, ) + if for_update: + statement = statement.with_for_update() + row = (await connection.execute(statement)).mappings().one_or_none() if row is None: raise RepositoryNotFoundError("artifact-head", (scope_id, family, artifact_id)) return _governance(row) + async def require_summary( + self, connection: AsyncConnection, scope_id: str, family: str, artifact_id: str, expected: ArtifactGovernance + ) -> ArtifactGovernance: + current = await self.get(connection, scope_id, family, artifact_id, for_update=True) + if ( + current.lifecycle_state != expected.lifecycle_state + or current.merged_into_id != expected.merged_into_id + or current.replacement_artifact_id != expected.replacement_artifact_id + or current.governance_generation != expected.governance_generation + ): + raise StoredPayloadConflictError("artifact-governance", (scope_id, family, artifact_id)) + return current + + async def transition_merge( + self, + connection: AsyncConnection, + scope_id: str, + family: str, + artifact_id: str, + expected: ArtifactGovernance, + lifecycle_state: ArtifactLifecycleState, + merged_into_id: str | None = None, + replacement_artifact_id: str | None = None, + ) -> ArtifactGovernance: + """Apply a service-validated group transition while retaining unchanged state generations.""" + + _validate_transition(expected.lifecycle_state, lifecycle_state) + current = await self.require_summary(connection, scope_id, family, artifact_id, expected) + if ( + expected.lifecycle_state == lifecycle_state + and expected.merged_into_id == merged_into_id + and expected.replacement_artifact_id == replacement_artifact_id + ): + return current + requested = ArtifactGovernance( + artifact=current.artifact, + lifecycle_state=lifecycle_state, + merged_into_id=merged_into_id, + replacement_artifact_id=replacement_artifact_id, + governance_generation=expected.governance_generation + 1, + ) + result = await connection.execute( + update(ARTIFACT_HEADS_TABLE) + .where( + ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, + ARTIFACT_HEADS_TABLE.c.family == family, + ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, + ARTIFACT_HEADS_TABLE.c.governance_generation == expected.governance_generation, + ) + .values( + lifecycle_state=lifecycle_state.value, + merged_into_id=merged_into_id, + replacement_artifact_id=replacement_artifact_id, + governance_generation=requested.governance_generation, + ) + ) + if result.rowcount != 1: + raise StoredPayloadConflictError("artifact-governance", (scope_id, family, artifact_id)) + return requested + async def transition( self, connection: AsyncConnection, @@ -97,6 +161,20 @@ async def transition( /, ) -> ArtifactGovernance: current = await self.get(connection, scope_id, family, artifact_id) + if current.merged_into_id is not None: + raise InvalidArtifactLifecycleError("frozen merge inputs require whole-group restoration") # noqa: TRY003 + if lifecycle_state is ArtifactLifecycleState.RETIRED: + frozen_input = await connection.scalar( + select(ARTIFACT_HEADS_TABLE.c.artifact_id) + .where( + ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, + ARTIFACT_HEADS_TABLE.c.family == family, + ARTIFACT_HEADS_TABLE.c.merged_into_id == artifact_id, + ) + .limit(1) + ) + if frozen_input is not None: + raise InvalidArtifactLifecycleError("an effective merge result cannot strand frozen inputs") # noqa: TRY003 _validate_transition(current.lifecycle_state, lifecycle_state) if replacement_artifact_id is not None: if replacement_artifact_id == artifact_id: @@ -154,6 +232,7 @@ def _governance(row) -> ArtifactGovernance: replacement_artifact_id=( None if row["replacement_artifact_id"] is None else str(row["replacement_artifact_id"]) ), + merged_into_id=None if row["merged_into_id"] is None else str(row["merged_into_id"]), governance_generation=int(row["governance_generation"]), ) diff --git a/src/powercontext/builtin/persistence/artifacts.py b/src/powercontext/builtin/persistence/artifacts.py index 9b52790a0b..8e16a20afa 100644 --- a/src/powercontext/builtin/persistence/artifacts.py +++ b/src/powercontext/builtin/persistence/artifacts.py @@ -31,12 +31,14 @@ ArtifactLineage, ArtifactRef, ) +from powercontext.builtin.persistence.artifact_governance import ArtifactGovernance, InvalidArtifactLifecycleError from powercontext.builtin.persistence.codec import dump_model, load_model, stored_bytes, validate_json_model from powercontext.builtin.persistence.errors import ( IdentityMismatchError, InvalidPublicationLineageError, InvalidRepositoryArgumentError, RepositoryNotFoundError, + StoredPayloadConflictError, ) from powercontext.builtin.persistence.tables import ( ARTIFACT_HEADS_TABLE, @@ -90,6 +92,12 @@ def families(self) -> frozenset[str]: return frozenset(self._by_family) + @property + def source_repository(self) -> Any | None: + """Expose the composed Source repository to services recording revision provenance.""" + + return self._sources + async def create( self, connection: AsyncConnection, @@ -135,6 +143,105 @@ async def create( raise conflict from None return artifact + async def create_merge_result( + self, + connection: AsyncConnection, + scope_id: str, + artifact_id: str, + draft: ArtifactDraft[Any] | RepositoryArtifactDraft, + inputs: Sequence[tuple[ArtifactRef, int]], + /, + ) -> Artifact[Any]: + """Create a result with explicit exact input membership; the caller freezes its inputs.""" + + refs = tuple(ref for ref, _ in inputs) + identities = {ref.artifact_id for ref in refs} + if ( + len(refs) < 2 + or len(identities) != len(refs) + or artifact_id in identities + or any(ref.family != draft.family for ref in refs) + or any(ref not in draft.artifacts for ref in refs) + or any(sum(item == ref for item in draft.artifacts) != 1 for ref in refs) + ): + raise InvalidArtifactLifecycleError("merge inputs must be distinct exact references in the result Family") # noqa: TRY003 + await self.lock_heads(connection, scope_id, refs) + for ref, generation in inputs: + head = ( + ( + await connection.execute( + select(ARTIFACT_HEADS_TABLE).where( + ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, + ARTIFACT_HEADS_TABLE.c.family == ref.family, + ARTIFACT_HEADS_TABLE.c.artifact_id == ref.artifact_id, + ) + ) + ) + .mappings() + .one_or_none() + ) + if ( + head is None + or head["revision"] != ref.revision + or head["governance_generation"] != generation + or head["lifecycle_state"] != "active" + or head["merged_into_id"] is not None + ): + raise InvalidArtifactLifecycleError( # noqa: TRY003 + "merge inputs must retain their expected active head and generation" + ) + result = await self.create(connection, scope_id, artifact_id, draft) + await connection.execute( + update(ARTIFACT_LINEAGE_ARTIFACTS_TABLE) + .where( + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.scope_id == scope_id, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.family == result.family, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.artifact_id == result.artifact_id, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.revision == 1, + tuple_( + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.upstream_family, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.upstream_artifact_id, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.upstream_revision, + ).in_(tuple((ref.family, ref.artifact_id, ref.revision) for ref in refs)), + ) + .values(is_merge_input=True) + ) + return result + + async def merge_inputs( + self, + connection: AsyncConnection, + scope_id: str, + result: ArtifactRef, + /, + ) -> tuple[ArtifactRef, ...]: + """Read selected creation inputs in their immutable lineage ordinal order.""" + + if result.revision != 1: + return () + await self.get(connection, scope_id, result) + rows = ( + await connection.execute( + select(ARTIFACT_LINEAGE_ARTIFACTS_TABLE) + .where( + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.scope_id == scope_id, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.family == result.family, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.artifact_id == result.artifact_id, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.revision == 1, + ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.is_merge_input.is_(True), + ) + .order_by(ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.ordinal) + ) + ).mappings() + return tuple( + ArtifactRef( + family=str(row["upstream_family"]), + artifact_id=str(row["upstream_artifact_id"]), + revision=int(row["upstream_revision"]), + ) + for row in rows + ) + async def revise( self, connection: AsyncConnection, @@ -145,6 +252,36 @@ async def revise( ) -> Artifact[Any]: """Commit a next revision only when ``artifact`` remains the head.""" + return await self._revise(connection, scope_id, artifact, draft) + + async def revise_for_restoration( + self, + connection: AsyncConnection, + scope_id: str, + artifact: Artifact[Any], + draft: ArtifactDraft[Any] | RepositoryArtifactDraft, + expected: ArtifactGovernance, + /, + ) -> Artifact[Any]: + """Append to a frozen input under the restoration service's exact governance dependency. + + The caller must apply its validated final group state in the same + transaction. Already retired identities remain immutable. + """ + + if expected.artifact != artifact.as_ref(): + raise StoredPayloadConflictError("artifact-governance", (scope_id, artifact.family, artifact.artifact_id)) + return await self._revise(connection, scope_id, artifact, draft, restoration=expected) + + async def _revise( + self, + connection: AsyncConnection, + scope_id: str, + artifact: Artifact[Any], + draft: ArtifactDraft[Any] | RepositoryArtifactDraft, + *, + restoration: ArtifactGovernance | None = None, + ) -> Artifact[Any]: _require_scope(scope_id) artifact_type = self._artifact_type(artifact.family) if type(artifact) is not artifact_type or draft.family != artifact.family: @@ -170,6 +307,28 @@ async def revise( if locked.rowcount != 1: current = await self.latest(connection, scope_id, artifact.family, artifact.artifact_id) raise RevisionConflictError(artifact, current) + head = ( + ( + await connection.execute( + select(ARTIFACT_HEADS_TABLE).where( + ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, + ARTIFACT_HEADS_TABLE.c.family == artifact.family, + ARTIFACT_HEADS_TABLE.c.artifact_id == artifact.artifact_id, + ) + ) + ) + .mappings() + .one() + ) + if restoration is not None and ( + head["lifecycle_state"] != restoration.lifecycle_state.value + or head["merged_into_id"] != restoration.merged_into_id + or head["replacement_artifact_id"] != restoration.replacement_artifact_id + or head["governance_generation"] != restoration.governance_generation + ): + raise StoredPayloadConflictError("artifact-governance", (scope_id, artifact.family, artifact.artifact_id)) + if head["lifecycle_state"] == "retired" or (head["merged_into_id"] is not None and restoration is None): + raise InvalidArtifactLifecycleError("frozen and retired Artifact identities cannot be revised") # noqa: TRY003 ref = ArtifactRef( family=artifact.family, diff --git a/src/powercontext/builtin/persistence/atomic_memory.py b/src/powercontext/builtin/persistence/atomic_memory.py index 04e63f5665..d871100818 100644 --- a/src/powercontext/builtin/persistence/atomic_memory.py +++ b/src/powercontext/builtin/persistence/atomic_memory.py @@ -12,117 +12,77 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Caller-transaction Atomic Memory state persistence and common head summaries.""" +"""Atomic Memory state adapter over authoritative shared Artifact heads.""" -from sqlalchemy import insert, select, update -from sqlalchemy.ext.asyncio import AsyncConnection +from sqlalchemy import case from powercontext.builtin.artifacts.atomic_memory.errors import AtomicMemoryConflictError, AtomicMemoryRelationError from powercontext.builtin.artifacts.atomic_memory.models import AtomicMemoryState, AtomicMemoryStateValue -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_STATES_TABLE -from powercontext.builtin.persistence.errors import RepositoryNotFoundError +from powercontext.builtin.persistence.artifact_governance import ( + ArtifactGovernance, + ArtifactGovernanceRepository, + ArtifactLifecycleState, + InvalidArtifactLifecycleError, +) +from powercontext.builtin.persistence.errors import StoredPayloadConflictError from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE class AtomicMemoryStateRepository: - async def get( - self, connection: AsyncConnection, scope_id: str, artifact_id: str, /, *, for_update: bool = False - ) -> AtomicMemoryState: - statement = select(ATOMIC_MEMORY_STATES_TABLE).where( - ATOMIC_MEMORY_STATES_TABLE.c.scope_id == scope_id, - ATOMIC_MEMORY_STATES_TABLE.c.artifact_id == artifact_id, - ) - if for_update: - statement = statement.with_for_update() - row = (await connection.execute(statement)).mappings().one_or_none() - if row is None: - raise RepositoryNotFoundError("atomic-memory-state", (scope_id, artifact_id)) - return AtomicMemoryState( - state=AtomicMemoryStateValue(str(row["state"])), - state_version=int(row["state_version"]), - merged_into_id=row["merged_into_id"], - ) - - async def create(self, connection: AsyncConnection, scope_id: str, artifact_id: str, /) -> AtomicMemoryState: - state = AtomicMemoryState() - await connection.execute( - insert(ATOMIC_MEMORY_STATES_TABLE).values( - scope_id=scope_id, artifact_id=artifact_id, **state.model_dump(mode="json") - ) - ) - await self.require_summary(connection, scope_id, artifact_id, state) + """Translate Atomic vocabulary through the shared governance CAS boundary.""" + + def __init__(self) -> None: + self.governance = ArtifactGovernanceRepository() + + async def get(self, connection, scope_id, artifact_id, /, *, for_update=False) -> AtomicMemoryState: + current = await self.governance.get(connection, scope_id, "atomic-memory", artifact_id, for_update=for_update) + return _atomic_state(current) + + async def create(self, connection, scope_id, artifact_id, /) -> AtomicMemoryState: + state = await self.get(connection, scope_id, artifact_id, for_update=True) + if state != AtomicMemoryState(): + raise AtomicMemoryRelationError("a new Atomic Memory requires an unchanged active head") # noqa: TRY003 return state - async def require_summary( - self, connection: AsyncConnection, scope_id: str, artifact_id: str, state: AtomicMemoryState, / - ) -> None: - row = ( - await connection.execute( - select( - ARTIFACT_HEADS_TABLE.c.lifecycle_state, - ARTIFACT_HEADS_TABLE.c.governance_generation, - ARTIFACT_HEADS_TABLE.c.replacement_artifact_id, - ) - .where( - ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, - ARTIFACT_HEADS_TABLE.c.family == "atomic-memory", - ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, - ) - .with_for_update() - ) - ).one_or_none() - if ( - row is None - or row.lifecycle_state != _summary(state.state) - or row.governance_generation != state.state_version - or row.replacement_artifact_id is not None - ): - raise AtomicMemoryRelationError("Atomic Memory state and common governance summary disagree") # noqa: TRY003 + async def require_summary(self, connection, scope_id, artifact_id, state, /) -> None: + if await self.get(connection, scope_id, artifact_id, for_update=True) != state: + raise AtomicMemoryRelationError("Atomic Memory state changed from its expected shared head") # noqa: TRY003 async def transition( - self, - connection: AsyncConnection, - scope_id: str, - artifact_id: str, - expected: AtomicMemoryState, - state: AtomicMemoryStateValue, - merged_into_id: str | None = None, - /, + self, connection, scope_id, artifact_id, expected, state, merged_into_id=None, / ) -> AtomicMemoryState: - if expected.state is AtomicMemoryStateValue.RETIRED and state is not AtomicMemoryStateValue.RETIRED: - raise AtomicMemoryRelationError("retired memory identities cannot be restored") # noqa: TRY003 - if expected.state == state and expected.merged_into_id == merged_into_id: - return expected - requested = AtomicMemoryState( - state=state, state_version=expected.state_version + 1, merged_into_id=merged_into_id - ) await self.require_summary(connection, scope_id, artifact_id, expected) - updated = await connection.execute( - update(ATOMIC_MEMORY_STATES_TABLE) - .where( - ATOMIC_MEMORY_STATES_TABLE.c.scope_id == scope_id, - ATOMIC_MEMORY_STATES_TABLE.c.artifact_id == artifact_id, - ATOMIC_MEMORY_STATES_TABLE.c.state_version == expected.state_version, - ) - .values(**requested.model_dump(mode="json")) - ) - summarized = await connection.execute( - update(ARTIFACT_HEADS_TABLE) - .where( - ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, - ARTIFACT_HEADS_TABLE.c.family == "atomic-memory", - ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, - ARTIFACT_HEADS_TABLE.c.governance_generation == expected.state_version, + current = await self.governance.get(connection, scope_id, "atomic-memory", artifact_id, for_update=True) + try: + updated = await self.governance.transition_merge( + connection, + scope_id, + "atomic-memory", + artifact_id, + current, + ArtifactLifecycleState(_summary(state)), + merged_into_id, ) - .values( - lifecycle_state=_summary(state), - governance_generation=requested.state_version, - replacement_artifact_id=None, - ) - ) - if updated.rowcount != 1 or summarized.rowcount != 1: - raise AtomicMemoryConflictError("Atomic Memory state changed during publication") # noqa: TRY003 - return requested + except StoredPayloadConflictError as error: + raise AtomicMemoryConflictError("Atomic Memory state changed during publication") from error # noqa: TRY003 + except InvalidArtifactLifecycleError as error: + raise AtomicMemoryRelationError(str(error)) from error + return _atomic_state(updated) + + +def _atomic_state(current: ArtifactGovernance) -> AtomicMemoryState: + if current.replacement_artifact_id is not None: + raise AtomicMemoryRelationError("Atomic Memory does not use generic governance replacements") # noqa: TRY003 + state = { + ArtifactLifecycleState.ACTIVE: AtomicMemoryStateValue.ACTIVE, + ArtifactLifecycleState.DEPRECATED: AtomicMemoryStateValue.FORGOTTEN + if current.merged_into_id is None + else AtomicMemoryStateValue.MERGED, + ArtifactLifecycleState.RETIRED: AtomicMemoryStateValue.RETIRED, + }[current.lifecycle_state] + return AtomicMemoryState( + state=state, state_version=current.governance_generation, merged_into_id=current.merged_into_id + ) def _summary(state: AtomicMemoryStateValue) -> str: @@ -131,3 +91,14 @@ def _summary(state: AtomicMemoryStateValue) -> str: if state is AtomicMemoryStateValue.RETIRED: return "retired" return "deprecated" + + +def atomic_memory_state_expression(): + """Expose Atomic vocabulary for SQL selection against the single shared head.""" + + return case( + (ARTIFACT_HEADS_TABLE.c.lifecycle_state == "active", "active"), + (ARTIFACT_HEADS_TABLE.c.lifecycle_state == "retired", "retired"), + (ARTIFACT_HEADS_TABLE.c.merged_into_id.is_not(None), "merged"), + else_="forgotten", + ) diff --git a/src/powercontext/builtin/persistence/atomic_memory_index.py b/src/powercontext/builtin/persistence/atomic_memory_index.py index 857532a336..e452658620 100644 --- a/src/powercontext/builtin/persistence/atomic_memory_index.py +++ b/src/powercontext/builtin/persistence/atomic_memory_index.py @@ -519,11 +519,11 @@ async def probe_recoverable( floor: AdmissionFloor, /, ) -> bool: - """Probe at most one eligible row admitted only by the permitted lower floor. + """Probe at most one eligible row gaining a channel under the lower floor. Qualification precedes LIMIT. This is an existence signal, not an invented - pre-admission collection count. Cross-channel admission excludes a row - already admitted by either channel, even if the delivered pool was capped. + pre-admission collection count. Gaining a second channel can change fusion + order and surface a row outside the delivered pool's cap. The same current row carries its body, vectors and content filters. """ @@ -568,10 +568,10 @@ async def probe_recoverable( recoverable.append(f"({similarity} >= :probe_lower_similarity)") if not recoverable: return False + new_channels = [f"({lower} AND NOT {admitted})" for admitted, lower in zip(current, recoverable, strict=True)] statement = text( "SELECT 1 FROM pc_atomic_memory_current WHERE scope_id = :scope_id " # noqa: S608 - f"AND ({eligibility}) AND ({' OR '.join(recoverable)}) " - f"AND NOT ({' OR '.join(current)}) LIMIT 1" + f"AND ({eligibility}) AND ({' OR '.join(new_channels)}) LIMIT 1" ) if request.mode in {"vector", "hybrid"} and dialect == "mysql": statement = statement.bindparams(bindparam("probe_vector", type_=self.table.c.embedding.type)) diff --git a/src/powercontext/builtin/persistence/atomic_memory_readiness.py b/src/powercontext/builtin/persistence/atomic_memory_readiness.py index 70fcca0e5c..fdd46b5b74 100644 --- a/src/powercontext/builtin/persistence/atomic_memory_readiness.py +++ b/src/powercontext/builtin/persistence/atomic_memory_readiness.py @@ -41,6 +41,16 @@ def __init__(self, errors: tuple[str, ...]) -> None: super().__init__("Atomic Memory offline migration is not ready: " + "; ".join(errors)) +async def ensure_supported_atomic_memory_schema(connection: AsyncConnection) -> None: + """Reject unreleased split authority without changing its evidence.""" + + tables = await connection.run_sync(lambda value: inspect(value).get_table_names()) + if "pc_atomic_memory_states" in tables: + raise AtomicMemoryMigrationError(( + "unsupported unreleased schema pc_atomic_memory_states; preserve the database and use a supported backup", + )) + + async def legacy_citation_columns(connection: AsyncConnection, tables: set[str]) -> list[str]: """Return public tables that still declare the retired citation column.""" @@ -112,6 +122,7 @@ async def atomic_memory_readiness_issues( async def assert_atomic_memory_migration_ready(connection: AsyncConnection) -> None: """Reject public migration residuals without reading frozen historical records.""" + await ensure_supported_atomic_memory_schema(connection) errors = await atomic_memory_readiness_issues(connection) if errors: raise AtomicMemoryMigrationError(tuple(errors)) diff --git a/src/powercontext/builtin/persistence/atomic_memory_schema.py b/src/powercontext/builtin/persistence/atomic_memory_schema.py deleted file mode 100644 index 1eeda691b9..0000000000 --- a/src/powercontext/builtin/persistence/atomic_memory_schema.py +++ /dev/null @@ -1,42 +0,0 @@ -# Copyright (c) 2026 OceanBase. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""The authoritative Atomic Memory Family state table.""" - -from sqlalchemy import BigInteger, CheckConstraint, Column, Index, Table - -from powercontext.builtin.persistence.tables import SHARED_METADATA, identity_string -from powercontext.limits import MAX_ARTIFACT_ID_LENGTH, MAX_SCOPE_ID_LENGTH - -ATOMIC_MEMORY_STATES_TABLE = Table( - "pc_atomic_memory_states", - SHARED_METADATA, - Column("scope_id", identity_string(MAX_SCOPE_ID_LENGTH), primary_key=True), - Column("artifact_id", identity_string(MAX_ARTIFACT_ID_LENGTH), primary_key=True), - Column("state", identity_string(16), nullable=False), - Column("state_version", BigInteger, nullable=False), - Column("merged_into_id", identity_string(MAX_ARTIFACT_ID_LENGTH)), - CheckConstraint("state IN ('active', 'forgotten', 'merged', 'retired')", name="ck_pc_atomic_memory_state"), - CheckConstraint("state_version >= 0", name="ck_pc_atomic_memory_state_version"), - CheckConstraint( - "(state = 'merged' AND merged_into_id IS NOT NULL AND merged_into_id <> artifact_id) " - "OR (state <> 'merged' AND merged_into_id IS NULL)", - name="ck_pc_atomic_memory_merge_target", - ), -) -Index( - "ix_pc_atomic_memory_states_management", ATOMIC_MEMORY_STATES_TABLE.c.scope_id, ATOMIC_MEMORY_STATES_TABLE.c.state -) - -ATOMIC_MEMORY_TABLES = (ATOMIC_MEMORY_STATES_TABLE,) diff --git a/src/powercontext/builtin/persistence/experience_index.py b/src/powercontext/builtin/persistence/experience_index.py index 76e0c6da84..8abf494cff 100644 --- a/src/powercontext/builtin/persistence/experience_index.py +++ b/src/powercontext/builtin/persistence/experience_index.py @@ -77,6 +77,7 @@ ), "replacement_artifact_id": ("VARCHAR(128) NULL", "VARCHAR(128) NULL"), "governance_generation": ("BIGINT NOT NULL DEFAULT 0", "BIGINT NOT NULL DEFAULT 0"), + "merged_into_id": ("VARCHAR(128) NULL", "VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL"), } @@ -93,6 +94,8 @@ async def replace( /, ) -> None: ... + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str, /) -> None: ... + async def search( self, connection: AsyncConnection, @@ -158,6 +161,9 @@ async def search( ) -> ExperienceSearchOutcome: return ExperienceSearchOutcome() + async def remove(self, _connection: AsyncConnection, _scope_id: str, _artifact_id: str, /) -> None: + pass + async def replace_skill( self, _connection: AsyncConnection, @@ -184,7 +190,7 @@ async def search_skills( async def ensure_artifact_head_searchable_text(connection: AsyncConnection, /) -> None: - """Upgrade a pre-Experience Artifact head table with its rebuildable search projection.""" + """Upgrade existing shared heads and lineage with their current persisted columns.""" dialect = connection.dialect.name if dialect == "sqlite": @@ -210,6 +216,17 @@ async def ensure_artifact_head_searchable_text(connection: AsyncConnection, /) - if int(exists or 0) == 0: definition = definitions[0 if dialect == "sqlite" else 1] await connection.exec_driver_sql(f"ALTER TABLE pc_artifact_heads ADD COLUMN {column} {definition}") + lineage_column_sql = text( + "SELECT COUNT(*) FROM pragma_table_info('pc_artifact_lineage_artifacts') WHERE name = 'is_merge_input'" + if dialect == "sqlite" + else "SELECT COUNT(*) FROM information_schema.columns " + "WHERE table_schema = DATABASE() AND table_name = 'pc_artifact_lineage_artifacts' " + "AND column_name = 'is_merge_input'" + ) + if int(await connection.scalar(lineage_column_sql) or 0) == 0: + await connection.exec_driver_sql( + "ALTER TABLE pc_artifact_lineage_artifacts ADD COLUMN is_merge_input BOOLEAN NOT NULL DEFAULT 0" + ) async def rebuild_experience_projections(connection: AsyncConnection, /) -> None: @@ -300,6 +317,20 @@ async def replace_experience_projection( ) +async def remove_experience_projection(connection: AsyncConnection, scope_id: str, artifact_id: str, /) -> None: + """Remove current searchable content without changing exact Artifact history.""" + + await connection.execute( + update(ARTIFACT_HEADS_TABLE) + .where( + ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, + ARTIFACT_HEADS_TABLE.c.family == Experience.family, + ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, + ) + .values(searchable_text=None) + ) + + async def replace_skill_projection( connection: AsyncConnection, scope_id: str, @@ -497,6 +528,7 @@ async def _load_skill_package( "experience_search_hits", "rebuild_experience_projections", "rebuild_skill_projections", + "remove_experience_projection", "replace_experience_projection", "replace_skill_projection", "skill_search_hits", diff --git a/src/powercontext/builtin/persistence/migrations/atomic_memory_v1.py b/src/powercontext/builtin/persistence/migrations/atomic_memory_v1.py index 63bfebcdec..17ee00228d 100644 --- a/src/powercontext/builtin/persistence/migrations/atomic_memory_v1.py +++ b/src/powercontext/builtin/persistence/migrations/atomic_memory_v1.py @@ -40,7 +40,7 @@ import rfc8785 from pydantic import BaseModel, ConfigDict, Field, StrictInt, ValidationError -from sqlalchemy import BigInteger, CheckConstraint, Column, Index, MetaData, Table, text +from sqlalchemy import inspect, text from sqlalchemy.ext.asyncio import AsyncConnection from powercontext.artifacts import Artifact @@ -57,22 +57,23 @@ from powercontext.builtin.persistence.atomic_memory_readiness import ( AtomicMemoryMigrationError, assert_atomic_memory_migration_ready, + ensure_supported_atomic_memory_schema, ) from powercontext.builtin.persistence.database import AsyncDatabase +from powercontext.builtin.persistence.experience_index import ensure_artifact_head_searchable_text from powercontext.builtin.persistence.migrations.atomic_memory_archive import ( ARCHIVE_TABLE_NAME, LEGACY_CITATION_COLUMN, LegacyCollection, archive_collection, ensure_archive_table, - frozen_identity, load_collections, ) from powercontext.builtin.persistence.migrations.atomic_memory_archive import ( BINDING_COLUMNS as _BINDING_COLUMNS, ) from powercontext.builtin.persistence.migrations.atomic_memory_archive import ( - HEAD_COLUMNS as _HEAD_COLUMNS, + HEAD_COLUMNS as _LEGACY_HEAD_COLUMNS, ) from powercontext.builtin.persistence.migrations.atomic_memory_archive import ( IDEMPOTENCY_COLUMNS as _IDEMPOTENCY_COLUMNS, @@ -193,28 +194,7 @@ class _Inventory: collections: Mapping[tuple[str, str], LegacyCollection] = field(default_factory=dict) -# Only the two Family tables are new. A separate metadata and fixed types keep -# the domain conversion stable when runtime declarations evolve. -_STATE_METADATA = MetaData() - - -_STATES = Table( - "pc_atomic_memory_states", - _STATE_METADATA, - Column("scope_id", frozen_identity(256), primary_key=True), - Column("artifact_id", frozen_identity(128), primary_key=True), - Column("state", frozen_identity(16), nullable=False), - Column("state_version", BigInteger, nullable=False), - Column("merged_into_id", frozen_identity(128)), - CheckConstraint("state IN ('active', 'forgotten', 'merged', 'retired')", name="ck_pc_atomic_memory_state"), - CheckConstraint("state_version >= 0", name="ck_pc_atomic_memory_state_version"), - CheckConstraint( - "(state = 'merged' AND merged_into_id IS NOT NULL AND merged_into_id <> artifact_id) " - "OR (state <> 'merged' AND merged_into_id IS NULL)", - name="ck_pc_atomic_memory_merge_target", - ), -) -Index("ix_pc_atomic_memory_states_management", _STATES.c.scope_id, _STATES.c.state) +_HEAD_COLUMNS = (*_LEGACY_HEAD_COLUMNS, "merged_into_id") _LINEAGE_ARTIFACT_COLUMNS = ( "scope_id", @@ -225,6 +205,7 @@ class _Inventory: "upstream_family", "upstream_artifact_id", "upstream_revision", + "is_merge_input", ) _VERSION_COLUMNS = ( "scope_id", @@ -329,6 +310,19 @@ class _Inventory: } +async def _merge_schema_errors(connection: AsyncConnection, tables: set[str]) -> tuple[str, ...]: + missing = [] + for table, column in ( + ("pc_artifact_heads", "merged_into_id"), + ("pc_artifact_lineage_artifacts", "is_merge_input"), + ): + if table in tables: + columns = await connection.run_sync(lambda value, table_name=table: inspect(value).get_columns(table_name)) + if column not in {item["name"] for item in columns}: + missing.append(f"{table}.{column}") + return () if not missing else ("Artifact merge columns require explicit upgrade: " + ", ".join(missing),) + + def _json_bytes(value: Any) -> bytes: return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8") @@ -870,17 +864,23 @@ async def plan_atomic_memory_migration( ) -> AtomicMemoryMigrationReport: """Scan every legacy container, entry version and legacy reference without changing data.""" + try: + await ensure_supported_atomic_memory_schema(connection) + except AtomicMemoryMigrationError as error: + return AtomicMemoryMigrationReport(action="plan", errors=error.errors) inventory = await _inventory(connection) tables = await _tables(connection) - pending = len(inventory.entries) - if "pc_atomic_memory_states" in tables: - pending = 0 - for entry in inventory.entries: - present = await connection.scalar( - text("SELECT 1 FROM pc_atomic_memory_states WHERE scope_id = :scope AND artifact_id = :id"), - {"scope": entry.scope_id, "id": entry.artifact_id}, - ) - pending += present is None + pending = 0 + for entry in inventory.entries: + present = await connection.scalar( + text( + "SELECT 1 FROM pc_artifact_heads WHERE scope_id = :scope AND family = 'atomic-memory' " + "AND artifact_id = :id" + ), + {"scope": entry.scope_id, "id": entry.artifact_id}, + ) + pending += present is None + schema_errors = await _merge_schema_errors(connection, tables) reference_counts: dict[str, int] = {} reference_errors: tuple[str, ...] = () if not inventory.errors: @@ -888,7 +888,7 @@ async def plan_atomic_memory_migration( dict(inventory.collections), dict(decisions or {}), connection=connection ) verification = await verify_atomic_memory_migration(connection, index=index) if pending == 0 else None - errors = (inventory.errors if verification is None else verification.errors) + reference_errors + errors = (inventory.errors + schema_errors if verification is None else verification.errors) + reference_errors return AtomicMemoryMigrationReport( action="plan", ready=verification is not None and verification.ready and not reference_errors, @@ -972,6 +972,7 @@ async def _insert_lineage(connection: AsyncConnection, entry: _Entry, row: Mappi "upstream_family": ref["family"], "upstream_artifact_id": ref["artifact_id"], "upstream_revision": ref["revision"], + "is_merge_input": False, }, ) @@ -992,12 +993,14 @@ async def _imported_lineage( refs = await _rows( connection, "pc_artifact_lineage_artifacts", - ("ordinal", "upstream_family", "upstream_artifact_id", "upstream_revision"), + ("ordinal", "upstream_family", "upstream_artifact_id", "upstream_revision", "is_merge_input"), f"{where} ORDER BY ordinal", scope=entry.scope_id, id=entry.artifact_id, revision=revision, ) + if any(item["is_merge_input"] for item in refs): + raise AtomicMemoryMigrationError((f"{entry.entry_id}@{revision}: imported exact evidence differs",)) if [item["ordinal"] for item in sources] != list(range(len(sources))) or [item["ordinal"] for item in refs] != list( range(len(refs)) ): @@ -1051,9 +1054,7 @@ async def _history_issues(connection: AsyncConnection, entry: _Entry, *, repair: return errors -async def _head_and_state( - connection: AsyncConnection, entry: _Entry -) -> tuple[dict[str, Any] | None, dict[str, Any] | None]: +async def _head(connection: AsyncConnection, entry: _Entry) -> dict[str, Any] | None: heads = await _rows( connection, "pc_artifact_heads", @@ -1062,22 +1063,26 @@ async def _head_and_state( scope=entry.scope_id, id=entry.artifact_id, ) - states = await _rows( - connection, - "pc_atomic_memory_states", - ("scope_id", "artifact_id", "state", "state_version", "merged_into_id"), - "WHERE scope_id = :scope AND artifact_id = :id", - scope=entry.scope_id, - id=entry.artifact_id, + return None if not heads else heads[0] + + +def _head_state(head: Mapping[str, Any]) -> str: + lifecycle = head["lifecycle_state"] + target = head["merged_into_id"] + _require(lifecycle in {"active", "deprecated", "retired"}, "invalid Atomic Memory lifecycle") + _require( + target is None or (lifecycle == "deprecated" and target != head["artifact_id"]), + "invalid merge destination", ) - return (None if not heads else heads[0], None if not states else states[0]) + _require(head["replacement_artifact_id"] is None, "Atomic Memory has an unsupported governance replacement") + return "merged" if target is not None else "forgotten" if lifecycle == "deprecated" else lifecycle async def _import_entry(connection: AsyncConnection, entry: _Entry) -> bool: - head, state = await _head_and_state(connection, entry) + head = await _head(connection, entry) if head is not None: errors = await _history_issues(connection, entry, repair=True) - if errors or state is None or head["revision"] < entry.tail["version"]: + if errors or head["revision"] < entry.tail["version"]: raise AtomicMemoryMigrationError(tuple(errors) or (f"{entry.entry_id}: existing target is incomplete",)) # A completed import may since have been revised, merged or retagged. # Its head and mutable metadata never return to the legacy snapshot. @@ -1088,10 +1093,8 @@ async def _import_entry(connection: AsyncConnection, entry: _Entry) -> bool: ), {"scope": entry.scope_id, "id": entry.artifact_id}, ) - if existing or state is not None: - raise AtomicMemoryMigrationError(( - f"{entry.entry_id}: target has orphan content or state; no overwrite allowed", - )) + if existing: + raise AtomicMemoryMigrationError((f"{entry.entry_id}: target has orphan content; no overwrite allowed",)) for row in entry.versions: await _insert( connection, @@ -1120,17 +1123,6 @@ async def _import_entry(connection: AsyncConnection, entry: _Entry) -> bool: "lifecycle_state": summary, "replacement_artifact_id": None, "governance_generation": entry.state_version, - }, - ) - await _insert( - connection, - "pc_atomic_memory_states", - ("scope_id", "artifact_id", "state", "state_version", "merged_into_id"), - { - "scope_id": entry.scope_id, - "artifact_id": entry.artifact_id, - "state": entry.state, - "state_version": entry.state_version, "merged_into_id": None, }, ) @@ -1210,6 +1202,8 @@ async def apply_atomic_memory_migration( # noqa: C901 - One ordered offline mai if not maintenance_confirmed: raise AtomicMemoryMigrationError(("apply requires stopped old writers and --maintenance-confirmed",)) started = perf_counter() + async with database.transaction() as connection: + await ensure_supported_atomic_memory_schema(connection) async with database.transaction() as connection: inventory = await _inventory(connection) reference_errors: tuple[str, ...] = () @@ -1225,7 +1219,9 @@ async def apply_atomic_memory_migration( # noqa: C901 - One ordered offline mai processing_snapshot_hash=inventory.processing_snapshot_hash, ) async with database.transaction() as connection: - await connection.run_sync(lambda value: _STATE_METADATA.create_all(value, tables=[_STATES], checkfirst=True)) + await ensure_supported_atomic_memory_schema(connection) + if "pc_artifact_heads" in await _tables(connection): + await ensure_artifact_head_searchable_text(connection) await ensure_archive_table(connection) rebuilt_projection = await drop_obsolete_atomic_memory_projection(connection, index.table) await index.initialize(connection) @@ -1245,7 +1241,7 @@ async def apply_atomic_memory_migration( # noqa: C901 - One ordered offline mai migrated_grant_receipts = 0 for entry in archived_inventory.entries: async with database.transaction() as connection: - head, _state = await _head_and_state(connection, entry) + head = await _head(connection, entry) prepared: PreparedAtomicMemoryProjection | None = None if head is None and entry.state == "active": prepared = await publisher.prepare(_content(entry.tail)) @@ -1374,15 +1370,22 @@ async def _verify_atomic_memory_migration( # noqa: C901 - One frozen import ver collections_removed: bool = True, ) -> AtomicMemoryMigrationReport: + try: + await ensure_supported_atomic_memory_schema(connection) + except AtomicMemoryMigrationError as error: + return AtomicMemoryMigrationReport(action="verify", errors=error.errors) inventory = await _inventory(connection) errors = list(inventory.errors) tables = await _tables(connection) errors.extend(await residual_issues(connection, tables, thorough=thorough, collections_removed=collections_removed)) - required = {"pc_atomic_memory_states"} + schema_errors = await _merge_schema_errors(connection, tables) + required = {"pc_artifact_heads", "pc_artifact_lineage_artifacts"} if check_projection: required.add("pc_atomic_memory_current") - if inventory.entries and not required <= tables: - errors.append("Atomic Memory Family state/current tables are absent") + if schema_errors or (inventory.entries and not required <= tables): + errors.extend(schema_errors) + if inventory.entries and not required <= tables: + errors.append("Atomic Memory head/lineage/current tables are absent") return AtomicMemoryMigrationReport( action="verify", errors=tuple(errors), @@ -1396,26 +1399,18 @@ async def _verify_atomic_memory_migration( # noqa: C901 - One frozen import ver previous_errors = len(errors) try: errors.extend(f"{prefix}: {issue}" for issue in await _history_issues(connection, entry)) - head, state = await _head_and_state(connection, entry) - _require(head is not None and state is not None, "mapped head or Family state is missing") - if head is None or state is None: - raise ValueError("mapped head or Family state is missing") # noqa: TRY003, TRY301 + head = await _head(connection, entry) + _require(head is not None, "mapped head is missing") + if head is None: + raise ValueError("mapped head is missing") # noqa: TRY003, TRY301 + state = _head_state(head) _require(head["revision"] >= entry.tail["version"], "mapped head is behind imported history") _require( - state["state_version"] >= entry.state_version, "mapped lifecycle generation is behind legacy history" - ) - if head["revision"] == entry.tail["version"] and state["state_version"] == entry.state_version: - _require(state["state"] == entry.state, "initial mapped lifecycle differs") - summary = ( - "active" if state["state"] == "active" else "retired" if state["state"] == "retired" else "deprecated" - ) - _require( - head["lifecycle_state"] == summary - and head["governance_generation"] == state["state_version"] - and head["replacement_artifact_id"] is None, - "public governance summary disagrees with Family state", + head["governance_generation"] >= entry.state_version, + "mapped lifecycle generation is behind legacy history", ) - _require((state["state"] == "merged") == (state["merged_into_id"] is not None), "invalid merge destination") + if head["revision"] == entry.tail["version"] and head["governance_generation"] == entry.state_version: + _require(state == entry.state, "initial mapped lifecycle differs") legacy_bindings = await _rows( connection, "pc_access_relationships", @@ -1455,7 +1450,7 @@ async def _verify_atomic_memory_migration( # noqa: C901 - One frozen import ver .mappings() .one_or_none() ) - if state["state"] != "active": + if state != "active": _require(projection is None, "nonactive memory remains in current projection") else: _require(projection is not None, "active memory has no current projection") @@ -1475,7 +1470,7 @@ async def _verify_atomic_memory_migration( # noqa: C901 - One frozen import ver content = _decode(content_row["content"]) _require( projection["revision"] == head["revision"] - and projection["state_version"] == state["state_version"], + and projection["state_version"] == head["governance_generation"], "projection versions differ", ) _require( diff --git a/src/powercontext/builtin/persistence/oceanbase/experience_index.py b/src/powercontext/builtin/persistence/oceanbase/experience_index.py index 48806edc4e..467475ff5d 100644 --- a/src/powercontext/builtin/persistence/oceanbase/experience_index.py +++ b/src/powercontext/builtin/persistence/oceanbase/experience_index.py @@ -29,6 +29,7 @@ experience_search_hits, rebuild_experience_projections, rebuild_skill_projections, + remove_experience_projection, replace_experience_projection, replace_skill_projection, skill_search_hits, @@ -78,6 +79,9 @@ async def replace( ) -> None: await replace_experience_projection(connection, scope_id, experience) + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str, /) -> None: + await remove_experience_projection(connection, scope_id, artifact_id) + async def search( self, connection: AsyncConnection, diff --git a/src/powercontext/builtin/persistence/records.py b/src/powercontext/builtin/persistence/records.py index 259d5cd607..e5e547981a 100644 --- a/src/powercontext/builtin/persistence/records.py +++ b/src/powercontext/builtin/persistence/records.py @@ -551,6 +551,8 @@ async def query_artifacts( ARTIFACT_HEADS_TABLE.c.scope_id == scope_id, ARTIFACT_HEADS_TABLE.c.family == family, ARTIFACT_HEADS_TABLE.c.artifact_id > after, + ARTIFACT_HEADS_TABLE.c.merged_into_id.is_(None), + ARTIFACT_HEADS_TABLE.c.lifecycle_state != "retired", ) .order_by(ARTIFACT_HEADS_TABLE.c.artifact_id) .limit(limit + 1) diff --git a/src/powercontext/builtin/persistence/sqlite/experience_index.py b/src/powercontext/builtin/persistence/sqlite/experience_index.py index 9875069940..f720774c4a 100644 --- a/src/powercontext/builtin/persistence/sqlite/experience_index.py +++ b/src/powercontext/builtin/persistence/sqlite/experience_index.py @@ -35,6 +35,7 @@ experience_search_hits, rebuild_experience_projections, rebuild_skill_projections, + remove_experience_projection, replace_experience_projection, replace_skill_projection, skill_search_hits, @@ -170,6 +171,13 @@ async def search( rows, query, limit, scope_id, admission=admission, min_score=min_score, require_scores=require_scores ) + async def remove(self, connection: AsyncConnection, scope_id: str, artifact_id: str, /) -> None: + await remove_experience_projection(connection, scope_id, artifact_id) + await connection.execute( + _DELETE_FTS_SQL, + {"scope_id": scope_id, "family": Experience.family, "artifact_id": artifact_id}, + ) + async def replace_skill( self, connection: AsyncConnection, diff --git a/src/powercontext/builtin/persistence/statistics.py b/src/powercontext/builtin/persistence/statistics.py index 211e7cf82d..c2ccf01cd7 100644 --- a/src/powercontext/builtin/persistence/statistics.py +++ b/src/powercontext/builtin/persistence/statistics.py @@ -26,7 +26,7 @@ from sqlalchemy.ext.asyncio import AsyncConnection from powercontext.builtin.inference import InferenceUsage -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_STATES_TABLE +from powercontext.builtin.persistence.atomic_memory import atomic_memory_state_expression from powercontext.builtin.persistence.database import SELECTION_BATCH_SIZE from powercontext.builtin.persistence.errors import InvalidRepositoryArgumentError from powercontext.builtin.persistence.tables import ( @@ -157,18 +157,9 @@ async def inventory_many( # SQL so neither content nor lineage is expanded in the runtime. for scope, memory_kind, state, total in ( await connection.execute( - select( - ATOMIC_MEMORY_STATES_TABLE.c.scope_id, kind, ATOMIC_MEMORY_STATES_TABLE.c.state, func.count() - ) + select(ARTIFACT_HEADS_TABLE.c.scope_id, kind, atomic_memory_state_expression(), func.count()) .select_from( - ATOMIC_MEMORY_STATES_TABLE.join( - ARTIFACT_HEADS_TABLE, - and_( - ARTIFACT_HEADS_TABLE.c.scope_id == ATOMIC_MEMORY_STATES_TABLE.c.scope_id, - ARTIFACT_HEADS_TABLE.c.artifact_id == ATOMIC_MEMORY_STATES_TABLE.c.artifact_id, - ARTIFACT_HEADS_TABLE.c.family == "atomic-memory", - ), - ).join( + ARTIFACT_HEADS_TABLE.join( ARTIFACTS_TABLE, and_( ARTIFACTS_TABLE.c.scope_id == ARTIFACT_HEADS_TABLE.c.scope_id, @@ -178,9 +169,9 @@ async def inventory_many( ), ) ) - .where(ATOMIC_MEMORY_STATES_TABLE.c.scope_id.in_(batch)) - .group_by(ATOMIC_MEMORY_STATES_TABLE.c.scope_id, kind, ATOMIC_MEMORY_STATES_TABLE.c.state) - .order_by(ATOMIC_MEMORY_STATES_TABLE.c.scope_id, kind, ATOMIC_MEMORY_STATES_TABLE.c.state) + .where(ARTIFACT_HEADS_TABLE.c.scope_id.in_(batch), ARTIFACT_HEADS_TABLE.c.family == "atomic-memory") + .group_by(ARTIFACT_HEADS_TABLE.c.scope_id, kind, atomic_memory_state_expression()) + .order_by(ARTIFACT_HEADS_TABLE.c.scope_id, kind, atomic_memory_state_expression()) ) ).all(): memories[str(scope)].append((str(memory_kind), str(state), int(total))) diff --git a/src/powercontext/builtin/persistence/tables.py b/src/powercontext/builtin/persistence/tables.py index 7f73c08295..6725a5a34e 100644 --- a/src/powercontext/builtin/persistence/tables.py +++ b/src/powercontext/builtin/persistence/tables.py @@ -206,6 +206,7 @@ def _entry_text_type(): Column("searchable_text", _entry_text_type()), Column("lifecycle_state", identity_string(16), nullable=False, server_default="active"), Column("replacement_artifact_id", identity_string(MAX_ARTIFACT_ID_LENGTH)), + Column("merged_into_id", identity_string(MAX_ARTIFACT_ID_LENGTH)), Column("governance_generation", BigInteger, nullable=False, server_default="0"), ForeignKeyConstraint( ("scope_id", "family", "artifact_id", "revision"), @@ -230,6 +231,11 @@ def _entry_text_type(): "replacement_artifact_id IS NULL OR lifecycle_state = 'deprecated'", name="ck_pc_artifact_heads_replacement_deprecated", ), + CheckConstraint( + "merged_into_id IS NULL OR (lifecycle_state = 'deprecated' AND merged_into_id <> artifact_id " + "AND replacement_artifact_id IS NULL)", + name="ck_pc_artifact_heads_merge_target", + ), ) ARTIFACT_LINEAGE_SOURCES_TABLE = Table( @@ -270,6 +276,7 @@ def _entry_text_type(): Column("upstream_family", identity_string(MAX_ARTIFACT_FAMILY_LENGTH), nullable=False), Column("upstream_artifact_id", identity_string(MAX_ARTIFACT_ID_LENGTH), nullable=False), Column("upstream_revision", Integer, nullable=False), + Column("is_merge_input", Boolean, nullable=False, server_default="0"), ForeignKeyConstraint( ("scope_id", "family", "artifact_id", "revision"), ( diff --git a/src/powercontext/builtin/persistence/topic_memory.py b/src/powercontext/builtin/persistence/topic_memory.py index 9fb0c2cf3f..504b6a1fdb 100644 --- a/src/powercontext/builtin/persistence/topic_memory.py +++ b/src/powercontext/builtin/persistence/topic_memory.py @@ -383,6 +383,8 @@ async def browse_current( .where( TOPIC_MEMORY_ACTIVE_TOPICS_TABLE.c.scope_id == scope_id, TOPIC_MEMORY_ACTIVE_TOPICS_TABLE.c.family == TopicMemory.family, + ARTIFACT_HEADS_TABLE.c.merged_into_id.is_(None), + ARTIFACT_HEADS_TABLE.c.lifecycle_state != "retired", ) ) if tag_filter is not None: diff --git a/src/powercontext/builtin/runtime/__init__.py b/src/powercontext/builtin/runtime/__init__.py index 64019f7ed5..120c9a16e6 100644 --- a/src/powercontext/builtin/runtime/__init__.py +++ b/src/powercontext/builtin/runtime/__init__.py @@ -76,6 +76,7 @@ TopicMemoryApplication, WorkApplication, ) +from powercontext.builtin.runtime.artifact_merge import ArtifactMergeApplication, ScopedArtifactMerge from powercontext.builtin.runtime.artifact_processing import ( ArtifactProcessingBinding, ArtifactProcessingSupervisor, @@ -211,6 +212,7 @@ "ApproveArtifactCandidateRequest", "ArtifactApplication", "ArtifactInventoryStatistics", + "ArtifactMergeApplication", "ArtifactProcessingBinding", "ArtifactProcessingSupervisor", "ArtifactProcessingSupervisorStatus", @@ -344,6 +346,7 @@ "ScheduledSourceProcessor", "ScopeStatistics", "ScopedArtifactApplication", + "ScopedArtifactMerge", "ScopedAtomicMemoryApplication", "ScopedDreamApplication", "ScopedExperienceApplication", diff --git a/src/powercontext/builtin/runtime/application.py b/src/powercontext/builtin/runtime/application.py index 075b09503b..9bfca705cb 100644 --- a/src/powercontext/builtin/runtime/application.py +++ b/src/powercontext/builtin/runtime/application.py @@ -30,7 +30,7 @@ from pydantic import BaseModel, JsonValue, ValidationError from powercontext._logging import log_safely -from powercontext.artifacts import ArtifactRef +from powercontext.artifacts import ArtifactLineage, ArtifactRef from powercontext.artifacts.search import ( ArtifactSearchExecutionContext, ArtifactSearchOutcome, @@ -65,6 +65,13 @@ PrepareHandoffHint, ) from powercontext.builtin.artifacts.memory import EmbeddingProfile, MemoryQueryEmbedding +from powercontext.builtin.artifacts.merge_models import ( + ArtifactMergeMutationResult, + ArtifactMergeRead, + ArtifactMergeRecord, + ArtifactMergeRestorationPreview, + ArtifactMergeRestoreOperation, +) from powercontext.builtin.artifacts.profile.service import RelationalProfileService from powercontext.builtin.artifacts.prompt import ( GeneratePromptDemonstrations, @@ -154,6 +161,7 @@ ScopeCacheObserver, ScopeEvictor, ) +from powercontext.builtin.runtime.artifact_merge import ArtifactMergeApplication from powercontext.builtin.runtime.artifact_search import ArtifactSearchService from powercontext.builtin.runtime.atomic_memory import AtomicMemorySearchHit from powercontext.builtin.runtime.decision_model import DecisionModel @@ -2582,6 +2590,88 @@ def for_scope(self, scope_id: str, /) -> ScopedAtomicMemoryApplication: return ScopedAtomicMemoryApplication(self._runtime, self._application, scope_id) +class ScopedArtifactMergeApplication: + """Keep shared merge operations inside the Runtime's Scope lease.""" + + def __init__( + self, runtime: BuiltinRuntime, application: ArtifactMergeApplication, scope_id: str, family: str + ) -> None: + self._runtime = runtime + self._scoped = application.for_scope(scope_id, family) + self.scope_id = self._scoped.scope_id + self.family = family + + @staticmethod + def _context(context: Any) -> Any: + return _PREPARE_ATOMIC_CONTEXT.get() if context is None else context + + async def get(self, artifact_id: str, *, revision: int | None = None, context: Any = None) -> ArtifactMergeRecord: + async with self._runtime._scope_operation(self.scope_id): + return await self._scoped.get(artifact_id, revision=revision, context=self._context(context)) + + async def restoration_outcome(self, artifact_id: str, *, revision: int, context: Any = None): + async with self._runtime._scope_operation(self.scope_id): + return await self._scoped.restoration_outcome( + artifact_id, revision=revision, context=self._context(context) + ) + + async def merge( + self, + inputs: Sequence[ArtifactMergeRead], + content: BaseModel, + *, + artifact_id: str | None = None, + lineage: ArtifactLineage | None = None, + context: Any = None, + ) -> ArtifactMergeMutationResult: + async with self._runtime._scoped_operation(self.scope_id, embedding_purpose=ModelUsagePurpose.MEMORY_INDEXING): + return await self._scoped.merge( + inputs, content, artifact_id=artifact_id, lineage=lineage, context=self._context(context) + ) + + async def preview_restoration( + self, + artifact_id: str, + *, + operation: ArtifactMergeRestoreOperation = "restore", + revision: int | None = None, + context: Any = None, + ) -> ArtifactMergeRestorationPreview: + async with self._runtime._scope_operation(self.scope_id): + return await self._scoped.preview_restoration( + artifact_id, operation=operation, revision=revision, context=self._context(context) + ) + + async def restore( + self, + artifact_id: str, + *, + operation: ArtifactMergeRestoreOperation = "restore", + revision: int | None = None, + preview_token: str | None = None, + context: Any = None, + ) -> ArtifactMergeMutationResult: + async with self._runtime._scoped_operation(self.scope_id, embedding_purpose=ModelUsagePurpose.MEMORY_INDEXING): + return await self._scoped.restore( + artifact_id, + operation=operation, + revision=revision, + preview_token=preview_token, + context=self._context(context), + ) + + +class ArtifactMergeRuntimeApplication: + """Select a registered Family; authenticated hosts supply the caller context.""" + + def __init__(self, runtime: BuiltinRuntime, application: ArtifactMergeApplication) -> None: + self._runtime = runtime + self._application = application + + def for_scope(self, scope_id: str, family: str, /) -> ScopedArtifactMergeApplication: + return ScopedArtifactMergeApplication(self._runtime, self._application, scope_id, family) + + class ScopedTopicMemoryApplication: """Search, exactly read, and request processing for Topic Memory in one scope.""" @@ -2886,6 +2976,7 @@ def __init__( statistics_service: StatisticsServiceFactory | None = None, record_service: RecordService | None = None, atomic_memory_application=None, + artifact_merge_application: ArtifactMergeApplication | None = None, prompt_service: PromptService | None = None, recall_token_estimator: RecallTokenEstimator | None = None, recall_effort_sink: RecallEffortSink | None = None, @@ -2958,6 +3049,11 @@ def __init__( if atomic_memory_application is None else AtomicMemoryRuntimeApplication(self, atomic_memory_application) ) + self.artifact_merge = ( + None + if artifact_merge_application is None + else ArtifactMergeRuntimeApplication(self, artifact_merge_application) + ) self._prompt_service = prompt_service self._recall_token_estimator = recall_token_estimator self._recall_effort_sink = recall_effort_sink diff --git a/src/powercontext/builtin/runtime/artifact_merge.py b/src/powercontext/builtin/runtime/artifact_merge.py new file mode 100644 index 0000000000..22d3c6560c --- /dev/null +++ b/src/powercontext/builtin/runtime/artifact_merge.py @@ -0,0 +1,306 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Explicit scoped merge and restoration for registered Artifact Family services.""" + +from __future__ import annotations + +from collections.abc import Callable, Sequence +from contextlib import nullcontext +from datetime import UTC, datetime +from hashlib import sha256 +from typing import Any +from uuid import uuid4 + +from pydantic import BaseModel +from sqlalchemy import insert, select +from sqlalchemy.ext.asyncio import AsyncConnection + +from powercontext.artifacts import ArtifactLineage, ArtifactRef +from powercontext.artifacts.search import ArtifactSearchExecutionContext +from powercontext.builtin.artifacts.merge import ArtifactMergeService +from powercontext.builtin.artifacts.merge_models import ( + ArtifactMergeMutationResult, + ArtifactMergeRead, + ArtifactMergeRecord, + ArtifactMergeRestorationPreview, + ArtifactMergeRestoreOperation, +) +from powercontext.builtin.persistence.database import AsyncDatabase +from powercontext.builtin.persistence.tables import ARTIFACT_TAGS_TABLE +from powercontext.builtin.records import BaseOperationNotSupportedError +from powercontext.builtin.sources import validate_scope_id +from powercontext.builtin.sources.content import ArtifactRestorationOutcome +from powercontext.builtin.tags import normalize_tags + + +class ArtifactMergeAccess: + """Apply the shared access service; a missing context means access control is disabled.""" + + def __init__(self, family: str) -> None: + self.family = family + + @staticmethod + def subject(context: ArtifactSearchExecutionContext | None) -> str: + principal = None if context is None else context.principal + return "" if principal is None else f"{principal.type}:{principal.id}" + + async def authorize( + self, + connection: AsyncConnection, + scope_id: str, + context: ArtifactSearchExecutionContext | None, + action: str, + ref: ArtifactRef | None = None, + ) -> None: + from powercontext.server.authz import AccessAction, AccessDeniedError, ResourceRef + + if context is None: + return + if context.access is None: + if not context.trusted_local: + raise AccessDeniedError + return + # Prompt and Topic Memory are Scope-owned and have no Artifact owner. + if action == "read" and (ref is None or ref.family in {"prompt", "topic-memory"}): + await context.access.require_scope_read( + context.principal, scope_id, connection=connection, context=context.audit + ) + return + access = context.access.with_connection(connection) + if ref is None or action == "create": + permission = AccessAction.SCOPE_READ if action == "read" else AccessAction.SCOPE_CONTRIBUTE + resource = ResourceRef.scope(scope_id) + await access.bootstrap_static_scope(context.principal, scope_id, context=context.audit) + else: + permission = AccessAction.ARTIFACT_READ if action == "read" else AccessAction.ARTIFACT_WRITE + resource = ResourceRef.artifact(scope_id, family=ref.family, artifact_id=ref.artifact_id) + await access.require(context.principal, permission, resource, context=context.audit) + + async def authorize_sources(self, connection: AsyncConnection, scope_id: str, context, sources) -> None: + if sources: + await self.authorize(connection, scope_id, context, "read") + + async def establish_owner(self, connection: AsyncConnection, scope_id: str, artifact_id: str, context) -> None: + if context is None or context.access is None: + return + from powercontext.server.authz import ResourceRef + + await context.access.with_connection(connection).establish_artifact_owner( + ResourceRef.artifact(scope_id, family=self.family, artifact_id=artifact_id), + context.principal, + idempotency_key=f"{self.family}-owner:{scope_id}:{artifact_id}", + context=context.audit, + ) + + +class ArtifactMergeApplication: + """Register Family services without choosing inputs or generating merge content. + + Each service supplies its own content, projection and permission adapter. + Authenticated hosts pass their execution context explicitly; the default + context belongs to the composed local application. + When access control is enabled, the shared policy requires write authority + over existing inputs. Creation establishes the new result's owner. + """ + + def __init__( + self, + database: AsyncDatabase, + services: Sequence[ArtifactMergeService], + *, + default_context: Any, + id_factory: Callable[[str], str] | None = None, + restore_retry_budget: int = 3, + ) -> None: + self.database = database + self.default_context = default_context + self.id_factory = id_factory or (lambda _family: f"art_{uuid4().hex}") + self.restore_retry_budget = restore_retry_budget + self._services: dict[str, ArtifactMergeService] = {} + for service in services: + self.register(service) + + def register(self, service: ArtifactMergeService) -> None: + """Register one explicitly configured service per Family.""" + + if service.family in self._services: + raise ValueError(f"Artifact merge Family is already registered: {service.family}") # noqa: TRY003 + self._services[service.family] = service + + def for_scope(self, scope_id: str, family: str, /) -> ScopedArtifactMerge: + scope_id = validate_scope_id(scope_id) + try: + service = self._services[family] + except KeyError: + raise BaseOperationNotSupportedError( + "artifact_family", family, "registered Artifact merge adapter" + ) from None + return ScopedArtifactMerge(self, service, scope_id) + + +class ScopedArtifactMerge: + """Explicit merge, signed restoration preview and whole-group restoration.""" + + def __init__(self, application: ArtifactMergeApplication, service: ArtifactMergeService, scope_id: str) -> None: + self.application = application + self.service = service + self.scope_id = scope_id + self.family = service.family + + def _context(self, context: Any) -> Any: + return self.application.default_context if context is None else context + + async def get(self, artifact_id: str, *, revision: int | None = None, context: Any = None) -> ArtifactMergeRecord: + context = self._context(context) + access = getattr(context, "access", None) + async with ( + access.defer_decision_audit() if access is not None else nullcontext(), + self.application.database.transaction(consistent_snapshot=True) as connection, + ): + return await self.service.get(connection, self.scope_id, artifact_id, context, revision=revision) + + async def merge( + self, + inputs: Sequence[ArtifactMergeRead], + content: BaseModel, + *, + artifact_id: str | None = None, + lineage: ArtifactLineage | None = None, + context: Any = None, + ) -> ArtifactMergeMutationResult: + """Create a new identity from explicitly selected exact current reads.""" + + context = self._context(context) + async with self.application.database.transaction() as connection: + plan = await self.service.inspect_merge( + connection, + self.scope_id, + self.application.id_factory(self.family) if artifact_id is None else artifact_id, + inputs, + content, + context, + lineage=lineage, + ) + prepared = await self.service.prepare_merge(plan) + async with self.application.database.transaction() as connection: + return await self.service.commit(connection, prepared, context) + + async def restoration_outcome( + self, artifact_id: str, *, revision: int, context: Any = None + ) -> ArtifactRestorationOutcome | None: + """Read the saved exact group outcome from a restoration's primary revision.""" + + context = self._context(context) + access = getattr(context, "access", None) + async with ( + access.defer_decision_audit() if access is not None else nullcontext(), + self.application.database.transaction(consistent_snapshot=True) as connection, + ): + return await self.service.restoration_outcome(connection, self.scope_id, artifact_id, revision, context) + + async def preview_restoration( + self, + artifact_id: str, + *, + operation: ArtifactMergeRestoreOperation = "restore", + revision: int | None = None, + context: Any = None, + ) -> ArtifactMergeRestorationPreview: + context = self._context(context) + access = getattr(context, "access", None) + async with ( + access.defer_decision_audit() if access is not None else nullcontext(), + self.application.database.transaction(consistent_snapshot=True) as connection, + ): + plan = await self.service.inspect_restore( + connection, self.scope_id, artifact_id, context, operation=operation, revision=revision + ) + return self.service.restoration_preview(plan) + + async def restore( + self, + artifact_id: str, + *, + operation: ArtifactMergeRestoreOperation = "restore", + revision: int | None = None, + preview_token: str | None = None, + context: Any = None, + ) -> ArtifactMergeMutationResult: + context = self._context(context) + for attempt in range(self.application.restore_retry_budget + 1): + try: + async with self.application.database.transaction() as connection: + plan = await self.service.inspect_restore( + connection, + self.scope_id, + artifact_id, + context, + operation=operation, + revision=revision, + preview_token=preview_token, + ) + prepared = await self.service.prepare_restore(plan) + async with self.application.database.transaction() as connection: + return await self.service.commit(connection, prepared, context) + except self.service.errors.conflict as error: + if preview_token is not None: + if isinstance(error, self.service.errors.preview_stale): + raise + raise self.service.errors.preview_stale("restoration preview is stale") from error # noqa: TRY003 + if attempt == self.application.restore_retry_budget: + raise + raise AssertionError("restoration retry loop ended without a result") # noqa: TRY003 + + +async def merge_artifact_tags( + connection: AsyncConnection, + scope_id: str, + result_id: str, + input_ids: tuple[str, ...], + *, + family: str, +) -> None: + """Copy the normalized union of tags within one Scope and Family.""" + + rows = ( + await connection.execute( + select(ARTIFACT_TAGS_TABLE.c.tag_key, ARTIFACT_TAGS_TABLE.c.tag) + .where( + ARTIFACT_TAGS_TABLE.c.scope_id == scope_id, + ARTIFACT_TAGS_TABLE.c.family == family, + ARTIFACT_TAGS_TABLE.c.artifact_id.in_(input_ids), + ARTIFACT_TAGS_TABLE.c.target_type == "artifact", + ) + .order_by(ARTIFACT_TAGS_TABLE.c.tag_key, ARTIFACT_TAGS_TABLE.c.artifact_id) + .with_for_update() + ) + ).all() + labels: dict[str, str] = {} + for row in rows: + labels.setdefault(str(row.tag_key), str(row.tag)) + for key, label in normalize_tags(tuple(labels.values())).items(): + await connection.execute( + insert(ARTIFACT_TAGS_TABLE).values( + scope_id=scope_id, + family=family, + artifact_id=result_id, + target_type="artifact", + target_id=result_id, + tag_key=key, + tag_key_hash=sha256(key.encode()).digest(), + tag=label, + assigned_at=datetime.now(UTC), + ) + ) diff --git a/src/powercontext/builtin/runtime/atomic_memory.py b/src/powercontext/builtin/runtime/atomic_memory.py index aade6c6f6a..fbe12d3ecc 100644 --- a/src/powercontext/builtin/runtime/atomic_memory.py +++ b/src/powercontext/builtin/runtime/atomic_memory.py @@ -24,10 +24,9 @@ from typing import cast from uuid import uuid4 -from sqlalchemy import insert, select +from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncConnection -from powercontext.artifacts import ArtifactRef from powercontext.artifacts.search import ArtifactSearchExecutionContext from powercontext.builtin.artifacts.atomic_memory.errors import AtomicMemoryConflictError, AtomicMemoryPreviewStaleError from powercontext.builtin.artifacts.atomic_memory.models import ( @@ -56,7 +55,7 @@ embed_query, ) from powercontext.builtin.persistence.artifacts import ArtifactRepository -from powercontext.builtin.persistence.atomic_memory import AtomicMemoryStateRepository +from powercontext.builtin.persistence.atomic_memory import AtomicMemoryStateRepository, atomic_memory_state_expression from powercontext.builtin.persistence.atomic_memory_index import ( AtomicMemoryIndex, AtomicMemoryIndexError, @@ -70,70 +69,18 @@ combine_atomic_memory_channels, ) from powercontext.builtin.persistence.atomic_memory_index_schema import ATOMIC_MEMORY_PROJECTION_FORMAT -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_STATES_TABLE from powercontext.builtin.persistence.cursor_codec import SignedCursorCodec from powercontext.builtin.persistence.database import AsyncDatabase -from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, ARTIFACT_TAGS_TABLE +from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE from powercontext.builtin.persistence.tags import tag_predicate from powercontext.builtin.records import BaseOperationNotSupportedError, InvalidBaseAccessRequestError -from powercontext.builtin.tags import TagFilter, normalize_tags +from powercontext.builtin.runtime.artifact_merge import ArtifactMergeAccess, merge_artifact_tags +from powercontext.builtin.tags import TagFilter -class AtomicMemoryAccess: - """Apply the shared access service; a missing context means access control is disabled.""" - - @staticmethod - def subject(context: ArtifactSearchExecutionContext | None) -> str: - principal = None if context is None else context.principal - return "" if principal is None else f"{principal.type}:{principal.id}" - - async def authorize( - self, - connection: AsyncConnection, - scope_id: str, - context: ArtifactSearchExecutionContext | None, - action: str, - ref: ArtifactRef | None = None, - ) -> None: - from powercontext.server.authz import AccessAction, AccessDeniedError, ResourceRef - - if context is None: - return - if context.access is None: - if not context.trusted_local: - raise AccessDeniedError - return - # Prompt and Topic Memory are Scope-owned and have no Artifact owner. - if action == "read" and (ref is None or ref.family in {"prompt", "topic-memory"}): - await context.access.require_scope_read( - context.principal, scope_id, connection=connection, context=context.audit - ) - return - access = context.access.with_connection(connection) - if ref is None or action == "create": - permission = AccessAction.SCOPE_READ if action == "read" else AccessAction.SCOPE_CONTRIBUTE - resource = ResourceRef.scope(scope_id) - await access.bootstrap_static_scope(context.principal, scope_id, context=context.audit) - else: - permission = AccessAction.ARTIFACT_READ if action == "read" else AccessAction.ARTIFACT_WRITE - resource = ResourceRef.artifact(scope_id, family=ref.family, artifact_id=ref.artifact_id) - await access.require(context.principal, permission, resource, context=context.audit) - - async def authorize_sources(self, connection: AsyncConnection, scope_id: str, context, sources) -> None: - if sources: - await self.authorize(connection, scope_id, context, "read") - - async def establish_owner(self, connection: AsyncConnection, scope_id: str, artifact_id: str, context) -> None: - if context is None or context.access is None: - return - from powercontext.server.authz import ResourceRef - - await context.access.with_connection(connection).establish_artifact_owner( - ResourceRef.artifact(scope_id, family="atomic-memory", artifact_id=artifact_id), - context.principal, - idempotency_key=f"atomic-memory-owner:{scope_id}:{artifact_id}", - context=context.audit, - ) +class AtomicMemoryAccess(ArtifactMergeAccess): + def __init__(self) -> None: + super().__init__("atomic-memory") async def filters( self, connection: AsyncConnection, scope_id: str, context, *, tags: TagFilter | None = None @@ -241,40 +188,7 @@ def for_scope(self, scope_id: str) -> ScopedAtomicMemory: return ScopedAtomicMemory(self, scope_id) async def merge_tags(self, connection, scope_id: str, result_id: str, input_ids: tuple[str, ...]) -> None: - from datetime import UTC, datetime - from hashlib import sha256 - - rows = ( - await connection.execute( - select(ARTIFACT_TAGS_TABLE.c.tag_key, ARTIFACT_TAGS_TABLE.c.tag) - .where( - ARTIFACT_TAGS_TABLE.c.scope_id == scope_id, - ARTIFACT_TAGS_TABLE.c.family == "atomic-memory", - ARTIFACT_TAGS_TABLE.c.artifact_id.in_(input_ids), - ARTIFACT_TAGS_TABLE.c.target_type == "artifact", - ) - .order_by(ARTIFACT_TAGS_TABLE.c.tag_key, ARTIFACT_TAGS_TABLE.c.artifact_id) - .with_for_update() - ) - ).all() - labels: dict[str, str] = {} - for row in rows: - labels.setdefault(str(row.tag_key), str(row.tag)) - labels = normalize_tags(tuple(labels.values())) - for key, label in labels.items(): - await connection.execute( - insert(ARTIFACT_TAGS_TABLE).values( - scope_id=scope_id, - family="atomic-memory", - artifact_id=result_id, - target_type="artifact", - target_id=result_id, - tag_key=key, - tag_key_hash=sha256(key.encode()).digest(), - tag=label, - assigned_at=datetime.now(UTC), - ) - ) + await merge_artifact_tags(connection, scope_id, result_id, input_ids, family=self.security.family) class ScopedAtomicMemory: @@ -326,8 +240,7 @@ async def list( items: list[AtomicMemoryRecord] = [] last = after has_more = False - table = ATOMIC_MEMORY_STATES_TABLE - head = ARTIFACT_HEADS_TABLE + table = ARTIFACT_HEADS_TABLE async with ( deferred_decision_audit(context), self.application.database.transaction(consistent_snapshot=True) as connection, @@ -341,13 +254,12 @@ async def list( while len(items) <= limit: statement = ( select(table.c.artifact_id) - .join( - head, - (head.c.scope_id == table.c.scope_id) - & (head.c.artifact_id == table.c.artifact_id) - & (head.c.family == "atomic-memory"), + .where( + table.c.scope_id == self.scope_id, + table.c.family == "atomic-memory", + table.c.artifact_id > last, + atomic_memory_state_expression().in_(states), ) - .where(table.c.scope_id == self.scope_id, table.c.artifact_id > last, table.c.state.in_(states)) .order_by(table.c.artifact_id) .limit(100) ) @@ -514,32 +426,18 @@ async def _validate_search_hits(self, connection, hits, *, verify_vectors): ) ).mappings() } - states = { - row["artifact_id"]: row - for row in ( - await connection.execute( - select(ATOMIC_MEMORY_STATES_TABLE).where( - ATOMIC_MEMORY_STATES_TABLE.c.scope_id == self.scope_id, - ATOMIC_MEMORY_STATES_TABLE.c.artifact_id.in_(ids), - ) - ) - ).mappings() - } for hit in hits: identity = hit.artifact_ref.artifact_id - projection, head, state = projected.get(identity), heads.get(identity), states.get(identity) + projection, head = projected.get(identity), heads.get(identity) if ( projection is None or head is None - or state is None or hit.artifact_ref.family != "atomic-memory" or head["revision"] != hit.artifact_ref.revision or projection["revision"] != hit.artifact_ref.revision - or state["state"] != "active" - or state["merged_into_id"] is not None + or head["merged_into_id"] is not None or head["lifecycle_state"] != "active" or head["replacement_artifact_id"] is not None - or state["state_version"] != hit.state_version or projection["state_version"] != hit.state_version or head["governance_generation"] != hit.state_version ): diff --git a/src/powercontext/builtin/runtime/atomic_memory_rebuild.py b/src/powercontext/builtin/runtime/atomic_memory_rebuild.py index 470e98a88d..90757129b0 100644 --- a/src/powercontext/builtin/runtime/atomic_memory_rebuild.py +++ b/src/powercontext/builtin/runtime/atomic_memory_rebuild.py @@ -42,8 +42,10 @@ load_atomic_memory_tags, ) from powercontext.builtin.persistence.atomic_memory_index_schema import ATOMIC_MEMORY_PROJECTION_FORMAT -from powercontext.builtin.persistence.atomic_memory_readiness import atomic_memory_readiness_issues -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_STATES_TABLE +from powercontext.builtin.persistence.atomic_memory_readiness import ( + atomic_memory_readiness_issues, + ensure_supported_atomic_memory_schema, +) from powercontext.builtin.persistence.database import AsyncDatabase from powercontext.builtin.persistence.oceanbase.atomic_memory_index import OceanBaseAtomicMemoryIndex from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, ARTIFACTS_TABLE @@ -57,11 +59,11 @@ class AtomicMemoryProjectionRebuildReport(BaseModel): async def _require_authority(connection: AsyncConnection) -> None: + await ensure_supported_atomic_memory_schema(connection) tables = set(await connection.run_sync(lambda sync: inspect(sync).get_table_names())) required = { "pc_artifact_heads", "pc_artifacts", - "pc_atomic_memory_states", "pc_artifact_tags", "pc_access_owners", "pc_access_relationships", @@ -74,20 +76,7 @@ async def _require_authority(connection: AsyncConnection) -> None: issues = await atomic_memory_readiness_issues(connection, tables) if issues: raise AtomicMemoryIndexError("migration-pending", "Atomic Memory migration is not ready: " + "; ".join(issues)) - heads, states = ARTIFACT_HEADS_TABLE, ATOMIC_MEMORY_STATES_TABLE - relation = and_(heads.c.scope_id == states.c.scope_id, heads.c.artifact_id == states.c.artifact_id) - missing_state = await connection.scalar( - select(heads.c.artifact_id) - .where(heads.c.family == AtomicMemory.family, ~exists(select(states.c.artifact_id).where(relation))) - .limit(1) - ) - missing_head = await connection.scalar( - select(states.c.artifact_id) - .where(~exists(select(heads.c.artifact_id).where(relation, heads.c.family == AtomicMemory.family))) - .limit(1) - ) - if missing_state is not None or missing_head is not None: - raise AtomicMemoryIndexError("authority-orphan", "Atomic Memory has an orphan head or Family state") + heads = ARTIFACT_HEADS_TABLE artifacts = ARTIFACTS_TABLE identity = and_( artifacts.c.scope_id == heads.c.scope_id, @@ -139,21 +128,14 @@ async def _record( def _obsolete_current(index: AtomicMemoryIndex): current = index.table - states, heads = ATOMIC_MEMORY_STATES_TABLE, ARTIFACT_HEADS_TABLE + heads = ARTIFACT_HEADS_TABLE return ~exists( - select(states.c.artifact_id) - .join( - heads, - and_( - heads.c.scope_id == states.c.scope_id, - heads.c.artifact_id == states.c.artifact_id, - heads.c.family == AtomicMemory.family, - ), - ) - .where( - states.c.scope_id == current.c.scope_id, - states.c.artifact_id == current.c.artifact_id, - states.c.state == "active", + select(heads.c.artifact_id).where( + heads.c.scope_id == current.c.scope_id, + heads.c.artifact_id == current.c.artifact_id, + heads.c.family == AtomicMemory.family, + heads.c.lifecycle_state == "active", + heads.c.merged_into_id.is_(None), ) ) diff --git a/src/powercontext/builtin/runtime/composition.py b/src/powercontext/builtin/runtime/composition.py index d6ba6fc5fa..a5da80e56d 100644 --- a/src/powercontext/builtin/runtime/composition.py +++ b/src/powercontext/builtin/runtime/composition.py @@ -30,6 +30,7 @@ from typing_extensions import override from powercontext._logging import log_safely +from powercontext.artifacts.search import ArtifactSearchExecutionContext from powercontext.builtin.artifacts.atomic_memory.extraction import ( AtomicMemoryExtractionInput, AtomicMemoryExtractionOutput, @@ -43,6 +44,7 @@ AtomicMemoryReconciliationOutput, ) from powercontext.builtin.artifacts.experience import ExperienceCandidatePipeline, ExperienceGenerator +from powercontext.builtin.artifacts.experience.merge import ExperienceMergeAdapter from powercontext.builtin.artifacts.handoff import ( DefaultHandoffEvidenceProjector, HandoffGenerationPipeline, @@ -55,6 +57,8 @@ MemoryWriteGate, ) from powercontext.builtin.artifacts.memory.reranking import MemoryRerankText +from powercontext.builtin.artifacts.merge import ArtifactMergeService +from powercontext.builtin.artifacts.merge_restoration import ArtifactMergePreviewSigner from powercontext.builtin.artifacts.profile.generation import PROFILE_INSTRUCTIONS, LLMProfileGenerator from powercontext.builtin.artifacts.profile.service import ( ProfileGenerationInput, @@ -90,8 +94,10 @@ UsageReportingStructuredGenerator, ) from powercontext.builtin.persistence.atomic_memory_index import AtomicMemoryIndexCapabilities -from powercontext.builtin.persistence.atomic_memory_readiness import assert_atomic_memory_migration_ready -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_TABLES +from powercontext.builtin.persistence.atomic_memory_readiness import ( + assert_atomic_memory_migration_ready, + ensure_supported_atomic_memory_schema, +) from powercontext.builtin.persistence.experience_index import ensure_artifact_head_searchable_text from powercontext.builtin.persistence.oceanbase.atomic_memory_index import OceanBaseAtomicMemoryIndex from powercontext.builtin.persistence.oceanbase.experience_index import OceanBaseExperienceFTSIndex @@ -128,6 +134,11 @@ ScheduledExperienceRunner, ScheduledSourceRunner, ) +from powercontext.builtin.runtime.artifact_merge import ( + ArtifactMergeAccess, + ArtifactMergeApplication, + merge_artifact_tags, +) from powercontext.builtin.runtime.artifact_processing import ( ArtifactProcessingBinding, ArtifactProcessingSupervisor, @@ -380,6 +391,36 @@ def _configured_memory_write_gate( return gate +def _artifact_merge_application(contexts: RelationalContexts, config: RuntimeConfig) -> ArtifactMergeApplication: + signer = ( + None + if config.atomic_memory_preview_signing_secret is None + else ArtifactMergePreviewSigner( + keys={ + config.atomic_memory_preview_signing_key_id: config.atomic_memory_preview_signing_secret.get_secret_value().encode( + "utf-8" + ) + }, + active_key_id=config.atomic_memory_preview_signing_key_id, + ttl_seconds=config.atomic_memory_preview_ttl_seconds, + ) + ) + experience = ArtifactMergeService( + artifacts=contexts.repositories.artifacts, + adapter=ExperienceMergeAdapter(contexts.experience_index), + security=ArtifactMergeAccess("experience"), + merge_tags=partial(merge_artifact_tags, family="experience"), + preview_signer=signer, + ) + return ArtifactMergeApplication( + contexts.database, + (contexts.atomic_memory.service.shared, experience), + default_context=ArtifactSearchExecutionContext(trusted_local=True), + id_factory=contexts.atomic_memory.id_factory, + restore_retry_budget=config.atomic_memory_restore_retry_budget, + ) + + @asynccontextmanager async def open_builtin_runtime( config: BuiltinConfig, @@ -711,6 +752,7 @@ async def run_profile(scope_id, high): statistics_service=contexts.statistics, record_service=contexts.records, atomic_memory_application=contexts.atomic_memory, + artifact_merge_application=_artifact_merge_application(contexts, config.runtime), prompt_service=contexts.prompts, recall_token_estimator=contexts.estimate_recall_tokens, recall_effort_sink=recall_effort_sink, @@ -963,10 +1005,11 @@ async def open_builtin_contexts( atomic_index = SQLiteAtomicMemoryIndex(None if embedding_model is None else embedding_model.profile) async with SQLiteProfile.open( database, - tables=BUILTIN_TABLES + topic_index.tables + ATOMIC_MEMORY_TABLES + atomic_index.tables, + tables=BUILTIN_TABLES + topic_index.tables + atomic_index.tables, load_vector_extension=embedding_model is not None, ) as profile: async with profile.database.transaction() as connection: + await ensure_supported_atomic_memory_schema(connection) await bootstrap_processing_schema(connection, canonical_processing_manifest(config)) await assert_processing_schema_ready(connection, canonical_processing_manifest(config)) await ensure_skill_distribution_schema(connection) @@ -1034,7 +1077,7 @@ async def open_builtin_contexts( topic_indexes.append(OceanBaseTopicMemoryVectorIndex(embedding_model.profile)) topic_index = CompositeTopicMemoryIndex(*topic_indexes) atomic_index = OceanBaseAtomicMemoryIndex(None if embedding_model is None else embedding_model.profile) - tables = BUILTIN_TABLES + topic_index.tables + ATOMIC_MEMORY_TABLES + atomic_index.tables + tables = BUILTIN_TABLES + topic_index.tables + atomic_index.tables if isinstance(database, OceanBaseConfig): profile_context = OceanBaseProfile.open(database, tables=tables) elif isinstance(database, SeekDBConfig): @@ -1043,6 +1086,7 @@ async def open_builtin_contexts( raise BuiltinConfigurationError("database") async with profile_context as profile: async with profile.database.transaction() as connection: + await ensure_supported_atomic_memory_schema(connection) await bootstrap_processing_schema(connection, canonical_processing_manifest(config)) await assert_processing_schema_ready(connection, canonical_processing_manifest(config)) await ensure_skill_distribution_schema(connection) diff --git a/src/powercontext/builtin/runtime/relational.py b/src/powercontext/builtin/runtime/relational.py index 45e817043d..f5651897c1 100644 --- a/src/powercontext/builtin/runtime/relational.py +++ b/src/powercontext/builtin/runtime/relational.py @@ -1108,6 +1108,7 @@ async def list_skills( ARTIFACT_HEADS_TABLE.c.scope_id == scope, ARTIFACT_HEADS_TABLE.c.family == Skill.family, ARTIFACT_HEADS_TABLE.c.lifecycle_state.in_(states), + ARTIFACT_HEADS_TABLE.c.merged_into_id.is_(None), ) .order_by(ARTIFACT_HEADS_TABLE.c.artifact_id) .limit(limit) diff --git a/src/powercontext/builtin/sources/content.py b/src/powercontext/builtin/sources/content.py index c1ce6bc71b..b8ccc4000c 100644 --- a/src/powercontext/builtin/sources/content.py +++ b/src/powercontext/builtin/sources/content.py @@ -18,8 +18,9 @@ from typing import Annotated, Literal -from pydantic import BaseModel, Field, JsonValue, field_validator +from pydantic import BaseModel, ConfigDict, Field, JsonValue, field_validator +from powercontext.artifacts import ArtifactRef from powercontext.limits import MAX_ARTIFACT_FAMILY_LENGTH from powercontext.sources import TEXT_EVIDENCE_PROJECTION_KEY, AdapterSourceDefinition, TextEvidence from powercontext.sources.models import Source, SourceMaterialization @@ -59,12 +60,35 @@ def require_canonical_family(cls, value: str) -> str: return value +class ArtifactRestorationWrite(BaseModel): + """Exact content source and committed result of one restoration write.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + before_ref: ArtifactRef + content_from_ref: ArtifactRef + after_ref: ArtifactRef + lifecycle_state: Literal["active", "retired"] + + +class ArtifactRestorationOutcome(BaseModel): + """One successful restoration group retained with its primary revision.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + operation: Literal["restore", "undo_merge"] + target: ArtifactRef + writes: tuple[ArtifactRestorationWrite, ...] + undo_merge_results: tuple[ArtifactRef, ...] + + class ContentSourceInternal(BaseModel): """Server-owned Source purpose data never exposed by the base REST API.""" role: Literal["lineage_only"] - operation: Literal["artifact_create", "artifact_replace"] + operation: Literal["artifact_create", "artifact_replace", "artifact_restore", "artifact_undo_merge"] target: ContentSourceTarget + restoration_outcome: ArtifactRestorationOutcome | None = None class ContentSource(Source): diff --git a/tests/builtin/artifacts/atomic_memory/test_forgetting.py b/tests/builtin/artifacts/atomic_memory/test_forgetting.py index fcd8e30d63..43c31b74b6 100644 --- a/tests/builtin/artifacts/atomic_memory/test_forgetting.py +++ b/tests/builtin/artifacts/atomic_memory/test_forgetting.py @@ -24,10 +24,9 @@ from powercontext.builtin.artifacts.atomic_memory import AtomicMemoryStateValue from powercontext.builtin.artifacts.atomic_memory.errors import AtomicMemoryConflictError -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_STATES_TABLE from powercontext.builtin.persistence.oceanbase import OceanBaseConfig from powercontext.builtin.persistence.sqlite import SQLiteConfig -from powercontext.builtin.persistence.tables import ARTIFACTS_TABLE +from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, ARTIFACTS_TABLE from powercontext.builtin.records import ArtifactWrite from powercontext.builtin.runtime import BuiltinConfig, open_builtin_contexts from powercontext.builtin.runtime.config import RuntimeConfig @@ -52,7 +51,7 @@ async def atomic_row_counts(contexts, scope_id): async with contexts.database.connection() as connection: return tuple([ await connection.scalar(select(func.count()).select_from(table).where(table.c.scope_id == scope_id)) - for table in (ARTIFACTS_TABLE, ATOMIC_MEMORY_STATES_TABLE, contexts.atomic_memory.index.table) + for table in (ARTIFACTS_TABLE, ARTIFACT_HEADS_TABLE, contexts.atomic_memory.index.table) ]) @@ -105,9 +104,11 @@ async def scenario(): assert await atomic_row_counts(contexts, scope_id) == (before[0], before[1], 1) restored = await memory.restore(tagged.artifact_id) assert restored.primary.state.state is AtomicMemoryStateValue.ACTIVE - assert restored.primary.ref.revision == 1 + assert restored.primary.ref.revision == 2 + assert restored.primary.artifact.content.text == fact(0) + assert (await memory.get(tagged.artifact_id, revision=1)).artifact.content.text == fact(0) assert await contexts.records.get_tags(scope_id, target) == tags - assert await atomic_row_counts(contexts, scope_id) == (*before[:2], 2) + assert await atomic_row_counts(contexts, scope_id) == (before[0] + 1, before[1], 2) asyncio.run(scenario()) @@ -207,8 +208,11 @@ async def concurrent_tag(plan): restored = await memory.restore(created[0].artifact_id) assert restored.primary.state.state is AtomicMemoryStateValue.ACTIVE hits = (await memory.search("Capacity project", tag_filter=TagFilter(tags=("newly protected",)))).hits - assert [hit.hit.artifact_ref for hit in hits] == [current.ref] - assert await atomic_row_counts(contexts, scope_id) == before + assert restored.primary.ref.revision == current.ref.revision + 1 + assert restored.primary.artifact.content == current.artifact.content.without_creation() + assert [hit.hit.artifact_ref for hit in hits] == [restored.primary.ref] + assert (await memory.get(current.ref.artifact_id, revision=1)).artifact == current.artifact + assert await atomic_row_counts(contexts, scope_id) == (before[0] + 1, *before[1:]) assert (await contexts.records.get_tags(scope_id, target)).tags == ("newly protected",) asyncio.run(scenario()) diff --git a/tests/builtin/artifacts/test_artifact_merge.py b/tests/builtin/artifacts/test_artifact_merge.py new file mode 100644 index 0000000000..5a894fdb20 --- /dev/null +++ b/tests/builtin/artifacts/test_artifact_merge.py @@ -0,0 +1,292 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""A registered ordinary Family reuses exact merge relations and group restoration.""" + +from __future__ import annotations + +import asyncio + +import pytest + +from powercontext.artifacts import ArtifactLineage +from powercontext.builtin.persistence.sources import SourceRepository +from powercontext.builtin.sources.content import CONTENT_SOURCE_ADAPTER +from tests.builtin.persistence.contract import HandoffContent, HandoffDraft, repository_profile + + +class _LocalSecurity: + def subject(self, context): + return context + + async def lock_transaction(self, connection, scope_id, context): + pass + + async def authorize(self, connection, scope_id, context, action, ref=None): + pass + + async def authorize_sources(self, connection, scope_id, context, sources): + pass + + async def establish_owner(self, connection, scope_id, artifact_id, context): + pass + + +class _HandoffAdapter: + family = "handoff" + + def draft(self, content, lineage, *, merge_inputs=(), historical=False): + return HandoffDraft( + content=HandoffContent.model_validate(content), **lineage.model_dump(include={"sources", "artifacts"}) + ) + + async def prepare(self, content): + return content + + def validate_prepared(self, prepared): + pass + + async def publish(self, connection, scope_id, record, prepared, execution_context): + pass + + async def remove(self, connection, scope_id, artifact_id): + pass + + +async def _merge_tags(connection, scope_id, artifact_id, inputs): + pass + + +def test_shared_family_cascade_restores_selected_inputs_and_preserves_ordinary_references() -> None: + async def scenario() -> None: + from powercontext.builtin.artifacts.merge import ArtifactMergeService + from powercontext.builtin.runtime.artifact_merge import ArtifactMergeApplication + + async with repository_profile() as (profile, repositories): + service = ArtifactMergeService( + artifacts=repositories.artifacts, + sources=SourceRepository((CONTENT_SOURCE_ADAPTER,)), + adapter=_HandoffAdapter(), + security=_LocalSecurity(), + merge_tags=_merge_tags, + ) + scoped = ArtifactMergeApplication(profile.database, (service,), default_context="owner").for_scope( + "project", "handoff" + ) + async with profile.database.transaction() as connection: + for identity in ("a", "b", "d", "evidence"): + await repositories.artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + + async def merge(result, identities, evidence=()): + inputs = tuple([(await scoped.get(identity)).as_read() for identity in identities]) + return await scoped.merge( + inputs, + HandoffContent(summary=result), + artifact_id=result, + lineage=ArtifactLineage(artifacts=(*evidence, *(read.ref for read in reversed(inputs)))), + ) + + evidence = (await scoped.get("evidence")).ref + first = await merge("c", ("a", "b"), (evidence,)) + await merge("e", ("c", "d")) + frozen = await scoped.get("a", revision=1) + assert frozen.artifact.content.summary == "a" + assert frozen.state.merged_into_id == "c" + async with profile.database.transaction() as connection: + assert await repositories.artifacts.merge_inputs(connection, "project", first.primary.ref) == tuple( + reversed(tuple(record.ref for record in first.records if record.ref.artifact_id in {"a", "b"})) + ) + restored = await scoped.restore("b") + assert restored.undo_merge_results == ("e", "c") + assert {ref.artifact_id: ref.revision for ref in restored.restored} == {"a": 2, "b": 2, "d": 2} + assert {ref.artifact_id: ref.revision for ref in restored.retired} == {"c": 2, "e": 2} + for identity in ("a", "b", "d", "evidence"): + current = await scoped.get(identity) + assert current.state.lifecycle_state == "active" + assert current.state.merged_into_id is None + if identity != "evidence": + assert current.artifact.lineage.artifacts == ((await scoped.get(identity, revision=1)).ref,) + for identity in ("c", "e"): + current = await scoped.get(identity) + assert current.state.lifecycle_state == "retired" + historical = await scoped.get(identity, revision=1) + assert current.artifact.content == historical.artifact.content + assert current.artifact.lineage.artifacts == (historical.ref,) + async with profile.database.transaction() as connection: + assert await repositories.artifacts.merge_inputs(connection, "project", current.ref) == () + async with profile.database.transaction() as connection: + assert await repositories.artifacts.merge_inputs(connection, "project", first.primary.ref) == tuple( + reversed(tuple(record.ref for record in first.records if record.ref.artifact_id in {"a", "b"})) + ) + assert (await scoped.get("evidence")).state.governance_generation == 0 + + asyncio.run(scenario()) + + +def test_shared_active_restore_is_noop_unless_a_revision_is_selected() -> None: + async def scenario() -> None: + from powercontext.builtin.artifacts.merge import ArtifactMergeService + from powercontext.builtin.artifacts.merge_restoration import ArtifactMergePreviewSigner + from powercontext.builtin.runtime.artifact_merge import ArtifactMergeApplication + + async with repository_profile() as (profile, repositories): + service = ArtifactMergeService( + artifacts=repositories.artifacts, + sources=SourceRepository((CONTENT_SOURCE_ADAPTER,)), + adapter=_HandoffAdapter(), + security=_LocalSecurity(), + merge_tags=_merge_tags, + preview_signer=ArtifactMergePreviewSigner(keys={"test": b"x" * 32}, active_key_id="test"), + ) + scoped = ArtifactMergeApplication(profile.database, (service,), default_context="owner").for_scope( + "project", "handoff" + ) + async with profile.database.transaction() as connection: + original = await repositories.artifacts.create( + connection, "project", "original", HandoffDraft(content=HandoffContent(summary="original")) + ) + preview = await scoped.preview_restoration("original") + assert len(preview.restore) == 1 and not preview.restore[0].creates_revision + unchanged = await scoped.restore("original", preview_token=preview.preview_token) + assert not unchanged.changed + assert unchanged.primary.artifact == original + selected = await scoped.preview_restoration("original", revision=1) + assert selected.restore[0].creates_revision + restored = await scoped.restore("original", revision=1, preview_token=selected.preview_token) + assert restored.changed and restored.primary.ref.revision == 2 + assert restored.primary.artifact.content == original.content + assert restored.primary.artifact.lineage.artifacts == (original.as_ref(),) + earlier = await scoped.restore("original", revision=1) + assert earlier.primary.ref.revision == 3 + assert earlier.primary.artifact.lineage.artifacts == (restored.primary.ref, original.as_ref()) + assert (await scoped.get("original", revision=1)).artifact == original + selected_receipt = await scoped.restore("original", revision=2) + assert selected_receipt.primary.ref.revision == 4 + assert selected_receipt.primary.artifact.lineage.artifacts == (earlier.primary.ref, restored.primary.ref) + assert len(selected_receipt.primary.artifact.lineage.sources) == 1 + assert selected_receipt.primary.artifact.lineage.sources != restored.primary.artifact.lineage.sources + outcome = await scoped.restoration_outcome("original", revision=4) + assert outcome is not None and outcome.writes[0].content_from_ref == restored.primary.ref + + asyncio.run(scenario()) + + +def test_shared_restoration_publication_failure_rolls_back_all_appended_revisions() -> None: + async def scenario() -> None: + from powercontext.builtin.artifacts.merge import ArtifactMergeService + from powercontext.builtin.persistence.errors import RepositoryNotFoundError + from powercontext.builtin.runtime.artifact_merge import ArtifactMergeApplication + + class RejectPublication(_HandoffAdapter): + reject = False + + async def publish(self, connection, scope_id, record, prepared, execution_context): + if self.reject and record.ref.artifact_id == "b": + raise ValueError("publication unavailable") # noqa: TRY003 + + async with repository_profile() as (profile, repositories): + adapter = RejectPublication() + service = ArtifactMergeService( + artifacts=repositories.artifacts, + sources=SourceRepository((CONTENT_SOURCE_ADAPTER,)), + adapter=adapter, + security=_LocalSecurity(), + merge_tags=_merge_tags, + ) + scoped = ArtifactMergeApplication(profile.database, (service,), default_context="owner").for_scope( + "project", "handoff" + ) + async with profile.database.transaction() as connection: + for identity in ("a", "b"): + await repositories.artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + inputs = tuple([(await scoped.get(identity)).as_read() for identity in ("a", "b")]) + await scoped.merge(inputs, HandoffContent(summary="combined"), artifact_id="c") + before = {identity: await scoped.get(identity) for identity in ("a", "b", "c")} + adapter.reject = True + with pytest.raises(ValueError, match="publication unavailable"): + await scoped.restore("a") + for identity, current in before.items(): + assert await scoped.get(identity) == current + assert (await scoped.get(identity, revision=1)).artifact == current.artifact + with pytest.raises(RepositoryNotFoundError): + await scoped.get(identity, revision=2) + async with profile.database.transaction() as connection: + assert service.sources is not None + assert await service.sources.list(connection, "project") == () + + asyncio.run(scenario()) + + +def test_shared_ordinary_deprecated_revision_preserves_replacement_until_explicit_restore() -> None: + async def scenario() -> None: + from powercontext.builtin.artifacts.merge import ArtifactMergeService + from powercontext.builtin.persistence.artifact_governance import ( + ArtifactGovernanceRepository, + ArtifactLifecycleState, + ) + + async with repository_profile() as (profile, repositories): + service = ArtifactMergeService( + artifacts=repositories.artifacts, + sources=SourceRepository((CONTENT_SOURCE_ADAPTER,)), + adapter=_HandoffAdapter(), + security=_LocalSecurity(), + merge_tags=_merge_tags, + ) + governance = ArtifactGovernanceRepository() + async with profile.database.transaction() as connection: + for identity in ("original", "replacement"): + await repositories.artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + await governance.transition( + connection, "project", "handoff", "original", 0, ArtifactLifecycleState.DEPRECATED, "replacement" + ) + read = await service.get(connection, "project", "original", "owner") + assert read.state.replacement_artifact_id == "replacement" and read.state.merged_into_id is None + plan = await service.inspect_change( + connection, + "project", + "original", + HandoffContent(summary="revised"), + "owner", + expected_revision=1, + expected_state_version=1, + ) + prepared = await service.prepare_change(plan) + async with profile.database.transaction() as connection: + result = await service.commit(connection, prepared, "owner") + assert result.primary.ref.revision == 2 + assert result.primary.state.lifecycle_state == "deprecated" + assert result.primary.state.replacement_artifact_id == "replacement" + assert result.primary.state.governance_generation == 1 + plan = await service.inspect_restore(connection, "project", "original", "owner") + prepared = await service.prepare_restore(plan) + async with profile.database.transaction() as connection: + restored = await service.commit(connection, prepared, "owner") + assert restored.primary.state.lifecycle_state == "active" + assert restored.primary.state.replacement_artifact_id is None + assert restored.primary.state.governance_generation == 2 + assert restored.primary.ref.revision == 3 + assert restored.primary.artifact.content == result.primary.artifact.content + assert restored.primary.artifact.lineage.artifacts == (result.primary.ref,) + assert ( + await repositories.artifacts.get(connection, "project", result.primary.ref) + ) == result.primary.artifact + + asyncio.run(scenario()) diff --git a/tests/builtin/persistence/test_artifact_current_eligibility.py b/tests/builtin/persistence/test_artifact_current_eligibility.py new file mode 100644 index 0000000000..b58ccbcc0e --- /dev/null +++ b/tests/builtin/persistence/test_artifact_current_eligibility.py @@ -0,0 +1,225 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import annotations + +import asyncio +from datetime import UTC, datetime + +from sqlalchemy import update +from sqlalchemy.ext.asyncio import AsyncConnection + +from powercontext.builtin.artifacts.skill import Skill, SkillContent, SkillDraft +from powercontext.builtin.artifacts.topic_memory import ( + TopicMemory, + TopicMemoryContent, + TopicMemoryDraft, + prepare_topic_memory_projection, +) +from powercontext.builtin.persistence.artifact_readers import TopicMemoryArtifactListReader +from powercontext.builtin.persistence.artifacts import ArtifactRepository +from powercontext.builtin.persistence.family_management import FamilyManagementWriterRegistry +from powercontext.builtin.persistence.records import RelationalRecordService +from powercontext.builtin.persistence.sources import SourceRepository +from powercontext.builtin.persistence.sqlite import SQLiteConfig, SQLiteProfile +from powercontext.builtin.persistence.sqlite.topic_memory_index import SQLiteTopicMemoryFTSIndex +from powercontext.builtin.persistence.tables import ( + ARTIFACT_HEADS_TABLE, + BUILTIN_TABLES, + TOPIC_MEMORY_REVISION_PUBLICATIONS_TABLE, +) +from powercontext.builtin.persistence.topic_memory import TopicMemoryRepository +from powercontext.builtin.persistence.topic_memory_index import CompositeTopicMemoryIndex +from powercontext.builtin.runtime.relational import RelationalContexts +from powercontext.builtin.sources import CONTENT_SOURCE_ADAPTER + +_ARTIFACT_IDS = ( + "a-frozen", + "b-retired", + "c-deprecated", + "d-active", + "e-frozen", + "f-retired", + "g-active", + "h-frozen", +) + + +def _skill_draft(label: str) -> SkillDraft: + return SkillDraft( + content=SkillContent( + name=label, + description=f"Instructions for {label}.", + instructions="Read the current evidence before acting.", + validation=("Check the reported result.",), + ) + ) + + +async def _set_lifecycle_fixture(connection: AsyncConnection, family: str) -> None: + for artifact_id in _ARTIFACT_IDS: + if artifact_id.endswith("frozen"): + values = {"lifecycle_state": "deprecated", "merged_into_id": "d-active"} + elif artifact_id.endswith("retired"): + values = {"lifecycle_state": "retired"} + elif artifact_id.endswith("deprecated"): + values = {"lifecycle_state": "deprecated", "replacement_artifact_id": "d-active"} + else: + continue + await connection.execute( + update(ARTIFACT_HEADS_TABLE) + .where( + ARTIFACT_HEADS_TABLE.c.scope_id == "scope", + ARTIFACT_HEADS_TABLE.c.family == family, + ARTIFACT_HEADS_TABLE.c.artifact_id == artifact_id, + ) + .values(**values) + ) + + +def _records(profile: SQLiteProfile, artifacts: ArtifactRepository, sources: SourceRepository, **kwargs): + return RelationalRecordService( + profile.database, + sources, + artifacts, + FamilyManagementWriterRegistry(()), + cursor_secret=b"current-eligibility-test-secret", + **kwargs, + ) + + +def test_common_current_list_filters_ineligible_heads_before_pagination() -> None: + async def scenario() -> None: + sources = SourceRepository((CONTENT_SOURCE_ADAPTER,)) + artifacts = ArtifactRepository((Skill,), sources=sources) + async with SQLiteProfile.open(SQLiteConfig(), tables=BUILTIN_TABLES) as profile: + records = _records(profile, artifacts, sources) + async with profile.database.transaction() as connection: + for artifact_id in _ARTIFACT_IDS: + await artifacts.create(connection, "scope", artifact_id, _skill_draft(artifact_id)) + await _set_lifecycle_fixture(connection, Skill.family) + + first = await records.query_artifacts("scope", Skill.family, limit=2, cursor=None) + assert [item.artifact_id for item in first.items] == ["c-deprecated", "d-active"] + assert first.next_cursor is not None + second = await records.query_artifacts("scope", Skill.family, limit=2, cursor=first.next_cursor) + assert [item.artifact_id for item in second.items] == ["g-active"] + assert second.next_cursor is None + + asyncio.run(scenario()) + + +def test_frozen_and_retired_identities_retain_latest_history_and_permission_catalog_reads() -> None: + async def scenario() -> None: + sources = SourceRepository((CONTENT_SOURCE_ADAPTER,)) + artifacts = ArtifactRepository((Skill,), sources=sources) + async with SQLiteProfile.open(SQLiteConfig(), tables=BUILTIN_TABLES) as profile: + records = _records(profile, artifacts, sources) + async with profile.database.transaction() as connection: + for artifact_id in _ARTIFACT_IDS: + original = await artifacts.create(connection, "scope", artifact_id, _skill_draft(artifact_id)) + if artifact_id in {"a-frozen", "b-retired"}: + await artifacts.revise(connection, "scope", original, _skill_draft(f"updated-{artifact_id}")) + await _set_lifecycle_fixture(connection, Skill.family) + + for artifact_id in ("a-frozen", "b-retired"): + latest = await records.get_artifact("scope", Skill.family, artifact_id) + exact = await records.get_artifact_revision("scope", Skill.family, artifact_id, 1) + history = await records.list_artifact_revisions( + "scope", Skill.family, artifact_id, limit=1, cursor=None + ) + assert latest.revision == 2 + assert exact.revision == 1 + assert [item.revision for item in history.items] == [2] + assert history.next_cursor is not None + older = await records.list_artifact_revisions( + "scope", Skill.family, artifact_id, limit=1, cursor=history.next_cursor + ) + assert [item.revision for item in older.items] == [1] + assert older.next_cursor is None + + catalog = await records.logical_artifacts("scope") + assert {item.artifact_id for item in catalog} == set(_ARTIFACT_IDS) + + asyncio.run(scenario()) + + +def test_topic_current_pages_and_has_more_filter_ineligible_heads() -> None: + async def scenario() -> None: + sources = SourceRepository((CONTENT_SOURCE_ADAPTER,)) + artifacts = ArtifactRepository((TopicMemory,), sources=sources) + index = CompositeTopicMemoryIndex(SQLiteTopicMemoryFTSIndex()) + repository = TopicMemoryRepository(artifacts=artifacts, index=index) + async with SQLiteProfile.open(SQLiteConfig(), tables=BUILTIN_TABLES + index.tables) as profile: + records = _records( + profile, + artifacts, + sources, + topic_memory_list_reader=TopicMemoryArtifactListReader( + database=profile.database, artifacts=artifacts, topics=repository + ), + ) + async with profile.database.transaction() as connection: + await repository.initialize(connection) + for artifact_id in _ARTIFACT_IDS: + content = TopicMemoryContent( + title=artifact_id, + summary=f"Current evidence for {artifact_id}.", + detail="The exact evidence remains available for historical review.", + ) + await repository.publish_create( + connection, + "scope", + artifact_id, + TopicMemoryDraft(content=content), + prepare_topic_memory_projection(content), + ) + await connection.execute( + update(TOPIC_MEMORY_REVISION_PUBLICATIONS_TABLE).values( + published_at=datetime(2026, 9, 5, 3, 4, 5, tzinfo=UTC) + ) + ) + await _set_lifecycle_fixture(connection, TopicMemory.family) + + first = await records.query_artifacts("scope", TopicMemory.family, limit=2, cursor=None) + assert [item.artifact_id for item in first.items] == ["c-deprecated", "d-active"] + assert first.next_cursor is not None + second = await records.query_artifacts("scope", TopicMemory.family, limit=1, cursor=first.next_cursor) + assert [item.artifact_id for item in second.items] == ["g-active"] + assert second.next_cursor is None + for artifact_id in ("a-frozen", "b-retired"): + assert (await records.get_artifact("scope", TopicMemory.family, artifact_id)).revision == 1 + assert (await records.get_artifact_revision("scope", TopicMemory.family, artifact_id, 1)).revision == 1 + + asyncio.run(scenario()) + + +def test_skill_library_includes_ordinary_deprecated_heads_without_frozen_inputs() -> None: + async def scenario() -> None: + async with SQLiteProfile.open(SQLiteConfig(), tables=BUILTIN_TABLES) as profile: + contexts = RelationalContexts(database=profile.database) + async with profile.database.transaction() as connection: + for artifact_id in _ARTIFACT_IDS: + await contexts.repositories.artifacts.create( + connection, "scope", artifact_id, _skill_draft(artifact_id) + ) + await _set_lifecycle_fixture(connection, Skill.family) + + included = await contexts.list_skills("scope", True, 2) + assert [skill.artifact_id for skill, _ in included] == ["c-deprecated", "d-active"] + assert included[0][1].replacement_artifact_id == "d-active" + active = await contexts.list_skills("scope", False, 2) + assert [skill.artifact_id for skill, _ in active] == ["d-active", "g-active"] + + asyncio.run(scenario()) diff --git a/tests/builtin/persistence/test_artifact_merge_storage.py b/tests/builtin/persistence/test_artifact_merge_storage.py new file mode 100644 index 0000000000..cc6874d78b --- /dev/null +++ b/tests/builtin/persistence/test_artifact_merge_storage.py @@ -0,0 +1,282 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Shared heads and exact lineage protect successful merge business state.""" + +from __future__ import annotations + +import asyncio + +import pytest +from sqlalchemy import select, update + +from powercontext.artifacts import ArtifactRef +from powercontext.builtin.artifacts.atomic_memory import AtomicMemory, AtomicMemoryContent, AtomicMemoryDraft +from powercontext.builtin.artifacts.atomic_memory.models import AtomicMemoryStateValue +from powercontext.builtin.persistence.artifact_governance import ( + ArtifactGovernanceRepository, + ArtifactLifecycleState, + InvalidArtifactLifecycleError, +) +from powercontext.builtin.persistence.artifacts import ArtifactRepository +from powercontext.builtin.persistence.atomic_memory import AtomicMemoryStateRepository +from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, ARTIFACT_LINEAGE_ARTIFACTS_TABLE +from powercontext.builtin.runtime import BuiltinConfig, open_builtin_contexts +from tests.builtin.persistence.contract import HandoffContent, HandoffDraft, repository_profile + + +def test_merge_creation_marks_only_selected_exact_inputs_in_lineage_order() -> None: + async def scenario() -> None: + async with repository_profile() as (profile, repositories), profile.database.transaction() as connection: + artifacts = repositories.artifacts + first, second, evidence = tuple([ + await artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + for identity in ("first", "second", "evidence") + ]) + result = await artifacts.create_merge_result( + connection, + "project", + "result", + HandoffDraft( + content=HandoffContent(summary="merged"), + artifacts=(evidence.as_ref(), second.as_ref(), first.as_ref()), + ), + ((first.as_ref(), 0), (second.as_ref(), 0)), + ) + assert await artifacts.merge_inputs(connection, "project", result.as_ref()) == ( + second.as_ref(), + first.as_ref(), + ) + rows = ( + ( + await connection.execute( + select(ARTIFACT_LINEAGE_ARTIFACTS_TABLE) + .where(ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.artifact_id == "result") + .order_by(ARTIFACT_LINEAGE_ARTIFACTS_TABLE.c.ordinal) + ) + ) + .mappings() + .all() + ) + assert [row["is_merge_input"] for row in rows] == [False, True, True] + ordinary = await artifacts.create( + connection, + "project", + "ordinary", + HandoffDraft(content=result.content, artifacts=result.lineage.artifacts), + ) + assert await artifacts.merge_inputs(connection, "project", ordinary.as_ref()) == () + + asyncio.run(scenario()) + + +@pytest.mark.parametrize("lifecycle,merged_into", (("deprecated", "result"), ("retired", None))) +def test_ordinary_revision_rejects_frozen_or_retired_identity(lifecycle: str, merged_into: str | None) -> None: + async def scenario() -> None: + async with repository_profile() as (profile, repositories), profile.database.transaction() as connection: + original = await repositories.artifacts.create( + connection, "project", "input", HandoffDraft(content=HandoffContent(summary="original")) + ) + await connection.execute( + update(ARTIFACT_HEADS_TABLE) + .where(ARTIFACT_HEADS_TABLE.c.artifact_id == "input") + .values( + lifecycle_state=lifecycle, + merged_into_id=merged_into, + ) + ) + with pytest.raises(InvalidArtifactLifecycleError): + await repositories.artifacts.revise( + connection, "project", original, HandoffDraft(content=HandoffContent(summary="edited")) + ) + if lifecycle == "retired": + expected = await ArtifactGovernanceRepository().get(connection, "project", "handoff", "input") + with pytest.raises(InvalidArtifactLifecycleError): + await repositories.artifacts.revise_for_restoration( + connection, "project", original, HandoffDraft(content=original.content), expected + ) + assert await repositories.artifacts.get(connection, "project", original.as_ref()) == original + + asyncio.run(scenario()) + + +def test_ordinary_governance_cannot_unfreeze_inputs_or_retire_effective_result() -> None: + async def scenario() -> None: + async with repository_profile() as (profile, repositories), profile.database.transaction() as connection: + for identity in ("input", "result"): + await repositories.artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + await connection.execute( + update(ARTIFACT_HEADS_TABLE) + .where(ARTIFACT_HEADS_TABLE.c.artifact_id == "input") + .values( + lifecycle_state="deprecated", + merged_into_id="result", + governance_generation=1, + ) + ) + governance = ArtifactGovernanceRepository() + with pytest.raises(InvalidArtifactLifecycleError): + await governance.transition( + connection, "project", "handoff", "input", 1, ArtifactLifecycleState.ACTIVE, None + ) + with pytest.raises(InvalidArtifactLifecycleError): + await governance.transition( + connection, "project", "handoff", "result", 0, ArtifactLifecycleState.RETIRED, None + ) + + asyncio.run(scenario()) + + +@pytest.mark.parametrize("invalid", ("duplicate", "stale", "inactive", "different-family", "missing-lineage")) +def test_merge_creation_requires_distinct_current_active_same_family_inputs(invalid: str) -> None: + async def scenario() -> None: + async with repository_profile() as (profile, repositories), profile.database.transaction() as connection: + first, second = tuple([ + await repositories.artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + for identity in ("first", "second") + ]) + inputs = ((first.as_ref(), 0), (second.as_ref(), 0)) + lineage = (first.as_ref(), second.as_ref()) + if invalid == "duplicate": + inputs = ((first.as_ref(), 0), (first.as_ref(), 0)) + elif invalid == "stale": + inputs = ((first.as_ref(), 1), (second.as_ref(), 0)) + elif invalid == "inactive": + await connection.execute( + update(ARTIFACT_HEADS_TABLE) + .where(ARTIFACT_HEADS_TABLE.c.artifact_id == "first") + .values(lifecycle_state="deprecated") + ) + elif invalid == "different-family": + inputs = ((ArtifactRef(family="report", artifact_id="first", revision=1), 0), (second.as_ref(), 0)) + elif invalid == "missing-lineage": + lineage = (first.as_ref(),) + with pytest.raises(InvalidArtifactLifecycleError): + await repositories.artifacts.create_merge_result( + connection, + "project", + "result", + HandoffDraft(content=HandoffContent(summary="merged"), artifacts=lineage), + inputs, + ) + assert await repositories.artifacts.revisions(connection, "project", "handoff", "result") == () + + asyncio.run(scenario()) + + +def test_atomic_state_adapter_uses_heads_without_separate_state_authority() -> None: + async def scenario() -> None: + async with repository_profile() as (profile, _), profile.database.transaction() as connection: + artifacts = ArtifactRepository((AtomicMemory,)) + states = AtomicMemoryStateRepository() + original = await artifacts.create( + connection, + "project", + "memory", + AtomicMemoryDraft(content=AtomicMemoryContent(kind="decision", text="Keep the exact evidence.")), + ) + state = await states.create(connection, "project", "memory") + assert state == await states.get(connection, "project", "memory") + assert ( + await states.transition(connection, "project", "memory", state, AtomicMemoryStateValue.ACTIVE) == state + ) + forgotten = await states.transition( + connection, "project", "memory", state, AtomicMemoryStateValue.FORGOTTEN + ) + assert forgotten.state_version == 1 + revised = await artifacts.revise( + connection, + "project", + original, + AtomicMemoryDraft( + content=AtomicMemoryContent(kind="decision", text="Preserve exact historical evidence.") + ), + ) + assert revised.revision == 2 + assert await states.get(connection, "project", "memory") == forgotten + active = await states.transition(connection, "project", "memory", forgotten, AtomicMemoryStateValue.ACTIVE) + assert active.state_version == 2 + head = ( + ( + await connection.execute( + select(ARTIFACT_HEADS_TABLE).where(ARTIFACT_HEADS_TABLE.c.artifact_id == "memory") + ) + ) + .mappings() + .one() + ) + assert head["lifecycle_state"] == "active" and head["merged_into_id"] is None + assert head["governance_generation"] == 2 + + asyncio.run(scenario()) + + +@pytest.mark.parametrize("historical", (False, True)) +def test_atomic_restoration_uses_marked_refs_without_creation_payload(historical: bool) -> None: + async def scenario() -> None: + async with open_builtin_contexts(BuiltinConfig()) as contexts: + application = contexts.atomic_memory + memory = application.for_scope("project") + await contexts.records.create_atomic_memories( + "project", + ( + {"kind": "decision", "text": "Preserve the first exact rule."}, + {"kind": "decision", "text": "Preserve the second exact rule."}, + ), + ) + originals = (await memory.list()).items + async with contexts.database.transaction() as connection: + result = await application.artifacts.create_merge_result( + connection, + "project", + "shared-result", + AtomicMemoryDraft( + content=AtomicMemoryContent(kind="decision", text="Preserve both rules."), + artifacts=tuple(record.ref for record in originals), + ), + tuple((record.ref, record.state.state_version) for record in reversed(originals)), + ) + await application.security.establish_owner(connection, "project", result.artifact_id, None) + for record in originals: + await application.service.states.transition( + connection, + "project", + record.ref.artifact_id, + record.state, + AtomicMemoryStateValue.MERGED, + result.artifact_id, + ) + await application.publisher.remove(connection, "project", record.ref.artifact_id) + assert result.content.creation is None + if historical: + restored = await memory.restore(originals[0].ref.artifact_id, revision=1) + assert restored.primary.ref.revision == 2 + assert restored.primary.artifact.content == originals[0].artifact.content.without_creation() + else: + restored = await memory.restore(result.artifact_id, operation="undo_merge") + assert restored.primary.state.state is AtomicMemoryStateValue.RETIRED + for original in originals: + current = await memory.get(original.ref.artifact_id) + assert current.state.state is AtomicMemoryStateValue.ACTIVE + assert current.state.state_version == 2 + assert (await memory.get(original.ref.artifact_id, revision=1)).artifact == original.artifact + assert len((await memory.list()).items) == 2 + + asyncio.run(scenario()) diff --git a/tests/builtin/persistence/test_atomic_memory_startup_legacy_schema.py b/tests/builtin/persistence/test_atomic_memory_startup_legacy_schema.py index a6ad7b8511..1a511f6378 100644 --- a/tests/builtin/persistence/test_atomic_memory_startup_legacy_schema.py +++ b/tests/builtin/persistence/test_atomic_memory_startup_legacy_schema.py @@ -130,7 +130,53 @@ async def scenario() -> None: connection.execute("SELECT COUNT(*) FROM pc_artifacts WHERE family = 'atomic-memory'").fetchone()[0] == 0 ) - assert connection.execute("SELECT COUNT(*) FROM pc_atomic_memory_states").fetchone()[0] == 0 + assert ( + connection.execute("SELECT name FROM sqlite_master WHERE name = 'pc_atomic_memory_states'").fetchone() + is None + ) assert connection.execute("SELECT COUNT(*) FROM pc_atomic_memory_current").fetchone()[0] == 0 asyncio.run(scenario()) + + +@pytest.mark.parametrize("entrypoint", ["startup", "apply", "authority"]) +def test_unreleased_atomic_state_table_is_rejected_without_data_loss(tmp_path: Path, entrypoint: str) -> None: + database = tmp_path / "unreleased.db" + with sqlite3.connect(database) as connection: + connection.execute("CREATE TABLE pc_atomic_memory_states(artifact_id TEXT, state TEXT)") + connection.execute("INSERT INTO pc_atomic_memory_states VALUES ('retained', 'merged')") + config = SQLiteConfig(url=f"sqlite+aiosqlite:///{database}") + + async def scenario() -> None: + with pytest.raises(AtomicMemoryMigrationError, match=r"unsupported unreleased.*pc_atomic_memory_states"): + if entrypoint == "startup": + async with open_builtin_contexts(BuiltinConfig(database=config)): + pytest.fail("Unreleased Atomic Memory authority must be rejected") + else: + from powercontext.builtin.persistence.migrations.atomic_memory_v1 import ( + apply_atomic_memory_migration, + assert_atomic_memory_migration_ready, + verify_atomic_memory_migration_authority, + ) + from powercontext.builtin.persistence.sqlite.atomic_memory_index import SQLiteAtomicMemoryIndex + + async with SQLiteProfile.open(config, tables=()) as profile: + if entrypoint == "apply": + await apply_atomic_memory_migration( + profile.database, SQLiteAtomicMemoryIndex(), maintenance_confirmed=True + ) + else: + async with profile.database.transaction() as connection: + report = await verify_atomic_memory_migration_authority(connection) + assert not report.ready + assert any("unsupported unreleased" in error for error in report.errors) + await assert_atomic_memory_migration_ready(connection) + + asyncio.run(scenario()) + with sqlite3.connect(database) as connection: + assert connection.execute("SELECT * FROM pc_atomic_memory_states").fetchall() == [("retained", "merged")] + current = connection.execute( + "SELECT name FROM sqlite_master WHERE name = 'pc_atomic_memory_current'" + ).fetchone() + if current is not None: + assert connection.execute("SELECT COUNT(*) FROM pc_atomic_memory_current").fetchone()[0] == 0 diff --git a/tests/builtin/persistence/test_experience_index.py b/tests/builtin/persistence/test_experience_index.py index 9e1a6d163d..7a12817381 100644 --- a/tests/builtin/persistence/test_experience_index.py +++ b/tests/builtin/persistence/test_experience_index.py @@ -36,7 +36,7 @@ skill_search_hits, ) from powercontext.builtin.persistence.oceanbase.experience_index import OceanBaseExperienceFTSIndex -from powercontext.builtin.persistence.sqlite import SQLiteConfig +from powercontext.builtin.persistence.sqlite import SQLiteConfig, SQLiteProfile from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, BUILTIN_TABLES from powercontext.builtin.runtime import BuiltinConfig, open_builtin_contexts from powercontext.builtin.sources import ContentCapture @@ -92,7 +92,13 @@ async def scenario() -> None: ): columns = tuple((await connection.exec_driver_sql("PRAGMA table_info('pc_artifact_heads')")).mappings()) names = tuple(column["name"] for column in columns) - for name in ("searchable_text", "lifecycle_state", "replacement_artifact_id", "governance_generation"): + for name in ( + "searchable_text", + "lifecycle_state", + "replacement_artifact_id", + "governance_generation", + "merged_into_id", + ): assert names.count(name) == 1 asyncio.run(scenario()) @@ -112,9 +118,52 @@ def test_oceanbase_startup_upgrades_legacy_artifact_heads_with_mediumtext() -> N "ALTER TABLE pc_artifact_heads ADD COLUMN lifecycle_state VARCHAR(16) NOT NULL DEFAULT 'active'", "ALTER TABLE pc_artifact_heads ADD COLUMN replacement_artifact_id VARCHAR(128) NULL", "ALTER TABLE pc_artifact_heads ADD COLUMN governance_generation BIGINT NOT NULL DEFAULT 0", + "ALTER TABLE pc_artifact_heads ADD COLUMN merged_into_id VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL", + "ALTER TABLE pc_artifact_lineage_artifacts ADD COLUMN is_merge_input BOOLEAN NOT NULL DEFAULT 0", ] +def test_explicit_shared_schema_upgrade_preserves_existing_heads_and_lineage(tmp_path) -> None: + database = tmp_path / "released-shared.db" + with sqlite3.connect(database) as connection: + connection.execute( + "CREATE TABLE pc_artifact_heads (scope_id TEXT, family TEXT, artifact_id TEXT, revision INTEGER, searchable_text TEXT, lifecycle_state TEXT, replacement_artifact_id TEXT, governance_generation INTEGER)" + ) + connection.execute( + "INSERT INTO pc_artifact_heads VALUES ('project', 'experience', 'old', 4, 'evidence', 'deprecated', 'replacement', 7)" + ) + connection.execute( + "CREATE TABLE pc_artifact_lineage_artifacts (scope_id TEXT, family TEXT, artifact_id TEXT, revision INTEGER, ordinal INTEGER, upstream_family TEXT, upstream_artifact_id TEXT, upstream_revision INTEGER)" + ) + connection.execute( + "INSERT INTO pc_artifact_lineage_artifacts VALUES ('project', 'experience', 'old', 1, 0, 'experience', 'evidence', 2)" + ) + + async def scenario() -> None: + async with SQLiteProfile.open(SQLiteConfig(url=f"sqlite+aiosqlite:///{database}"), tables=()) as profile: + for _ in range(2): + async with profile.database.transaction() as connection: + await ensure_artifact_head_searchable_text(connection) + head = (await connection.execute(text("SELECT * FROM pc_artifact_heads"))).mappings().one() + assert ( + head["revision"], + head["lifecycle_state"], + head["governance_generation"], + head["replacement_artifact_id"], + head["merged_into_id"], + ) == (4, "deprecated", 7, "replacement", None) + lineage = ( + (await connection.execute(text("SELECT * FROM pc_artifact_lineage_artifacts"))).mappings().one() + ) + assert ( + lineage["upstream_artifact_id"], + lineage["upstream_revision"], + lineage["is_merge_input"], + ) == ("evidence", 2, False) + + asyncio.run(scenario()) + + def test_sqlite_experience_fts_tracks_only_approved_current_heads_and_rebuilds() -> None: async def scenario() -> None: async with open_builtin_contexts(BuiltinConfig(database=SQLiteConfig())) as contexts: diff --git a/tests/builtin/persistence/test_records.py b/tests/builtin/persistence/test_records.py index 9e35c64944..350b9e20cb 100644 --- a/tests/builtin/persistence/test_records.py +++ b/tests/builtin/persistence/test_records.py @@ -30,7 +30,6 @@ from powercontext.builtin.artifacts.handoff import Handoff from powercontext.builtin.artifacts.skill import Skill from powercontext.builtin.persistence.artifacts import ArtifactRepository -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_STATES_TABLE from powercontext.builtin.persistence.experience_index import ExperienceIndex, NoExperienceIndex from powercontext.builtin.persistence.family_management import ( ExperienceManagementWriter, @@ -448,8 +447,20 @@ async def scenario() -> None: } lineage = (await connection.execute(select(ARTIFACT_LINEAGE_SOURCES_TABLE))).mappings().one() assert lineage["ordinal"] == 0 - state = (await connection.execute(select(ATOMIC_MEMORY_STATES_TABLE))).mappings().one() - assert (state["artifact_id"], state["state"], state["state_version"]) == ("mem-1", "active", 0) + state = ( + ( + await connection.execute( + select(ARTIFACT_HEADS_TABLE).where(ARTIFACT_HEADS_TABLE.c.family == "atomic-memory") + ) + ) + .mappings() + .one() + ) + assert (state["artifact_id"], state["lifecycle_state"], state["governance_generation"]) == ( + "mem-1", + "active", + 0, + ) projection = (await connection.execute(select(contexts.atomic_memory.index.table))).mappings().one() assert (projection["kind"], projection["text"]) == ("preference", "用户偏好使用中文回答") assert projection["searchable_text"] diff --git a/tests/builtin/review/test_service.py b/tests/builtin/review/test_service.py index b04baa032e..e929398311 100644 --- a/tests/builtin/review/test_service.py +++ b/tests/builtin/review/test_service.py @@ -85,6 +85,9 @@ class _FailingExperienceIndex: async def initialize(self, _connection: AsyncConnection, /) -> None: pass + async def remove(self, _connection: AsyncConnection, _scope_id: str, _artifact_id: str, /) -> None: + raise _InjectedExperienceProjectionError + async def replace( self, _connection: AsyncConnection, diff --git a/tests/builtin/runtime/test_atomic_merge_restoration.py b/tests/builtin/runtime/test_atomic_merge_restoration.py new file mode 100644 index 0000000000..038e43abae --- /dev/null +++ b/tests/builtin/runtime/test_atomic_merge_restoration.py @@ -0,0 +1,197 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Atomic public API retains preview and cascade behavior through the shared service.""" + +from __future__ import annotations + +import asyncio + +import pytest +from pydantic import SecretStr + +from powercontext.builtin.artifacts.atomic_memory import AtomicMemoryContent +from powercontext.builtin.artifacts.atomic_memory.errors import AtomicMemoryConflictError, AtomicMemoryPreviewStaleError +from powercontext.builtin.artifacts.atomic_memory.restoration import AtomicMemoryPreviewSigner +from powercontext.builtin.records import ArtifactWrite +from powercontext.builtin.runtime import BuiltinConfig, RuntimeConfig, open_builtin_contexts + + +@pytest.mark.parametrize("target,historical", (("e", False), ("b", False), ("b", True))) +def test_atomic_cascade_restore_appends_revisions_and_preserves_preview_contract(target, historical) -> None: + async def scenario() -> None: + async with open_builtin_contexts( + BuiltinConfig( + runtime=RuntimeConfig( + atomic_memory_preview_signing_secret=SecretStr("x" * 32), + atomic_memory_preview_signing_key_id="test", + ) + ) + ) as contexts: + application = contexts.atomic_memory + signer = AtomicMemoryPreviewSigner(keys={"test": b"x" * 32}, active_key_id="test") + memory = application.for_scope("project") + originals = {} + for identity in ("a", "b", "d"): + created = await contexts.records.create_atomic_memories( + "project", ({"kind": "rule", "text": f"Keep rule {identity}."},) + ) + originals[identity] = await memory.get(created[0].artifact_id) + first = await memory.merge( + tuple(originals[item].as_read() for item in ("a", "b")), + AtomicMemoryContent(kind="rule", text="Keep a and b."), + ) + await contexts.records.replace_artifact( + "project", + "atomic-memory", + first.primary.ref.artifact_id, + '"revision:1"', + ArtifactWrite(content={"kind": "rule", "text": "Keep revised a and b."}), + ) + revised = await memory.get(first.primary.ref.artifact_id) + final = await memory.merge( + (revised.as_read(), originals["d"].as_read()), AtomicMemoryContent(kind="rule", text="Keep a, b and d.") + ) + identity = final.primary.ref.artifact_id if target == "e" else originals["b"].ref.artifact_id + kwargs = {"operation": "undo_merge"} if target == "e" else {"revision": 1} if historical else {} + preview = await memory.preview_restoration(identity, **kwargs) + # Atomic's original token can still be decoded by the public signer. + assert ( + signer.validate( + preview.preview_token, + scope_id="project", + subject=application.security.subject(None), + operation=kwargs.get("operation", "restore"), + artifact_id=identity, + revision=kwargs.get("revision"), + ) + == preview.endpoint + ) + result = await memory.restore(identity, preview_token=preview.preview_token, **kwargs) + assert all(item.creates_revision for item in preview.restore) + assert result.undo_merge_results == ( + (final.primary.ref.artifact_id,) + if target == "e" + else (final.primary.ref.artifact_id, first.primary.ref.artifact_id) + ) + retired = await memory.get(final.primary.ref.artifact_id) + assert retired.state.state == "retired" + assert retired.ref.revision == 2 + restored_first = await memory.get(first.primary.ref.artifact_id) + assert restored_first.ref.revision == 3 + assert restored_first.state.state == ("active" if target == "e" else "retired") + assert restored_first.artifact.content == revised.artifact.content.without_creation() + assert restored_first.artifact.lineage.artifacts == (revised.ref,) + assert (await memory.get(revised.ref.artifact_id, revision=2)).artifact == revised.artifact + for name in ("a", "b"): + record = await memory.get(originals[name].ref.artifact_id) + assert record.state.state == ("merged" if target == "e" else "active") + assert record.ref.revision == (1 if target == "e" else 2) + restored_d = await memory.get(originals["d"].ref.artifact_id) + assert restored_d.state.state == "active" + assert restored_d.ref.revision == 2 + assert {ref.artifact_id: ref.revision for ref in result.restored} == ( + {revised.ref.artifact_id: 3, originals["d"].ref.artifact_id: 2} + if target == "e" + else {originals[item].ref.artifact_id: 2 for item in ("a", "b", "d")} + ) + assert retired.ref in result.retired + if target == "e": + second_undo = await memory.restore(first.primary.ref.artifact_id, operation="undo_merge") + assert {ref.artifact_id: ref.revision for ref in second_undo.restored} == { + originals[item].ref.artifact_id: 2 for item in ("a", "b") + } + assert second_undo.primary.ref.revision == 4 + again = await memory.merge( + tuple([(await memory.get(originals[item].ref.artifact_id)).as_read() for item in ("a", "b")]), + AtomicMemoryContent(kind="rule", text="Keep a and b again."), + ) + third_undo = await memory.restore(again.primary.ref.artifact_id, operation="undo_merge") + assert {ref.artifact_id: ref.revision for ref in third_undo.restored} == { + originals[item].ref.artifact_id: 3 for item in ("a", "b") + } + assert third_undo.primary.ref.revision == 2 + + asyncio.run(scenario()) + + +def test_atomic_endpoint_change_rejects_signed_preview() -> None: + async def scenario() -> None: + async with open_builtin_contexts( + BuiltinConfig( + runtime=RuntimeConfig( + atomic_memory_preview_signing_secret=SecretStr("x" * 32), + atomic_memory_preview_signing_key_id="test", + ) + ) + ) as contexts: + application = contexts.atomic_memory + memory = application.for_scope("project") + created = await contexts.records.create_atomic_memories( + "project", ({"kind": "rule", "text": "Keep this rule."},) + ) + original = await memory.get(created[0].artifact_id) + preview = await memory.preview_restoration(original.ref.artifact_id) + await contexts.records.replace_artifact( + "project", + "atomic-memory", + original.ref.artifact_id, + '"revision:1"', + ArtifactWrite(content={"kind": "rule", "text": "Keep the revised rule."}), + ) + with pytest.raises(AtomicMemoryPreviewStaleError): + await memory.restore(original.ref.artifact_id, preview_token=preview.preview_token) + + asyncio.run(scenario()) + + +@pytest.mark.parametrize("change", ("revision", "state")) +def test_atomic_merge_commit_rejects_changed_input_without_partial_result(change: str) -> None: + async def scenario() -> None: + async with open_builtin_contexts(BuiltinConfig()) as contexts: + application = contexts.atomic_memory + memory = application.for_scope("project") + created = await contexts.records.create_atomic_memories( + "project", ({"kind": "rule", "text": "First rule."}, {"kind": "rule", "text": "Second rule."}) + ) + originals = tuple([await memory.get(item.artifact_id) for item in created]) + async with contexts.database.transaction() as connection: + plan = await application.service.inspect_merge( + connection, + "project", + "new-result", + tuple(record.as_read() for record in originals), + AtomicMemoryContent(kind="rule", text="Keep both rules."), + None, + ) + prepared = await application.service.prepare_merge(plan) + if change == "revision": + await contexts.records.replace_artifact( + "project", + "atomic-memory", + originals[0].ref.artifact_id, + '"revision:1"', + ArtifactWrite(content={"kind": "rule", "text": "Updated first rule."}), + ) + else: + await memory.forget(originals[0].ref.artifact_id, expected_revision=1, expected_state_version=0) + with pytest.raises(AtomicMemoryConflictError): + async with contexts.database.transaction() as connection: + await application.service.commit(connection, prepared, None) + current = (await memory.list(states=("active", "forgotten", "merged", "retired"))).items + assert {record.ref.artifact_id for record in current} == {record.ref.artifact_id for record in originals} + assert all(record.state.merged_into_id is None for record in current) + assert await memory.get(originals[1].ref.artifact_id) == originals[1] + + asyncio.run(scenario()) diff --git a/tests/builtin/runtime/test_family_processing.py b/tests/builtin/runtime/test_family_processing.py index f2ffcbbf9c..f4f5c3658b 100644 --- a/tests/builtin/runtime/test_family_processing.py +++ b/tests/builtin/runtime/test_family_processing.py @@ -31,11 +31,9 @@ from powercontext.builtin.inference import InferenceTimeoutError from powercontext.builtin.inference.models import GenerationResult, InferenceUsage from powercontext.builtin.inference.usage import UsageReportingStructuredGenerator -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_TABLES from powercontext.builtin.persistence.cursors import SourceCursorRepository from powercontext.builtin.persistence.processing_intents import ArtifactProcessingIntentRepository from powercontext.builtin.persistence.processing_migration import bootstrap_processing_schema -from powercontext.builtin.persistence.schema import create_tables from powercontext.builtin.persistence.sqlite import SQLiteConfig, SQLiteProfile from powercontext.builtin.persistence.supervision import ArtifactProcessingFence, ArtifactProcessingLeaseRepository from powercontext.builtin.persistence.tables import ( @@ -143,7 +141,6 @@ async def prepare(profile, family): ) async with profile.database.transaction() as connection: await _initialize_atomic_memory_authority(connection) - await create_tables(connection, ATOMIC_MEMORY_TABLES) await contexts.atomic_memory.index.initialize(connection) contexts.profiles.generator = ProfileGenerator() scope = ( diff --git a/tests/builtin/runtime/test_memory_window_recovery.py b/tests/builtin/runtime/test_memory_window_recovery.py index d9694a512f..a5d79d99ce 100644 --- a/tests/builtin/runtime/test_memory_window_recovery.py +++ b/tests/builtin/runtime/test_memory_window_recovery.py @@ -21,8 +21,6 @@ from powercontext.builtin.artifacts.atomic_memory.extraction import AtomicMemoryCandidate, AtomicMemoryExtractionOutput from powercontext.builtin.inference import GenerationResult, InferenceTimeoutError, InferenceUnavailableError -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_TABLES -from powercontext.builtin.persistence.schema import create_tables from powercontext.builtin.persistence.sqlite import SQLiteConfig, SQLiteProfile from powercontext.builtin.persistence.tables import BUILTIN_TABLES from powercontext.builtin.runtime.composition import _initialize_atomic_memory_authority @@ -58,7 +56,6 @@ async def _ready_contexts(profile, **kwargs): contexts = RelationalContexts(database=profile.database, **kwargs) async with profile.database.transaction() as connection: await _initialize_atomic_memory_authority(connection) - await create_tables(connection, ATOMIC_MEMORY_TABLES) await contexts.atomic_memory.index.initialize(connection) return contexts diff --git a/tests/builtin/test_publication.py b/tests/builtin/test_publication.py index a90e26f453..fb763969af 100644 --- a/tests/builtin/test_publication.py +++ b/tests/builtin/test_publication.py @@ -45,6 +45,9 @@ class _FailingExperienceIndex: async def initialize(self, _connection, /) -> None: pass + async def remove(self, _connection, _scope_id, _artifact_id, /) -> None: + raise _IndexUnavailableError + async def replace(self, _connection, _scope_id, _experience, /) -> None: raise _IndexUnavailableError diff --git a/tests/e2e/test_artifact_merge_experience.py b/tests/e2e/test_artifact_merge_experience.py new file mode 100644 index 0000000000..cf1f48577d --- /dev/null +++ b/tests/e2e/test_artifact_merge_experience.py @@ -0,0 +1,628 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Explicit Experience merge uses the runtime's shared service and real indexes.""" + +from __future__ import annotations + +import asyncio +import os +from collections.abc import Iterable, Iterator +from datetime import UTC, datetime +from pathlib import Path +from typing import cast +from uuid import uuid4 + +import pytest +from pydantic import SecretStr +from sqlalchemy.engine import make_url + +from powercontext.artifacts import ArtifactAddress, ArtifactLineage, ArtifactRef, ArtifactSearchExecutionContext +from powercontext.builtin.artifacts.atomic_memory import AtomicMemoryContent +from powercontext.builtin.artifacts.experience import ExperienceContent, FailureRecord +from powercontext.builtin.artifacts.merge_models import ArtifactMergeRelationError +from powercontext.builtin.persistence.artifact_governance import InvalidArtifactLifecycleError +from powercontext.builtin.persistence.errors import RepositoryNotFoundError +from powercontext.builtin.persistence.experience_index import ensure_artifact_head_searchable_text +from powercontext.builtin.persistence.oceanbase import OceanBaseConfig, OceanBaseProfile +from powercontext.builtin.persistence.sqlite import SQLiteConfig, SQLiteProfile +from powercontext.builtin.records import ArtifactWrite, BaseOperationNotSupportedError +from powercontext.builtin.runtime import ( + ApproveArtifactCandidateRequest, + BuiltinConfig, + GetArtifactCandidateRequest, + ProposeExperienceRequest, + RuntimeConfig, + open_builtin_runtime, +) +from powercontext.builtin.runtime.relational import RelationalContexts +from powercontext.builtin.scope import ScopeDraft +from powercontext.builtin.tags import ArtifactTagTarget +from powercontext.server.authz import ( + AccessAuditContext, + AccessBinding, + AccessBindingState, + AccessControlService, + AccessDeniedError, + AccessRole, + BuiltinAuthorizationProvider, + PrincipalRef, + ResourceRef, +) +from powercontext.server.authz.repository import RelationalAccessRepository +from powercontext.sources import SourceRef + +OWNER = PrincipalRef(type="service", id="experience-owner") + + +def _execution_context(runtime, principal: PrincipalRef = OWNER) -> ArtifactSearchExecutionContext: + contexts = cast(RelationalContexts, runtime._provider) + repository = RelationalAccessRepository(contexts.database) + access = AccessControlService( + BuiltinAuthorizationProvider(repository), + relationships=repository, + audit=repository, + static_scope_principal=OWNER, + ) + return ArtifactSearchExecutionContext( + principal=principal, access=access, audit=AccessAuditContext(transport="test", operation="experience-merge") + ) + + +@pytest.fixture(params=("sqlite", "oceanbase")) +def database(request: pytest.FixtureRequest, tmp_path: Path) -> Iterator[SQLiteConfig | OceanBaseConfig]: + if request.param == "sqlite": + yield SQLiteConfig(url=f"sqlite+aiosqlite:///{tmp_path / 'artifact-merge.db'}") + return + configured_url = os.environ.get("POWERCONTEXT_TEST_OCEANBASE_URL") + if configured_url is None: + pytest.skip("set POWERCONTEXT_TEST_OCEANBASE_URL with test database creation and deletion privileges") + configured = OceanBaseConfig(url=SecretStr(configured_url)) + database_name = "pc_artifact_merge_test_" + uuid4().hex[:16] + + async def manage(statement: str) -> None: + async with ( + OceanBaseProfile.open(configured, tables=()) as admin, + admin.database.transaction() as connection, + ): + await connection.exec_driver_sql(statement) + + asyncio.run(manage(f"CREATE DATABASE `{database_name}` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin")) + try: + url = make_url(configured_url).set(database=database_name) + yield OceanBaseConfig(url=SecretStr(url.render_as_string(hide_password=False))) + finally: + asyncio.run(manage(f"DROP DATABASE `{database_name}`")) + + +def _content(name: str, *, failure: FailureRecord | None = None) -> ExperienceContent: + return ExperienceContent( + situation=f"The rollout needs {name} checks.", + action=f"Perform the {name} rollout check.", + outcome=f"The {name} rollout check passed.", + lesson=f"Keep the {name} rollout evidence.", + failure=failure, + ) + + +def _refs(refs: Iterable[ArtifactRef]) -> set[tuple[str, str, int]]: + return {(ref.family, ref.artifact_id, ref.revision) for ref in refs} + + +async def _scope(runtime) -> str: + scope = await runtime.scopes.create( + ScopeDraft(title="Merge acceptance", summary="Experience merge", idempotency_key=uuid4().hex) + ) + assert getattr(runtime, "artifact_merge", None) is not None, "Runtime must expose registered shared Artifact merge" + return scope.scope_id + + +async def _create(runtime, scope: str, name: str): + created = await runtime.records.for_scope(scope).create_artifact( + "experience", ArtifactWrite(content=_content(name).model_dump(mode="json")) + ) + contexts = cast(RelationalContexts, runtime._provider) + resource = ResourceRef.artifact(scope, family="experience", artifact_id=created.artifact_id) + context = _execution_context(runtime) + # Ordinary local management writes leave ownership to their host's authorization boundary. + async with contexts.database.transaction() as connection: + await context.access.with_connection(connection).establish_artifact_owner( + resource, OWNER, idempotency_key="test-owner:" + created.artifact_id, context=context.audit + ) + return await runtime.artifact_merge.for_scope(scope, "experience").get(created.artifact_id) + + +async def _search(runtime, scope: str, query: str): + contexts = cast(RelationalContexts, runtime._provider) + return (await contexts.search_experience_outcome(scope, query, 20)).hits + + +def test_experience_management_merge_freezes_inputs_preserves_history_and_candidate_rollback( + database, tmp_path +) -> None: + async def scenario() -> None: + async with open_builtin_runtime( + BuiltinConfig(database=database), scheduler_path=tmp_path / "scheduler.db" + ) as runtime: + scope = await _scope(runtime) + assert runtime.artifact_merge is not None + records = runtime.records.for_scope(scope) + merge = runtime.artifact_merge.for_scope(scope, "experience") + contexts = cast(RelationalContexts, runtime._provider) + context = _execution_context(runtime) + a, b = await _create(runtime, scope, "canary"), await _create(runtime, scope, "pause") + await records.replace_artifact( + "experience", + a.ref.artifact_id, + '"revision:1"', + ArtifactWrite(content=_content("revised").model_dump(mode="json")), + ) + a = await merge.get(a.ref.artifact_id) + assert a.ref.revision == 2 + assert len((await records.query_artifacts("experience", limit=20, cursor=None)).items) == 2 + assert _refs(hit.artifact_ref for hit in await _search(runtime, scope, "rollout")) == _refs((a.ref, b.ref)) + for original, labels in ((a, ("Rollout", "canary")), (b, ("rollout", "pause"))): + target = ArtifactTagTarget(family="experience", artifact_id=original.ref.artifact_id) + tags = await records.get_tags(target) + await records.replace_tags(target, labels, expected_etag=tags.etag) + source = await records.create_source("content", "The rollout pause check passed.") + candidate = await runtime.experience.for_scope(scope).propose( + ProposeExperienceRequest( + proposal=_content("candidate"), + target=a.ref, + artifacts=(a.ref,), + sources=(SourceRef(source_type="content", source_id=source.source_id),), + ) + ) + merged_content = _content( + "combined", + failure=FailureRecord.model_validate({ + "signature": {"recall_cue": "Rollout check failed"}, + "repair_surface": "acceptance_check", + "verification": {"condition": "Rollout check passed", "check_subject": "rollout-check"}, + }), + ) + result = await merge.merge((a.as_read(), b.as_read()), merged_content, context=context) + merged = result.primary + assert merged.ref.revision == 1 + assert merged.artifact.content == merged_content + assert merged.artifact.lineage.artifacts == (a.ref, b.ref) + tags = await records.get_tags(ArtifactTagTarget(family="experience", artifact_id=merged.ref.artifact_id)) + assert {tag.casefold() for tag in tags.tags} == {"rollout", "canary", "pause"} + async with contexts.database.transaction() as connection: + access = RelationalAccessRepository(contexts.database, connection=connection) + owner = await access.get_artifact_owner( + ResourceRef.artifact(scope, family="experience", artifact_id=merged.ref.artifact_id) + ) + assert owner is not None and owner.owner == OWNER + assert ( + await access.get_artifact_owner( + ResourceRef.artifact(scope, family="atomic-memory", artifact_id=merged.ref.artifact_id) + ) + is None + ) + for original in (a, b): + frozen = await merge.get(original.ref.artifact_id) + assert frozen.artifact == original.artifact + assert frozen.state.lifecycle_state == "deprecated" + assert frozen.state.merged_into_id == merged.ref.artifact_id + assert frozen.as_read().state_version == original.as_read().state_version + 1 + historical = await records.get_artifact_revision( + "experience", original.ref.artifact_id, original.ref.revision + ) + assert historical.content == original.artifact.content.model_dump(mode="json") + with pytest.raises(InvalidArtifactLifecycleError): + await records.replace_artifact( + "experience", + original.ref.artifact_id, + f'"revision:{original.ref.revision}"', + ArtifactWrite(content=_content("illegal").model_dump(mode="json")), + ) + with pytest.raises(InvalidArtifactLifecycleError): + await runtime.review.for_scope(scope).approve( + ApproveArtifactCandidateRequest( + candidate_id=candidate.candidate_id, expected_version=candidate.version + ) + ) + pending = await runtime.review.for_scope(scope).get( + GetArtifactCandidateRequest(candidate_id=candidate.candidate_id) + ) + assert pending.status == "pending" and pending.version == candidate.version + assert (await merge.get(a.ref.artifact_id)).artifact == a.artifact + assert tuple( + item.artifact_id for item in (await records.query_artifacts("experience", limit=1, cursor=None)).items + ) == (merged.ref.artifact_id,) + assert tuple(hit.artifact_ref for hit in await _search(runtime, scope, "rollout")) == (merged.ref,) + assert {item.artifact_id for item in await records.logical_artifacts()} >= { + a.ref.artifact_id, + b.ref.artifact_id, + merged.ref.artifact_id, + } + with pytest.raises(BaseOperationNotSupportedError): + runtime.artifact_merge.for_scope(scope, "prompt") + + asyncio.run(scenario()) + + +def test_registered_families_keep_same_result_identity_and_projections_separate(database, tmp_path) -> None: + async def scenario() -> None: + async with open_builtin_runtime( + BuiltinConfig(database=database), scheduler_path=tmp_path / "scheduler.db" + ) as runtime: + scope = await _scope(runtime) + assert runtime.artifact_merge is not None + assert runtime.atomic_memory is not None + records = runtime.records.for_scope(scope) + merged_api = runtime.artifact_merge.for_scope(scope, "atomic-memory") + originals = [] + for name in ("canary", "pause"): + created = await records.create_artifact( + "atomic-memory", ArtifactWrite(content={"kind": "rule", "text": f"Keep the rollout {name} check."}) + ) + originals.append(await merged_api.get(created.artifact_id)) + target = ArtifactTagTarget(family="atomic-memory", artifact_id=created.artifact_id) + tags = await records.get_tags(target) + await records.replace_tags(target, ("Atomic",), expected_etag=tags.etag) + result = await merged_api.merge( + tuple(item.as_read() for item in originals), + AtomicMemoryContent(kind="rule", text="Keep both rollout checks."), + ) + atomic = await runtime.atomic_memory.for_scope(scope).get(result.primary.ref.artifact_id) + assert atomic.artifact == result.primary.artifact + assert atomic.artifact.content.creation is not None + assert atomic.artifact.content.creation.input_artifact_ids == tuple( + item.ref.artifact_id for item in originals + ) + experience = runtime.artifact_merge.for_scope(scope, "experience") + inputs = (await _create(runtime, scope, "canary"), await _create(runtime, scope, "pause")) + for original in inputs: + target = ArtifactTagTarget(family="experience", artifact_id=original.ref.artifact_id) + tags = await records.get_tags(target) + await records.replace_tags(target, ("Experience",), expected_etag=tags.etag) + merged_experience = ( + await experience.merge( + tuple(item.as_read() for item in inputs), + _content("combined"), + artifact_id=result.primary.ref.artifact_id, + ) + ).primary + assert merged_experience.ref.artifact_id == atomic.ref.artifact_id + for family, label in (("atomic-memory", "Atomic"), ("experience", "Experience")): + tags = await records.get_tags(ArtifactTagTarget(family=family, artifact_id=atomic.ref.artifact_id)) + assert tags.tags == (label,) + await experience.restore(merged_experience.ref.artifact_id, operation="undo_merge") + assert (await merged_api.get(atomic.ref.artifact_id)).artifact == atomic.artifact + assert (await merged_api.get(atomic.ref.artifact_id)).state.lifecycle_state == "active" + assert tuple( + hit.hit.artifact_ref for hit in (await runtime.atomic_memory.for_scope(scope).search("rollout")).hits + ) == (atomic.ref,) + assert _refs(hit.artifact_ref for hit in await _search(runtime, scope, "rollout")) == _refs( + item.ref.model_copy(update={"revision": item.ref.revision + 1}) for item in inputs + ) + restored = await merged_api.restore(result.primary.ref.artifact_id, operation="undo_merge") + assert _refs(restored.restored) == _refs( + item.ref.model_copy(update={"revision": item.ref.revision + 1}) for item in originals + ) + assert ( + await runtime.atomic_memory.for_scope(scope).get(result.primary.ref.artifact_id) + ).state.state == "retired" + + asyncio.run(scenario()) + + +@pytest.mark.parametrize("target,historical", (("endpoint", False), ("input", False), ("input", True))) +def test_experience_nested_restore_republishes_the_selected_group(database, tmp_path, target, historical) -> None: + async def scenario() -> None: + config = BuiltinConfig( + database=database, + runtime=RuntimeConfig(atomic_memory_preview_signing_secret=SecretStr("x" * 32)), + ) + async with open_builtin_runtime(config, scheduler_path=tmp_path / "scheduler.db") as runtime: + scope = await _scope(runtime) + assert runtime.artifact_merge is not None + merge = runtime.artifact_merge.for_scope(scope, "experience") + records = runtime.records.for_scope(scope) + a, b, d = ( + await _create(runtime, scope, "canary"), + await _create(runtime, scope, "pause"), + await _create(runtime, scope, "health"), + ) + await records.replace_artifact( + "experience", + b.ref.artifact_id, + '"revision:1"', + ArtifactWrite(content=_content("newpause").model_dump(mode="json")), + ) + b_current = await merge.get(b.ref.artifact_id) + first = (await merge.merge((a.as_read(), b_current.as_read()), _content("combined"))).primary + await records.replace_artifact( + "experience", + first.ref.artifact_id, + '"revision:1"', + ArtifactWrite(content=_content("updatedcombined").model_dump(mode="json")), + ) + first = await merge.get(first.ref.artifact_id) + last = (await merge.merge((first.as_read(), d.as_read()), _content("complete"))).primary + identity = last.ref.artifact_id if target == "endpoint" else b.ref.artifact_id + kwargs = {"operation": "undo_merge"} if target == "endpoint" else {"revision": 1} if historical else {} + preview = await merge.preview_restoration(identity, **kwargs) + assert preview.endpoint.ref == last.ref + result = await merge.restore(identity, preview_token=preview.preview_token, **kwargs) + outcome = await merge.restoration_outcome(identity, revision=result.primary.ref.revision) + assert outcome is not None and outcome.operation == kwargs.get("operation", "restore") + assert outcome.undo_merge_results == ((last.ref,) if target == "endpoint" else (last.ref, first.ref)) + assert _refs(item.after_ref for item in outcome.writes) == _refs((*result.restored, *result.retired)) + source = next(item.content_from_ref for item in outcome.writes if item.after_ref.artifact_id == identity) + assert source == (last.ref if target == "endpoint" else b.ref if historical else b_current.ref) + assert len(result.primary.artifact.lineage.sources) == 1 + assert all(not item.artifact.lineage.sources for item in result.records if item.ref.artifact_id != identity) + expected_refs = ( + (first.ref.model_copy(update={"revision": 3}), d.ref.model_copy(update={"revision": 2})) + if target == "endpoint" + else ( + a.ref.model_copy(update={"revision": 2}), + d.ref.model_copy(update={"revision": 2}), + ArtifactRef(family="experience", artifact_id=b.ref.artifact_id, revision=3), + ) + ) + assert _refs(result.restored) == _refs(expected_refs) + retired = await merge.get(last.ref.artifact_id) + assert retired.state.lifecycle_state == "retired" + assert retired.ref.revision == 2 + assert retired.ref in result.retired + assert all(item.creates_revision for item in preview.restore) + assert result.undo_merge_results == ( + (last.ref.artifact_id,) if target == "endpoint" else (last.ref.artifact_id, first.ref.artifact_id) + ) + assert _refs(hit.artifact_ref for hit in await _search(runtime, scope, "rollout")) == _refs(expected_refs) + assert { + item.artifact_id for item in (await records.query_artifacts("experience", limit=20, cursor=None)).items + } == {ref.artifact_id for ref in expected_refs} + if target == "endpoint": + assert (await merge.get(b.ref.artifact_id)).state.merged_into_id == first.ref.artifact_id + restored_first = await merge.get(first.ref.artifact_id) + assert restored_first.ref.revision == 3 + assert restored_first.artifact.content == first.artifact.content + assert restored_first.artifact.lineage.artifacts == (first.ref,) + assert (await merge.get(first.ref.artifact_id, revision=2)).artifact == first.artifact + else: + assert (await merge.get(first.ref.artifact_id)).state.lifecycle_state == "retired" + restored = await merge.get(b.ref.artifact_id) + assert restored.artifact.content == (b.artifact.content if historical else b_current.artifact.content) + assert (await merge.get(b.ref.artifact_id, revision=1)).artifact == b.artifact + if historical: + assert b.ref in restored.artifact.lineage.artifacts + assert b_current.ref in restored.artifact.lineage.artifacts + + asyncio.run(scenario()) + + +def test_experience_merge_preserves_explicit_caller_identity_and_denies_foreign_inputs(database, tmp_path) -> None: + async def scenario() -> None: + async with open_builtin_runtime( + BuiltinConfig(database=database), scheduler_path=tmp_path / "scheduler.db" + ) as runtime: + scope = await _scope(runtime) + assert runtime.artifact_merge is not None + merge = runtime.artifact_merge.for_scope(scope, "experience") + a, b = await _create(runtime, scope, "canary"), await _create(runtime, scope, "pause") + foreign = _execution_context(runtime, PrincipalRef(type="user", id="foreign")) + contexts = cast(RelationalContexts, runtime._provider) + async with contexts.database.transaction() as connection: + await RelationalAccessRepository(contexts.database, connection=connection).create_binding( + AccessBinding( + binding_id="foreign-contributor", + subject=foreign.principal, + resource=ResourceRef.scope(scope), + role=AccessRole.SCOPE_CONTRIBUTOR, + granted_by=OWNER, + reason=None, + created_at=datetime.now(UTC), + expires_at=None, + state=AccessBindingState.ACTIVE, + version=1, + policy_revision="pending", + idempotency_key="foreign-contributor", + ) + ) + assert await merge.get(a.ref.artifact_id, context=foreign) == a + with pytest.raises(AccessDeniedError): + await merge.merge( + (a.as_read(), b.as_read()), _content("combined"), artifact_id="denied-result", context=foreign + ) + assert await merge.get(a.ref.artifact_id) == a + assert await merge.get(b.ref.artifact_id) == b + with pytest.raises(RepositoryNotFoundError): + await merge.get("denied-result") + assert ( + len((await runtime.records.for_scope(scope).query_artifacts("experience", limit=20, cursor=None)).items) + == 2 + ) + owner_context = _execution_context(runtime) + merged = ( + await merge.merge((a.as_read(), b.as_read()), _content("combined"), context=owner_context) + ).primary + owner = await owner_context.access.artifact_owner( + ResourceRef.artifact(scope, family="experience", artifact_id=merged.ref.artifact_id) + ) + assert owner is not None and owner.owner == OWNER + + asyncio.run(scenario()) + + +def test_restoration_outcome_survives_reopen_and_requires_group_read_access(database, tmp_path) -> None: + async def scenario() -> None: + config = BuiltinConfig(database=database) + scheduler_path = tmp_path / "scheduler.db" + async with open_builtin_runtime(config, scheduler_path=scheduler_path) as runtime: + scope = await _scope(runtime) + assert runtime.artifact_merge is not None + merge = runtime.artifact_merge.for_scope(scope, "experience") + owner_context = _execution_context(runtime) + a, b = await _create(runtime, scope, "canary"), await _create(runtime, scope, "pause") + merged = ( + await merge.merge((a.as_read(), b.as_read()), _content("combined"), context=owner_context) + ).primary + assert await merge.restoration_outcome(a.ref.artifact_id, revision=1) is None + restored = await merge.restore(a.ref.artifact_id, context=owner_context) + outcome = await merge.restoration_outcome(a.ref.artifact_id, revision=2) + assert outcome is not None + assert outcome.operation == "restore" and outcome.target == a.ref + assert outcome.undo_merge_results == (merged.ref,) + assert _refs(item.before_ref for item in outcome.writes) == _refs((a.ref, b.ref, merged.ref)) + assert _refs(item.after_ref for item in outcome.writes) == _refs((*restored.restored, *restored.retired)) + assert all(item.content_from_ref == item.before_ref for item in outcome.writes) + assert {item.after_ref.artifact_id: item.lifecycle_state for item in outcome.writes} == { + a.ref.artifact_id: "active", + b.ref.artifact_id: "active", + merged.ref.artifact_id: "retired", + } + frozen_outcome = outcome.model_dump(mode="json") + await runtime.records.for_scope(scope).replace_artifact( + "experience", a.ref.artifact_id, '"revision:2"', ArtifactWrite(content=_content("latest").model_dump()) + ) + again = await merge.merge( + ((await merge.get(a.ref.artifact_id)).as_read(), (await merge.get(b.ref.artifact_id)).as_read()), + _content("again"), + context=owner_context, + ) + reader = PrincipalRef(type="user", id="artifact-only-reader") + contexts = cast(RelationalContexts, runtime._provider) + async with contexts.database.transaction() as connection: + await RelationalAccessRepository(contexts.database, connection=connection).create_binding( + AccessBinding( + binding_id="primary-only", + subject=reader, + resource=ResourceRef.artifact(scope, family="experience", artifact_id=a.ref.artifact_id), + role=AccessRole.ARTIFACT_VIEWER, + granted_by=OWNER, + reason=None, + created_at=datetime.now(UTC), + expires_at=None, + state=AccessBindingState.ACTIVE, + version=1, + policy_revision="pending", + idempotency_key="primary-only", + ) + ) + context = _execution_context(runtime, reader) + assert (await merge.get(a.ref.artifact_id, revision=2, context=context)).ref == restored.primary.ref + with pytest.raises(AccessDeniedError): + await merge.restoration_outcome(a.ref.artifact_id, revision=2, context=context) + async with open_builtin_runtime(config, scheduler_path=scheduler_path) as reopened: + assert reopened.artifact_merge is not None + merge = reopened.artifact_merge.for_scope(scope, "experience") + recovered = await merge.restoration_outcome(a.ref.artifact_id, revision=2) + assert recovered is not None and recovered.model_dump(mode="json") == frozen_outcome + assert (await merge.get(a.ref.artifact_id)).ref.revision == 3 + assert (await merge.get(a.ref.artifact_id)).state.merged_into_id == again.primary.ref.artifact_id + + asyncio.run(scenario()) + + +def test_existing_shared_tables_upgrade_preserves_exact_rows_and_defaults(database) -> None: + async def scenario() -> None: + profile_factory = SQLiteProfile if isinstance(database, SQLiteConfig) else OceanBaseProfile + async with profile_factory.open(database, tables=()) as profile, profile.database.transaction() as connection: + identity = ( + "VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin" + if connection.dialect.name == "mysql" + else "VARCHAR(128)" + ) + table_default = ( + " DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci" + if connection.dialect.name == "mysql" + else "" + ) + await connection.exec_driver_sql( + "CREATE TABLE pc_artifact_heads (" + f"scope_id {identity} NOT NULL, family {identity} NOT NULL, artifact_id {identity} NOT NULL, " + "revision BIGINT NOT NULL, searchable_text TEXT NULL, lifecycle_state VARCHAR(16) NOT NULL DEFAULT 'active', " + "replacement_artifact_id VARCHAR(128) NULL, governance_generation BIGINT NOT NULL DEFAULT 0, " + f"PRIMARY KEY (scope_id, family, artifact_id)){table_default}" + ) + await connection.exec_driver_sql( + "CREATE TABLE pc_artifact_lineage_artifacts (" + "scope_id VARCHAR(128) NOT NULL, family VARCHAR(128) NOT NULL, artifact_id VARCHAR(128) NOT NULL, " + "revision BIGINT NOT NULL, ordinal BIGINT NOT NULL, upstream_family VARCHAR(128) NOT NULL, " + "upstream_artifact_id VARCHAR(128) NOT NULL, upstream_revision BIGINT NOT NULL, " + "PRIMARY KEY (scope_id, family, artifact_id, revision, ordinal))" + ) + await connection.exec_driver_sql( + "INSERT INTO pc_artifact_heads (scope_id, family, artifact_id, revision, searchable_text, governance_generation) " + "VALUES ('historical', 'experience', 'existing', 3, 'existing searchable evidence', 7)" + ) + await connection.exec_driver_sql( + "INSERT INTO pc_artifact_lineage_artifacts " + "VALUES ('historical', 'experience', 'existing', 3, 0, 'experience', 'upstream', 2)" + ) + await ensure_artifact_head_searchable_text(connection) + head = ( + await connection.exec_driver_sql( + "SELECT revision, searchable_text, governance_generation, merged_into_id FROM pc_artifact_heads" + ) + ).one() + assert tuple(head) == (3, "existing searchable evidence", 7, None) + lineage = ( + await connection.exec_driver_sql( + "SELECT revision, ordinal, upstream_family, upstream_artifact_id, upstream_revision, is_merge_input " + "FROM pc_artifact_lineage_artifacts" + ) + ).one() + assert tuple(lineage) == (3, 0, "experience", "upstream", 2, 0) + await ensure_artifact_head_searchable_text(connection) + await connection.exec_driver_sql( + "UPDATE pc_artifact_heads SET lifecycle_state = 'deprecated', merged_into_id = 'ResultA'" + ) + assert ( + await connection.exec_driver_sql( + "SELECT artifact_id FROM pc_artifact_heads WHERE merged_into_id = 'resulta'" + ) + ).all() == [] + assert ( + await connection.exec_driver_sql( + "SELECT artifact_id FROM pc_artifact_heads WHERE merged_into_id = 'ResultA'" + ) + ).scalar_one() == "existing" + + asyncio.run(scenario()) + + +def test_experience_merge_rejects_unsupported_lineage_before_writing(database, tmp_path) -> None: + async def scenario() -> None: + async with open_builtin_runtime( + BuiltinConfig(database=database), scheduler_path=tmp_path / "scheduler.db" + ) as runtime: + scope = await _scope(runtime) + assert runtime.artifact_merge is not None + merge = runtime.artifact_merge.for_scope(scope, "experience") + a, b = await _create(runtime, scope, "canary"), await _create(runtime, scope, "pause") + with pytest.raises(ArtifactMergeRelationError, match="direct Sources and exact in-Scope Artifacts"): + await merge.merge( + (a.as_read(), b.as_read()), + _content("combined"), + lineage=ArtifactLineage( + artifacts=(a.ref, b.ref), + publication_source=ArtifactAddress(scope_id=scope, artifact=a.ref), + publication_digest="a" * 64, + ), + ) + assert await merge.get(a.ref.artifact_id) == a + assert await merge.get(b.ref.artifact_id) == b + assert ( + len((await runtime.records.for_scope(scope).query_artifacts("experience", limit=20, cursor=None)).items) + == 2 + ) + + asyncio.run(scenario()) diff --git a/tests/e2e/test_artifact_merge_registration.py b/tests/e2e/test_artifact_merge_registration.py new file mode 100644 index 0000000000..b6e4171668 --- /dev/null +++ b/tests/e2e/test_artifact_merge_registration.py @@ -0,0 +1,119 @@ +# Copyright (c) 2026 OceanBase. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Constructor registration admits an ordinary test Family to the public merge API.""" + +from __future__ import annotations + +import asyncio + +import pytest + +from powercontext.builtin.artifacts.merge import ArtifactMergeService +from powercontext.builtin.artifacts.merge_restoration import ArtifactMergePreviewSigner +from powercontext.builtin.persistence.sources import SourceRepository +from powercontext.builtin.records import BaseOperationNotSupportedError +from powercontext.builtin.runtime import ArtifactMergeApplication +from powercontext.builtin.sources.content import CONTENT_SOURCE_ADAPTER +from tests.builtin.persistence.contract import HandoffContent, HandoffDraft, repository_profile + + +class _LocalSecurity: + def subject(self, context): + return context + + async def lock_transaction(self, connection, scope_id, context): + pass + + async def authorize(self, connection, scope_id, context, action, ref=None): + pass + + async def authorize_sources(self, connection, scope_id, context, sources): + pass + + async def establish_owner(self, connection, scope_id, artifact_id, context): + pass + + +class _TestFamilyAdapter: + family = "handoff" + + def draft(self, content, lineage, *, merge_inputs=(), historical=False): + return HandoffDraft( + content=HandoffContent.model_validate(content), sources=lineage.sources, artifacts=lineage.artifacts + ) + + async def prepare(self, content): + return content + + def validate_prepared(self, prepared): + pass + + async def publish(self, connection, scope_id, record, prepared, execution_context): + pass + + async def remove(self, connection, scope_id, artifact_id): + pass + + +async def _tags(connection, scope_id, artifact_id, inputs): + pass + + +def test_constructor_registration_reuses_shared_merge_preview_and_restore() -> None: + async def scenario() -> None: + async with repository_profile() as (profile, repositories): + service = ArtifactMergeService( + artifacts=repositories.artifacts, + sources=SourceRepository((CONTENT_SOURCE_ADAPTER,)), + adapter=_TestFamilyAdapter(), + security=_LocalSecurity(), + merge_tags=_tags, + preview_signer=ArtifactMergePreviewSigner(keys={"test": b"x" * 32}, active_key_id="test"), + ) + application = ArtifactMergeApplication( + profile.database, + (service,), + default_context="local-owner", + id_factory=lambda family: family + "-result", + ) + scoped = application.for_scope("project", "handoff") + with pytest.raises(BaseOperationNotSupportedError): + application.for_scope("project", "experience") + async with profile.database.transaction() as connection: + for identity in ("a", "b"): + await repositories.artifacts.create( + connection, "project", identity, HandoffDraft(content=HandoffContent(summary=identity)) + ) + a, b = await scoped.get("a"), await scoped.get("b") + result = (await scoped.merge((a.as_read(), b.as_read()), HandoffContent(summary="combined"))).primary + assert result.ref.artifact_id == "handoff-result" + assert (await scoped.get("a", revision=1)).artifact == a.artifact + assert (await scoped.get("a")).state.merged_into_id == result.ref.artifact_id + preview = await scoped.preview_restoration(result.ref.artifact_id, operation="undo_merge") + restored = await scoped.restore( + result.ref.artifact_id, operation="undo_merge", preview_token=preview.preview_token + ) + assert restored.undo_merge_results == (result.ref.artifact_id,) + for original in (a, b): + current = await scoped.get(original.ref.artifact_id) + assert current.artifact.content == original.artifact.content + assert current.ref.revision == 2 + assert (await scoped.get(original.ref.artifact_id, revision=1)).artifact == original.artifact + assert (await scoped.get(result.ref.artifact_id)).state.lifecycle_state == "retired" + assert restored.primary.ref.revision == 2 + outcome = await scoped.restoration_outcome(result.ref.artifact_id, revision=2) + assert outcome is not None and outcome.undo_merge_results == (result.ref,) + + asyncio.run(scenario()) diff --git a/tests/e2e/test_atomic_memory_cascade_restoration.py b/tests/e2e/test_atomic_memory_cascade_restoration.py index a0641f90c0..e71b0af6d6 100644 --- a/tests/e2e/test_atomic_memory_cascade_restoration.py +++ b/tests/e2e/test_atomic_memory_cascade_restoration.py @@ -135,23 +135,36 @@ async def scenario() -> None: assert result.changed assert result.undo_merge_results == (e.ref.artifact_id, c.ref.artifact_id) assert {_ref_identity(ref) for ref in result.restored} == { - _ref_identity(record.ref) for record in (a, b, d) + (record.ref.family, record.ref.artifact_id, record.ref.revision + 1) for record in (a, b, d) + } + assert {_ref_identity(ref) for ref in result.retired} == { + (record.ref.family, record.ref.artifact_id, record.ref.revision + 1) for record in (c, e) } - assert {_ref_identity(ref) for ref in result.retired} == {_ref_identity(c.ref), _ref_identity(e.ref)} assert {record.ref.artifact_id for record in result.records} == { record.ref.artifact_id for record in (a, b, c, d, e) } + current_records = {} + restored_inputs = [] for frozen in (a, b, d): restored = await memory.get(frozen.ref.artifact_id) - assert restored.artifact == frozen.artifact + assert restored.ref.revision == frozen.ref.revision + 1 + assert restored.artifact.content == frozen.artifact.content.without_creation() + assert restored.artifact.lineage.artifacts == (frozen.ref,) assert restored.state.state == "active" assert restored.state.merged_into_id is None + current_records[restored.ref.artifact_id] = restored + restored_inputs.append(restored) for merged in (c, e): retired = await memory.get(merged.ref.artifact_id) - assert retired.artifact == merged.artifact + assert retired.ref.revision == merged.ref.revision + 1 + assert retired.artifact.content == merged.artifact.content.without_creation() + assert retired.artifact.lineage.artifacts == (merged.ref,) assert retired.state.state == "retired" assert retired.state.merged_into_id is None - await _assert_active_visibility(memory, (a, b, d)) + current_records[retired.ref.artifact_id] = retired + assert {record.ref.artifact_id: record for record in result.records} == current_records + assert result.primary == current_records[result.primary_artifact_id] + await _assert_active_visibility(memory, tuple(restored_inputs)) await _assert_immutable_history(contexts, chain) asyncio.run(scenario()) @@ -181,9 +194,11 @@ async def scenario() -> None: assert result.undo_merge_results == (e.ref.artifact_id,) assert {_ref_identity(ref) for ref in result.restored} == { _ref_identity(restored.ref), - _ref_identity(d.ref), + (d.ref.family, d.ref.artifact_id, d.ref.revision + 1), } - assert result.retired == (e.ref,) + assert result.retired == ( + ArtifactRef(family=e.ref.family, artifact_id=e.ref.artifact_id, revision=e.ref.revision + 1), + ) assert {record.ref.artifact_id for record in result.records} == { record.ref.artifact_id for record in (c, d, e) } @@ -193,13 +208,21 @@ async def scenario() -> None: assert current.state.state == "merged" assert current.state.merged_into_id == c.ref.artifact_id current_d = await memory.get(d.ref.artifact_id) - assert current_d.artifact == d.artifact + assert current_d.ref.revision == d.ref.revision + 1 + assert current_d.artifact.content == d.artifact.content.without_creation() + assert current_d.artifact.lineage.artifacts == (d.ref,) assert current_d.state.state == "active" assert current_d.state.merged_into_id is None retired = await memory.get(e.ref.artifact_id) - assert retired.artifact == e.artifact + assert retired.ref.revision == e.ref.revision + 1 + assert retired.artifact.content == e.artifact.content.without_creation() + assert retired.artifact.lineage.artifacts == (e.ref,) assert retired.state.state == "retired" - await _assert_active_visibility(memory, (restored, d)) + assert retired.state.merged_into_id is None + assert {record.ref.artifact_id: record for record in result.records} == { + record.ref.artifact_id: record for record in (restored, current_d, retired) + } + await _assert_active_visibility(memory, (restored, current_d)) await _assert_immutable_history(contexts, chain) asyncio.run(scenario()) diff --git a/tests/e2e/test_atomic_memory_generation_merge.py b/tests/e2e/test_atomic_memory_generation_merge.py index e0fb9be354..4e0013409d 100644 --- a/tests/e2e/test_atomic_memory_generation_merge.py +++ b/tests/e2e/test_atomic_memory_generation_merge.py @@ -19,6 +19,8 @@ import asyncio from pathlib import Path +import pytest + from powercontext.builtin.artifacts.atomic_memory.extraction import ( AtomicMemoryCandidate, AtomicMemoryEvidence, @@ -467,3 +469,52 @@ async def scenario() -> None: ) asyncio.run(scenario()) + + +def test_failed_merge_projection_rolls_back_entire_source_window_and_recomputes(tmp_path: Path, monkeypatch) -> None: + pipeline = AtomicMemoryGenerationPipeline( + extractor=_PolicyExtractor(), reconciler=_PolicyReconciler(), estimator=character_token_estimator() + ) + config = BuiltinConfig( + database=SQLiteConfig(url=f"sqlite+aiosqlite:///{tmp_path / 'merge-rollback.db'}"), + runtime=RuntimeConfig(atomic_memory_related_mode="fts"), + ) + + async def scenario() -> None: + async with open_builtin_contexts(config, candidate_pipeline=pipeline) as contexts: + context = await contexts.get(SCOPE) + application = contexts.atomic_memory + memory = application.for_scope(SCOPE) + for source_id, text in (("canary-rule", CANARY), ("pause-rule", PAUSE)): + await contexts.records.capture_source(SCOPE, "content", source_id, text, {}) + await context.triggers.flush(limit=1) + originals = (await memory.list()).items + await contexts.records.capture_source(SCOPE, "content", "consolidate-rules", CONSOLIDATION, {}) + remove = application.publisher.remove + + async def fail_projection(connection, scope_id, artifact_id): + await remove(connection, scope_id, artifact_id) + if artifact_id == originals[-1].ref.artifact_id: + raise RuntimeError("projection publication failed") # noqa: TRY003 + + with monkeypatch.context() as patch: + patch.setattr(application.publisher, "remove", fail_projection) + with pytest.raises(RuntimeError, match="projection publication failed"): + await context.triggers.flush(limit=1) + assert (await context.triggers.cursor()).sequence == 2 + assert (await memory.list(states=("active", "forgotten", "merged", "retired"))).items == originals + assert { + (hit.hit.artifact_ref.artifact_id, hit.hit.artifact_ref.revision) + for hit in (await memory.search("Database rollout")).hits + } == {(original.ref.artifact_id, original.ref.revision) for original in originals} + retry = await context.triggers.flush(limit=1) + assert retry.previous_cursor == 2 and retry.current_cursor == 3 + (result,) = (await memory.list()).items + assert result.artifact.content.text == POLICY + assert result.artifact.lineage.sources == (SourceRef(source_type="content", source_id="consolidate-rules"),) + for original in originals: + frozen = await memory.get(original.ref.artifact_id) + assert frozen.state.merged_into_id == result.ref.artifact_id + assert frozen.artifact == original.artifact + + asyncio.run(scenario()) diff --git a/tests/e2e/test_atomic_memory_grant_migration.py b/tests/e2e/test_atomic_memory_grant_migration.py index bbba5b71a9..5d20b82bd0 100644 --- a/tests/e2e/test_atomic_memory_grant_migration.py +++ b/tests/e2e/test_atomic_memory_grant_migration.py @@ -16,10 +16,12 @@ import asyncio import json +import sqlite3 from contextlib import AsyncExitStack from dataclasses import replace from datetime import UTC, datetime from hashlib import sha256 +from pathlib import Path from typing import Any, cast import httpx @@ -36,8 +38,10 @@ apply_atomic_memory_migration, plan_atomic_memory_migration, verify_atomic_memory_migration, + verify_atomic_memory_migration_authority, ) from powercontext.builtin.persistence.schema import create_tables +from powercontext.builtin.persistence.sqlite import SQLiteConfig, SQLiteProfile from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, ARTIFACTS_TABLE, MEMORY_ENTRY_VERSIONS_TABLE from powercontext.builtin.runtime import BuiltinConfig, open_builtin_runtime from powercontext.builtin.runtime.config import RuntimeConfig @@ -98,7 +102,12 @@ async def _execute(migration_backend: MigrationBackend, statement: str, values: async def _seed_legacy( - migration_backend: MigrationBackend, scope_id: str, *, revoked: bool = False, replacement: bool = False + migration_backend: MigrationBackend, + scope_id: str, + *, + revoked: bool = False, + replacement: bool = False, + artifact_evidence: bool = False, ): resources = tuple( ResourceRef.artifact( @@ -106,11 +115,14 @@ async def _seed_legacy( ) for entry_id in ("legacy-entry-1", "legacy-entry-2") ) + evidence = ( + [{"family": "experience", "artifact_id": "migration-evidence", "revision": 1}] if artifact_evidence else [] + ) versions = [] for ordinal, resource in enumerate(resources, 1): assert resource.selector is not None entry_id = resource.selector.entry_id - value = {"kind": "fact", "text": f"Legacy fact {ordinal}.", "source_refs": [], "artifact_refs": []} + value = {"kind": "fact", "text": f"Legacy fact {ordinal}.", "source_refs": [], "artifact_refs": evidence} versions.append({ **value, "entry_id": entry_id, @@ -142,6 +154,33 @@ async def _seed_legacy( migration_profile(migration_backend.config) as profile, profile.database.transaction() as connection, ): + if artifact_evidence: + await connection.execute( + insert(ARTIFACTS_TABLE).values( + scope_id=scope_id, + family="experience", + artifact_id="migration-evidence", + revision=1, + content=json.dumps({ + "situation": "Preserve migration evidence.", + "action": "Verify exact imports.", + "outcome": "Evidence remains readable.", + "lesson": "Keep lineage separate from merge inputs.", + }).encode(), + ) + ) + await connection.execute( + insert(ARTIFACT_HEADS_TABLE).values( + scope_id=scope_id, + family="experience", + artifact_id="migration-evidence", + revision=1, + searchable_text=None, + lifecycle_state="active", + replacement_artifact_id=None, + governance_generation=0, + ) + ) # Frozen legacy records exercise conversion independently of the current Memory models. await create_tables(connection, (MEMORY_ENTRY_VERSIONS_TABLE,)) await connection.execute( @@ -173,7 +212,7 @@ async def _seed_legacy( "(:scope, 'memory', 'memory', :entry_id, :entry_version_id, 1, NULL, :kind, :text, " ":source_refs, :artifact_refs, :entry_content_hash, 1)" ), - {**version, "scope": scope_id, "source_refs": b"[]", "artifact_refs": b"[]"}, + {**version, "scope": scope_id, "source_refs": b"[]", "artifact_refs": json.dumps(evidence).encode()}, ) repository = RelationalAccessRepository(profile.database, connection=connection) for resource in resources: @@ -480,6 +519,130 @@ async def verify() -> None: asyncio.run(scenario()) +def test_offline_inspection_and_apply_upgrade_released_common_columns(tmp_path: Path) -> None: + config = SQLiteConfig(url=f"sqlite+aiosqlite:///{tmp_path / 'migration.db'}") + backend = MigrationBackend(config, tmp_path) + _prepare(backend, artifact_evidence=True) + database = tmp_path / "migration.db" + with sqlite3.connect(database) as connection: + for table, columns in ( + ( + "pc_artifact_heads", + "scope_id, family, artifact_id, revision, searchable_text, lifecycle_state, " + "replacement_artifact_id, governance_generation", + ), + ( + "pc_artifact_lineage_artifacts", + "scope_id, family, artifact_id, revision, ordinal, " + "upstream_family, upstream_artifact_id, upstream_revision", + ), + ): + connection.execute(f"CREATE TABLE released AS SELECT {columns} FROM {table}") # noqa: S608 + connection.execute(f"DROP TABLE {table}") + connection.execute(f"ALTER TABLE released RENAME TO {table}") + if table == "pc_artifact_heads": + connection.execute( + "CREATE UNIQUE INDEX released_head_identity ON pc_artifact_heads(scope_id, family, artifact_id)" + ) + connection.execute("PRAGMA journal_mode = DELETE") + original = database.read_bytes() + + async def inspect() -> None: + async with ( + SQLiteProfile.open_readonly(config) as profile, + profile.database.transaction() as connection, + ): + plan = await plan_atomic_memory_migration(connection) + assert plan.counts["pending_entries"] == 2 + assert not plan.ready + for report in ( + await verify_atomic_memory_migration(connection), + await verify_atomic_memory_migration_authority(connection), + ): + assert not report.ready + assert any("merge columns" in error for error in report.errors), report.errors + + asyncio.run(inspect()) + assert database.read_bytes() == original + result = asyncio.run(_apply(backend)) + assert result.ready, result.errors + assert result.counts["imported_entries"] == 2 + with sqlite3.connect(database) as connection: + heads = connection.execute( + "SELECT lifecycle_state, governance_generation, merged_into_id, revision " + "FROM pc_artifact_heads WHERE family = 'atomic-memory' ORDER BY artifact_id" + ).fetchall() + assert heads == [("active", 0, None, 1), ("active", 0, None, 1)] + assert connection.execute( + "SELECT DISTINCT is_merge_input FROM pc_artifact_lineage_artifacts WHERE family = 'atomic-memory'" + ).fetchall() == [(0,)] + assert ( + connection.execute("SELECT name FROM sqlite_master WHERE name = 'pc_atomic_memory_states'").fetchone() + is None + ) + + +def test_migration_replay_preserves_evolved_head_and_imported_history(tmp_path: Path) -> None: + config = SQLiteConfig(url=f"sqlite+aiosqlite:///{tmp_path / 'migration.db'}") + backend = MigrationBackend(config, tmp_path) + _prepare(backend) + assert asyncio.run(_apply(backend)).ready + with sqlite3.connect(tmp_path / "migration.db") as connection: + target = connection.execute( + "SELECT artifact_id FROM pc_artifact_heads WHERE family = 'atomic-memory' ORDER BY artifact_id LIMIT 1" + ).fetchone()[0] + connection.execute( + "UPDATE pc_artifact_heads SET lifecycle_state = 'deprecated', governance_generation = 4 " + "WHERE family = 'atomic-memory' AND artifact_id = ?", + (target,), + ) + connection.execute("DELETE FROM pc_atomic_memory_current WHERE artifact_id = ?", (target,)) + connection.execute("DELETE FROM pc_atomic_memory_current_fts WHERE artifact_id = ?", (target,)) + retained = { + table: _snapshot(backend, table) + for table in ( + "pc_artifacts", + "pc_artifact_heads", + "pc_artifact_lineage_artifacts", + "pc_access_owners", + "pc_artifact_tags", + "pc_access_relationships", + "pc_access_idempotency", + ) + } + repeated = asyncio.run(_apply(backend)) + assert repeated.ready, repeated.errors + assert repeated.counts["imported_entries"] == 0 + for table, rows in retained.items(): + assert _snapshot(backend, table) == rows + + +def test_verification_rejects_imported_evidence_marked_as_merge_input(tmp_path: Path) -> None: + config = SQLiteConfig(url=f"sqlite+aiosqlite:///{tmp_path / 'migration.db'}") + backend = MigrationBackend(config, tmp_path) + _prepare(backend, artifact_evidence=True) + assert asyncio.run(_apply(backend)).ready + with sqlite3.connect(tmp_path / "migration.db") as connection: + assert ( + connection.execute( + "SELECT COUNT(*) FROM pc_artifact_lineage_artifacts WHERE family = 'atomic-memory' AND is_merge_input = 0" + ).fetchone()[0] + == 2 + ) + connection.execute("UPDATE pc_artifact_lineage_artifacts SET is_merge_input = 1 WHERE family = 'atomic-memory'") + + async def scenario() -> None: + async with ( + SQLiteProfile.open_readonly(config) as profile, + profile.database.transaction() as connection, + ): + report = await verify_atomic_memory_migration(connection) + assert not report.ready + assert any("imported exact evidence differs" in error for error in report.errors), report.errors + + asyncio.run(scenario()) + + def _atomic_payload(payload: dict[str, Any]) -> dict[str, Any]: resource = payload["resource"] artifact_id = legacy_entry_artifact_id( diff --git a/tests/e2e/test_atomic_memory_migration_evidence.py b/tests/e2e/test_atomic_memory_migration_evidence.py index 87c7003357..1f2f5bdd07 100644 --- a/tests/e2e/test_atomic_memory_migration_evidence.py +++ b/tests/e2e/test_atomic_memory_migration_evidence.py @@ -270,6 +270,8 @@ async def scenario() -> None: def test_merge_keeps_migrated_input_history_without_unrelated_collection_sources(migrated) -> None: async def scenario() -> None: async with open_builtin_contexts(BuiltinConfig(database=migrated[0])) as contexts: + async with contexts.database.transaction() as connection: + imported = await _imported_snapshot(connection) memory = contexts.atomic_memory.for_scope(SCOPE) alpha = await memory.get(_atomic("alpha", 2).artifact_id) (created,) = await contexts.records.create_atomic_memories( @@ -283,6 +285,15 @@ async def scenario() -> None: resolved = await _resolver(contexts).resolve(connection, artifacts=(merged.primary.ref,)) assert _projected_sources(resolved) == (A, C) assert any(node.artifact == alpha.ref and node.historical for node in resolved.manifest.nodes) + heads = tuple((await connection.execute(select(ARTIFACT_HEADS_TABLE))).all()) + repeated = await apply_atomic_memory_migration( + contexts.database, contexts.atomic_memory.index, maintenance_confirmed=True + ) + assert repeated.ready, repeated.errors + assert repeated.counts["imported_entries"] == 0 + async with contexts.database.transaction() as connection: + assert tuple((await connection.execute(select(ARTIFACT_HEADS_TABLE))).all()) == heads + assert await _imported_snapshot(connection) == imported asyncio.run(scenario()) diff --git a/tests/e2e/test_atomic_memory_normalization.py b/tests/e2e/test_atomic_memory_normalization.py index 52600f8299..c62b4f0877 100644 --- a/tests/e2e/test_atomic_memory_normalization.py +++ b/tests/e2e/test_atomic_memory_normalization.py @@ -245,7 +245,7 @@ def snapshot() -> dict[str, list[tuple[object, ...]]]: for table in ( "pc_artifacts", "pc_artifact_heads", - "pc_atomic_memory_states", + "pc_artifact_lineage_artifacts", "pc_atomic_memory_current", ) } diff --git a/tests/e2e/test_atomic_memory_projection_rebuild.py b/tests/e2e/test_atomic_memory_projection_rebuild.py index 20d3a869bd..3075699659 100644 --- a/tests/e2e/test_atomic_memory_projection_rebuild.py +++ b/tests/e2e/test_atomic_memory_projection_rebuild.py @@ -116,7 +116,7 @@ async def scenario() -> None: @pytest.mark.parametrize( "corruption", - ["missing_state", "missing_head", "missing_head_and_state", "head_behind", "summary", "content"], + ["missing_head", "orphan_history", "head_behind", "content"], ) def test_rebuild_rejects_damaged_current_authority_without_legacy_history(tmp_path: Path, corruption: str) -> None: config = SQLiteConfig(url=f"sqlite+aiosqlite:///{tmp_path / 'authority.db'}") @@ -129,30 +129,24 @@ async def scenario() -> None: await connection.execute(text("DROP TABLE pc_memory_entry_heads")) await connection.execute(text("DROP TABLE pc_memory_entry_versions")) identity = {"scope": SCOPE, "id": _atomic("alpha", 2).artifact_id} - if corruption == "missing_state": - statement = "DELETE FROM pc_atomic_memory_states WHERE scope_id = :scope AND artifact_id = :id" - elif corruption in {"missing_head", "missing_head_and_state"}: - if corruption == "missing_head_and_state": + if corruption in {"missing_head", "orphan_history"}: + if corruption == "orphan_history": await connection.execute( - text("DELETE FROM pc_atomic_memory_states WHERE scope_id = :scope AND artifact_id = :id"), + text("DELETE FROM pc_atomic_memory_current WHERE scope_id = :scope AND artifact_id = :id"), identity, ) statement = "DELETE FROM pc_artifact_heads WHERE scope_id = :scope AND family = 'atomic-memory' AND artifact_id = :id" elif corruption == "head_behind": statement = "UPDATE pc_artifact_heads SET revision = 1 WHERE scope_id = :scope AND family = 'atomic-memory' AND artifact_id = :id" - elif corruption == "summary": - statement = "UPDATE pc_artifact_heads SET governance_generation = governance_generation + 1 WHERE scope_id = :scope AND family = 'atomic-memory' AND artifact_id = :id" else: statement = "UPDATE pc_artifacts SET content = :bad WHERE scope_id = :scope AND family = 'atomic-memory' AND artifact_id = :id AND revision = 2" identity["bad"] = b"{}" await connection.execute(text(statement), identity) before = (await connection.execute(select(ARTIFACTS_TABLE))).all() expected = { - "missing_state": "orphan head or Family state", - "missing_head": "orphan head or Family state", - "missing_head_and_state": "history has no head", + "missing_head": "history has no head", + "orphan_history": "history has no head", "head_behind": "head does not select its latest revision", - "summary": "governance summary disagree", "content": "invalid stored artifact payload", } with pytest.raises( diff --git a/tests/e2e/test_observability.py b/tests/e2e/test_observability.py index a8b8f4b53f..5371b51e59 100644 --- a/tests/e2e/test_observability.py +++ b/tests/e2e/test_observability.py @@ -373,7 +373,8 @@ def test_database_failure_log_does_not_include_memory_content(caplog, tmp_path) with sqlite3.connect(database_path) as connection: connection.executescript(""" CREATE TRIGGER reject_memory_insert - BEFORE INSERT ON pc_atomic_memory_states + BEFORE INSERT ON pc_artifact_heads + WHEN NEW.family = 'atomic-memory' BEGIN SELECT RAISE(ABORT, 'forced persistence failure'); END; @@ -639,7 +640,8 @@ def test_memory_commit_failure_is_traced_and_rolls_back(tmp_path) -> None: with sqlite3.connect(database_path) as connection: connection.executescript(""" CREATE TRIGGER reject_memory_insert - BEFORE INSERT ON pc_atomic_memory_states + BEFORE INSERT ON pc_artifact_heads + WHEN NEW.family = 'atomic-memory' BEGIN SELECT RAISE(ABORT, 'forced Memory commit failure'); END; diff --git a/tests/e2e/test_prompt_management_live.py b/tests/e2e/test_prompt_management_live.py index 920fc0f592..fa12d7a017 100644 --- a/tests/e2e/test_prompt_management_live.py +++ b/tests/e2e/test_prompt_management_live.py @@ -27,7 +27,6 @@ from sqlalchemy import delete, select, text from sqlalchemy.sql.ddl import sort_tables -from powercontext.builtin.persistence.atomic_memory_schema import ATOMIC_MEMORY_TABLES from powercontext.builtin.persistence.sqlite import SQLiteConfig from powercontext.builtin.persistence.tables import BUILTIN_TABLES from powercontext.builtin.runtime import CaptureSource @@ -100,7 +99,6 @@ async def _cleanup(contexts: RelationalContexts, scopes: list[str]) -> None: tables = sort_tables(( *BUILTIN_TABLES, *ACCESS_TABLES, - *ATOMIC_MEMORY_TABLES, *contexts.atomic_memory.index.tables, )) scoped_tables = [table for table in tables if "scope_id" in table.c] diff --git a/tests/e2e/test_recall_sufficiency_gate.py b/tests/e2e/test_recall_sufficiency_gate.py index df573e9722..badb19d510 100644 --- a/tests/e2e/test_recall_sufficiency_gate.py +++ b/tests/e2e/test_recall_sufficiency_gate.py @@ -47,9 +47,10 @@ prepare_topic_memory_projection, ) from powercontext.builtin.inference import EmbeddingResult, InferenceUnavailableError +from powercontext.builtin.persistence.records import RelationalRecordService from powercontext.builtin.persistence.sqlite import SQLiteConfig from powercontext.builtin.persistence.statistics import StatisticsRepository -from powercontext.builtin.persistence.tables import ARTIFACTS_TABLE, MEMORY_ENTRY_HEADS_TABLE, RECALL_EFFORT_DAILY_TABLE +from powercontext.builtin.persistence.tables import ARTIFACT_HEADS_TABLE, ARTIFACTS_TABLE, RECALL_EFFORT_DAILY_TABLE from powercontext.builtin.runtime import ( BuiltinConfig, PrepareContextRequest, @@ -131,6 +132,8 @@ async def counted( reuse=reuse, topic_reuse=topic_reuse, ) + log.calls[-1]["memory_recoverable"] = result.memory_recoverable + log.calls[-1]["memory_texts"] = tuple(hit.text for group in result.memory for hit in group.hits) if log.force_recoverable_family is None: return result return _RecallRoundOutcome( @@ -236,7 +239,7 @@ async def _revision_state(runtime: BuiltinRuntime, scope_id: str) -> tuple[list[ provider = cast(Any, runtime._provider) async with provider.database.transaction() as connection: snapshots = [] - for table in (ARTIFACTS_TABLE, MEMORY_ENTRY_HEADS_TABLE): + for table in (ARTIFACTS_TABLE, ARTIFACT_HEADS_TABLE): snapshots.append((await connection.execute(select(table).where(table.c.scope_id == scope_id))).all()) return tuple(snapshots) @@ -314,25 +317,26 @@ async def scenario() -> None: assert effort is not None assert effort.assessment == "weak-top-1" assert effort.rounds == 2 + assert effort.candidates_by_round == (1, 2) assert effort.signals is not None assert effort.signals.top_score == pytest.approx(0.31, abs=0.005) assert len(log.calls) == 2 assert "unrelated vector evidence" in (build.context.content or "") # Atomic Memory reports recoverability as an existence probe, not pre-admission counts. - assert all(count.family != MEMORY_FAMILY for count in effort.admission_by_family) + assert effort.admission_by_family == () + assert [call["memory_recoverable"] for call in log.calls] == [True, False] asyncio.run(scenario()) -def test_strong_vector_hit_is_delivered_beside_a_window_of_weak_dual_channel_hits(tmp_path, monkeypatch) -> None: - """Atomic Memory qualifies and limits each channel before fusion, so a strong vector-only - hit competes in the fused window with weak dual-channel hits; the gate scores its cosine - relevance rather than its fused rank, and the committed candidates deliver it.""" +def test_bounded_expansion_surfaces_a_strong_hit_hidden_by_the_fusion_limit(tmp_path, monkeypatch) -> None: + """The fused window is capped before the gate sees hits, so a weak top cannot be + declared unimprovable: a strong vector-only hit outside the window gains an FTS + channel under the lowered floor and RRF moves it into view.""" embedding_profile = EmbeddingProfile( profile_id="recall-gate-hidden", model="deterministic", dimension=2, distance="l2", normalization="unit" ) - # One fewer weak hit than the window, so a fused-rank tie cannot cut the strong hit. - weak_texts = tuple(f"alpha beta gamma weak {index:02d}" for index in range(15)) + weak_texts = tuple(f"beta gamma weak {index:02d}" for index in range(16)) strong_text = "alpha window hidden" cosines = {_QUERY: 1.0, strong_text: 0.90} cosines.update({text: 0.31 + index * 0.001 for index, text in enumerate(weak_texts)}) @@ -371,19 +375,37 @@ async def scenario() -> None: embedding_model=DeterministicEmbedding(), ) as runtime: scope_id = await _create_scope(runtime, "hidden-hit") + # Each channel is capped at sixteen. The weakest vector row and the + # strong vector-only row tie for the last fused slot; stable IDs put + # the FTS-only weak row first. Under the lower lexical floor, the + # strong row's rare alpha term ranks first and gives it both channels. + records = cast(RelationalRecordService, runtime._records()) + original_id_factory = records._id_factory + ids = iter(f"hidden-memory-{index:02d}" for index in range(17)) + monkeypatch.setattr( + records, + "_id_factory", + lambda kind: next(ids) if kind == "atomic-memory" else original_id_factory(kind), + ) await _seed(runtime, scope_id, [*weak_texts, strong_text]) _, effort = await _prepare_build(runtime, scope_id, _memory_request()) - # The fused rank of the vector-only hit trails every dual-channel weak hit, but the - # gate reads its 0.90 cosine relevance, so round zero is already sufficient. + # Round 0: fifteen dual-channel and one FTS-only weak hit fill the window and truncate + # the vector-only 0.90 hit, so the gate sees a weak top. Round 1's lower lexical + # floor admits the strong hit's FTS row; the dual-channel hit then enters the + # window, the top score reflects it, and the committed candidates keep the latest + # round's fused order so the surfaced hit is actually delivered to the Builder. assert effort is not None assert effort.assessment == REASON_SUFFICIENT - assert effort.rounds == 1 - assert effort.candidates_by_round == (16,) + assert effort.rounds == 2 + assert effort.candidates_by_round == (16, 17) assert effort.signals is not None assert effort.signals.top_score == pytest.approx(0.90, abs=0.005) - assert len(log.calls) == 1 + assert len(log.calls) == 2 + assert strong_text not in log.calls[0]["memory_texts"] + assert strong_text in log.calls[1]["memory_texts"] assert any(strong_text in hit.text for group in delivered for hit in group.hits) + assert [call["memory_recoverable"] for call in log.calls] == [True, False] asyncio.run(scenario()) @@ -450,7 +472,7 @@ async def scenario() -> None: asyncio.run(scenario()) -def test_scoped_first_hit_rule_applies_per_scope_through_the_runtime(tmp_path) -> None: +def test_scoped_first_hit_rule_applies_per_scope_through_the_runtime(tmp_path, monkeypatch) -> None: embedding_profile = EmbeddingProfile( profile_id="recall-gate-scopes", model="deterministic", dimension=2, distance="l2", normalization="unit" ) @@ -462,6 +484,9 @@ async def embed(self, texts: tuple[str, ...], /) -> EmbeddingResult: vectors = {_QUERY: (1.0, 0.0), "alpha beta gamma vector evidence": (0.31, math.sqrt(1 - 0.31**2))} return EmbeddingResult(vectors=tuple(vectors.get(text, (0.0, 1.0)) for text in map(_embedded_body, texts))) + log = _RecallRoundLog() + log.install(monkeypatch) + async def scenario() -> None: database = tmp_path / "scoped-first-hit.db" async with _runtime( @@ -486,6 +511,7 @@ async def scenario() -> None: # lower round-two floor would add, so expansion stops; the weak-top-1 reason is kept. assert effort.rounds == 2 assert effort.candidates_by_round == (2, 3) + assert [call["memory_recoverable"] for call in log.calls] == [True, False] assert effort.signals is not None assert effort.signals.scored_families == 1 assert effort.signals.top_score == pytest.approx(0.31, abs=0.005) diff --git a/tests/test_atomic_memory_migration_cli.py b/tests/test_atomic_memory_migration_cli.py index adc2130dd7..8ddf8aa57f 100644 --- a/tests/test_atomic_memory_migration_cli.py +++ b/tests/test_atomic_memory_migration_cli.py @@ -163,7 +163,29 @@ async def initialize_authority(): with closing(sqlite3.connect(database)) as connection: assert connection.execute("PRAGMA journal_mode").fetchone()[0] == "wal" tables = {row[0] for row in connection.execute("SELECT name FROM sqlite_master WHERE type = 'table'")} - assert {"pc_atomic_memory_states", "pc_atomic_memory_current", "pc_atomic_memory_current_fts"} <= tables + assert {"pc_atomic_memory_current", "pc_atomic_memory_current_fts"} <= tables + assert "pc_atomic_memory_states" not in tables + + +@pytest.mark.parametrize("action", ["plan", "verify"]) +def test_inspection_rejects_unreleased_atomic_state_table_without_writes( + tmp_path, monkeypatch, maintenance_cli, action +): + database = tmp_path / "unreleased.db" + with closing(sqlite3.connect(database)) as connection: + connection.execute("CREATE TABLE pc_atomic_memory_states(artifact_id TEXT, state TEXT)") + connection.execute("INSERT INTO pc_atomic_memory_states VALUES ('retained', 'merged')") + connection.commit() + original = database.read_bytes() + monkeypatch.setenv("POWERCONTEXT_SERVER_DATABASE_URL", f"sqlite+aiosqlite:///{database}") + + result = CliRunner().invoke(maintenance_cli, ["server", "atomic-memory-migrate", "--action", action]) + + assert result.exit_code == 1, result.output + report = json.loads(result.output) + assert report["ready"] is False + assert any("unsupported unreleased" in error and "pc_atomic_memory_states" in error for error in report["errors"]) + assert database.read_bytes() == original def test_obsolete_development_projection_requires_explicit_rebuild(tmp_path, monkeypatch, maintenance_cli): @@ -192,7 +214,6 @@ async def seed(): for name in [ "pc_artifacts", "pc_artifact_heads", - "pc_atomic_memory_states", "pc_access_owners", "pc_access_relationships", ]