Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions doc/architecture/SPEC/engine/steps/knowledge.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# engine/steps/knowledge.py —— 规范

<!-- verified-against: 2026-09-28 -->
<!-- verified-against: 2026-09-30 -->

`知识服务的 knowledge.* 步骤 · refactor-status: new`

Expand All @@ -12,10 +12,17 @@ auto_merge 仓库写 outbox 的 `open_pr` 项,shadow 只记录)。逻辑全
## 不变量
- 只处理 `params.repo` 指定的一个仓库;未知或未启用的仓库 → BLOCKED。
- 步骤之间只经 `state_updates` 传递纯数据(`kb_repo`、`kb_changeset`、`kb_changeset_status`)。
- 从不直接写 GitHub;发布只经签名 outbox 由 GPU 盒发布器执行。
- 从不直接写 GitHub;发布只经签名 outbox 由 GPU 盒发布器执行(唯一例外:`knowledge.init`,见下)。

## 测试
`test_kb_intake_gate.py`。

## 2026-09-28 合并、巡检、激活步骤
新增 `knowledge.advance_merges`、`knowledge.sweep`、`knowledge.activate`;`kb run` 路径自行持有租约,调度器路径复用其租约。

## 2026-09-30 `knowledge.init`(kb init)
参数 `repo`、`stage`、`dry_run`(缺省 true)、`pin`。用自己的 `InitRuntime`(`kb_service/init_support.py`),**不**经
`_runtime`/`_lifecycle`:服务运行时会打开 `kb.db`,且 `_lifecycle` 拒绝未启用的仓库,而为未启用的仓库建库正是 init 的目的。
调 `init_stages.run_stage`;`InitError`/`NotImplementedError` 与状态 `blocked` 的记录都返回 BLOCKED。`state_updates`:
`kb_init_stage`、`kb_init_status`、`kb_init_pr`。它是本模块里唯一直接写 GitHub 的步骤:经仓库主人的 `gh` 开一个由人合并的
PR,且只在 `ALLOW_PUSH=1` 与 `ALLOW_POST=1` 同时成立、非 dry run、上游非私有时。测试:`test_kb_init_skeleton.py`。
148 changes: 146 additions & 2 deletions doc/architecture/SPEC/kb_service.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions doc/architecture/SPEC/knowledge_service/l1.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
- 共享的 `knowledge/repos/_index.md` 可以被**修改**(登记新仓库),新增/删除仍拒绝。
其余规则(尾注、frontmatter、`sources`、白名单其余部分、保护规则)不变;默认 `False`
时行为与原来逐字节一致。
- `link_targets(directory, text)`:公开的只读帮助函数,返回 `text`(位于 `directory` 的页面)按 CommonMark 渲染后实际链接到的知识库相对路径集合(与 `check_index_links` 同一解析器);kb init 用它确认新索引导语不再链接目录清单已列出的页面。
- `check_index_links(base, head)`:init 侧的索引链接检查(索引改成表格或跨目录链接时
L1 只把它当 prose 块)。**链接 = 渲染器实际生成的链接**:用 markdown-it-py 的 CommonMark 解析器
(开启 GitHub 管道表格)取所有 `link_open` 的 href(图片及其 alt 文本里的不算);先按 URL 拆分(编码的 `#`、`:` 属于文件名),
Expand Down
6 changes: 5 additions & 1 deletion doc/architecture/SPEC/playbooks/PLAYBOOKS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# playbooks/*.yaml —— 规范

<!-- verified-against: 2026-09-29 -->
<!-- verified-against: 2026-09-30 -->

`9 个文件 · 声明式编排数据 · refactor-status: ok`

Expand Down Expand Up @@ -57,3 +57,7 @@ success, steps[]`。

## kb-merge / kb-sweep / kb-activate(candidate)
知识服务的合并推进、发版巡检与快照激活;candidate 状态,由 `kb serve` 调度或 `kb run` 手动运行。

## kb-init(candidate)
为一个仓库建知识库:单步 `knowledge.init`,每次运行一个阶段(skeleton → modules → deepen + 生命周期开关 →
harvest-calibration),每阶段一个由人合并的 PR。只由 `kb init REPO --stage S` 运行,不经 `kb serve`,不写服务账本。
7 changes: 5 additions & 2 deletions doc/architecture/SPEC/profiles/establish.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

## 公开契约
`fact_id`、`build_doc_corpus`、`is_redundant`、`extract_directives`、
`scan_modules`、`scan_modules_at_depth`、`normalize_root`、`ROOT_MODULE`、`HUMAN_DOC_NAMES`。
`scan_modules`、`scan_modules_at_depth`、`normalize_root`、`ROOT_MODULE`、`HUMAN_DOC_NAMES`、`doc_files`。

## 不变量
- `is_redundant`(对 README+docs 做 6 词 shingle)会丢弃任何**仓库自己的文档已经写过**
Expand All @@ -19,6 +19,9 @@
- `scan_modules_at_depth`(kb init 的模块定义):源根以下至多 `depth` 层目录为模块,更深的文件归最深的那层祖先;
非空行少于 `min_loc` 的模块由深到浅折入父目录模块,直到稳定;源根永不折叠(`normalize_root` 先把 `.`、`./pkg`、`/pkg/` 等写法规范化,仓库根为 `""` 且包含其他根,保证折叠必然终止);嵌套源根各自拥有自己的文件;
每一层都跳过隐藏目录与非代码目录;`exclude` 按仓库相对路径做 fnmatch;未知语言返回 `{}`;结果确定、有序。
- `build_doc_corpus(..., globs=None)`:不给 `globs` 时语料与原来完全相同(README* + docs/**/*.md);给出时读
`doc_files(repo, globs)` —— 按 glob 顺序、再按路径排序,每个文件只读一次,只收普通文件,解析后逃出仓库的
符号链接一律跳过。kb init 用它按 adapter 的 `doc_globs` 建 D5 语料,归一化与 briefing 路径是同一份。

## 边界 —— 不属于这里
纯确定性 helper —— 不含 LLM、不写 store、不含 step 逻辑。
Expand All @@ -27,7 +30,7 @@
仅 stdlib。

## 测试
`test_profile_steps.py`(冗余过滤、模块扫描、指令抽取);`test_kb_init_coverage.py`(按深度的模块扫描)。
`test_profile_steps.py`(冗余过滤、模块扫描、指令抽取);`test_kb_init_coverage.py`(按深度的模块扫描);`test_kb_init_skeleton.py`(`doc_files` 与带 `globs` 的语料)。

## 重构备注
纯函数 —— 易测、易复用。冗余过滤器是承载 ETH 研究结论的关键防线;**保持它确定性**。
Expand Down
21 changes: 21 additions & 0 deletions playbooks/kb-init.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# kb init for ONE repository: one stage per run (skeleton, then modules, then
# deepen + lifecycle flip, then harvest-calibration), each opened as a
# human-merged PR (design kb-init v3). Runs outside the knowledge service: it
# never writes the service ledger. Candidate.
name: kb-init
version: 1
status: candidate
provenance:
created_by: human
source: kb init design v3 (bootstrap a repository's knowledge base)
task_kinds: [knowledge_maintenance]
repos: []
requires: []
params:
repo: "knowledge repository name"
stage: "skeleton | modules | deepen | harvest-calibration"
dry_run: "true (default): write the tree and PR body locally; false needs ALLOW_PUSH=1 and ALLOW_POST=1"
pin: "upstream commit to pin (default: the default branch head)"
steps:
- {id: init, step: knowledge.init}
success: the stage's PR is open (or its dry run is written), or the record says why it is blocked
37 changes: 37 additions & 0 deletions src/infermatrix_copilot/engine/steps/knowledge.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,40 @@ async def activate_snapshot(ctx: StepContext) -> StepResult:
return StepResult(False, FailureKind.ESCALATE, f"activation refused: {exc}")
return StepResult(True, summary=f"active snapshot {sha}",
outputs={"state_updates": {"kb_active_snapshot": sha, "kb_snapshot_path": str(snapshot)}})


def _init_runtime(ctx: StepContext):
"""kb init's own runtime: never the service's (that opens kb.db), and it
serves disabled repositories — bootstrapping one is the point."""
from pathlib import Path

from ...kb_service.cli import DEFAULT_STATE_DIR
from ...kb_service.init_support import InitRuntime

return InitRuntime.from_env(ctx.settings, state_dir=Path(_STATE_DIR or DEFAULT_STATE_DIR))


@step("knowledge.init", kind="agent", risk="knowledge",
description="kb init: bootstrap one repository's knowledge base, one human-merged stage at a time")
async def init_stage(ctx: StepContext) -> StepResult:
from ...kb_service.init_stages import run_stage
from ...kb_service.init_support import InitError

repo = str(ctx.params.get("repo") or ctx.state.get("kb_repo") or "")
stage = str(ctx.params.get("stage") or "")
dry_run = str(ctx.params.get("dry_run", "true")).lower() not in ("0", "false", "no")
pin = str(ctx.params.get("pin") or "") or None
try:
rt = _init_runtime(ctx)
lifecycle = rt.registry.get(repo)
if lifecycle is None:
return StepResult(False, FailureKind.BLOCKED, f"no adapter declares knowledge repo {repo!r}")
record = run_stage(rt, lifecycle, stage, dry_run=dry_run, pin=pin)
except (InitError, NotImplementedError) as exc:
return StepResult(False, FailureKind.BLOCKED, str(exc))
updates = {"kb_init_stage": stage, "kb_init_status": record.status, "kb_init_pr": dict(record.pr)}
if record.status == "blocked":
return StepResult(False, FailureKind.BLOCKED, "; ".join(record.problems)[:2000],
outputs={"state_updates": updates})
return StepResult(True, summary=f"kb init {stage} for {repo}: {record.status}",
outputs={"state_updates": updates})
45 changes: 45 additions & 0 deletions src/infermatrix_copilot/kb_service/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@
kb traces [--kind K] [--changeset ID] [--rule ID] [--limit N] query trace/1 records
kb replay --record ID --model PROVIDER:MODEL[:EFFORT] re-ask a recorded call
kb export --out FILE [--role judge|generator] dataset (calibration-safe)
kb init REPO --stage STAGE [--dry-run] [--pin SHA]
bootstrap a repository's knowledge base, one
human-merged stage at a time (skeleton, modules,
deepen, harvest-calibration); never touches kb.db
kb init REPO --suggest-seeds rank existing knowledge pages worth seeding from (no model call)
kb publish (--remote HOST:/STATE_DIR | --local DIR) [--once]
the publisher (GPU box, owner's gh login): perform
the signed outbox items; writes need ALLOW_POST=1
Expand Down Expand Up @@ -103,6 +108,36 @@ def _refresh(state_dir: Path, ledger, registry) -> None:
_outbox(state_dir, ledger).transition(lambda: None)


def _init_command(args, state_dir: Path) -> int:
from ..config import Settings

if args.suggest_seeds == bool(args.stage):
print("kb init: pass exactly one of --stage or --suggest-seeds", file=sys.stderr)
return 2
if args.suggest_seeds:
from .init_stages import suggest_seeds
from .init_support import InitError, InitRuntime

try:
rt = InitRuntime.from_env(Settings(), state_dir=state_dir)
lifecycle = rt.registry.get(args.repo)
if lifecycle is None:
print(f"no adapter declares knowledge repo {args.repo!r}", file=sys.stderr)
return 2
for path, score, shared in suggest_seeds(rt, lifecycle):
print(f"{score:6.3f} {path} ({', '.join(shared[:8])})")
except InitError as exc:
print(str(exc), file=sys.stderr)
return 1
return 0
from .runner import run_playbook

params = {"stage": args.stage, "dry_run": "true" if args.dry_run else "false", "pin": args.pin or ""}
outcome, run_dir = run_playbook(Settings(), "kb-init", args.repo, state_dir=state_dir, params=params)
print(f"kb init {args.repo} {args.stage}: {outcome.status} ({run_dir})")
return 0 if outcome.status == "done" else 1


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="infermatrix-copilot kb")
parser.add_argument("--state-dir", help="default: $KB_STATE_DIR or ~/.infermatrix-copilot/kb")
Expand Down Expand Up @@ -142,6 +177,13 @@ def main(argv: list[str] | None = None) -> int:
export = sub.add_parser("export")
export.add_argument("--out", required=True)
export.add_argument("--role", default="judge", choices=("judge", "generator"))
init = sub.add_parser("init")
init.add_argument("repo")
init.add_argument("--stage", choices=("skeleton", "modules", "deepen", "harvest-calibration"))
init.add_argument("--dry-run", action="store_true",
help="write the tree and PR body under the state directory instead of opening a PR")
init.add_argument("--pin", help="upstream commit to pin (default: the default branch head)")
init.add_argument("--suggest-seeds", action="store_true")
publish = sub.add_parser("publish")
where = publish.add_mutually_exclusive_group(required=True)
where.add_argument("--remote", help="host:/absolute/path of the service state directory (over ssh)")
Expand All @@ -163,6 +205,9 @@ def main(argv: list[str] | None = None) -> int:
return _publish(args)
if args.command in {"traces", "replay", "export"}:
return _traces_command(args, _state_dir(args.state_dir))
if args.command == "init":
# before any ledger is opened: kb init never touches the service's kb.db
return _init_command(args, _state_dir(args.state_dir))

state_dir = _state_dir(args.state_dir)
ledger = _ledger(state_dir)
Expand Down
11 changes: 11 additions & 0 deletions src/infermatrix_copilot/kb_service/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
_INIT_KEYS = {
"seeds", "doc_globs", "source_roots", "exclude", "module_depth", "min_module_loc",
"pr_window", "coverage_target", "budget_usd", "judge_call_usd",
"generator_call_usd", "harness_overhead_bytes",
}
# README* + docs/**/*.md is what profiles/establish.build_doc_corpus reads;
# the contribution and agent guides carry most cross-doc invariants.
Expand Down Expand Up @@ -68,6 +69,10 @@ class InitConfig:
coverage_target: float = 0.85
budget_usd: float = 30.0
judge_call_usd: float = 0.50
# per generator call: the spend threshold handed to the transport, and the
# bytes its own harness adds to the prompt (for the one-request overshoot bound)
generator_call_usd: float = 2.0
harness_overhead_bytes: int = 200_000


@dataclass(frozen=True)
Expand Down Expand Up @@ -176,6 +181,9 @@ def parse_init(section: object, where: str, *, repo: str, manifest: dict) -> Ini
judge_call = _number(data.get("judge_call_usd", 0.50), f"{where}.judge_call_usd")
if judge_call < 0:
raise LifecycleConfigError(f"{where}.judge_call_usd must be >= 0")
generator_call = _number(data.get("generator_call_usd", 2.0), f"{where}.generator_call_usd")
if generator_call <= 0:
raise LifecycleConfigError(f"{where}.generator_call_usd must be > 0")
return InitConfig(
seeds=seeds,
doc_globs=(_strings(data["doc_globs"], f"{where}.doc_globs") if "doc_globs" in data
Expand All @@ -190,6 +198,9 @@ def parse_init(section: object, where: str, *, repo: str, manifest: dict) -> Ini
coverage_target=coverage,
budget_usd=budget,
judge_call_usd=judge_call,
generator_call_usd=generator_call,
harness_overhead_bytes=_int(data.get("harness_overhead_bytes", 200_000),
f"{where}.harness_overhead_bytes", minimum=0),
)


Expand Down
Loading
Loading