Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .agents/skills/maintain/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
name: maintain
description: Analyze, fix, validate, and ship Markkit maintenance through green CI and merge. Trigger on run maintenance, maintain, maintainace, or maintaiance; use analysis-only mode only when explicitly requested.
user_invocable: true
metadata:
internal: true
---

# Markkit maintenance

A request to run maintenance authorizes the full outcome: analyze the repository,
fix findings, validate, push, open a PR, resolve review/CI failures, and squash-merge
when every required check is green. A local commit or a list of follow-up issues
is not completion. Explicit analysis-only requests omit edits and shipping.

Read `knowledge/operations/maintenance.md` and execute its checklist, then use the
`ship` skill. Sync from latest `origin/main` before editing. Split independent
fixes into separate small PRs (one for structure/docs, one per behavior fix).

Keep findings in the active pass and fix root causes. A large diff, missing audit,
failed local check, or long build is work to complete, not a reason to defer.
Never weaken tests, resource limits, or `deny.toml` policy to ship an upgrade.
If the newest upstream version breaks a security contract, establish that, keep
the newest safe version with a tested and documented pin, and finish the pass.

Audit CI definitions, not only results: run the Workflow credential audit in
`knowledge/operations/maintenance.md` against `.github/workflows/` (top-level
`permissions`, `persist-credentials: false`, secrets via step `env:` only, no
`pull_request_target`/`workflow_run`/`issue_comment`, no `${{ github.event.* }}`
inside `run:`). A finding there is a fix in this pass.

Smoke test real-world shaped input, not only fixtures: run the CLI on indented,
multi-block HTML and on each office format, and read the output.

Only stop for a missing permission, unavailable service, or user decision that
cannot be resolved within the task. Report evidence and continue independent work.
Never merge red CI or claim success while a required check is unfinished.

Finish by appending a factual entry to the Pass record in
`knowledge/operations/maintenance.md` and a `knowledge/log.md` entry. The final
response names the merged PRs, key fixes, verification, and accepted constraints.
10 changes: 9 additions & 1 deletion .agents/skills/process-issues/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
---
name: process-issues
description: Resolve open GitHub issues for Markkit, one shipped PR per issue. Trigger when user says process issues, work through issues, resolve issues, or handle open issues.
user_invocable: true
metadata:
internal: true
---

Process open GitHub issues and resolve them via merged pull requests.

## Qualifying Issues
Expand Down Expand Up @@ -29,7 +37,7 @@ One issue = one PR. No bundling. No skipping.

### 4. Ship
- Use the `/ship` skill for:
- Spec updates
- Knowledge updates (`knowledge/`)
- Quality gates
- PR creation
- CI verification
Expand Down
172 changes: 111 additions & 61 deletions .agents/skills/ship/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,61 +1,111 @@
Run the full ship flow: verify quality, ensure test coverage, update
artifacts, smoke test, then push, create PR, address review comments,
and merge only when CI is green, all threads resolved, and user confirms.

## Phases

### 1. Pre-flight
- Verify on a feature branch (not main)
- Check for uncommitted changes
- Fetch latest main and rebase if needed

### 2. Test Coverage
- Identify changed code paths
- Ensure tests exist for:
- Positive cases (happy path)
- Negative cases (errors, edge cases)
- Security scenarios (malicious input)
- Add missing tests

### 3. Artifact Updates
- Update specs if behavior changed
- Update CHANGELOG.md for user-facing changes
- Update documentation for API changes

### 4. Simplification
- Remove dead code introduced by changes
- Reduce duplication
- Simplify where possible without changing behavior

### 5. Security Review
- Check for injection risks in new parsing code
- Verify resource limits are enforced
- Review input validation

### 6. Smoke Test
- Run `just test` for full test suite
- Run `just lint` for code quality
- Build CLI and verify basic extraction works

### 7. Quality Gates
- `cargo fmt --all -- --check`
- `cargo clippy --workspace -- -D warnings`
- `cargo deny check`

### 8. Push & PR
- Push branch to remote
- Create or update PR with summary
- Wait for CI to pass

### 9. Review & Merge
- Check for unresolved review comments — address or reply to each
- Ensure all review threads are resolved
- Confirm no changes-requested reviews remain outstanding
- Ask the user whether to merge now or wait for reviewer feedback
- Only merge when: CI green, no unresolved threads, no pending change requests

**IMPORTANT**: Phases 2-5 (tests, artifacts, simplification, security review,
smoke testing) are the quality core — do NOT skip them.

**IMPORTANT**: Never auto-merge without confirming with the user. PRs should
have time for human review before merging.
---
name: ship
description: Run the full Markkit ship flow — verify quality, ensure test coverage, update artifacts, smoke test, push, create PR, and merge when CI is green. Trigger when user says ship, ship it, fix and ship, asks to push and merge a branch, or runs maintenance.
user_invocable: true
metadata:
internal: true
---

Run the full ship flow: verify quality, ensure test coverage, update artifacts,
smoke test, then push, create PR, and merge when CI is green.

Implements the Pre-PR Checklist in `AGENTS.md`. Execute ALL phases, not just push/merge.

## Arguments

- `$ARGUMENTS` - Optional: what is being shipped (PR title/body context, scopes checks)

## Phase 1: Pre-flight

1. Confirm not on `main`
2. Detached `HEAD` with task changes: create a branch first
3. Uncommitted changes belong to the task: keep going. Unrelated: stop and tell the user
4. `git config user.name`/`user.email` are human (see `AGENTS.md` Commit Attribution)

## Phase 2: Test Coverage

Review `git diff origin/main...HEAD` and `git log origin/main..HEAD`:

1. Identify every changed code path
2. `just test`; fix failures
3. Add missing tests:
- **Positive**: happy path, valid input
- **Negative**: invalid input, errors, boundaries, limits
- **Security**: parsing, limits, or URL fetching changes add tests per
`knowledge/security/threat-model.md` and update its Test Coverage map
- **Golden fixtures**: HTML conversion changes add `tests/fixtures/html/<name>.{html,md}`
4. `just test` green

## Phase 3: Artifact Updates

Skip what isn't affected.

1. **Knowledge** (`knowledge/`): update affected concepts in the same change; add a `knowledge/log.md` entry; `just check-okf`
2. **Threat model**: new input surface or limit → new/updated entry, keep `T*`/`S*` ids stable
3. **AGENTS.md**: new commands, knowledge concepts, workflow changes
4. **README.md / docs/**: user-facing behavior, flags, API
5. **Binding READMEs**: exported function changes in `markkit-python` / `markkit-js`
6. **CHANGELOG.md**: Unreleased entry for user-facing changes

## Phase 3b: Simplification

Duplication, deep nesting, dead code, unclear names, needless abstraction. If changed, back to Phase 2.

## Phase 3c: Security Review

1. Untrusted input validated and bounded before use (size, depth, counts)
2. No unbounded allocation driven by attacker-controlled headers or sizes
3. Errors don't leak raw input or internal paths
4. No code execution paths (scripts, macros, formulas, entities)
5. `unsafe` justified and sound

Fix, add regression tests, update the threat model.

## Phase 3d: Design Quality

Real domain types over stringly-typed interfaces; specific error variants; no
compatibility shims (fix call sites); no premature generics; visibility protects invariants.

## Phase 4: Smoke Testing

1. CLI: `just run extract --file <file>` on realistic input (indented HTML, office docs); read the output
2. Examples: `just examples`
3. Bindings changed: build them (`maturin develop` / `napi build`) and call the changed function

## Phase 5: Quality Gates

```bash
git fetch origin main && git rebase origin/main
just pre-pr
```

Fix root causes; never bypass hooks or weaken tests/audit policy.

## Phase 6: Push and PR

```bash
git push -u origin <branch>
```

Create or update the PR:

- **Title**: conventional commit style
- **Body**: `.github/pull_request_template.md`; functional change and impact, not a file walkthrough
- **Evidence**: Before / After (CLI output, test results); say so when nothing observable changed

## Phase 7: CI and Merge

- Wait for CI with bounded checks; elapsed time alone is not a blocker
- Failure: read logs, fix root cause, push, wait again
- All required checks green and review threads resolved: squash-merge, confirm it merged
- **NEVER** merge red CI

## Phase 8: Post-merge

Report the merged PR URL.

## Rules

- Phases 2-4 are the quality core; never skip them
- "Fix and ship": implement first, then ship
- Partial issue coverage: `Part of #N`, not `Closes #N`
55 changes: 8 additions & 47 deletions .claude/commands/maintain.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,10 @@
Run pre-release maintenance checks on the markkit codebase.
Analyze, fix, and ship Markkit maintenance. "Run maintenance" (including
"maintainace" and "maintaiance") requests the complete shipped outcome.

Each section below is an outcome to achieve, not a script to follow.
Use whatever tools and approaches make sense to verify and fix.
Read `.agents/skills/maintain/SKILL.md`, execute the checklist in
`knowledge/operations/maintenance.md`, then run `.agents/skills/ship/SKILL.md`
through green CI and squash-merge. `$ARGUMENTS` may scope the maintenance area;
an explicit analysis-only request is the only exception to fixing and shipping.

Fix issues as you find them — don't just report. Use conventional
commit messages for any fixes.

Sections (run all unless $ARGUMENTS specifies a subset):

## 1. Dependencies
- Run `cargo update` and verify tests pass
- Check for outdated dependencies
- Review Dependabot PRs if any

## 2. Security
- Run `cargo audit` for vulnerabilities
- Run `cargo deny check` for license compliance
- Review any new advisories

## 3. Tests
- Run full test suite: `just test`
- Check coverage isn't regressing
- Identify any `#[ignore]` tests that should be re-enabled

## 4. Documentation
- Verify README is current
- Check rustdoc builds without warnings
- Ensure specs match implementation

## 5. Examples
- Verify examples compile and run
- Update examples for any API changes

## 6. Specs
- Review specs/ for accuracy
- Update implementation status

## 7. Simplification
- Look for dead code, unused dependencies
- Identify duplication opportunities
- Reduce complexity where possible

## 8. CI Health
- Check recent GitHub Actions runs
- Verify nightly builds are passing
- Review any flaky tests

Deliver a summary of findings and actions taken.
Do not replace fixes, audits, or validation with deferral issues or stop at a
local commit. Preserve security contracts, fix CI failures, never merge red checks.
20 changes: 20 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## What changed
Describe the change functionally — what behavior changes and its impact on users
or callers. Lead with outcomes; don't walk through code locations, the diff shows
where and how. Keep any code-level notes short and specific.

## Why
Problem or motivation.

## Before / After
Show the effect with evidence. Include before and after whenever behavior changes —
CLI/API output, logs, metrics, or screenshots for UI (attach working screenshots
when possible). For changes with no observable behavior (pure refactor, docs), say so.

## Risk
- Low / Medium / High
- What can break

## Checklist
- [ ] Tests added or updated
- [ ] Backward compatibility considered
25 changes: 25 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,17 @@ env:
CARGO_TERM_COLOR: always
RUSTFLAGS: -Dwarnings

permissions:
contents: read

jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
Expand All @@ -29,12 +34,25 @@ jobs:
run: cargo doc --workspace --no-deps
env:
RUSTDOCFLAGS: -Dwarnings
# OKF v0.2 conformance of knowledge/. okf-lint enforces the spec;
# check_okf.py enforces bundle-local conventions it does not cover.
# Keep the version and --max-line-length in lockstep with the justfile.
- name: Install okf-lint
run: cargo install okf-lint --version 0.1.1 --locked
- name: Check knowledge bundle (OKF)
run: |
python3 scripts/check_okf.py knowledge
okf-lint knowledge --max-line-length 10000
- name: Test repo scripts
run: python3 -m unittest discover -s scripts/tests -p 'test_*.py'

audit:
name: Audit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- name: Install cargo-audit
run: cargo install cargo-audit
Expand All @@ -50,6 +68,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Run tests
Expand All @@ -63,6 +83,11 @@ jobs:
echo "HTTP client leaked into the docx,spreadsheet feature set"
exit 1
fi
- name: Run examples
run: |
for ex in basic custom_selectors configuration ir_access; do
cargo run -q -p markkit --example "$ex"
done

check:
name: Check
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,17 @@ on:
pull_request:
branches: [main]

permissions:
contents: read

jobs:
coverage:
name: Code Coverage
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Install cargo-tarpaulin
Expand Down
Loading
Loading