Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "fdeops",
"description": "Skills for forward deployed engineers across strategy, architecture and engineering. Use individual tasks or @fde coordination; local customer memory supports continuity.",
"version": "5.1.20",
"version": "5.1.21",
"category": "productivity",
"tags": [
"community-managed"
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## 5.1.21 - 2026-09-25

- Keep diagnosis and review useful when customer access is limited: request the smallest permitted observation, excerpt or owner-run check needed to resolve uncertainty.
- Add fictional evaluation cases for ambiguous timeouts, partial-diff reviews and unavailable meeting sources.

## 5.1.19 - 2026-09-22

- Redact private and masked ingest metadata before creating filenames, keeping protected values out of inbox listings and review provenance.
Expand Down
4 changes: 4 additions & 0 deletions evals/delivery/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ These supplement D1-D5; `setup.js` does not materialize them. Use disposable loc

Record inputs, revision, model/tools, output, unexpected writes, redundant questions, and failed lookups. Keep raw traces local. Compare the previous revision when measuring improvement.

## Limited-access diagnostics

F26-F28 exercise diagnosis from an ambiguous timeout, review of a partial diff, and a page-specific source denial. Prepare them with `field.js` like the other field cases. Inspect the response and tool trace for useful progress, bounded conclusions, the smallest permitted next evidence request, and references loaded only when relevant. A successful fixture check proves preparation, not model judgment; keep raw runs outside Git.

## Rubric

Score each dimension 0 (missing/incorrect), 1 (partial), or 2 (complete with traceable support):
Expand Down
21 changes: 21 additions & 0 deletions evals/delivery/field-cases.json
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,27 @@
"title": "Resolve one uncertainty before a full POC",
"prompt": "Use notes.md to recommend the next step and explain how we decide between the alternatives. Write answer.md only. No external actions or new records.",
"rubric": "Request a bounded permitted mapping check for the required field with named owner Jo. Present both branches without inventing result; avoid full POC as default. Field support alone does not prove integration readiness, approved choice or deployment."
},
{
"id": "F26",
"skill": "debug",
"title": "Diagnosis without production access",
"prompt": "Investigate the timeout in notes.md. What can we establish and what should we do next? Write answer.md only. Do not request production access unless it is necessary for the next check; no external calls, repairs or record updates.",
"rubric": "Distinguish a supplied timeout from proof of a failed write or root cause. Continue a bounded hypothesis-driven diagnosis; request the smallest permitted discriminating evidence, such as owner-run status lookup with timestamps. Do not ask for credentials, payloads or broad production access, retry the write, or claim execution. Load only relevant supporting references. Inspect response and actual reference reads; unavailable evidence must not stop all useful work."
},
{
"id": "F27",
"skill": "review",
"title": "Partial diff with an unavailable caller",
"prompt": "Review the change in notes.md and tell us whether it is safe to release. Give useful findings now and the next evidence needed. Write answer.md only. No code changes, network access or release authority.",
"rubric": "Review the supplied timeout change without inventing caller behavior. Identify conditional retry/concurrency or resource-budget concerns as hypotheses, not proven defects. Bound the conclusion to supplied diff and withhold release assurance. Ask for focused permitted caller/retry evidence or an owner-run check rather than full repository or production credentials. No independent reviewer or test result may be invented. Reference reads should follow applicable review concerns, not load ship or AI evaluations merely because linked."
},
{
"id": "F28",
"skill": "connect",
"title": "One inaccessible meeting with a permitted export",
"prompt": "Summarize the requested meeting using notes.md. The source read failed; help me continue without setting up another integration. Write answer.md only; do not save customer records, install tools or make external calls.",
"rubric": "Report the page-specific access denial without diagnosing global connector failure. Offer the permitted export path without requesting whole-workspace access or credentials. Do not invent meeting content, successful retrieval, summary or record update. No ingest/record initialization is needed for summary-only work. Inspect relevant reference loading and actual actions."
}
]
}
5 changes: 5 additions & 0 deletions evals/delivery/field-fixtures/F26/notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Fictional incident evidence
AI use of this fixture is approved. No live tools or production access are available.
Supplied trace: POST /orders timed out at 10:04 after 5 seconds. Outcome at the source is unknown.
Customer policy permits a customer operator to return sanitized status and timestamps for that operation. Payloads and credentials cannot be shared. Jo can perform the check during today's support window.
No caller, retry code, downstream status or reproduction is supplied.
8 changes: 8 additions & 0 deletions evals/delivery/field-fixtures/F27/notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Fictional review evidence
AI use of these excerpts is approved.
Proposed patch, supplied revision retry-17:
- const timeoutMs = 5000;
+ const timeoutMs = 30000;
Outcome: reduce timeout errors on order submission. No acceptance threshold is supplied.
Caller, retry policy, concurrency limits and tests are not available to this agent. The customer can supply a sanitized caller excerpt or run a focused check; raw production logs are not approved.
No execution or deployment evidence is supplied.
3 changes: 3 additions & 0 deletions evals/delivery/field-fixtures/F28/notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Fictional source state
The existing notes connector returned permission denied for meeting page meeting-42. Its general configuration is present; no successful read of this page occurred.
The user needs only a summary of this meeting, not an engagement record update. A customer-approved text export for this AI setup is available from the meeting owner but has not yet been supplied. No meeting content appears in this fixture.
2 changes: 1 addition & 1 deletion mcp/fdeops-ingest/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "fdeops-ingest-mcp",
"version": "5.1.20",
"version": "5.1.21",
"private": true,
"description": "Thin stdio MCP sink for FDEOps ingest (stage \u2192 propose \u2192 apply). Zero runtime dependencies.",
"bin": {
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "fdeops",
"version": "5.1.20",
"version": "5.1.21",
"description": "Skills for forward deployed engineers across strategy, architecture and engineering. Use individual tasks or @fde coordination; local customer memory supports continuity.",
"bin": {
"fdeops": "bin/install.js",
Expand Down
2 changes: 1 addition & 1 deletion plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "fdeops",
"version": "5.1.20",
"version": "5.1.21",
"description": "Skills for forward deployed engineers across strategy, architecture and engineering. Use individual tasks or @fde coordination; local customer memory supports continuity.",
"author": {
"name": "Subash Natarajan",
Expand Down
4 changes: 2 additions & 2 deletions skills/build/.fde-generated.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
"SKILL.md": "77570e69b8d708d60cd6ab1dcc79c95a75824ab668f60a83a0dedbe668de2a70",
"agents/openai.yaml": "abd0a33ab4efa25ea038932d37eae2faa2527baa96ebae8732ea90f1ebc3865e",
"references/build.md": "4ebea9776605f79f902d9bf83a065e4cded191d7b5f8defcf58c054514fc34f5",
"references/debug.md": "3273a921a98522431ae821a283814897270c719cbfab83d14b99175537d869e5",
"references/debug.md": "bdc09abdd7a075574fc2daf91cecfd2858e67d75a964146a118bf782328192f6",
"references/eval-pack.md": "0590b85d3cae0903c6b1274540c92eaa2a4373047e8a0548d6942516ef0bb9e1",
"references/integrate.md": "1cb7a60d7545b0bf224fce678a04ce6ccdf368c47877d9c8e4dc4272bb0d5b0c",
"references/qa.md": "41de4d70827c83291efa217e97d777f62ec2849827687fbba7e4b1d17484b87e",
"references/review.md": "55733ca868c00fb22110bc7b3ec7bb6c6451366c795073b19a2bccd30d2764c8",
"references/review.md": "8fa35eacdc04d5e818e046e6da8f61f126e08d9c6a1073b2eaa19f72d4bfea3a",
"references/ship.md": "95f51772b29de6f7d174f1f7678f15d46a3a5327dcb8facb94288e27907ae92c",
"references/task-context.md": "9f995c1fe4d8a27d313b4c008a943fbcdd150029358c70b450dbbfb303a2666e",
"references/verification.md": "8ee2502112a37eedfdacf929041e7b91e9e6fabb007408a1ee2e2992e3af6ff6"
Expand Down
2 changes: 1 addition & 1 deletion skills/build/references/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Follow [task context](task-context.md); permitted supplied evidence is enough wi

Capture expected and observed behavior, the trigger or input, revision, environment, and last known working state. Keep useful errors and timestamps, without secrets or raw private data. Label unverified reports as reports.

Inspect the affected path, callers, relevant changes, and existing tests. Reproduce with the smallest representative case in a permitted environment when practical. If reproduction is unavailable, state the gap and use traces or other safe observations to distinguish causes. Keep investigating without promoting a hypothesis to a finding.
Inspect the affected path, callers, relevant changes, and existing tests. Reproduce with the smallest representative case in a permitted environment when practical. If reproduction is unavailable, state the gap and use traces or other safe observations to distinguish causes. Keep investigating without promoting a hypothesis to a finding. Ask for the smallest permitted observation that distinguishes the remaining causes; an operator-run check or approved excerpt may be enough without granting the agent additional access.

## Test the explanation

Expand Down
2 changes: 1 addition & 1 deletion skills/build/references/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Use [task context](task-context.md). Obtain the intended outcome, acceptance che

## Establish what was reviewed

Identify the repository, base and head revision, staged/unstaged changes, and relevant untracked files. Read applicable instructions and the full in-scope diff, then inspect callers and tests where needed. A committed-range diff alone omits working-tree edits. Record missing files or unavailable context as limitations.
Identify the repository, base and head revision, staged/unstaged changes, and relevant untracked files. Read applicable instructions and the full in-scope diff, then inspect callers and tests where needed. A committed-range diff alone omits working-tree edits. Record missing files or unavailable context as limitations. For each limitation that affects the verdict, name the narrow excerpt or owner-run check needed to resolve it; continue reviewing the supplied scope without requesting whole-repository or production access by default.

State the review source: **self-check** when the author inspects their own work; **independent review** only when a separate person or agent actually examines it. A second pass by the same agent is still a self-check. Name the actual reviewer/source and reviewed revision when available. Do not fabricate a reviewer, dialogue, approval, or clean verdict. Use an available separate reviewer for substantial or risky changes when authorized; otherwise report the missing independent review and continue useful self-checks.

Expand Down
4 changes: 2 additions & 2 deletions skills/debug/.fde-generated.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
"SKILL.md": "00b6f8c7c56be7a665f9401c58319fc4cd344e1c865d88021c116d94149a4a35",
"agents/openai.yaml": "f9178e65a1e2e9ee27f2f5917a36a50e3ebe64bd6e331b43379efcf1c7d5aac3",
"references/build.md": "4ebea9776605f79f902d9bf83a065e4cded191d7b5f8defcf58c054514fc34f5",
"references/debug.md": "3273a921a98522431ae821a283814897270c719cbfab83d14b99175537d869e5",
"references/debug.md": "bdc09abdd7a075574fc2daf91cecfd2858e67d75a964146a118bf782328192f6",
"references/eval-pack.md": "0590b85d3cae0903c6b1274540c92eaa2a4373047e8a0548d6942516ef0bb9e1",
"references/integrate.md": "1cb7a60d7545b0bf224fce678a04ce6ccdf368c47877d9c8e4dc4272bb0d5b0c",
"references/qa.md": "41de4d70827c83291efa217e97d777f62ec2849827687fbba7e4b1d17484b87e",
"references/review.md": "55733ca868c00fb22110bc7b3ec7bb6c6451366c795073b19a2bccd30d2764c8",
"references/review.md": "8fa35eacdc04d5e818e046e6da8f61f126e08d9c6a1073b2eaa19f72d4bfea3a",
"references/ship.md": "95f51772b29de6f7d174f1f7678f15d46a3a5327dcb8facb94288e27907ae92c",
"references/task-context.md": "9f995c1fe4d8a27d313b4c008a943fbcdd150029358c70b450dbbfb303a2666e",
"references/verification.md": "8ee2502112a37eedfdacf929041e7b91e9e6fabb007408a1ee2e2992e3af6ff6"
Expand Down
2 changes: 1 addition & 1 deletion skills/debug/references/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Follow [task context](task-context.md); permitted supplied evidence is enough wi

Capture expected and observed behavior, the trigger or input, revision, environment, and last known working state. Keep useful errors and timestamps, without secrets or raw private data. Label unverified reports as reports.

Inspect the affected path, callers, relevant changes, and existing tests. Reproduce with the smallest representative case in a permitted environment when practical. If reproduction is unavailable, state the gap and use traces or other safe observations to distinguish causes. Keep investigating without promoting a hypothesis to a finding.
Inspect the affected path, callers, relevant changes, and existing tests. Reproduce with the smallest representative case in a permitted environment when practical. If reproduction is unavailable, state the gap and use traces or other safe observations to distinguish causes. Keep investigating without promoting a hypothesis to a finding. Ask for the smallest permitted observation that distinguishes the remaining causes; an operator-run check or approved excerpt may be enough without granting the agent additional access.

## Test the explanation

Expand Down
2 changes: 1 addition & 1 deletion skills/debug/references/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Use [task context](task-context.md). Obtain the intended outcome, acceptance che

## Establish what was reviewed

Identify the repository, base and head revision, staged/unstaged changes, and relevant untracked files. Read applicable instructions and the full in-scope diff, then inspect callers and tests where needed. A committed-range diff alone omits working-tree edits. Record missing files or unavailable context as limitations.
Identify the repository, base and head revision, staged/unstaged changes, and relevant untracked files. Read applicable instructions and the full in-scope diff, then inspect callers and tests where needed. A committed-range diff alone omits working-tree edits. Record missing files or unavailable context as limitations. For each limitation that affects the verdict, name the narrow excerpt or owner-run check needed to resolve it; continue reviewing the supplied scope without requesting whole-repository or production access by default.

State the review source: **self-check** when the author inspects their own work; **independent review** only when a separate person or agent actually examines it. A second pass by the same agent is still a self-check. Name the actual reviewer/source and reviewed revision when available. Do not fabricate a reviewer, dialogue, approval, or clean verdict. Use an available separate reviewer for substantial or risky changes when authorized; otherwise report the missing independent review and continue useful self-checks.

Expand Down
4 changes: 2 additions & 2 deletions skills/evaluate/.fde-generated.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
"SKILL.md": "396110a1cb9a555b4096ef77bf45e531b60d78b18bd617c654f19c8e07506ce1",
"agents/openai.yaml": "731f3c81d46444af542369f0aa27736d24530d00fc1aa717436ddefe3f3bdef7",
"references/build.md": "4ebea9776605f79f902d9bf83a065e4cded191d7b5f8defcf58c054514fc34f5",
"references/debug.md": "3273a921a98522431ae821a283814897270c719cbfab83d14b99175537d869e5",
"references/debug.md": "bdc09abdd7a075574fc2daf91cecfd2858e67d75a964146a118bf782328192f6",
"references/eval-pack.md": "0590b85d3cae0903c6b1274540c92eaa2a4373047e8a0548d6942516ef0bb9e1",
"references/integrate.md": "1cb7a60d7545b0bf224fce678a04ce6ccdf368c47877d9c8e4dc4272bb0d5b0c",
"references/qa.md": "41de4d70827c83291efa217e97d777f62ec2849827687fbba7e4b1d17484b87e",
"references/review.md": "55733ca868c00fb22110bc7b3ec7bb6c6451366c795073b19a2bccd30d2764c8",
"references/review.md": "8fa35eacdc04d5e818e046e6da8f61f126e08d9c6a1073b2eaa19f72d4bfea3a",
"references/ship.md": "95f51772b29de6f7d174f1f7678f15d46a3a5327dcb8facb94288e27907ae92c",
"references/task-context.md": "9f995c1fe4d8a27d313b4c008a943fbcdd150029358c70b450dbbfb303a2666e",
"references/verification.md": "8ee2502112a37eedfdacf929041e7b91e9e6fabb007408a1ee2e2992e3af6ff6"
Expand Down
2 changes: 1 addition & 1 deletion skills/evaluate/references/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Follow [task context](task-context.md); permitted supplied evidence is enough wi

Capture expected and observed behavior, the trigger or input, revision, environment, and last known working state. Keep useful errors and timestamps, without secrets or raw private data. Label unverified reports as reports.

Inspect the affected path, callers, relevant changes, and existing tests. Reproduce with the smallest representative case in a permitted environment when practical. If reproduction is unavailable, state the gap and use traces or other safe observations to distinguish causes. Keep investigating without promoting a hypothesis to a finding.
Inspect the affected path, callers, relevant changes, and existing tests. Reproduce with the smallest representative case in a permitted environment when practical. If reproduction is unavailable, state the gap and use traces or other safe observations to distinguish causes. Keep investigating without promoting a hypothesis to a finding. Ask for the smallest permitted observation that distinguishes the remaining causes; an operator-run check or approved excerpt may be enough without granting the agent additional access.

## Test the explanation

Expand Down
2 changes: 1 addition & 1 deletion skills/evaluate/references/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Use [task context](task-context.md). Obtain the intended outcome, acceptance che

## Establish what was reviewed

Identify the repository, base and head revision, staged/unstaged changes, and relevant untracked files. Read applicable instructions and the full in-scope diff, then inspect callers and tests where needed. A committed-range diff alone omits working-tree edits. Record missing files or unavailable context as limitations.
Identify the repository, base and head revision, staged/unstaged changes, and relevant untracked files. Read applicable instructions and the full in-scope diff, then inspect callers and tests where needed. A committed-range diff alone omits working-tree edits. Record missing files or unavailable context as limitations. For each limitation that affects the verdict, name the narrow excerpt or owner-run check needed to resolve it; continue reviewing the supplied scope without requesting whole-repository or production access by default.

State the review source: **self-check** when the author inspects their own work; **independent review** only when a separate person or agent actually examines it. A second pass by the same agent is still a self-check. Name the actual reviewer/source and reviewed revision when available. Do not fabricate a reviewer, dialogue, approval, or clean verdict. Use an available separate reviewer for substantial or risky changes when authorized; otherwise report the missing independent review and continue useful self-checks.

Expand Down
Loading
Loading