Skip to content

Add dg-obsidian-cdp-verify skill - #1327

Draft
trangdoan982 wants to merge 1 commit into
mainfrom
obsidian-cdp-verify-skill
Draft

Add dg-obsidian-cdp-verify skill#1327
trangdoan982 wants to merge 1 commit into
mainfrom
obsidian-cdp-verify-skill

Conversation

@trangdoan982

@trangdoan982 trangdoan982 commented Aug 21, 2026

Copy link
Copy Markdown
Member

Scope check

  • Ran $scope-check against the ENG ticket and final diff. — not run: no Linear ticket backs this PR, so there are no Done When criteria to compare a diff against.
  • Scope beyond Done When: n/a. This is tooling extracted from ENG-2114's verification work at the team's request. The diff adds one skill directory and touches nothing else.

What

Adds skills/dg-obsidian-cdp-verify, following the dg-roam-* pattern: SKILL.md, agents/openai.yaml, and the scripts the skill drives.

apps/obsidian has no test runner — roam, website, database and content-model do — so changes there get verified by driving the real app. Obsidian is Electron, so it speaks the Chrome DevTools Protocol; ~150 lines covers evaluate, input injection and condition polling, and Playwright is not needed.

File Role
scripts/driver.mjs CDP primitives: evaluate, waitFor, key, typeText, reloadPlugin, pressEscape
scripts/harness.mjs Scenario runner — plugin reload, stray-modal cleanup, assertions, exit code
scripts/preflight.mjs Checks the three things that silently invalidate a run
scripts/websocket.mjs Resolves a WebSocket impl without adding a dependency
examples/insert-link-at-cursor.mjs The real ENG-2114 verification, as a worked example

A verification declares scenarios and hands them to runVerification, which owns everything order-dependent, so each scenario only describes its own behaviour:

scenarios: [
  {
    name: "01-does-the-thing",
    body: async ({ client, check, state }) => {
      await client.evaluate(`return app.commands.executeCommandById("…");`);
      await client.waitFor(`!!document.querySelector(".my-modal")`, { label: "modal" });
      check("the thing happened", await client.evaluate(`…`), "detail on failure");
    },
  },
]

The gotchas are the real payload

SKILL.md documents 9 failure modes and their causes. Two produced confident, wrong diagnoses during ENG-2114 that survived until they were deliberately tested. The one most likely to bite others:

Every worktree's dev watcher mirrors into the same vault plugin dir. Another branch's watcher can silently replace the bundle you are testing, and it presents as "my feature is missing from the build."

That happened repeatedly in one session, which is why preflight.mjs takes a marker string and greps the mirrored bundle for it.

The other two worth calling out, because both look like product bugs:

  • editor.hasFocus() ANDs with document.hasFocus(), so it reads false whenever Obsidian is not the frontmost macOS app — running a build in the terminal flips it. Assert on document.activeElement instead.
  • Obsidian saves on a debounce, so a file-content assertion made straight after an action fails while the editor is visibly correct. Poll the file.

Verification

15/15 assertions from the worked example, run from the committed path after each pre-commit prettier pass reformatted the scripts.

Note for reviewers

The existing dg-roam-* skills are SKILL.md + agents/openai.yaml only. This is the first skill in skills/ to ship executable scripts, so it is worth a look at whether that belongs here or under apps/obsidian/. Happy to move it.

🤖 Generated with Claude Code

@graphite-app

graphite-app Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

PR size/scope check

This PR is over our review-size guideline.

  • Recommended: ~200 lines changed
  • Acceptable limit: up to 400 lines when well-scoped/self-contained
  • Preferred file count: fewer than 5 files

Please split this into smaller PRs unless there is a clear reason the changes need to land together.

If keeping it as one PR, please add a brief justification covering:

  • What single problem this PR solves
  • Why the files/changes are coupled

@vercel

vercel Bot commented Aug 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
discourse-graph Ready Ready Preview Aug 21, 2026 5:23pm

Request Review

@supabase

supabase Bot commented Aug 21, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

Comment on lines +96 to +125
await client.startScreencast(({ data, timestamp }) => {
const file = framePath();
frames.push({ file, timestamp });
// Fire-and-forget: awaiting here stalls the ack and drops frames.
void writeFile(file, Buffer.from(data, "base64"));
});

/**
* Each beat appends its own explicitly-timed frame. Deriving caption spans
* from screencast frame indices instead is unreliable: several beats can land
* on one index, collapsing their captions to zero width.
*/
const beat = async (text) => {
const data = await client.screenshot();
const file = framePath();
await writeFile(file, Buffer.from(data, "base64"));
beats.push({ text, frameIndex: frames.length });
frames.push({ file, timestamp: null, duration: DWELL_MS });
console.log(` · ${text}`);
};

try {
return await body({ beat });
} finally {
// Let the closing frames land before tearing the screencast down.
await new Promise((r) => setTimeout(r, 400));
await client.stopScreencast();
await encode({ frames, beats, frameDir, name });
await rm(frameDir, { recursive: true, force: true });
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential race condition between fire-and-forget frame writes and encoding. The screencast callback at line 100 uses void writeFile(...) (fire-and-forget) to avoid stalling frame acknowledgment. However, encode() at line 123 immediately reads these files via ffmpeg, and the frame directory is deleted at line 124. While the 400ms delay at line 121 provides some buffer, there's no guarantee all async writes have completed before encoding starts, especially under system load.

// Current problematic flow:
await client.startScreencast(({ data, timestamp }) => {
  const file = framePath();
  frames.push({ file, timestamp });
  void writeFile(file, Buffer.from(data, "base64")); // async, no await
});
// ... scenario runs ...
await new Promise((r) => setTimeout(r, 400)); // hope writes finish
await client.stopScreencast();
await encode({ frames, beats, frameDir, name }); // reads files
await rm(frameDir, { recursive: true, force: true }); // deletes directory

Fix: Track pending writes and await them before encoding:

const pendingWrites = [];
await client.startScreencast(({ data, timestamp }) => {
  const file = framePath();
  frames.push({ file, timestamp });
  pendingWrites.push(
    writeFile(file, Buffer.from(data, "base64"))
  );
});
// ... scenario runs ...
await new Promise((r) => setTimeout(r, 400));
await client.stopScreencast();
await Promise.all(pendingWrites); // ensure all writes complete
await encode({ frames, beats, frameDir, name });
Suggested change
await client.startScreencast(({ data, timestamp }) => {
const file = framePath();
frames.push({ file, timestamp });
// Fire-and-forget: awaiting here stalls the ack and drops frames.
void writeFile(file, Buffer.from(data, "base64"));
});
/**
* Each beat appends its own explicitly-timed frame. Deriving caption spans
* from screencast frame indices instead is unreliable: several beats can land
* on one index, collapsing their captions to zero width.
*/
const beat = async (text) => {
const data = await client.screenshot();
const file = framePath();
await writeFile(file, Buffer.from(data, "base64"));
beats.push({ text, frameIndex: frames.length });
frames.push({ file, timestamp: null, duration: DWELL_MS });
console.log(` · ${text}`);
};
try {
return await body({ beat });
} finally {
// Let the closing frames land before tearing the screencast down.
await new Promise((r) => setTimeout(r, 400));
await client.stopScreencast();
await encode({ frames, beats, frameDir, name });
await rm(frameDir, { recursive: true, force: true });
}
const pendingWrites = [];
await client.startScreencast(({ data, timestamp }) => {
const file = framePath();
frames.push({ file, timestamp });
// Fire-and-forget: awaiting here stalls the ack and drops frames.
pendingWrites.push(writeFile(file, Buffer.from(data, "base64")));
});
/**
* Each beat appends its own explicitly-timed frame. Deriving caption spans
* from screencast frame indices instead is unreliable: several beats can land
* on one index, collapsing their captions to zero width.
*/
const beat = async (text) => {
const data = await client.screenshot();
const file = framePath();
await writeFile(file, Buffer.from(data, "base64"));
beats.push({ text, frameIndex: frames.length });
frames.push({ file, timestamp: null, duration: DWELL_MS });
console.log(` · ${text}`);
};
try {
return await body({ beat });
} finally {
// Let the closing frames land before tearing the screencast down.
await new Promise((r) => setTimeout(r, 400));
await client.stopScreencast();
await Promise.all(pendingWrites);
await encode({ frames, beats, frameDir, name });
await rm(frameDir, { recursive: true, force: true });
}

Spotted by Graphite

Fix in Graphite


Is this helpful? React 👍 or 👎 to let us know.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment on lines +10 to +12
const pluginDir =
process.env.PLUGIN_DIR ??
`${process.env.HOME}/Documents/${vault}/.obsidian/plugins/discourse-graphs`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Preflight checks the wrong plugin folder name

The default plugin directory ends in discourse-graphs, but the dev build mirrors into discourse-graph per apps/obsidian/.env.example. The bundle check then reports "no bundle" and preflight exits non-zero on a normal dev setup unless PLUGIN_DIR is overridden.

Suggested change
const pluginDir =
process.env.PLUGIN_DIR ??
`${process.env.HOME}/Documents/${vault}/.obsidian/plugins/discourse-graphs`;
const pluginDir =
process.env.PLUGIN_DIR ??
`${process.env.HOME}/Documents/${vault}/.obsidian/plugins/discourse-graph`;
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

apps/obsidian has no test runner, so changes there are verified by driving the
running app over the Chrome DevTools Protocol. This packages the driver and the
scenario harness used to verify ENG-2114, plus that verification as a worked
example.

Included gotchas are the ones that produced wrong diagnoses in practice — most
notably that every worktree's dev watcher mirrors into the same vault plugin
dir, so another branch's build can silently replace the code under test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant