Add dg-obsidian-cdp-verify skill - #1327
Conversation
PR size/scope checkThis PR is over our review-size guideline.
Please split this into smaller PRs unless there is a clear reason the changes need to land together. If keeping it as one PR, please add a brief justification covering:
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
| await client.startScreencast(({ data, timestamp }) => { | ||
| const file = framePath(); | ||
| frames.push({ file, timestamp }); | ||
| // Fire-and-forget: awaiting here stalls the ack and drops frames. | ||
| void writeFile(file, Buffer.from(data, "base64")); | ||
| }); | ||
|
|
||
| /** | ||
| * Each beat appends its own explicitly-timed frame. Deriving caption spans | ||
| * from screencast frame indices instead is unreliable: several beats can land | ||
| * on one index, collapsing their captions to zero width. | ||
| */ | ||
| const beat = async (text) => { | ||
| const data = await client.screenshot(); | ||
| const file = framePath(); | ||
| await writeFile(file, Buffer.from(data, "base64")); | ||
| beats.push({ text, frameIndex: frames.length }); | ||
| frames.push({ file, timestamp: null, duration: DWELL_MS }); | ||
| console.log(` · ${text}`); | ||
| }; | ||
|
|
||
| try { | ||
| return await body({ beat }); | ||
| } finally { | ||
| // Let the closing frames land before tearing the screencast down. | ||
| await new Promise((r) => setTimeout(r, 400)); | ||
| await client.stopScreencast(); | ||
| await encode({ frames, beats, frameDir, name }); | ||
| await rm(frameDir, { recursive: true, force: true }); | ||
| } |
There was a problem hiding this comment.
Potential race condition between fire-and-forget frame writes and encoding. The screencast callback at line 100 uses void writeFile(...) (fire-and-forget) to avoid stalling frame acknowledgment. However, encode() at line 123 immediately reads these files via ffmpeg, and the frame directory is deleted at line 124. While the 400ms delay at line 121 provides some buffer, there's no guarantee all async writes have completed before encoding starts, especially under system load.
// Current problematic flow:
await client.startScreencast(({ data, timestamp }) => {
const file = framePath();
frames.push({ file, timestamp });
void writeFile(file, Buffer.from(data, "base64")); // async, no await
});
// ... scenario runs ...
await new Promise((r) => setTimeout(r, 400)); // hope writes finish
await client.stopScreencast();
await encode({ frames, beats, frameDir, name }); // reads files
await rm(frameDir, { recursive: true, force: true }); // deletes directoryFix: Track pending writes and await them before encoding:
const pendingWrites = [];
await client.startScreencast(({ data, timestamp }) => {
const file = framePath();
frames.push({ file, timestamp });
pendingWrites.push(
writeFile(file, Buffer.from(data, "base64"))
);
});
// ... scenario runs ...
await new Promise((r) => setTimeout(r, 400));
await client.stopScreencast();
await Promise.all(pendingWrites); // ensure all writes complete
await encode({ frames, beats, frameDir, name });| await client.startScreencast(({ data, timestamp }) => { | |
| const file = framePath(); | |
| frames.push({ file, timestamp }); | |
| // Fire-and-forget: awaiting here stalls the ack and drops frames. | |
| void writeFile(file, Buffer.from(data, "base64")); | |
| }); | |
| /** | |
| * Each beat appends its own explicitly-timed frame. Deriving caption spans | |
| * from screencast frame indices instead is unreliable: several beats can land | |
| * on one index, collapsing their captions to zero width. | |
| */ | |
| const beat = async (text) => { | |
| const data = await client.screenshot(); | |
| const file = framePath(); | |
| await writeFile(file, Buffer.from(data, "base64")); | |
| beats.push({ text, frameIndex: frames.length }); | |
| frames.push({ file, timestamp: null, duration: DWELL_MS }); | |
| console.log(` · ${text}`); | |
| }; | |
| try { | |
| return await body({ beat }); | |
| } finally { | |
| // Let the closing frames land before tearing the screencast down. | |
| await new Promise((r) => setTimeout(r, 400)); | |
| await client.stopScreencast(); | |
| await encode({ frames, beats, frameDir, name }); | |
| await rm(frameDir, { recursive: true, force: true }); | |
| } | |
| const pendingWrites = []; | |
| await client.startScreencast(({ data, timestamp }) => { | |
| const file = framePath(); | |
| frames.push({ file, timestamp }); | |
| // Fire-and-forget: awaiting here stalls the ack and drops frames. | |
| pendingWrites.push(writeFile(file, Buffer.from(data, "base64"))); | |
| }); | |
| /** | |
| * Each beat appends its own explicitly-timed frame. Deriving caption spans | |
| * from screencast frame indices instead is unreliable: several beats can land | |
| * on one index, collapsing their captions to zero width. | |
| */ | |
| const beat = async (text) => { | |
| const data = await client.screenshot(); | |
| const file = framePath(); | |
| await writeFile(file, Buffer.from(data, "base64")); | |
| beats.push({ text, frameIndex: frames.length }); | |
| frames.push({ file, timestamp: null, duration: DWELL_MS }); | |
| console.log(` · ${text}`); | |
| }; | |
| try { | |
| return await body({ beat }); | |
| } finally { | |
| // Let the closing frames land before tearing the screencast down. | |
| await new Promise((r) => setTimeout(r, 400)); | |
| await client.stopScreencast(); | |
| await Promise.all(pendingWrites); | |
| await encode({ frames, beats, frameDir, name }); | |
| await rm(frameDir, { recursive: true, force: true }); | |
| } | |
Spotted by Graphite
Is this helpful? React 👍 or 👎 to let us know.
| const pluginDir = | ||
| process.env.PLUGIN_DIR ?? | ||
| `${process.env.HOME}/Documents/${vault}/.obsidian/plugins/discourse-graphs`; |
There was a problem hiding this comment.
🟡 Preflight checks the wrong plugin folder name
The default plugin directory ends in discourse-graphs, but the dev build mirrors into discourse-graph per apps/obsidian/.env.example. The bundle check then reports "no bundle" and preflight exits non-zero on a normal dev setup unless PLUGIN_DIR is overridden.
| const pluginDir = | |
| process.env.PLUGIN_DIR ?? | |
| `${process.env.HOME}/Documents/${vault}/.obsidian/plugins/discourse-graphs`; | |
| const pluginDir = | |
| process.env.PLUGIN_DIR ?? | |
| `${process.env.HOME}/Documents/${vault}/.obsidian/plugins/discourse-graph`; |
Was this helpful? React with 👍 or 👎 to provide feedback.
apps/obsidian has no test runner, so changes there are verified by driving the running app over the Chrome DevTools Protocol. This packages the driver and the scenario harness used to verify ENG-2114, plus that verification as a worked example. Included gotchas are the ones that produced wrong diagnoses in practice — most notably that every worktree's dev watcher mirrors into the same vault plugin dir, so another branch's build can silently replace the code under test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fbaa116 to
6eec941
Compare
Scope check
$scope-checkagainst the ENG ticket and final diff. — not run: no Linear ticket backs this PR, so there are noDone Whencriteria to compare a diff against.Done When: n/a. This is tooling extracted from ENG-2114's verification work at the team's request. The diff adds one skill directory and touches nothing else.What
Adds
skills/dg-obsidian-cdp-verify, following thedg-roam-*pattern:SKILL.md,agents/openai.yaml, and the scripts the skill drives.apps/obsidianhas no test runner — roam, website, database and content-model do — so changes there get verified by driving the real app. Obsidian is Electron, so it speaks the Chrome DevTools Protocol; ~150 lines coversevaluate, input injection and condition polling, and Playwright is not needed.scripts/driver.mjsevaluate,waitFor,key,typeText,reloadPlugin,pressEscapescripts/harness.mjsscripts/preflight.mjsscripts/websocket.mjsexamples/insert-link-at-cursor.mjsA verification declares scenarios and hands them to
runVerification, which owns everything order-dependent, so each scenario only describes its own behaviour:The gotchas are the real payload
SKILL.mddocuments 9 failure modes and their causes. Two produced confident, wrong diagnoses during ENG-2114 that survived until they were deliberately tested. The one most likely to bite others:That happened repeatedly in one session, which is why
preflight.mjstakes a marker string and greps the mirrored bundle for it.The other two worth calling out, because both look like product bugs:
editor.hasFocus()ANDs withdocument.hasFocus(), so it reads false whenever Obsidian is not the frontmost macOS app — running a build in the terminal flips it. Assert ondocument.activeElementinstead.Verification
15/15assertions from the worked example, run from the committed path after each pre-commit prettier pass reformatted the scripts.Note for reviewers
The existing
dg-roam-*skills areSKILL.md+agents/openai.yamlonly. This is the first skill inskills/to ship executable scripts, so it is worth a look at whether that belongs here or underapps/obsidian/. Happy to move it.🤖 Generated with Claude Code