Load card recognition when the video plays; free it after 10 idle minutes

PRIVACY.md

@@ -51,14 +51,15 @@ storage, uploaded or shared.
 ## What it sends, and to whom
 
 Card hovers start in a tab when the pointer comes over a video there, with DuelSight
-switched on in the popup.
+switched on in the popup. Card recognition loads in a tab when a video starts playing there with
+DuelSight switched on (or when card hovers start, for a video that has not played).
 
 | Request | To | When | What the host learns |
 | --- | --- | --- | --- |
 | The settings file (`config.json`) | DuelSight's card CDN | When the extension is installed or updated and when the browser starts, then at most once an hour while you use it (when you open the popup, or when card hovers start in a tab) | Your IP address and that the extension asked for its settings |
 | A card's picture | DuelSight's card CDN | When a card is shown and its picture is not already loaded | Your IP address and which card picture was asked for |
 | The card details file (the type, stats and text of the catalog's cards, in one file) | DuelSight's card CDN | When card hovers start in a tab with card details switched on and no copy of the file is kept yet: in practice once, and again when the settings file names a new version of it | Your IP address and that the extension asked for the details file (not which card you looked at: it is the same file whatever card you hover). A browser in another language DuelSight has asks for that language's file instead, so the host also learns that, as it does from the language every browser sends with its requests |
-| Updated card-recognition data (a release: its card list, card fingerprints and, after a model update, the recognition models; data files, never code) | DuelSight's card CDN | When card hovers start in a tab and the settings file names a release other than the one built into your version of the extension: a newer one, or an earlier one after a rollback (otherwise never). About 4.4 MB once per release when cards are added, about 19 MB when the models themselves are updated; afterwards it is read from your browser's cache. Your browser may keep a separate copy for each website, so watching on both YouTube and Twitch can download it twice. Nothing is downloaded at install or with card hovers off | Your IP address, the time, your browser's user agent and language, that this browser downloaded that release, and the extension's own id (sent as the request's origin) |
+| Updated card-recognition data (a release: its card list, card fingerprints and, after a model update, the recognition models; data files, never code) | DuelSight's card CDN | When card recognition loads in a tab and the settings file names a release other than the one built into your version of the extension: a newer one, or an earlier one after a rollback (otherwise never). About 4.4 MB once per release when cards are added, about 19 MB when the models themselves are updated; afterwards it is read from your browser's cache. Your browser may keep a separate copy for each website, so watching on both YouTube and Twitch can download it twice. Nothing is downloaded at install or with card hovers off | Your IP address, the time, your browser's user agent and language, that this browser downloaded that release, and the extension's own id (sent as the request's origin) |
 
 These requests carry no cookies, no account ID or per-user tracking identifier, and not the
 address of the page you watch. Settings, picture, details and release requests carry no `Referer` at all. Like any

docs/how-it-works.md

@@ -59,7 +59,7 @@ flowchart TD
     HS -- "capture request;<br/>card id, name, box" --> CS
     HS <-- "frame out,<br/>boxes and names back" --> W
     PKG -- "release.json<br/>at load" --> W
-    REL -. "checked files,<br/>when hovers start" .-> W
+    REL -. "checked files,<br/>when a video plays" .-> W
     CS -- "card id and name" --> TT
     TT -- "its size" --> CS
     CS -- "the named box" --> OUT
@@ -74,8 +74,12 @@ flowchart TD
 in [PRIVACY.md](../PRIVACY.md). Nothing else leaves the browser.
 
 - **The content script** is on for all supported videos (the popup's switch is the sole
-  control), but makes no frame and loads no model before the pointer comes over the
-  video's picture, so a video nobody points at costs nothing. From then on it reports the
+  control). When the page's main video starts playing it makes the compute frame, which loads
+  the model, so the load is usually over before the pointer comes to a card (a video that never
+  plays loads when the pointer comes over its picture). It makes the tooltip frame, and analyzes
+  nothing, before the pointer comes over the video's picture. After 10 minutes with no move over
+  the picture and no card showing it removes both frames, freeing the model's memory; the next
+  pointer move in the page while the video plays makes the compute frame again. From then on it reports the
   pointer and seeks to the compute frame,
   and, when asked, draws the video to a canvas at analysis size and transfers it as an
   `ImageBitmap`. It gives a picture on the authority of pointer events the browser marks as

docs/using.md

@@ -23,8 +23,11 @@ How card hovers work once the extension is installed, what to expect, and how to
 ## Turning card hovers on and off
 
 Card hovers are on for all supported videos: open a Yu-Gi-Oh! video on YouTube or Twitch and
-point at a card. Nothing is analyzed before the pointer comes over a video, so a video you just
-watch costs nothing. To turn card hovers off everywhere, use the switch in DuelSight's toolbar
+point at a card. When a video starts playing, DuelSight loads its card recognition in the
+background (a short burst of work, then some memory), so the card you point at is named sooner.
+After 10 minutes without pointing at the video it frees that memory, and loads again when you
+move the pointer. Nothing is analyzed before the pointer comes over the
+video. To turn card hovers off everywhere, use the switch in DuelSight's toolbar
 popup; there is no per-channel switch. The extension separately checks its settings file when it
 is installed or updated and when the browser starts, and at most once an hour while you use it.
 

scripts/build.mjs

@@ -1,13 +1,15 @@
 // SPDX-License-Identifier: Apache-2.0
 // Build the unpacked extension into dist/ (not kept in the repository).
 //
-//   npm run build -- --release <release folder> [--dev] [--store] [--target chrome|firefox] [--out dist]
+//   npm run build -- --release <release folder> [--dev [--idle-unload-ms N]] [--store] [--target chrome|firefox] [--out dist]
 //
 // --release       A release folder as `cardvision build-release` writes it: release.json and the
 //                 files it pins. It is checked with the runtime's own verification (every file
 //                 against its SHA-256 and size, the components against each other, the ONNX
 //                 Runtime version) and copied into dist/release/. It is never committed.
 // --dev           A development build: also runs on the local test page (http://127.0.0.1, http://localhost).
+// --idle-unload-ms  Dev builds only: free the model after N ms with nobody hovering, instead of
+//                 10 minutes (src/constants.ts IDLE_UNLOAD_MS), to check the unload.
 // --store         A Chrome Web Store build: the manifest has no `key` (the store assigns the ID).
 //                 Other builds carry the public key that fixes the ID of unpacked installs.
 // --target        chrome (the default; also Edge and Brave) or firefox: the same bundles with
@@ -43,6 +45,7 @@ const { values } = parseArgs({
     store: { type: "boolean", default: false },
     target: { type: "string", default: "chrome" },
     out: { type: "string", default: "dist" },
+    "idle-unload-ms": { type: "string" },
   },
 });
 if (!values.release) {
@@ -53,6 +56,12 @@ if (values.dev && values.store) {
   console.error("--dev and --store do not go together");
   process.exit(2);
 }
+// A dev build may shorten the idle time before the model is freed, to check the unload.
+const idleUnloadMs = values["idle-unload-ms"] === undefined ? null : Number(values["idle-unload-ms"]);
+if (idleUnloadMs !== null && (!values.dev || !(Number.isInteger(idleUnloadMs) && idleUnloadMs > 0))) {
+  console.error("--idle-unload-ms is a whole number of ms above 0, for a --dev build only");
+  process.exit(2);
+}
 if (values.target !== "chrome" && values.target !== "firefox") {
   console.error(`--target is chrome or firefox (got "${values.target}")`);
   process.exit(2);
@@ -138,7 +147,7 @@ const shared = {
   sourcemap: false,
   logLevel: "warning",
   legalComments: "linked",
-  define: { __DEV__: values.dev ? "true" : "false", __TARGET__: JSON.stringify(values.target) },
+  define: { __DEV__: values.dev ? "true" : "false", __TARGET__: JSON.stringify(values.target), __IDLE_UNLOAD_MS__: JSON.stringify(idleUnloadMs) },
   plugins: [configPlugin],
 };
 await build({ ...shared, format: "iife", entryPoints: { content: join(root, "src/content/index.ts") }, outdir: out });

src/compute/compute.ts

@@ -4,8 +4,8 @@
 // the toolkit's session, which decides when a picture is analyzed and what is shown at the
 // pointer. DuelSight's options and its card facts channel are in frame.ts.
 //
-// This frame is made on the first pointer move over a video's picture, so the release loads then
-// and never at install. Which release: the bundled one, or the one the settings file names (the
+// This frame is made when the page's main video starts playing (or on the first pointer move over
+// its picture, for a video that never plays), so the release loads then and never at install. Which release: the bundled one, or the one the settings file names (the
 // vendored remote-release module, cdn/SETTINGS.md "release"), read once per page.
 
 import { CONFIG } from "../config.js";

src/constants.ts

@@ -47,6 +47,10 @@ export const IMAGE_TIMEOUT_MS = 5_000;
 /** A source that failed this many times in a row is skipped until fresh settings are read. */
 export const SOURCE_FAILURE_LIMIT = 3;
 
+/** How long the model stays loaded with no trusted move over the picture and no card showing,
+ *  before the frames are removed to free its memory (owner, 2026-11-10). */
+export const IDLE_UNLOAD_MS = 10 * 60_000;
+
 /** Smallest video, in CSS pixels of width on the page, that the extension attaches to. */
 export const MIN_VIDEO_WIDTH = 320;
 

src/content/agent.ts

@@ -8,8 +8,16 @@
 //     pointer events the browser marks as trusted, checked again against the picture at each
 //     capture, and never more than 60 s after the last one. Pictures go only to the compute frame,
 //     which is the extension's own; nothing is sent over the network.
-//   - Nothing is analyzed until the pointer first comes over a picture: that is when the two
-//     frames are created and the model loads. So a video nobody points at costs nothing.
+//   - The model loads when the main video plays (owner, 2026-11-10: "Products load at video
+//     play"): the compute frame is made then, so the load is usually over by the first hover. A
+//     video that never plays loads on the first pointer move over its picture, as before. With
+//     the popup's switch off, or on a page with no video, nothing loads. Nothing is analyzed
+//     until the pointer comes over the picture; the tooltip frame is made then.
+//   - After IDLE_UNLOAD_MS with no trusted move over the picture and no card showing, both frames
+//     are removed, which frees the model's memory. The next trusted pointer move anywhere in the
+//     page while the video plays (or the video playing again after a pause) loads it again, so the
+//     load runs while the pointer travels to the card. A `playing` with no pause first (after a
+//     stall or a loop) does not.
 //   - It shows the tooltip frame (the card's picture alone, or its name) beside the card the
 //     compute frame names, placed off the cards around it and scaled down when the page has no
 //     room for it, and hides it shortly after. Nothing it shows takes the pointer: hovering goes
@@ -23,7 +31,7 @@
 //     frame (fromExtensionFrame): a page can make up a message event, not a trusted one.
 
 import { CONFIG } from "../config.js";
-import { MIN_VIDEO_WIDTH } from "../constants.js";
+import { IDLE_UNLOAD_MS, MIN_VIDEO_WIDTH } from "../constants.js";
 import { parseTooltipOut, tag, TAG, type TooltipIn, type TooltipSize } from "../shared/messages.js";
 import { type CaptureControl, computeProtocol, createCaptureControl, fromExtensionFrame, type ShownCard } from "../../vendor/cardvision-browser/page-capture.js";
 import { availableSpace, CARD_CORNER, fitBesideCard, fitsBesideCard, fitTooltipSize, fitWithPanel, fractionIn, pictureRect, placeTooltipAvoiding, type Rect, scaleTooltipSize, tooltipRadius } from "./geometry.js";
@@ -51,6 +59,9 @@ export interface AgentOptions {
   dev: boolean;
   /** The clock, in ms; default `performance.now()`. For tests. */
   now?: () => number;
+  /** How long the model stays loaded with nobody hovering; default IDLE_UNLOAD_MS. A dev build
+   *  can shorten it to check the unload. */
+  idleUnloadMs?: number;
 }
 
 export class Agent {
@@ -88,6 +99,14 @@ export class Agent {
   private loadError: string | null = null;
   /** The outline around the card the tooltip names (a shadow root in the page). */
   private outline: CardOutline | null = null;
+  /** When the viewer last hovered the picture (or the model loaded, or a card hid), by Date.now(). */
+  private activeAt = 0;
+  private idleTimer: ReturnType<typeof setTimeout> | null = null;
+  /** The frames were removed for idleness: a trusted move anywhere in the page loads again. */
+  private unloaded = false;
+  /** The video paused since the unload: playing again then is the viewer back, and loads again.
+   *  A `playing` with no pause first (after a stall, a loop, an ad) is not. */
+  private pausedSinceUnload = false;
 
   /** `chrome-extension://<id>`: the extension's base address without its trailing slash. */
   private readonly extensionOrigin = chrome.runtime.getURL("").replace(/\/$/, "");
@@ -131,6 +150,9 @@ export class Agent {
     // The page moving under a held card: the tooltip and the outline follow it.
     window.addEventListener("resize", this.onLayout, { passive: true });
     document.addEventListener("scroll", this.onScroll, { capture: true, passive: true });
+    // Media events do not bubble: a video starting to play is caught on the way down.
+    document.addEventListener("playing", this.onPlaying, { capture: true });
+    document.addEventListener("pause", this.onPause, { capture: true });
   }
 
   /** A scroll that can move the video: the page's own, or a box holding the player. Others (a
@@ -169,6 +191,8 @@ export class Agent {
     document.removeEventListener("fullscreenchange", this.onFullscreen);
     window.removeEventListener("resize", this.onLayout);
     document.removeEventListener("scroll", this.onScroll, { capture: true });
+    document.removeEventListener("playing", this.onPlaying, { capture: true });
+    document.removeEventListener("pause", this.onPause, { capture: true });
     this.enabled = false;
     this.teardown();
     this.control.stop();
@@ -186,7 +210,9 @@ export class Agent {
   setEnabled(enabled: boolean): void {
     if (enabled === this.enabled) return;
     this.enabled = enabled;
+    this.unloaded = false;
     if (!enabled) this.teardown();
+    else this.loadIfPlaying();
   }
 
   // ------------------------------------------------------------------ the pointer
@@ -203,9 +229,59 @@ export class Agent {
   private readonly onPointer = (event: PointerEvent): void => {
     if (event.isTrusted !== true || !this.enabled || !isActivePage(new URL(location.href), this.options.dev)) return;
     const video = this.currentVideo();
-    if (video && fractionIn(this.pictureOf(video), event.clientX, event.clientY)) this.ensureFrames();
+    if (video && fractionIn(this.pictureOf(video), event.clientX, event.clientY)) {
+      this.ensureFrames();
+      this.markActive();
+    } else if (this.unloaded && video && !video.paused) {
+      // Unloaded for idleness: the pointer is moving, so load while it travels to the card.
+      this.ensureCompute();
+    }
+  };
+
+  private readonly onPlaying = (): void => {
+    if (this.unloaded && !this.pausedSinceUnload) return;
+    this.loadIfPlaying();
   };
 
+  private readonly onPause = (): void => {
+    if (this.unloaded) this.pausedSinceUnload = true;
+  };
+
+  /** Makes the compute frame, which loads the model, when the page's main video is playing: on a
+   *  `playing` event, and when the switch is turned on (a video may already be playing). */
+  private loadIfPlaying(): void {
+    if (!this.enabled || !isActivePage(new URL(location.href), this.options.dev)) return;
+    // The video that just started may be a new one: pick again.
+    this.videoPickedAt = -Infinity;
+    const video = this.currentVideo();
+    if (video && !video.paused) this.ensureCompute();
+  }
+
+  // ------------------------------------------------------------------ idleness
+  /** The viewer is here (or the model just loaded, or a card just hid): the idle time starts again. */
+  private markActive(): void {
+    this.activeAt = Date.now();
+    if (this.compute && this.idleTimer === null) this.armIdle(this.idleMs());
+  }
+
+  private idleMs(): number {
+    return this.options.idleUnloadMs ?? IDLE_UNLOAD_MS;
+  }
+
+  private armIdle(ms: number): void {
+    this.idleTimer = setTimeout(() => {
+      this.idleTimer = null;
+      if (!this.compute) return;
+      // A card showing holds the model; its idle time starts when it hides.
+      if (this.displayedCardId !== null) return this.markActive();
+      const left = this.activeAt + this.idleMs() - Date.now();
+      if (left > 0) return this.armIdle(left);
+      this.teardown();
+      this.unloaded = true;
+      this.pausedSinceUnload = false;
+    }, ms);
+  }
+
   // ------------------------------------------------------------------ the video
   private currentVideo(): HTMLVideoElement | null {
     const now = performance.now();
@@ -244,14 +320,20 @@ export class Agent {
 
   // ------------------------------------------------------------------ the frames
   private ensureFrames(): void {
-    // A frame the page removed is made again.
-    if (this.compute && !this.compute.isConnected) this.compute = null;
+    this.ensureCompute();
     if (this.tooltip && !this.tooltip.isConnected) {
       this.tooltip = null;
       this.tooltipReady = false;
       this.displayedCardId = null;
       this.sizedCardId = null;
     }
+    if (!this.tooltip) this.createTooltip();
+  }
+
+  /** The compute frame, made once (and again if the page removed it). It loads the model as soon
+   *  as it is made. */
+  private ensureCompute(): void {
+    if (this.compute && !this.compute.isConnected) this.compute = null;
     if (!this.compute) {
       const frame = document.createElement("iframe");
       frame.setAttribute("aria-hidden", "true");
@@ -268,8 +350,9 @@ export class Agent {
       });
       document.documentElement.appendChild(frame);
       this.compute = frame;
+      this.unloaded = false;
+      this.markActive();
     }
-    if (!this.tooltip) this.createTooltip();
   }
 
   private createTooltip(): void {
@@ -460,6 +543,7 @@ export class Agent {
     this.sizedCardId = null;
     this.placedFor = null;
     this.outline?.hide();
+    this.markActive();
   }
 
   private teardown(): void {
@@ -479,5 +563,7 @@ export class Agent {
     this.pendingTooltip = null;
     this.outline?.remove();
     this.outline = null;
+    if (this.idleTimer) clearTimeout(this.idleTimer);
+    this.idleTimer = null;
   }
 }

src/content/index.ts

@@ -6,8 +6,10 @@ import { ENABLED_KEY, readEnabled } from "../shared/enabled.js";
 import { CARD_SIZE_KEY, clampCardSize, readCardSize } from "../shared/card-size.js";
 
 declare const __DEV__: boolean;
+// Null except in a dev build made with --idle-unload-ms.
+declare const __IDLE_UNLOAD_MS__: number | null;
 
-const agent = new Agent({ dev: __DEV__ });
+const agent = new Agent({ dev: __DEV__, ...(__IDLE_UNLOAD_MS__ !== null ? { idleUnloadMs: __IDLE_UNLOAD_MS__ } : {}) });
 agent.start();
 void readEnabled(chrome.storage.local).then((enabled) => agent.setEnabled(enabled));
 void readCardSize(chrome.storage.local).then((percent) => agent.setCardSize(percent));