Skip to content

Engine: per-query statement prepares, non-transactional paired writes, N+1 search lookups, crash traps #45

Closed

Description

@dowjames

Findings from the repo review of Sources/DigUpKit / Sources/LlamaRuntime:

  • Database.swift re-prepares every SQL statement on every call (no cache) — hot paths pay sqlite3_prepare_v2 per insert/update/count; transaction(_:) is not reentrant
  • IndexStore.swift: move, requeue, insert run their paired statements outside any transaction — a mid-way failure leaves the files table and the keyword index out of step; failed embeddings store a 0-byte blob instead of NULL (phantom vectors); pending() interpolates an unbounded IN (…) list
  • Searcher.swift does one files and one segments SELECT per hit (N+1)
  • MediaExtractors.swift force-unwrapped baseAddress on zero-length sample buffers (trap); ImageSizing.targetSize divides by height * width and traps on zero
  • LlamaEmbedder hardcodes the 30 s audio window instead of the configured audioWindow

Fixed by #38.

Activity

  1. ARahim3 commented on Oct 21, 2026

    @ARahim3
    Owner

    Thanks @dowjames. The ones that could bite are in 0.6.2: the paired index writes happen in one transaction (nested ones are savepoints now), and the two traps on odd inputs are guarded. The failed-embedding blob turned out not to exist: a failed file writes no segment. The statement cache and the per-hit lookups I'd still want measured first; search is tens of milliseconds and the keyword-search profile in #93 pointed at the page cache, not at prepares. Closing since it's released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions