Skip to content

Linux port (x86_64): portable digup CLI, poppler PDFs, deck support, binary releases - #6

Closed
ik0zy wants to merge 7 commits into
ARahim3:mainfrom
ik0zy:linux-port-x86_64
Closed

ik0zy wants to merge 7 commits into
ARahim3:mainfrom
ik0zy:linux-port-x86_64

Conversation

@ik0zy

@ik0zy ik0zy commented Oct 11, 2026

Copy link
Copy Markdown

Follows #5 (Linux CLI port proposal).

What

digup CLI + engine run on Linux x86_64; the app stays macOS-only. Details in docs/linux.md.

  • Manifest: app/Sparkle macOS-only, CSQLite + swift-crypto on Linux, native linking per OS (original llama archives with --start-group on Linux; the merge drops same-named members).
  • Portable fallbacks: Accelerate fast path kept, pure-Swift math on Linux; extension typing (no UTType); Linux stat/git/XDG paths; pdfinfo/ffprobe estimates.
  • Apple readers gated; Linux: PDFs via poppler (same page model), ppt/pptx/odp decks via LibreOffice (new slides kind), images/audio/video/rich-docs report why per file.
  • One portability fix in shared code: ICU has no \u{...} escapes and Linux answers bad patterns with "" (emptied every chunk list); now braceless.
  • Releases: package-linux.sh (tarball/.deb), AUR -bin + rpm spec, linux-release.yml CI (build/test/package on PRs, upload on releases).

Verified (Arch x86_64, Swift 6.4)

  • swift build --product digup, full swift test: 64 + 6 green, incl. vectorsMatchTheReference (Linux vectors match macOS >=0.999, indexes stay compatible).
  • Real folders: ~400 pages/slides/decks/code indexed and searched (meaning + keyword fusion checked).

Needs from review

  • macOS swift build && swift test (Xcode): I have no Mac; shared-file changes are the manifest, CLlama links, a behavior-identical PDF-prepare refactor, and an appended (not reordered) index kind.
  • Happy to split (engine vs PDFs/decks vs packaging) or rework to fit.

ik0zy added 7 commits October 11, 2026 09:30
- Package.swift: DigUpApp/Sparkle macOS-only; CSQLite system lib on Linux
- CLlama modulemap: drop framework links (now via linkerSettings, macOS-only)
- Vectors/Searcher: Accelerate fast path kept, pure-Swift fallback on Linux
- Database: SQLite3/CSQLite conditional import
- Rules/Crawler: extension-based typing, Linux stat/git/iCloud handling
- Estimator: pdfinfo/ffprobe fallbacks (no Spotlight)
- Extractors/MediaExtractors: Apple readers gated, Linux stubs that report
  why a file was left out; txt/md/html/code fully work
- CLI paths follow XDG on Linux; Glibc imports where needed
- build-llama.sh: Linux CPU build (ar merge, nproc, no Metal)
- scripts/package-linux.sh + packaging/ (PKGBUILD-bin, rpm spec)
  + linux-release CI (tarball/.deb/.rpm) + docs/linux.md
- Tests: mac-only PDF-output tests gated; UTType/CryptoKit/realpath fixed
- Link llama.cpp's original static archives with --start-group instead of
  merging (members share basenames; merges silently drop objects)
- CMAKE_POSITION_INDEPENDENT_CODE=ON (SwiftPM links test bundles as .so)
- stdc++/gomp links on Linux; all native linking via LlamaRuntime target
- Extractors.chunks: braceless NBSP escape (Linux answers bad patterns
  with empty string, which emptied every chunk list)
- LongFilesTests: remove duplicated signatures from gating edit
- swift-crypto pinned in Package.resolved; dist/ ignored
@ik0zy

ik0zy commented Oct 15, 2026

Copy link
Copy Markdown
Author

Closing as superseded: since this was opened, upstream shipped a fuller Linux (and Windows) port with a cleaner architecture (Platform/Apple + Platform/Portable, vendored SQLite/stb/miniz/PDFium, shell UI, updates, .deb, 0.7.0-beta.1). Merging this CLI-only branch on top would only conflict. The verification work here still holds (Linux vectors match macOS at >=0.999). The one piece upstream lacks is deck indexing (.pptx/.ppt/.odp via LibreOffice) -- proposed as a follow-up on #5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant