Repository navigation
Conversation
search_batch takes max_indels alongside max_subs, capped independently:
max_subs=2, max_indels=1 is two substitutions AND one gap, not three edits.
The index, the build and the probe enumeration do not change. With
b = max_subs + max_indels + 1 disjoint blocks the total edit count cannot reach
the block count, so the same pigeonhole argument still leaves a zero-error
block -- and a zero-error block matches the text exactly whether or not the
occurrence carries gaps, so the existing exact seed lookup finds it. That was
checked before any code was written: 2,800 planted occurrences with random
insertions and deletions, recall 1.000 from the unmodified lookup.
What changes is verification. A seed no longer pins the start, because the
prefix before the block may have gained or lost up to max_indels residues, so
every start in [pos - base - max_indels, pos - base + max_indels] is aligned by
a banded DP over (query position, net offset, indels spent). An alignment must
begin and end on an aligned pair: a gap at either edge would stretch the
reported interval over a residue that matches nothing, and a leading gap is the
same occurrence starting one residue over, so allowing it reported one match
two or three times -- which is exactly what the first draft did, caught by the
brute-force oracle.
TextHit gains n_ins, n_dels and length, since a gapped match is no longer
len(query) residues wide.
Two refusals rather than partial answers: a query must be at least
(max_subs + max_indels + 1) * k long, and mismatches is empty on this path with
matrix= ignored, because the substituted columns would need a traceback the
verifier does not keep.
Cost on a synthetic 5,000,000-residue corpus, k=4, one thread: L=16 max_subs=2
goes 0.001 -> 0.043 ms/query, L=24 max_subs=3 goes 0.002 -> 0.067. 25-37x an
ungapped search, still under 0.1 ms/query. An early exit once every banded state
has breached a cap is worth 2.5-3x of that.
Completeness held to the same standard as the substitution path: set equality
against an independent brute force over k in {3,4,5} x max_subs 0-3 x
max_indels 1-2 -- 6,732 occurrences, zero missing and zero extra. max_indels=0
is byte-identical to the previous path, hit for hit, and pays none of the cost.
4,130 C++ assertions, 422 Python tests, perf and oracle gates, warning-free docs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Run serial native batches on the caller, propagate scratch and body failures, and join partially started pools. Co-Authored-By: Codex <codex@openai.com>
Preserve the unreleased TextIndex feature and record the separately released worker bugfix. Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Add bounded exhaustive gap-block top-K and threshold counts
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prepare seqtree 1.0.2 with exhaustive memory-bounded gap-block top-K and cumulative threshold counts, including linked two-view scoring and batch reference access. All generic native APIs reuse the existing scorer and preserve stable ties and pre-selection exact exclusion.
The feature PR passed C++ tests on Linux/macOS/Windows, Python 3.10/3.12 on Linux/macOS, benchmark oracle/performance checks and appendix compilation. Local focused tests and zero-warning Sphinx also passed. This promotes the release candidate only; no publication is requested by this PR.