Repository navigation
fix(index): express comparison bounds in the indexed property's type (#222) - #228
Merged
Merged
Conversation
…222) IndexOptimizer.ParseSimplePredicate kept the runtime type of the constant in a comparison such as r.Count < 9.5 or r.Count >= 5L, where the compiler wraps the int property in a Convert to double or long. Index keys carry no type tag and compare bytewise, so a double or long bound against an int index was encoded in another key layout and selected the wrong range: Count(r => r.Count < 9.5) counted every document, Where(r => r.Count >= 2.5) returned none, and Max over such a filter could throw. The bound is now converted to the property's type when the property is widened by the comparison. For int and long properties an integer bound that selects the same values is chosen (Count < 9.5 becomes Count <= 9, Count >= 2.5 becomes Count >= 3, Count > 9.0 stays strict); a bound beyond the property's range, or NaN, turns the comparison into Count >= MinValue or Count > MaxValue, so plans stay exact for the count and aggregate push-downs and nulls of an int? property stay excluded. Pairs whose keys already share a layout (float/double, DateTime/DateTimeOffset, numeric bounds on a decimal-keyed index, enums) are kept. Anything else, such as a narrowing cast like (int)r.Total == 5 or a double bound beyond 2^53 on a long, plans no index, so the query evaluates the predicate per document. Only Convert/ConvertChecked unary nodes are treated as the property itself now, so Negate or ArrayLength over a property no longer becomes a key range. DynamicCollection does not use this planner and is unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #222.
Problem
In
r.Count < 9.5orr.Count >= 5Lthe compiler wraps the property inConvert(r.Count, Double/Int64). The twoConvertbranches ofIndexOptimizer.ParseSimplePredicatereturned the constant with its own type.ConvertToIndexKeythen encoded it as a double or long key, compared bytewise against the index's int keys, so the scan covered the wrong range: wrong counts and rows, and an exception forMax.Fix
Both
Convertbranches now callTryConvertBound. It expresses the bound in the property's own type, adjusting the operator where needed. Properties with a value converter are left as they were.int/long properties. Only a widening comparison is planned:
The bound becomes the integer that selects the same values.
< 9.5becomes<= 9,>= 2.5becomes>= 3and> 2.5becomes>= 3.== 5.5is always false.Out-of-range and NaN bounds. An always-true comparison becomes
>= MinValue, which keeps the plan exact and still leaves out the nulls of anint?. An always-false one becomes> MaxValue.HasResiduePredicatestays false, so the key-count push-down inCountByPredicateAsyncstays correct.Pairs already keyed alike are kept:
Anything else gets no index, and the predicate is evaluated per document. That covers NaN on float/double/decimal properties, narrowing casts such as
(int)r.Total == 5, and lossy bounds beyond the exact limit. In an AND, the other side becomes the plan, with a residue.Beyond the issue. Only
Convert/ConvertCheckedcount as "the property" now. Before,-r.Count > 5or anArrayLengthover an indexed property could be planned as a key range on that property.Aggregates. The type guard in
TryAggregateAsync(from #183) stays. Converted bounds now pass it, so these filters use the index-key aggregate. Only its comment changed, and one inLinqAggregateTests.DynamicCollectiondoesn't use this planner and is not affected.Tests
IndexBoundConversionTestshas 171 tests onAggregateDbContext/Reading(indexed intCount, longTotaland int?MaybeCount), with data 0..19 plus documents at the int and long minimum and maximum.Where().ToListAsync(),CountAsync(pred)andWhere().Max/Min(exceptions included). They cover the issue's table plus inclusive and strict bounds, in and out of range, NaN, ±infinity, float and decimal bounds, long and int? properties, AND merges of converted bounds, captured variables, and the reversed form9.5 > r.Count.Count < 9.5gives an exact plan with max 9 inclusive;Count >= 5Lgives an int min of 5;Total > 1e17gives no plan.Before the fix 112 of the 171 failed. Full
BLite.Testssuite: 2964 passed, 4 skipped, 0 failed.BLite.Corebuilds with no warnings.Left as is
CreateIndexAsync(r => (long)r.Units).CollectionIndexInfodoesn't carry the key type, so bounds are still expressed in the property type. Even a plainr.Units == 5against such an index was already mismatched before this PR.(double)r.Price >= x) is still converted by the index as before. It can be off by a hair for decimals with more than about 15 significant digits.🤖 Generated with Claude Code