Repository navigation
fix(index): -0.0 and +0.0 share one index key (#249) - #252
Merged
Merged
Conversation
IndexKey(double) is an order-preserving bit encoding: negative values have all bits inverted, positive values only the sign bit. -0.0 has the sign bit set, so it encoded to 0x7FFF...FF, just below +0.0's 0x8000...00. The point lookup planned for x.Ratio == 0.0 therefore missed documents storing -0.0, x.Ratio >= 0.0 excluded them and x.Ratio < 0.0 included them, while C#, LINQ-to-Objects and the plain collection use IEEE 754 equality (-0.0 == 0.0). Stored keys and query bounds both go through the constructor, float keys included, so normalising a zero to +0.0 there fixes both sides. An index written by an earlier release keeps -0.0 under the former key. No bound reaches it any more, and an update or delete of its document now computes the +0.0 key and leaves the old entry behind, pointing at a location that may be reused. The open-time key-format migration that already rebuilds legacy decimal and full-text indexes now also rebuilds a double- or float-keyed index that holds that key; finding out costs one seek per such index and open, and an index without it is left alone. No KeyFormat bump is needed: nothing else about the encoding changed, and the check is on the key itself, so it also catches a -0.0 written by an older release after this one rebuilt the index. SignedZeroIndexTests runs ==, the four range operators, Count and OrderBy over -0.0, +0.0, negative and positive values on a double and a float property, indexed and plain, against LINQ-to-Objects; and writes the former -0.0 key into an index to check that the next open rebuilds it, that the entry then goes away with its document and that the other index is not rebuilt. Found-By: OT Defect-Class: IDX Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 16, 2026
Closed
Contributor
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
With an index on a
double(orfloat), queries that hit zero did not treat -0.0 and +0.0 as equal:x.Ratio >= 0.0also left out -0.0, andx.Ratio < 0.0included it. C#, LINQ-to-Objects and the plain collection all use IEEE 754 equality, where-0.0 == 0.0.Mechanism
IndexKey(double)is an order-preserving bit encoding: a negative value has all its bits inverted, a positive value only its sign bit. -0.0 has the sign bit set, so it encoded to0x7FFF…FF, just below +0.0's0x8000…00. The equality plan is a point lookup on the key of +0.0, and range bounds fall between the two zeros.Fix
IndexKey(double)maps a zero to +0.0 before taking the bits (if (value == 0) value = 0.0;). Stored keys and query bounds both go through this constructor, and so dofloatkeys (ConvertToIndexKeyand the optimizer widen them todouble). One line fixes both sides.MigrateLegacyDecimalIndexes, which rebuilds legacy decimal and full-text indexes).NeedsKeyFormatMigrationis now also true for a single-propertydouble/floatB-tree index that holds the old -0.0 key. The check is one range seek per such index on each open; an index without that key is left alone, and the test checks that.KeyFormatbump. I checked this, as the issue suggested: no other key changes, and the trigger is the presence of the old key, not a format number. A bump would rebuild every double index on upgrade, whether or not it holds a -0.0. Checking the key also catches a -0.0 that an older release writes to the file after this one rebuilt it.Tests
SignedZeroIndexTests(6 tests) seeds -0.0, +0.0, negative and positive values on an indexed and a plain collection and compares each query with LINQ-to-Objects:==(both zeros, plus afloatproperty);>=,>,<,<=with a zero bound, and a closed range ending at 0.0;Count;OrderByascending and descending, aSkip/Takewindow over the two zeros, andWhere(>= 0.0).OrderBy;IndexKeyvalues themselves.One more test writes the old -0.0 key into an index, then checks that the next open rebuilds it, that the entry then goes away with its document, and that the second index (no -0.0 key) and a later open rebuild nothing.
Guard: with the
srcchange reverted, all 6 fail. With only the migration reverted (theIndexKeychange kept), the migration test fails. Fulldotnet test tests/BLite.Tests: 3098 passed, 0 failed, 4 skipped.Compatibility
DropIndexAsync+ recreate) if they hold a -0.0 written before this change.Not fixed here: dynamic double indexes (#254)
DynamicCollectionkeys doubles withnew IndexKey(BitConverter.GetBytes(d))(BsonValueToIndexKeyandCreateIndexKeyFromObject). That encoding is little-endian and not order-preserving. I checked on this branch with a throwaway test (not committed): with an index onratioover 0.1, 2.2, 1.3, 5.7, -3.3 and 1.0000001, BLQL{ratio: {$lt: 2.0}}returns nothing and{$gte: 1.0, $lt: 3.0}returns nothing, while the unindexed collection returns 4 and 3 documents.{ratio: 0.0}also misses -0.0 there.I did not touch the dynamic path. Normalising the zero alone would bring the same leftover-entry hazard as above, and the real fix (switching to the order-preserving
IndexKey(double)) means rebuilding every dynamic double index. That rebuild would fix -0.0 there for free. Filed as #254.Fixes #249
PR #250 (oracle tests) lists these rows as known candidates (
-0.0 is not found by an equality on an indexed double) and will drop the entries once this lands.Merge order
#282 (one key for every NaN, #257) is stacked on this PR: both change
IndexKey(double)at the same line. This one lands first. #282 is rebased onto it and carries both changes; I will retarget it tomainonce this merges.🤖 Generated with Claude Code