Skip to content

fix(index): -0.0 and +0.0 share one index key (#249) - #252

Merged
mrdevrobot merged 1 commit into
mainfrom
fix/249-negative-zero-index-key
Oct 18, 2026
Merged

mrdevrobot merged 1 commit into
mainfrom
fix/249-negative-zero-index-key

Conversation

@CaffeinatedCoder

@CaffeinatedCoder CaffeinatedCoder commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

What was wrong

With an index on a double (or float), queries that hit zero did not treat -0.0 and +0.0 as equal:

// d1: Ratio = -0.0;  d2: 0.0;  d3: 1.0   (HasIndex(x => x.Ratio))
Where(x => x.Ratio == 0.0)   // expected [d1, d2], was [d2]
Count(x => x.Ratio == 0.0)   // expected 2, was 1

x.Ratio >= 0.0 also left out -0.0, and x.Ratio < 0.0 included it. C#, LINQ-to-Objects and the plain collection all use IEEE 754 equality, where -0.0 == 0.0.

Mechanism

IndexKey(double) is an order-preserving bit encoding: a negative value has all its bits inverted, a positive value only its sign bit. -0.0 has the sign bit set, so it encoded to 0x7FFF…FF, just below +0.0's 0x8000…00. The equality plan is a point lookup on the key of +0.0, and range bounds fall between the two zeros.

Fix

  • IndexKey(double) maps a zero to +0.0 before taking the bits (if (value == 0) value = 0.0;). Stored keys and query bounds both go through this constructor, and so do float keys (ConvertToIndexKey and the optimizer widen them to double). One line fixes both sides.
  • Existing indexes. An index written by an earlier release keeps -0.0 under the old key. No query bound reaches that key any more. Worse, an update or delete of the document now computes the +0.0 key and leaves the old entry behind, still pointing at a location that may be reused for another document. So the fix also hooks into the key-format migration that already runs on open (MigrateLegacyDecimalIndexes, which rebuilds legacy decimal and full-text indexes). NeedsKeyFormatMigration is now also true for a single-property double/float B-tree index that holds the old -0.0 key. The check is one range seek per such index on each open; an index without that key is left alone, and the test checks that.
  • No KeyFormat bump. I checked this, as the issue suggested: no other key changes, and the trigger is the presence of the old key, not a format number. A bump would rebuild every double index on upgrade, whether or not it holds a -0.0. Checking the key also catches a -0.0 that an older release writes to the file after this one rebuilt it.

Tests

SignedZeroIndexTests (6 tests) seeds -0.0, +0.0, negative and positive values on an indexed and a plain collection and compares each query with LINQ-to-Objects:

  • == (both zeros, plus a float property);
  • >=, >, <, <= with a zero bound, and a closed range ending at 0.0;
  • Count;
  • OrderBy ascending and descending, a Skip/Take window over the two zeros, and Where(>= 0.0).OrderBy;
  • the IndexKey values themselves.

One more test writes the old -0.0 key into an index, then checks that the next open rebuilds it, that the entry then goes away with its document, and that the second index (no -0.0 key) and a later open rebuild nothing.

Guard: with the src change reverted, all 6 fail. With only the migration reverted (the IndexKey change kept), the migration test fails. Full dotnet test tests/BLite.Tests: 3098 passed, 0 failed, 4 skipped.

Compatibility

  • Index-served queries on double/float properties now match the plain path for -0.0.
  • The first open after upgrading rebuilds any double/float index that holds a -0.0 key, with the same machinery and failure behaviour as the decimal migration.
  • Compound keys (more than one property) are not checked. They would need a rebuild by hand (DropIndexAsync + recreate) if they hold a -0.0 written before this change.
  • NaN is out of scope, as in the issue.

Not fixed here: dynamic double indexes (#254)

DynamicCollection keys doubles with new IndexKey(BitConverter.GetBytes(d)) (BsonValueToIndexKey and CreateIndexKeyFromObject). That encoding is little-endian and not order-preserving. I checked on this branch with a throwaway test (not committed): with an index on ratio over 0.1, 2.2, 1.3, 5.7, -3.3 and 1.0000001, BLQL {ratio: {$lt: 2.0}} returns nothing and {$gte: 1.0, $lt: 3.0} returns nothing, while the unindexed collection returns 4 and 3 documents. {ratio: 0.0} also misses -0.0 there.

I did not touch the dynamic path. Normalising the zero alone would bring the same leftover-entry hazard as above, and the real fix (switching to the order-preserving IndexKey(double)) means rebuilding every dynamic double index. That rebuild would fix -0.0 there for free. Filed as #254.

Fixes #249

PR #250 (oracle tests) lists these rows as known candidates (-0.0 is not found by an equality on an indexed double) and will drop the entries once this lands.

Merge order

#282 (one key for every NaN, #257) is stacked on this PR: both change IndexKey(double) at the same line. This one lands first. #282 is rebased onto it and carries both changes; I will retarget it to main once this merges.

🤖 Generated with Claude Code

IndexKey(double) is an order-preserving bit encoding: negative values have
all bits inverted, positive values only the sign bit. -0.0 has the sign bit
set, so it encoded to 0x7FFF...FF, just below +0.0's 0x8000...00. The point
lookup planned for x.Ratio == 0.0 therefore missed documents storing -0.0,
x.Ratio >= 0.0 excluded them and x.Ratio < 0.0 included them, while C#,
LINQ-to-Objects and the plain collection use IEEE 754 equality
(-0.0 == 0.0). Stored keys and query bounds both go through the
constructor, float keys included, so normalising a zero to +0.0 there fixes
both sides.

An index written by an earlier release keeps -0.0 under the former key. No
bound reaches it any more, and an update or delete of its document now
computes the +0.0 key and leaves the old entry behind, pointing at a
location that may be reused. The open-time key-format migration that
already rebuilds legacy decimal and full-text indexes now also rebuilds a
double- or float-keyed index that holds that key; finding out costs one
seek per such index and open, and an index without it is left alone. No
KeyFormat bump is needed: nothing else about the encoding changed, and the
check is on the key itself, so it also catches a -0.0 written by an older
release after this one rebuilt the index.

SignedZeroIndexTests runs ==, the four range operators, Count and OrderBy
over -0.0, +0.0, negative and positive values on a double and a float
property, indexed and plain, against LINQ-to-Objects; and writes the former
-0.0 key into an index to check that the next open rebuilds it, that the
entry then goes away with its document and that the other index is not
rebuilt.

Found-By: OT
Defect-Class: IDX

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mrdevrobot

Copy link
Copy Markdown
Contributor

Verified: the tests fail on 0450b97 and pass here; diff stays within #249. On-disk note: -0.0 now takes the key of +0.0, and an index holding an old -0.0 key is detected and rebuilt on open.

@mrdevrobot
mrdevrobot merged commit fba9330 into main Oct 18, 2026
6 checks passed

This branch was successfully deployed

1 active deployment
Production — 8a6f3200 Deployed Oct 16, 2026 by CaffeinatedCoder via build_and_publish #748
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/indexing B-Tree and secondary indexes, index lifecycle bug Something isn't working package/core Affects BLite.Core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Indexed double equality misses -0.0: IndexKey(double) gives the two zeros different keys

2 participants