diff --git a/CHANGELOG.md b/CHANGELOG.md index 612c03e4..6fc35062 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,10 @@ Some calls answer differently: [Changed answers in 10.2.0](https://github.com/Ca - EF Core: `LowerBound()` and `UpperBound()` in a query are `NULL` for an empty range or set and on an unbounded side, as in memory. So `== null` selects those rows, where it selected none on a required column, and `?? fallback` returns the fallback, where the row was missing ([#354](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/354), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). - EF Core: a range, multirange or value set predicate over a NULL column is `false`, so its negation keeps the row, as EF Core does for `string` and `string[]`. `!b.Period.Overlaps(p)` dropped the NULL rows, and a projected predicate threw ([#357](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/357), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). +### Fixed + +- Docs: an ordering by `LowerBound()` or `UpperBound()` puts the empty rows and those without that bound where PostgreSQL puts NULLs, last ascending and first descending, also on a required column. A `!= null` key in front sorts them as memory does ([EF Core](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/efcore.md#read-bounds-merge-ranges-and-aggregate)). + ## [10.1.0] — 2026-10-19 One call answers differently, on the .NET 11 runtime only: [Changed answers in 10.1.0](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1010). diff --git a/README.md b/README.md index e1cfd02c..b1049f16 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ options.UseNpgsql(connectionString, npgsql => npgsql.UseValueRanges()); bookings.Where(b => b.Period.Contains(day)); // b."Period" @> @day bookings.Where(b => b.Period.Overlaps(request)); // b."Period" && @request -bookings.OrderBy(b => b.Period.LowerBound()); // ORDER BY lower(b."Period") +bookings.OrderBy(b => b.Period.LowerBound()); // ORDER BY lower(b."Period"); no start sorts last ``` **→ [Getting started](docs/getting-started.md)** takes you from an empty project to a first query diff --git a/docs/efcore.md b/docs/efcore.md index 7510aa37..046febfa 100644 --- a/docs/efcore.md +++ b/docs/efcore.md @@ -100,6 +100,7 @@ Since v4 you can also read bounds, merge ranges, aggregate per group and compare ```csharp // ORDER BY lower(b."Period") — bound accessors: lower / upper / lower_inc / upper_inc +// A row with no lower bound, or an empty one, sorts last (see "Good to know" below) bookings.OrderBy(b => b.Period.LowerBound()); // range_merge(b."Period", @other) and range_merge(b."BlockedDays") @@ -158,6 +159,15 @@ Good to know: for an unbounded or empty range, and the in-memory methods match that. So `== null`, `HasValue` and `??` on a bound select the same rows in a query as in memory, on a range or a `RangeSet` column. +- **Ordering by a bound puts the rows without it where PostgreSQL puts NULLs.** + `OrderBy(b => b.Period.LowerBound())` sorts the empty rows and the rows with no lower bound last, + and `OrderByDescending(b => b.Period.UpperBound())` sorts those with no upper bound first. That + holds on a required column too, because the bound is NULL there. In memory, `null` sorts first + ascending and last descending. To sort as memory does, put a `!= null` key in front: + `OrderBy(b => b.Period.LowerBound() != null).ThenBy(b => b.Period.LowerBound())`, or + `OrderByDescending` and `ThenByDescending` for the other direction. To sort open starts first, as a + range's own order does, order by the range: `OrderBy(b => b.Period)` + ([Sort ranges](ranges.md#sort-ranges)). - **Discrete upper bounds are adjusted for you.** For `int4range`, `int8range` and `daterange`, PostgreSQL normalizes to the half-open form `[lower, upper)`. This library normalizes to the closed form `[lower, upper]`. So `UpperBound()` becomes `upper(x) - 1`, and diff --git a/docs/getting-started.md b/docs/getting-started.md index c35b2b8b..c006113a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -193,7 +193,7 @@ var colliding = context.Bookings.Where(b => b.Period.Overlaps(request)).ToList() // WHERE b."Tags" @> ARRAY[@tag]::text[] — a plain GIN index serves this var tagged = context.Bookings.Where(b => b.Tags.Contains("vip")).ToList(); -// ORDER BY lower(b."Period") +// ORDER BY lower(b."Period"); a booking with no start, or an empty period, sorts last var chronological = context.Bookings.OrderBy(b => b.Period.LowerBound()).ToList(); ``` diff --git a/docs/ranges.md b/docs/ranges.md index e62557a6..e8c46c9d 100644 --- a/docs/ranges.md +++ b/docs/ranges.md @@ -448,7 +448,9 @@ Good to know: `<= 0`, and with a non-null operand it compares lower, so it matches `< 0`, `<= 0` and `!= 0`. In memory, calling `CompareTo` on a null throws. - **PostgreSQL puts `NULL` last in an ascending `ORDER BY`.** In memory, `null` sorts first. On a - nullable column, the two orders differ only in where the nulls go. + nullable column, the two orders differ only in where the nulls go. An ordering by `LowerBound()` or + `UpperBound()` meets the same difference on any column, because the bound of an empty or unbounded + range is NULL ([Read bounds](efcore.md#read-bounds-merge-ranges-and-aggregate)). - **This is not the set's lower-bound order.** `RangeSet.LowerBoundComparer` compares only the lower bound, so it ties `[1,5]` with `[1,9]` ([Sort ranges the way a set does](#sort-ranges-the-way-a-set-does)). diff --git a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md index aec9f254..5d49fc00 100644 --- a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md +++ b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md @@ -10,6 +10,10 @@ covers all four packages, which share one version number and release together. - EF Core: `LowerBound()` and `UpperBound()` in a query are `NULL` for an empty range or set and on an unbounded side, as in memory. So `== null` selects those rows, where it selected none on a required column, and `?? fallback` returns the fallback, where the row was missing ([#354](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/354), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). - EF Core: a range, multirange or value set predicate over a NULL column is `false`, so its negation keeps the row, as EF Core does for `string` and `string[]`. `!b.Period.Overlaps(p)` dropped the NULL rows, and a projected predicate threw ([#357](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/357), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). +### Fixed + +- Docs: an ordering by `LowerBound()` or `UpperBound()` puts the empty rows and those without that bound where PostgreSQL puts NULLs, last ascending and first descending, also on a required column. A `!= null` key in front sorts them as memory does ([EF Core](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/efcore.md#read-bounds-merge-ranges-and-aggregate)). + ## [10.1.0] — 2026-10-19 ### Fixed diff --git a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/README.md b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/README.md index 1ce330f7..8672f40f 100644 --- a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/README.md +++ b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/README.md @@ -75,7 +75,7 @@ var day = new DateOnly(2024, 6, 15); bookings.Where(b => b.Period.Contains(day)); // b."Period" @> @day bookings.Where(b => b.Period.Overlaps(other)); // b."Period" && @other bookings.Select(b => b.Period.Intersect(other)); // b."Period" * @other -bookings.OrderBy(b => b.Period.LowerBound()); // ORDER BY lower(b."Period") +bookings.OrderBy(b => b.Period.LowerBound()); // ORDER BY lower(b."Period"); no start sorts last bookings.Select(b => b.Period.Merge(other)); // range_merge(b."Period", @other) bookings.Select(b => b.BlockedDays.Gaps(month)); // datemultirange(@month) - b."BlockedDays" bookings.Where(b => b.BlockedDays == someSet); // b."BlockedDays" = @set diff --git a/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs b/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs index 17c3b41f..cc060380 100644 --- a/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs +++ b/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs @@ -633,6 +633,76 @@ await Seed( Assert.AreEqual(periods.Take(2).RangeIntersectAgg(), serverIntersectAgg); } + /// + /// An ordering by LowerBound() or UpperBound() puts the empty rows and the rows without + /// that bound where PostgreSQL puts NULLs, last ascending and first descending, also on a required + /// column; memory puts a first ascending and last descending (#359, + /// docs/efcore.md#read-bounds-merge-ranges-and-aggregate). With a Bound() != null key in + /// front, the database orders as memory does, in both directions. Over a range and a multirange + /// column. Ids 2700 to 2708. + /// + [TestMethod] + public async Task OrderingByABound_PutsTheRowsWithoutItWherePostgresPutsNulls() + { + ContainerLifecycle.RequireDatabase(); + + Int32Range[] shapes = + [ + Int32Range.CreateFinite(1, 20, true, true), Int32Range.CreateFinite(1, 1, true, true), Int32Range.CreateFinite(20, 20, true, true), + Int32Range.CreateUnboundedStart(1, true), Int32Range.CreateUnboundedStart(20, true), + Int32Range.CreateUnboundedEnd(1, true), Int32Range.CreateUnboundedEnd(20, true), + Int32Range.Infinite, Int32Range.Empty + ]; + await Seed([.. shapes.Select((shape, index) => new Reservation + { + Id = 2700 + index, Seats = shape, SeatBlocks = RangeSet.From(shape) + })]); + + await using var context = new IntegrationDbContext(); + var mine = context.Reservations.Where(r => r.Id >= 2700 && r.Id <= 2708); + var loaded = await mine.AsNoTracking().ToListAsync(); + var failures = new List(); + + foreach (var (name, key) in new (string, Expression>)[] + { + ("Seats.LowerBound()", r => r.Seats.LowerBound()), + ("Seats.UpperBound()", r => r.Seats.UpperBound()), + ("SeatBlocks.LowerBound()", r => r.SeatBlocks.LowerBound()), + ("SeatBlocks.UpperBound()", r => r.SeatBlocks.UpperBound()) + }) + { + var compiled = key.Compile(); + var present = Expression.Lambda>( + Expression.NotEqual(key.Body, Expression.Constant(null, typeof(int?))), key.Parameters); + + foreach (var descending in new[] { false, true }) + { + var order = descending ? $"OrderByDescending({name})" : $"OrderBy({name})"; + + var memory = (descending ? loaded.OrderByDescending(compiled) : loaded.OrderBy(compiled)) + .ThenBy(r => r.Id).Select(r => r.Id).ToList(); + + // The documented placement: the same order, with the rows whose bound is null moved to + // the end ascending and to the front descending. + var withoutBound = memory.Where(id => compiled(loaded.Single(r => r.Id == id)) is null).ToList(); + var withBound = memory.Except(withoutBound).ToList(); + List documented = descending ? [.. withoutBound, .. withBound] : [.. withBound, .. withoutBound]; + + var plain = await (descending ? mine.OrderByDescending(key) : mine.OrderBy(key)) + .ThenBy(r => r.Id).Select(r => r.Id).ToListAsync(); + if (!plain.SequenceEqual(documented)) + failures.Add($"{order}: [{string.Join(",", plain)}], documented [{string.Join(",", documented)}]"); + + var keyed = await (descending ? mine.OrderByDescending(present).ThenByDescending(key) : mine.OrderBy(present).ThenBy(key)) + .ThenBy(r => r.Id).Select(r => r.Id).ToListAsync(); + if (!keyed.SequenceEqual(memory)) + failures.Add($"{order} after a Bound() != null key: [{string.Join(",", keyed)}], memory [{string.Join(",", memory)}]"); + } + } + + Assert.AreEqual(0, failures.Count, string.Join(Environment.NewLine, failures)); + } + /// /// LowerBound() and UpperBound() compared with , read through /// HasValue, coalesced with ?? in a filter and a projection, and filtered after being diff --git a/test/oracles/claims.json b/test/oracles/claims.json index 5d798c98..1af7d9e4 100644 --- a/test/oracles/claims.json +++ b/test/oracles/claims.json @@ -1154,6 +1154,22 @@ "NullOperandTranslationTests" ] }, + { + "id": "E72-ordering-by-a-bound-puts-nulls-where-postgres-does", + "source": "docs/efcore.md#read-bounds-merge-ranges-and-aggregate", + "quote": "**Ordering by a bound puts the rows without it where PostgreSQL puts NULLs.** `OrderBy(b => b.Period.LowerBound())` sorts the empty rows and the rows with no lower bound last, and `OrderByDescending(b => b.Period.UpperBound())` sorts those with no upper bound first. That holds on a required column too, because the bound is NULL there.", + "claim": "Over a required range column and a required multirange column holding every shape, ordering by LowerBound() or UpperBound(), ascending or descending, gives the in-memory order with the rows whose bound is null moved to the end ascending and to the front descending; with a Bound() != null key in front (OrderBy/ThenBy, or OrderByDescending/ThenByDescending), it gives the in-memory order exactly.", + "oracle": "claim", + "surfaces": [ + "RangeExtensions.LowerBound", + "RangeExtensions.UpperBound", + "RangeSet.LowerBound", + "RangeSet.UpperBound" + ], + "heldBy": [ + "ExecutedQueryTests" + ] + }, { "id": "J01-tostring-is-postgres-literal", "source": "docs/serialization.md#print-a-range-as-a-literal", diff --git a/test/oracles/seeds/bound-declared-null-only-for-a-null-argument.patch b/test/oracles/seeds/bound-declared-null-only-for-a-null-argument.patch new file mode 100644 index 00000000..385c98bf --- /dev/null +++ b/test/oracles/seeds/bound-declared-null-only-for-a-null-argument.patch @@ -0,0 +1,16 @@ +Seed: lower() and upper() are declared NULL only for a NULL argument again, so EF Core takes the bound of a required column as never NULL (#349) +Oracles: ExecutedQueryTests +Fails: BoundsComparedWithNull_SelectTheRowsMemorySelects on every operation and column, and OrderingByABound_PutsTheRowsWithoutItWherePostgresPutsNulls on the Bound() != null key, which folds to TRUE + +diff --git a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs +--- a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs ++++ b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs +@@ -681,7 +681,7 @@ + name, + [argument], + nullable: true, +- argumentsPropagateNullability: [false], ++ argumentsPropagateNullability: [true], + typeof(Nullable<>).MakeGenericType(definition.ElementClrType), + ElementMapping(definition)); +