diff --git a/CHANGELOG.md b/CHANGELOG.md index dbac594e..3d583280 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,12 @@ has the rules. ## [10.2.0] +Some calls answer differently: [Changed answers in 10.2.0](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020). + +### Changed answers + +- EF Core: `LowerBound()` and `UpperBound()` in a query are `NULL` for an empty range or set and on an unbounded side, as in memory. So `== null` selects those rows, where it selected none on a required column, and `?? fallback` returns the fallback, where the row was missing ([#354](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/354), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). + ## [10.1.0] — 2026-10-19 One call answers differently, on the .NET 11 runtime only: [Changed answers in 10.1.0](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1010). diff --git a/docs/efcore.md b/docs/efcore.md index 8332088f..10275c79 100644 --- a/docs/efcore.md +++ b/docs/efcore.md @@ -147,7 +147,9 @@ Good to know: have a gap. PostgreSQL normalizes multiranges the same way this library does, so the equality is exact ([verified against live PostgreSQL](../README.md#verified-against-postgresql)). - **`LowerBound()` and `UpperBound()` return `T?`.** PostgreSQL's `lower` and `upper` return `NULL` - for an unbounded or empty range, and the in-memory methods match that. + for an unbounded or empty range, and the in-memory methods match that. So `== null`, `HasValue` + and `??` on a bound select the same rows in a query as in memory, on a range or a `RangeSet` + column. - **Discrete upper bounds are adjusted for you.** For `int4range`, `int8range` and `daterange`, PostgreSQL normalizes to the half-open form `[lower, upper)`. This library normalizes to the closed form `[lower, upper]`. So `UpperBound()` becomes `upper(x) - 1`, and diff --git a/docs/migration.md b/docs/migration.md index 0a604fb6..775bd3c2 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -785,3 +785,42 @@ runtimes. Look at the text instead: - **Say "the end of the day" another way.** Use an unbounded end, `[22:00,)`, or an inclusive `23:59:59.9999999` bound ([TimeRange and the custom timerange type](efcore.md#timerange-and-the-custom-timerange-type)). + +## Changed answers in 10.2.0 + +Some calls return a different value. Your code stays the same, so the compiler can't show you where. +Each section below says how to find the calls whose answer changed. + +### Check bound comparisons with `null` in queries + +Check EF Core queries that call `LowerBound()` or `UpperBound()` on a range or `RangeSet` column and +then test the result for `null`. That covers `== null`, `!= null`, `HasValue` and `??` with a +fallback, in a filter or a projection. It also covers a filter on a bound you projected first, which +is what a data grid sends for a "blank" filter. + +PostgreSQL's `lower` and `upper` return `NULL` for an empty range and for an unbounded side. The +in-memory methods return `null` for the same values. Before 10.2.0, the EF Core package declared +that a bound is `NULL` only when the column is, and EF Core simplified the query on that basis. On a +required column: + +| Query | Before 10.2.0 | Now | +|---|---|---| +| `Where(r => r.Period.LowerBound() == null)` | no rows | the empty rows and the rows with no lower bound | +| `Where(r => r.Period.UpperBound() != null)`, or `HasValue` | every row | the rows with an upper bound | +| `Where(r => (r.Period.UpperBound() ?? DateOnly.MaxValue) >= day)` | the empty rows and the rows with no upper bound are missing | those rows match through the fallback | +| `Select(r => r.Period.UpperBound() ?? DateOnly.MaxValue)` | throws (`InvalidOperationException`) at an empty row or a row with no upper bound | returns the fallback for it | +| `Select(r => new { To = r.Period.UpperBound() }).Where(x => x.To == null)` | no rows | the empty rows and the rows with no upper bound | + +On a nullable column, `LowerBound() == null` ran as the column's own `IS NULL`. It found the `NULL` +rows and missed the empty and unbounded ones. + +This holds for every range type and for `RangeSet`: the .NET and NodaTime ranges, `TimeRange`, the +float ranges and `YearMonthRange`. Code that runs in memory is unchanged. + +- **Search your queries for `LowerBound()` and `UpperBound()`.** Each one followed by `== null`, + `!= null`, `HasValue` or `??` now returns the rows the same code returns in memory. +- **Keep or drop the workarounds you wrote for it.** A query that adds `IsEmpty()`, + `IsUnboundedStart()` or `IsUnboundedEnd()` beside the `null` test returns the same rows as before, + so the extra test is now optional. +- **Check reports built on the old rows.** On a required column, a count of rows with no end date + was 0, and a filter on a date with a `??` fallback left out every open-ended row. diff --git a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL.NodaTime/CHANGELOG.md b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL.NodaTime/CHANGELOG.md index 0fe49ba1..8f73a247 100644 --- a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL.NodaTime/CHANGELOG.md +++ b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL.NodaTime/CHANGELOG.md @@ -5,7 +5,9 @@ covers all four packages, which share one version number and release together. ## [10.2.0] -No change in this package. +### Changed answers + +- EF Core: `LowerBound()` and `UpperBound()` of a NodaTime range or multirange column, `YearMonthRange` included, are `NULL` in a query for an empty value and on an unbounded side, as in memory. So `== null` selects those rows, where it selected none on a required column, and `?? fallback` returns the fallback, where the row was missing ([#354](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/354), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). ## [10.1.0] — 2026-10-19 diff --git a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md index 3bda39b5..51439083 100644 --- a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md +++ b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md @@ -5,7 +5,9 @@ covers all four packages, which share one version number and release together. ## [10.2.0] -No change in this package. +### Changed answers + +- EF Core: `LowerBound()` and `UpperBound()` in a query are `NULL` for an empty range or set and on an unbounded side, as in memory. So `== null` selects those rows, where it selected none on a required column, and `?? fallback` returns the fallback, where the row was missing ([#354](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/issues/354), [Migration](https://github.com/CaffeinatedCoder/CodoMetis.ValueRanges/blob/main/docs/migration.md#changed-answers-in-1020)). ## [10.1.0] — 2026-10-19 diff --git a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs index 95436f9c..fe6341cc 100644 --- a/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs +++ b/src/CodoMetis.ValueRanges.EFCore.PostgreSQL/Query/ValueRangesMethodCallTranslator.cs @@ -634,14 +634,17 @@ private SqlExpression BoolFunction(string name, SqlExpression argument) /// /// A bound-extraction function (lower/upper) on a range or multirange. /// PostgreSQL returns NULL for an empty or correspondingly unbounded operand, hence the - /// nullable CLR type. + /// nullable CLR type. That NULL comes from a non-null argument, so the argument must not be + /// declared as propagating nullability: EF Core would then take the bound of a required column + /// as never NULL, fold IS NULL to FALSE and drop the COALESCE of a + /// ??, and test a nullable column's bound with the column's own IS NULL. /// private SqlExpression BoundFunction(string name, SqlExpression argument, IRangeTypeDefinition definition) => sqlExpressionFactory.Function( name, [argument], nullable: true, - argumentsPropagateNullability: [true], + argumentsPropagateNullability: [false], typeof(Nullable<>).MakeGenericType(definition.ElementClrType), ElementMapping(definition)); diff --git a/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs b/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs index 74374eac..4455f43f 100644 --- a/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs +++ b/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/ExecutedQueryTests.cs @@ -1,4 +1,13 @@ +using System.Data.Common; +using System.Linq.Expressions; +using System.Reflection; +using System.Runtime.CompilerServices; +using CodoMetis.ValueRanges.Core; +using CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests.Fuzzing; +using CodoMetis.ValueRanges.Oracles; using Microsoft.EntityFrameworkCore; +using NodaTime; +using Npgsql; namespace CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests; @@ -6,6 +15,7 @@ namespace CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests; /// Executes the translated SQL against live PostgreSQL and asserts that the server's /// answers agree with the in-memory implementations — the core promise of the library. /// Each test seeds its own rows (unique Ids / GroupKeys) so tests can run in parallel. +/// The two sweeps at the end collect their disagreements and end in . /// [TestClass] public class ExecutedQueryTests @@ -622,4 +632,310 @@ await Seed( .SingleAsync(); Assert.AreEqual(periods.Take(2).RangeIntersectAgg(), serverIntersectAgg); } + + /// + /// LowerBound() and UpperBound() compared with , read through + /// HasValue, coalesced with ?? in a filter and a projection, and filtered after being + /// projected, for a range column and a multirange column of every family, against the same + /// expression over the rows as loaded. The bounds are NULL for the empty range and for the unbounded + /// side of a non-null one, so these forms only agree while EF Core knows lower and + /// upper can be NULL for a non-null argument (#349). Each failure line starts with the + /// operation, then the column. Ids run from 2400 to 2659, twenty per family. + /// + [TestMethod] + public async Task BoundsComparedWithNull_SelectTheRowsMemorySelects() + { + ContainerLifecycle.RequireDatabase(); + + var at = new DateTime(2026, 3, 1, 8, 0, 0); + var utc = new DateTimeOffset(at, TimeSpan.Zero); + LocalDate Day(int day) => new(2026, 3, day); + + var failures = new List(); + failures.AddRange(await BoundNullSweep(2400, r => r.Seats, r => r.SeatBlocks, 1, 10, 20, int.MinValue, int.MaxValue)); + failures.AddRange(await BoundNullSweep(2420, r => r.Tickets, r => r.TicketBlocks, 1L, 10L, 20L, long.MinValue, long.MaxValue)); + failures.AddRange(await BoundNullSweep(2440, r => r.Price, r => r.PriceBands, 1.5m, 10m, 20.25m, decimal.MinValue, decimal.MaxValue)); + failures.AddRange(await BoundNullSweep(2460, r => r.Period, r => r.BlockedDays, new DateOnly(2026, 3, 1), new DateOnly(2026, 3, 15), + new DateOnly(2026, 3, 31), DateOnly.MinValue, DateOnly.MaxValue)); + failures.AddRange(await BoundNullSweep(2480, r => r.LocalTime, r => r.WallClockWindows, at, at.AddDays(1), at.AddDays(2), + DateTime.MinValue, DateTime.MaxValue)); + failures.AddRange(await BoundNullSweep(2500, r => r.InstantTime, r => r.InstantWindows, utc, utc.AddDays(1), utc.AddDays(2), + DateTimeOffset.MinValue, DateTimeOffset.MaxValue)); + failures.AddRange(await BoundNullSweep(2520, r => r.NodaPeriod, r => r.NodaBlockedDays, Day(1), Day(15), Day(31), + LocalDate.MinIsoValue, LocalDate.MaxIsoValue)); + failures.AddRange(await BoundNullSweep(2540, r => r.NodaWallClock, r => r.NodaWallClocks, LocalDateTime.FromDateTime(at), + LocalDateTime.FromDateTime(at.AddDays(1)), LocalDateTime.FromDateTime(at.AddDays(2)), + LocalDateTime.MinIsoValue, LocalDateTime.MaxIsoValue)); + failures.AddRange(await BoundNullSweep(2560, r => r.NodaWindow, r => r.NodaWindows, Instant.FromDateTimeOffset(utc), + Instant.FromDateTimeOffset(utc.AddDays(1)), Instant.FromDateTimeOffset(utc.AddDays(2)), + Instant.MinValue, Instant.MaxValue)); + // time is microsecond-precise, so TimeOnly.MaxValue would round to 24:00; any fallback the column holds will do. + failures.AddRange(await BoundNullSweep(2580, r => r.OpeningHours, r => r.OpeningWindows, new TimeOnly(9, 0), new TimeOnly(12, 0), + new TimeOnly(17, 0), TimeOnly.MinValue, new TimeOnly(23, 59, 59))); + failures.AddRange(await BoundNullSweep(2600, r => r.Score, r => r.ScoreBands, 1.5, 10.0, 20.25, double.MinValue, double.MaxValue)); + failures.AddRange(await BoundNullSweep(2620, r => r.Load, r => r.LoadBands, 1.5f, 10f, 20.25f, float.MinValue, float.MaxValue)); + failures.AddRange(await BoundNullSweep(2640, r => r.BillingPeriod, r => r.BillingPeriods, new YearMonth(2026, 1), new YearMonth(2026, 6), + new YearMonth(2026, 12), new YearMonth(1, 1), new YearMonth(9999, 12))); + + Candidates.Verdict(nameof(ExecutedQueryTests), failures, + "LowerBound()/UpperBound() against null in a query: PostgreSQL and memory select different rows", + method: nameof(BoundsComparedWithNull_SelectTheRowsMemorySelects)); + } + + /// + /// Seeds one row per shape into , and the same shape as a one-element + /// multirange into — finite across, below and above , + /// unbounded on either side of it, infinite and empty — and runs every bound operation on both + /// columns, on the server and over the rows as loaded. and + /// are the ?? fallbacks; they must round-trip through the column's + /// element type, or the projection disagrees for that reason alone. + /// + private static async Task> BoundNullSweep( + int firstId, Expression> column, Expression>> set, + T a, T v, T b, T min, T max) + where TRange : class, IRangeFactory, IRange + where T : struct, IComparable, IEquatable + { + (string Name, TRange Range)[] shapes = + [ + ("[a,b]", TRange.CreateFinite(a, b, true, true)), + ("[a,a]", TRange.CreateFinite(a, a, true, true)), + ("[b,b]", TRange.CreateFinite(b, b, true, true)), + ("(,a]", TRange.CreateUnboundedStart(a, true)), + ("(,b]", TRange.CreateUnboundedStart(b, true)), + ("[a,)", TRange.CreateUnboundedEnd(a, true)), + ("[b,)", TRange.CreateUnboundedEnd(b, true)), + ("(,)", TRange.Infinite), + ("empty", TRange.Empty) + ]; + + var property = (PropertyInfo)((MemberExpression)column.Body).Member; + var setProperty = (PropertyInfo)((MemberExpression)set.Body).Member; + var names = new Dictionary(); + var rows = new List(); + foreach (var (shape, index) in shapes.Select((shape, index) => (shape, index))) + { + var row = new Reservation { Id = firstId + index }; + property.SetValue(row, shape.Range); + setProperty.SetValue(row, RangeSet.From(shape.Range)); + rows.Add(row); + names[row.Id] = shape.Name; + } + + await Seed([.. rows]); + + var row0 = column.Parameters[0]; + var setColumn = Expression.Property(row0, setProperty); + var lastId = firstId + shapes.Length - 1; + + await using var context = new IntegrationDbContext(); + var mine = context.Reservations.Where(r => r.Id >= firstId && r.Id <= lastId); + var loaded = await mine.AsNoTracking().OrderBy(r => r.Id).ToListAsync(); + var failures = new List(); + + // The range's bounds are the IRange extensions and the set's its own methods, each spliced + // onto its column, so the call is the one a consumer's r.Column.LowerBound() makes. + await Sweep($"{property.Name} ({typeof(TRange).Name})", + Bound>(column.Body, (_, x) => x.LowerBound()), Bound>(column.Body, (_, x) => x.UpperBound())); + await Sweep($"{setProperty.Name} (RangeSet<{typeof(TRange).Name}>)", + Bound>(setColumn, (_, x) => x.LowerBound()), Bound>(setColumn, (_, x) => x.UpperBound())); + + return failures; + + Expression> Bound(Expression on, Expression> bound) => + Expression.Lambda>(FuzzEngine.Splice(bound, row0, on), row0); + + // The operation written over the bound, spliced onto it. + Expression> Over(Expression> bound, Expression> operation) => + Expression.Lambda>(FuzzEngine.Splice(operation, row0, bound.Body), row0); + + // (bound ?? fallback) op v, with both values captured, so EF Core sends them as parameters. + Expression> Coalesced(ExpressionType comparison, Expression> bound, T fallback) => + Expression.Lambda>( + Expression.MakeBinary(comparison, Expression.Coalesce(bound.Body, Captured(fallback)), Captured(v)), row0); + + async Task Sweep(string label, Expression> lower, Expression> upper) + { + (string Name, Expression> Filter)[] filters = + [ + ("LowerBound() == null", Over(lower, (_, x) => x == null)), + ("LowerBound() != null", Over(lower, (_, x) => x != null)), + ("LowerBound().HasValue", Over(lower, (_, x) => x.HasValue)), + ("UpperBound() == null", Over(upper, (_, x) => x == null)), + ("UpperBound() != null", Over(upper, (_, x) => x != null)), + ("(UpperBound() ?? max) >= v", Coalesced(ExpressionType.GreaterThanOrEqual, upper, max)), + ("(LowerBound() ?? min) <= v", Coalesced(ExpressionType.LessThanOrEqual, lower, min)) + ]; + + foreach (var (name, filter) in filters) + { + var memory = loaded.Where(filter.Compile()).Select(r => r.Id).ToList(); + await Agree(label, name, () => mine.Where(filter).Select(r => r.Id).OrderBy(id => id).ToListAsync(), memory); + } + + var projection = Expression.Lambda>(Expression.Coalesce(upper.Body, Captured(max)), row0); + var projected = loaded.Select(projection.Compile()).ToList(); + try + { + var server = await mine.OrderBy(r => r.Id).Select(projection).ToListAsync(); + if (!server.SequenceEqual(projected)) + failures.Add($"Select(UpperBound() ?? max) on {label}: server [{string.Join(", ", server)}], memory [{string.Join(", ", projected)}]"); + } + catch (Exception e) when (e is InvalidOperationException or DbException) + { + failures.Add($"Select(UpperBound() ?? max) on {label}: server threw {e.GetType().Name}: {e.Message}"); + } + + // The filter a grid's "blank" header filter sends: the bound projected first, compared after. + var withUpper = Over(upper, (r, x) => new { r.Id, To = x }); + await Agree(label, "Select(new { Id, To = UpperBound() }).Where(To == null)", + () => mine.Select(withUpper).Where(t => t.To == null).Select(t => t.Id).OrderBy(id => id).ToListAsync(), + [.. loaded.Select(withUpper.Compile()).Where(t => t.To == null).Select(t => t.Id)]); + } + + async Task Agree(string label, string operation, Func>> server, List memory) + { + try + { + var rowsServer = await server(); + if (!rowsServer.SequenceEqual(memory)) + failures.Add($"{operation} on {label}: server rows {Named(rowsServer)}, memory rows {Named(memory)}"); + } + catch (Exception e) when (e is InvalidOperationException or DbException) + { + failures.Add($"{operation} on {label}: server threw {e.GetType().Name}: {e.Message}"); + } + } + + string Named(IEnumerable ids) => $"{{{string.Join(" ", ids.Select(id => names[id]))}}}"; + } + + /// + /// Range predicates over a nullable column holding NULL and every shape, against the same + /// expression over the rows as loaded. On a NULL column every call is NULL in SQL; in memory some + /// extensions answer a value on a null receiver, so their negated and vacuous forms keep or drop the + /// NULL row where SQL does the opposite (#350). A row whose in-memory evaluation throws has no answer + /// to compare and is left out on both sides. The column lives in a database of its own, so the + /// shared schema keeps its required columns. + /// + [TestMethod] + public async Task PredicatesOverANullableColumn_SelectTheRowsMemorySelects() + { + ContainerLifecycle.RequireServer(); + + var connectionString = await NullableRangeContext.CreateDatabase(); + + DateOnly March(int day) => new(2026, 3, day); + (string Name, DateRange? Range)[] shapes = + [ + ("null", null), + ("empty", DateRange.Empty), + ("(,)", DateRange.Infinite), + ("[1,10]", DateRange.CreateFinite(March(1), March(10), true, true)), + ("[20,25]", DateRange.CreateFinite(March(20), March(25), true, true)), + ("(,10]", DateRange.CreateUnboundedStart(March(10), true)), + ("[20,)", DateRange.CreateUnboundedEnd(March(20), true)) + ]; + var names = shapes.Select((shape, index) => (Id: index + 1, shape.Name)).ToDictionary(s => s.Id, s => s.Name); + + await using (var seed = new NullableRangeContext(connectionString)) + { + await seed.Database.EnsureCreatedAsync(); + seed.Rows.AddRange(shapes.Select((shape, index) => new NullableRangeRow { Id = index + 1, Period = shape.Range })); + await seed.SaveChangesAsync(); + } + + var p = DateRange.CreateFinite(March(5), March(22), true, true); + var v = March(7); + + (string Name, Expression> Filter)[] filters = + [ + ("!IsEmpty()", r => !r.Period!.IsEmpty()), + ("!IsFinite()", r => !r.Period!.IsFinite()), + ("!IsUnboundedEnd()", r => !r.Period!.IsUnboundedEnd()), + ("!UpperBoundInclusive()", r => !r.Period!.UpperBoundInclusive()), + ("!Overlaps(p)", r => !r.Period!.Overlaps(p)), + ("Contains(Empty)", r => r.Period!.Contains(DateRange.Empty)), + ("IsContainedBy(Infinite)", r => r.Period!.IsContainedBy(DateRange.Infinite)), + ("Intersect(p) != null", r => r.Period!.Intersect(p) != null), + ("Intersect(p) == Empty", r => r.Period!.Intersect(p) == DateRange.Empty), + ("Gap(Empty).IsEmpty()", r => r.Period!.Gap(DateRange.Empty).IsEmpty()), + ("Union(Infinite).Contains(v)", r => r.Period!.Union(DateRange.Infinite).Contains(v)), + ("Merge(Infinite) == x", r => r.Period!.Merge(DateRange.Infinite) == r.Period) + ]; + + await using var context = new NullableRangeContext(connectionString); + var loaded = await context.Rows.AsNoTracking().OrderBy(r => r.Id).ToListAsync(); + var failures = new List(); + + foreach (var (name, filter) in filters) + { + var test = filter.Compile(); + var judged = new HashSet(); + var memory = new List(); + foreach (var row in loaded) + { + bool holds; + try { holds = test(row); } + catch (Exception e) when (e is NullReferenceException or InvalidOperationException or ArgumentException) { continue; } + + judged.Add(row.Id); + if (holds) memory.Add(row.Id); + } + + // One line per row, so a candidate names the row it quarantines and every other row stays judged. + try + { + var server = (await context.Rows.Where(filter).Select(r => r.Id).ToListAsync()).ToHashSet(); + foreach (var id in judged.Order().Where(id => server.Contains(id) != memory.Contains(id))) + failures.Add($"{name} on DateRange? {names[id]}: server {Selects(server.Contains(id))} the row, memory {Selects(memory.Contains(id))} it"); + } + catch (Exception e) when (e is InvalidOperationException or DbException) + { + failures.Add($"{name} on DateRange?: server threw {e.GetType().Name}: {e.Message}"); + } + } + + Candidates.Verdict(nameof(ExecutedQueryTests), failures, + "Range predicates over a nullable column: PostgreSQL and memory select different rows", + method: nameof(PredicatesOverANullableColumn_SelectTheRowsMemorySelects)); + + static string Selects(bool selected) => selected ? "selects" : "drops"; + } + + /// A captured variable, which EF Core sends as a parameter. + private static MemberExpression Captured(T value) => + Expression.Field(Expression.Constant(new StrongBox(value)), nameof(StrongBox<>.Value)); + + public sealed class NullableRangeRow + { + public int Id { get; set; } + + public DateRange? Period { get; set; } + } + + /// A database of its own, so that a nullable range column does not change the shared schema. + private sealed class NullableRangeContext(string connectionString) : DbContext + { + public DbSet Rows => Set(); + + public static async Task CreateDatabase() + { + var name = $"nullable_range_{Guid.NewGuid():N}"; + await using (var connection = new NpgsqlConnection(ContainerLifecycle.ServerConnectionString)) + { + await connection.OpenAsync(); + await using var command = new NpgsqlCommand($"CREATE DATABASE {name}", connection); + await command.ExecuteNonQueryAsync(); + } + + return new NpgsqlConnectionStringBuilder(ContainerLifecycle.ServerConnectionString) { Database = name }.ConnectionString; + } + + protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) + => optionsBuilder.UseNpgsql(connectionString, npgsql => npgsql.UseValueRanges()); + + protected override void OnModelCreating(ModelBuilder modelBuilder) + => modelBuilder.Entity().Property(r => r.Id).ValueGeneratedNever(); + } } diff --git a/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.Tests/QueryTranslationTests.cs b/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.Tests/QueryTranslationTests.cs index 7a50a9e3..13231407 100644 --- a/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.Tests/QueryTranslationTests.cs +++ b/test/CodoMetis.ValueRanges.EFCore.PostgreSQL.Tests/QueryTranslationTests.cs @@ -262,6 +262,28 @@ public void LowerBound_InOrderBy() StringAssert.Contains(sql, "ORDER BY lower(b.\"Period\")"); } + /// + /// lower and upper are NULL for an empty or unbounded operand that is not NULL itself, + /// so a required column's bound can still be NULL. Declared as propagating its argument's + /// nullability, EF Core folded these to WHERE FALSE and dropped the COALESCE (#349). + /// ExecutedQueryTests.BoundsComparedWithNull_SelectTheRowsMemorySelects is the authority. + /// + [TestMethod] + public void BoundComparedWithNull_OnARequiredColumn_TestsTheBound() + { + var sql = Sql(db => db.Bookings.Where(b => b.Period.LowerBound() == null || b.BlockedDays.UpperBound() == null)); + StringAssert.Contains(sql, "lower(b.\"Period\") IS NULL"); + StringAssert.Contains(sql, "upper(b.\"BlockedDays\") IS NULL"); + Assert.IsFalse(sql.Contains("FALSE"), sql); + } + + [TestMethod] + public void BoundCoalesced_OnARequiredColumn_KeepsTheCoalesce() + { + var sql = Sql(db => db.Bookings.Where(b => (b.Period.UpperBound() ?? DateOnly.MaxValue) >= Day)); + StringAssert.Contains(sql, "COALESCE(upper(b.\"Period\") - 1, "); + } + [TestMethod] public void UpperBound_Discrete_CompensatesHalfOpenCanonicalization() { diff --git a/test/oracles/candidates.json b/test/oracles/candidates.json index 0e271757..d8638340 100644 --- a/test/oracles/candidates.json +++ b/test/oracles/candidates.json @@ -26,5 +26,89 @@ "family": "(,max] needs max + 1 to canonicalize; PostgreSQL raises 22003", "issue": "#132", "method": "EveryEntryPoint_ReadsTheLiteralAsPostgresReadsIt" + }, + { + "oracle": "ExecutedQueryTests", + "row": "!IsEmpty() on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "!IsFinite() on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "!IsUnboundedEnd() on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "!UpperBoundInclusive() on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "!Overlaps(p) on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "Contains(Empty) on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "IsContainedBy(Infinite) on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "Intersect(p) != null on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "Intersect(p) == Empty on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "Gap(Empty).IsEmpty() on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "Union(Infinite).Contains(v) on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" + }, + { + "oracle": "ExecutedQueryTests", + "row": "Merge(Infinite) == x on DateRange? null:", + "family": "a null receiver answers in memory where SQL answers NULL", + "issue": "#350", + "method": "PredicatesOverANullableColumn_SelectTheRowsMemorySelects" } ] diff --git a/test/oracles/claims.json b/test/oracles/claims.json index 4ed5d975..b90d43e3 100644 --- a/test/oracles/claims.json +++ b/test/oracles/claims.json @@ -1121,6 +1121,22 @@ "TimeAndYearMonthIntegrationTests" ] }, + { + "id": "E70-bounds-null-for-unbounded-or-empty-in-sql", + "source": "docs/efcore.md#read-bounds-merge-ranges-and-aggregate", + "quote": "**`LowerBound()` and `UpperBound()` return `T?`.** PostgreSQL's `lower` and `upper` return `NULL` for an unbounded or empty range, and the in-memory methods match that. So `== null`, `HasValue` and `??` on a bound select the same rows in a query as in memory, on a range or a `RangeSet` column.", + "claim": "In a query over a range or multirange column of any family, LowerBound() and UpperBound() are NULL exactly for the empty range or set and for the unbounded side, as in memory: compared with null, read through HasValue, coalesced with ?? in a filter or a projection, or compared with null after being projected, they select and return what the same expression gives over the rows as loaded.", + "oracle": "differential", + "surfaces": [ + "RangeExtensions.LowerBound", + "RangeExtensions.UpperBound", + "RangeSet.LowerBound", + "RangeSet.UpperBound" + ], + "heldBy": [ + "ExecutedQueryTests" + ] + }, { "id": "J01-tostring-is-postgres-literal", "source": "docs/serialization.md#print-a-range-as-a-literal",